SQuAD: Babban Tsarin Bayanai na Karatu da Fahimta don Nazarin Harshe ta Hanyar Kwamfuta (NLP)

1. Gabatarwa & Bayyani

Fahimtar Karatu (RC) kalubale ce ta asali a cikin Sarrafa Harshe ta Hanyar Kwamfuta (NLP), tana buƙatar injuna su fahimci rubutu kuma su amsa tambayoyi game da shi. Kafin SQuAD, fagen bai da babban tsarin bayanai mai inganci, mai girma wanda ke kwatanta ainihin fahimtar karatu ta ɗan adam. Tsarin bayanai da ake da su ko dai sun yi ƙanƙanta don horar da ƙirar zamani masu cike da bayanai (misali, MCTest) ko kuma ƙirar ƙirar ƙira ce, sun kasa ɗaukar ƙayyadaddun tambayoyi na gaske. An gabatar da Tsarin Bayanai na Tambayoyi na Stanford (SQuAD) don rage wannan gibi, yana ba da ma'auni wanda tun daga lokacin ya zama ginshiƙi don kimanta ƙirar fahimtar inji.

2. Tsarin Bayanai na SQuAD

2.1 Gina Tsarin Bayanai & Girma

An ƙirƙiri SQuAD v1.0 ta hanyar ma'aikatan jama'a waɗanda suka gabatar da tambayoyi bisa labaran Wikipedia 536. Amsar kowace tambaya wani ɓangare ne na rubutu daga sashin da ya dace. Wannan ya haifar da 107,785 tambayoyi-amsoshi biyu-biyu, wanda ya sa ya zama kusan sau biyu mafi girma fiye da tsarin bayanai na RC da aka yiwa lakabi da hannu kamar MCTest.

2.2 Muhimman Halaye & Tsarin Amsa

Siffa ta musamman ta SQuAD ita ce tsarin amsarta na tushen ɓangare. Ba kamar tambayoyi masu zaɓi da yawa ba, dole ne tsarin ya gano ainihin sashin rubutu daga cikin nassi wanda ke amsa tambayar. Wannan tsarin:

• Yana gabatar da aiki mai gaskiya da ƙalubale, saboda ƙirar dole ne ta kimanta duk yuwuwar ɓangarorin.
• Yana ba da damar ƙima madaidaiciya kuma mai ma'ana ta hanyar daidaitaccen daidaitawa da ma'aunin maki F1.
• Yana ɗaukar nau'ikan tambayoyi daban-daban, daga sauƙaƙan tambayoyin gaskiya zuwa waɗanda ke buƙatar tunani na ƙamus ko tsari.

Misali daga takardar shine tambayar "Me ke haifar da ruwan sama ya faɗo?" akan nassi na yanayin sama, inda ainihin ɓangaren amsa shine "nauyi".

3. Binciken Fasaha & Hanyoyin Aiki

3.1 Tsarin Ma'auni & Siffofi

Don kafa ma'auni, marubutan sun aiwatar da ƙirar koma baya na logistic. Muhimman siffofi sun haɗa da:

• Siffofin Ƙamus: Haɗuwar kalmomi da n-grams tsakanin tambaya da nassi.
• Siffofin Tsari: Hanyoyi a cikin bishiyoyin dogaro da ke haɗa kalmomin tambaya zuwa ga ɓangarorin amsa masu yuwuwa.
• Siffofin ɓangare: Halayen ɓangaren amsa mai yuwuwa da kansa (misali, tsayi, matsayi).

Ƙirar ta sami maki F1 na 51.0%, wanda ya fi sauƙaƙan ma'auni (20%) amma ya yi nisa da aikin ɗan adam (86.8%).

3.2 Rarraba Matsaloli

Marubutan sun haɓaka fasahohi ta atomatik don bincika wahalar tambaya, galibi ta amfani da nisa a cikin bishiyoyin rarraba dogaro. Sun gano cewa aikin ƙirar ya ragu tare da:

1. Ƙara rikitarwar nau'in amsa (misali, sunayen abubuwa da aka sanya suna da jimlolin bayani).
2. Bambance-bambancen tsari mafi girma tsakanin tambaya da jimlar da ke ɗauke da amsar.

Wannan rarrabuwar ya ba da hangen nesa mai zurfi game da ƙalubalen tsarin bayanai fiye da jimillar maki.

4. Sakamakon Gwaji & Aiki

Sakamakon farko ya nuna bambanci mai mahimmanci tsakanin aikin inji da na ɗan adam.

• Tsarin Ma'auni (Koma Baya na Logistic): 51.0% makin F1.
• Aikin Dan Adam: 86.8% makin F1.

Wannan tazarar kusan maki 36 ta nuna a sarari cewa SQuAD ya gabatar da babban ƙalubale da ba a warware ba, wanda ya sa ya zama ma'auni mai kyau don tafiyar da bincike na gaba. Takardar kuma ta haɗa da bincike da ke nuna raguwar aiki a cikin nau'ikan tambayoyi daban-daban da matakan wahala, kamar yadda aka fahimta daga ma'aunin bishiyar dogaro.

5. Cikakken Bincike & Hikimar Kwararru

Cikakken Hikima: Rajpurkar da sauransu ba kawai sun ƙirƙiri wani tsarin bayanai ba; sun ƙirƙiri kayan aikin bincike mai daidaito da filin gasa wanda ya fallasa zurfin zurfin ƙirar NLP na lokacin. Hazakar SQuAD tana cikin ƙayyadaddun tsarinta na tushen ɓangare—ya tilasta wa ƙirar su karanta da kuma gano shaida da gaske, suna motsawa bayan daidaitawar maɓalli ko dabarar zaɓi da yawa. Bayyanannen da aka yi nan da nan na tazarar maki 35.8 tsakanin mafi kyawun ƙirar su na koma baya na logistic da aikin ɗan adam ya kasance kira mai karfi, yana nuna ba kawai tazarar aiki ba amma ainihin tazarar fahimta.

Tsarin Hankali: Hankalin takardar yana da tasiri sosai. Ya fara da binciken cutar fagen: rashin babban ma'auni na RC mai inganci. Sannan ya ba da magani: SQuAD, wanda aka gina ta hanyar tara jama'a mai yawa akan abun ciki na Wikipedia mai daraja. An gabatar da tabbacin tasiri ta hanyar ƙirar ma'auni mai tsauri wanda ke amfani da siffofi masu fassara (haɗuwar ƙamus, hanyoyin dogaro), waɗanda yanayin gazawarsu daga nan aka raba su ta hanyar amfani da bishiyoyin tsari. Wannan ya haifar da zagaye mai kyau: tsarin bayanai ya fallasa raunuka, kuma binciken ya ba da taswirar farko na waɗannan raunukan don masu bincike na gaba su kai hari.

Ƙarfi & Kurakurai: Babban ƙarfi shine tasirin canjin SQuAD. Kamar ImageNet don hangen nesa, ya zama tauraro ta arewa don fahimtar inji, yana haɓaka haɓaka ƙirar da ke ƙara rikitarwa, daga BiDAF zuwa BERT. Kurakuransa, wanda aka yarda da shi a cikin bincike na baya da kuma marubutan da kansu a cikin SQuAD 2.0, yana cikin tsarin tushen ɓangare: baya buƙatar ainihin fahimta ko tunani bayan rubutu. Ƙirar na iya samun maki mai kyau ta hanyar zama ƙwararre a daidaitawar tsarin tsari ba tare da sanin duniya ba. Wannan iyakancewa yana kwatanta sukar sauran tsarin bayanai na ma'auni, inda ƙirar suke koyon amfani da son zuciya na tsarin bayanai maimakon warware ainihin aikin, wani abu da aka yi nazari sosai a cikin mahallin misalai na adawa da kayan aikin tsarin bayanai.

Hanyoyin Aiki masu Amfani: Ga masu aiki, wannan takarda ce babbar darasi a cikin ƙirƙirar ma'auni. Babban abin da za a ɗauka shine cewa ma'auni mai kyau dole ne ya zama mai wuya, mai yawa, kuma mai bincike. SQuAD ya cimma duka ukun. Hanyar aiki ga masu haɓaka ƙirar ita ce mai da hankali kan siffofin tunani, ba kawai na ƙamus ba. Amfani da hanyoyin dogaro na takardar ya nuna kai tsaye buƙatar ƙirar tsari da ma'ana mai zurfi, wata hanya wacce ta ƙare a cikin gine-ginen da suka dogara da canzawa waɗanda ke koyon irin waɗannan sifofi a ɓoye. A yau, darasin shine duba bayan makin F1 akan SQuAD 1.0 kuma a mai da hankali kan ƙarfi, ƙaddarar yanki, da ayyukan da ke buƙatar ainihin tunani, kamar yadda aka gani a cikin juyin halitta zuwa tsarin bayanai kamar DROP ko HotpotQA.

6. Cikakkun Bayanai na Fasaha & Tsarin Lissafi

Babbar hanyar ƙirar tana ɗaukar zaɓin ɓangaren amsa a matsayin aikin rarrabuwa akan duk yuwuwar ɓangarorin rubutu. Ga ɓangaren ɗan takara s a cikin nassi P da tambaya Q, ƙirar koma baya na logistic tana ƙididdige yuwuwar cewa s shine amsar.

Ƙirar Maki: Makin ɓangaren haɗuwa ne mai nauyi na ƙimar siffa: $$\text{maki}(s, Q, P) = \mathbf{w}^T \phi(s, Q, P)$$ inda $\mathbf{w}$ shine nauyin nauyin da aka koya kuma $\phi$ shine nauyin siffa.

Ƙirar Siffa:

• Daidaitawar Ƙamus: Siffofi kamar nauyin TF-IDF na haɗuwar kalma, $\sum_{q \in Q} \text{TF-IDF}(q, P)$.
• Hanyar Bishiyar Dogaro: Ga kalmar tambaya q da kalma a a cikin ɓangaren ɗan takara s, siffar tana ɓoye mafi gajeren hanya tsakanin su a cikin bishiyar rarraba dogaro, tana ɗaukar alaƙar tsari.
• Siffofin ɓangare: Ya haɗa da $\log(\text{tsayi}(s))$ da matsayin ɓangaren a cikin nassi.

Horarwa & Tunani: An horar da ƙirar don haɓaka yuwuwar ainihin ɓangaren. Yayin tunani, ana zaɓar ɓangaren da ya fi maki.

7. Tsarin Bincike: Nazarin Lamari

Yanayi: Bincika aikin ƙirar akan tambayoyi irin na SQuAD.

Matakan Tsari:

1. Cire ɓangare: Samar da duk yuwuwar ɓangarorin da ke ci gaba daga nassi har zuwa matsakaicin tsayin alama.
2. Ƙididdigar Siffa: Ga kowane ɓangaren ɗan takara, lissafta nauyin siffa $\phi$.
   • Ƙamus: Lissafta haɗuwar unigram/bigram tare da tambaya.
   • Tsari: Rarraba duka tambaya da nassi. Ga kowace kalmar tambaya (misali, "dalili") da kalmar shugaban ɓangare, lissafta nisa da tsarin hanyar dogaro.
   • Matsayi: Daidaita farkon da ƙarshen fihirisar ɓangaren.
3. Maki & Matsayi: Aiwatar da ƙirar koma baya na logistic da aka koya $\mathbf{w}^T \phi$ don maki kowane ɓangare. Matsayi ɓangarori ta maki.
4. Binciken Kuskure: Ga hasashen da ba daidai ba, bincika siffofin ɓangaren da ya fi matsayi. Shin kuskuren ya samo asali ne daga:
   • Rashin daidaitawar ƙamus? (Ma'ana ɗaya, sake fasalin jimla)
   • Rikitarwar tsari? (Hanyoyin dogaro masu tsayi, muryar wucewa)
   • Rudani nau'in amsa? (Zaɓin kwanan wata maimakon dalili)

Aiwatar Misali: Aiwatar da wannan tsari ga misalin ruwan sama zai nuna maki masu yawa ga ɓangarorin da ke ɗauke da "nauyi" saboda ƙaƙƙarfan hanyar dogaro daga "dalili" a cikin tambaya zuwa "ƙarƙashin" da "nauyi" a cikin nassi, wanda ya fi sauƙaƙan daidaitawar ƙamus tare da wasu kalmomi.

8. Ayyuka na Gaba & Hanyoyin Bincike

Gadon SQuAD ya wuce farkon sakin sa. Hanyoyin gaba sun haɗa da:

• Tambayoyi Masu Tsalle-tsalle & Masu Takardu Da Yawa: Faɗaɗa tsarin zuwa tambayoyin da ke buƙatar tunani a cikin jimloli ko takardu da yawa, kamar yadda aka gani a cikin tsarin bayanai kamar HotpotQA.
• Haɗawa da Ilimin Waje: Haɓaka ƙirar don haɗa tushen ilimi (misali, Wikidata) don amsa tambayoyin da ke buƙatar sanin duniya waɗanda ba a bayyana su a cikin nassi ba.
• Tambayoyi Masu Bayyanawa & Amincewa: Haɓaka ƙirar waɗanda ba kawai suka amsa daidai ba amma kuma suna ba da hanyoyin tunani masu bayyanawa, suna haɗa shawararsu zuwa takamaiman shaida a cikin rubutu.
• Ƙarfi & Ƙimar Adawa: Ƙirƙirar gwaje-gwaje masu wuya don kimanta ƙarfin ƙirar a kan sake fasalin jimla, cikakkun bayanai masu raɗaɗi, da sauye-sauyen adawa, suna motsawa bayan yuwuwar son zuciya na tsarin bayanai.
• Tambayoyi Masu Tsallaka Harshe & Ƙarancin Albarkatu: Aiwatar darasin daga SQuAD don gina ingantattun tsarin QA don harsunan da ke da ƙayyadaddun bayanan da aka yiwa lakabi, suna amfani da koyon canja wurin tsakanin harsuna.

Ka'idodin da SQuAD ya kafa—ma'anar aiki bayyananne, tattara bayanai mai yawa, da ƙima mai tsauri—suna ci gaba da jagorantar haɓaka ma'auni na zamani na NLP da tsarin.

9. Nassoshi

1. Rajpurkar, P., Zhang, J., Lopyrev, K., & Liang, P. (2016). SQuAD: Tambayoyi 100,000+ don Fahimtar Rubutu ta Injin. Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2383–2392.
2. Deng, J., Dong, W., Socher, R., Li, L. J., Li, K., & Fei-Fei, L. (2009). ImageNet: Babban tsarin bayanai na hoto mai tsari. 2009 IEEE Conference on Computer Vision and Pattern Recognition.
3. Marcus, M. P., Marcinkiewicz, M. A., & Santorini, B. (1993). Gina babban tarin rubutu na Ingilishi: Bankin Bishiyar Penn. Ilimin harshe na kwamfuta, 19(2), 313-330.
4. Richardson, M., Burges, C. J., & Renshaw, E. (2013). MCTest: Kalubalen Tsarin Bayanai don Fahimtar Rubutu ta Injin na Buɗe Yanki. Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing (EMNLP).
5. Hermann, K. M., Kocisky, T., Grefenstette, E., Espeholt, L., Kay, W., Suleyman, M., & Blunsom, P. (2015). Koyar da Injuna Karatu da Fahimta. Advances in Neural Information Processing Systems (NeurIPS).
6. Devlin, J., Chang, M. W., Lee, K., & Toutanova, K. (2019). BERT: Horar da Farko na Masu Canzawa Masu Gudana Biyu don Fahimtar Harshe. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT).