Teburin Abubuwan Ciki
- 1. Gabatarwa
- 2. Ayyukan da suka danganci
- 3. Hanyoyin Aiki
- 4. Gwaje-gwaje da Sakamako
- 5. Babban Fahimta, Tsarin Tunani, Ƙarfi da Rashi, da Shawarwari masu Amfani
- 6. Bincike na Asali
- 7. Bayanan Fasaha
- 8. Nazarin Harka: Misalin Canjin Harshe
- 9. Hanyoyi na gaba
- 10. Manazarta
1. Gabatarwa
Binciken koyon harshe na biyu (SLA) ya yi nazari sosai kan canjin harshe, wato tasirin tsarin harshen mahaifiyar mai magana (L1) akan nasarar koyon harshen waje (L2). Tasirin irin wannan canji na iya zama mai kyau (yana sauƙaƙe koyo) ko mara kyau (yana hana koyo). Wannan takarda ta gabatar da SLABERT, wani sabon tsari da ke yin amfani da BERT don ƙirƙirar koyon harshe na biyu a jere, tare da mai da hankali kan duka tasirin canji mai kyau da mara kyau.
2. Ayyukan da suka danganci
Duk da cewa canjin harshe ya sami kulawa sosai a binciken NLP, yawancin ayyukan sun fi mayar da hankali kan amfanin aiki kamar inganta tokenizer. Hanyar TILT (Papadimitriou da Jurafsky, 2020) ta mai da hankali kan canji mai kyau tare da saitin horarwa daban-daban. SLABERT ya faɗaɗa wannan ta hanyar ƙirƙirar alaƙar canji a jere da ke faruwa a cikin SLA na ɗan adam.
3. Hanyoyin Aiki
3.1 Gina Tarin Bayanai
Tarin bayanai na MAO-CHILDES ya ƙunshi harsuna 5 daban-daban na tsari: Jamusanci, Faransanci, Yaren Poland, Indonesiya, da Jafananci. Yana amfani da Maganar da aka yi wa Yara (CDS) don ƙirƙirar saitin horarwa na L1 na halitta wanda ya dace da muhalli kuma an daidaita shi don koyon harshe.
3.2 Tsarin Samfurin
SLABERT yana amfani da tsarin Transformer tare da BERT a matsayin tushe. An riga an horar da samfurin akan bayanan CDS na L1 sannan a sake daidaita shi akan bayanan Ingilishi na L2, yana kwaikwayon SLA a jere.
3.3 Hanyar Horarwa
Horarwar ta ƙunshi matakai biyu: na farko, horarwa ta farko akan bayanan CDS na L1; na biyu, sake daidaitawa akan bayanan Ingilishi na L2. Ana amfani da hanyar koyon canjin harshe ta TILT don nazarin tasirin CDS na asali.
4. Gwaje-gwaje da Sakamako
4.1 Kimantawar BLiMP
An gwada samfuran akan gwajin nahawu na BLiMP. Sakamako ya nuna cewa L1 na iya sauƙaƙe ko hana koyon L2. Nisan dangin harshe yana hasashen ƙarin canji mara kyau, wanda ya dace da SLA na ɗan adam.
4.2 Nazarin Nisan Dangin Harshe
Tebur na 1 yana nuna aikin samfuran SLABERT akan BLiMP a cikin harsunan L1 daban-daban. Jamusanci (mafi kusanci da Ingilishi) ya nuna daidaito mafi girma fiye da Jafananci (mafi nisa).
| Harshen L1 | Daidaiton BLiMP (%) |
|---|---|
| Jamusanci | 78.5 |
| Faransanci | 74.2 |
| Yaren Poland | 71.8 |
| Indonesiya | 68.3 |
| Jafananci | 65.1 |
5. Babban Fahimta, Tsarin Tunani, Ƙarfi da Rashi, da Shawarwari masu Amfani
Babban Fahimta: SLABERT ya nuna cewa canji mara kyau a SLA ba wai kawai al'amarin ɗan adam ba ne—ana iya ƙirƙira shi da auna shi a cikin LMs, tare da nisan dangin harshe a matsayin babban mai hasashen.
Tsarin Tunani: Takardar ta tashi daga ka'idar SLA zuwa gina tarin bayanai (MAO-CHILDES), zuwa horar da samfurin, zuwa kimantawa akan BLiMP, kuma a ƙarshe zuwa nazarin tasirin canji. Tsarin yana da daidaituwa amma yana iya zama mafi ƙarfi wajen haɗa ma'aunin NLP da ka'idar SLA.
Ƙarfi da Rashi: Ƙarfin ya haɗa da sabon amfani da bayanan CDS da mai da hankali kan canji mara kyau, wanda ba a yi nazari sosai ba. Rashi ya haɗa da iyakancewar harshe (harsuna 5 kawai) da rashin kwatanta da bayanan ɗaliban ɗan adam.
Shawarwari masu Amfani: Masu bincike ya kamata su faɗaɗa wannan zuwa ƙarin harsuna kuma su haɗa da ma'aunin ɗaliban ɗan adam. Masu aiki za su iya amfani da SLABERT don tsara ingantattun tsarin NLP na harshe daban-daban waɗanda ke la'akari da canji mara kyau.
6. Bincike na Asali
SLABERT yana wakiltar wani muhimmin mataki na haɗa ilimin harshe na kwamfuta da binciken koyon harshe na biyu. Ta hanyar ƙirƙirar canji mara kyau, yana magance gibin da ke cikin NLP inda yawancin ayyukan ke mai da hankali kan canji mai kyau. Amfani da Maganar da aka yi wa Yara yana da ƙwarewa musamman, saboda yana ba da bayanan horarwa masu dacewa da muhalli waɗanda ke kwaikwayon koyon harshe na halitta. Duk da haka, dogaron binciken akan BLiMP a matsayin ma'aunin kimantawa kaɗai na iya kama dukkan bangarorin SLA, kamar canji na aiki ko magana. Ayyuka na gaba ya kamata su haɗa da ƙarin cikakkun ma'auni kuma su kwatanta da bayanan ɗaliban ɗan adam don tabbatar da hasashen samfurin. Gano cewa bayanan magana ta tattaunawa suna nuna sauƙi mafi girma fiye da magana da aka rubuta ya dace da bincike kan mahimmancin shigar da hulɗa a SLA (misali, Long, 1996). Wannan yana nuna cewa ana iya amfani da SLABERT don inganta kayan koyon harshe ta hanyar ba da fifiko ga bayanan tattaunawa.
7. Bayanan Fasaha
Samfurin yana amfani da tsarin Transformer mai yadudduka 12, ɓoyayyun girma 768, da kawunan hankali 12. Aikin asara shine cross-entropy tare da ƙirar harshe mai rufe fuska. Manufar horarwa ita ce rage yiwuwar rashin jin daɗi na alamomin da aka rufe: $\mathcal{L} = -\sum_{i \in \text{masked}} \log P(x_i | x_{\backslash i})$.
8. Nazarin Harka: Misalin Canjin Harshe
Yi la'akari da mai magana da Jamusanci na L1 yana koyon Ingilishi. Jamusanci yana da tsari mai sassauƙa na kalmomi, yayin da Ingilishi ya fi tsauri. SLABERT da aka horar akan CDS na Jamusanci ya nuna daidaito mafi girma akan ayyukan tsarin kalmomi na Ingilishi (misali, batun-fi'ili-abu) idan aka kwatanta da samfuran da aka horar akan Jafananci, yana nuna canji mai kyau. Duk da haka, samfuran da aka horar akan Jamusanci sun nuna ƙarancin daidaito akan amfani da labarin Ingilishi (tunda Jamusanci yana da labaran jinsi), yana nuna canji mara kyau.
9. Hanyoyi na gaba
Ayyuka na gaba ya kamata su faɗaɗa SLABERT zuwa ƙarin harsuna, su haɗa da bayanan multimodal (misali, mahallin gani), da kuma haɓaka yanayin koyo na hulɗa. Hakanan za a iya amfani da tsarin don nazarin raguwar harshe da yawan harsuna. Bugu da ƙari, haɗa fahimta daga kimiyyar fahimta na iya inganta amincin tunanin samfurin.
10. Manazarta
- Papadimitriou, I., & Jurafsky, D. (2020). Learning Music Helps You Read: Using Transfer to Study Linguistic Structure in Language Models. EMNLP.
- Warstadt, A., et al. (2020). BLiMP: The Benchmark of Linguistic Minimal Pairs for English. Transactions of the ACL.
- Jarvis, S., & Pavlenko, A. (2007). Crosslinguistic Influence in Language and Cognition. Routledge.
- Long, M. (1996). The Role of the Linguistic Environment in Second Language Acquisition. In Handbook of Second Language Acquisition.