Alamomin da Tsarin Harshe na Haruffa ke Koyon Rukunin Nahawu da Tsarin Nahawun Turanci

Teburin Abubuwan Ciki

1.1 Gabatarwa

Tsarin harshe na matakin haruffa (LMs) sun nuna iyawa mai ban mamaki a cikin samar da ƙamus na buɗe ido, suna ba da damar aikace-aikace a cikin gane magana da fassarar inji. Waɗannan tsare-tsaren suna samun nasara ta hanyar raba sigogi a cikin kalmomin da aka saba da su, da na daɗewa, da waɗanda ba a gani ba, wanda ke haifar da iƙirarin game da ikonsu na koyon kaddarorin nahawu. Duk da haka, waɗannan iƙirarin sun kasance mafi yawa na hankali maimakon goyan bayan gwaji. Wannan binciken yana binciken abin da tsarin haruffa na LMs suka koya game da ilimin nahawu da kuma yadda suke koyonsa, tare da mai da hankali kan sarrafa harshen Turanci.

1.2 Tsarin Harshe

Binciken ya yi amfani da RNN na haruffa 'marasa kalma' tare da naúrar LSTM, inda ba a raba shigarwa zuwa kalmomi kuma ana ɗaukar sarari a matsayin haruffa na yau da kullun. Wannan tsarin yana ba da damar bincike na matakin nahawu ta hanyar ba da damar shigar da kalmomi na ɓangare da ayyukan kammalawa.

1.2.1 Tsarin Tsari

A kowane lokaci $t$, harafin $c_t$ ana tsara shi zuwa sararin maɗaukaki: $x_{c_t} = E^T v_{c_t}$, inda $E \in \mathbb{R}^{|V| \times d}$ shine matrix ɗin maɗaukakin haruffa, $|V|$ shine girman ƙamus na haruffa, $d$ shine girman maɗaukaki, kuma $v_{c_t}$ shine vector mai zafi guda ɗaya.

Ana ƙididdige yanayin ɓoye kamar haka: $h_t = \text{LSTM}(x_{c_t}; h_{t-1})$

Rarraba yuwuwar haruffa na gaba shine: $p(c_{t+1} = c | h_t) = \text{softmax}(W_o h_t + b_o)_i$ ga duk $c \in V$

1.2.2 Cikakkun Bayanai na Horarwa

An horar da tsarin a kan haruffa miliyan 7 na farko daga bayanan rubutu na Turanci, ta amfani da daidaitaccen baya-bayan baya-baya ta hanyar lokaci tare da asarar giciye-entropy.

2.1 Ayyukan Nahawu Masu Samarwa

Lokacin samar da rubutu, LM yana amfani da hanyoyin nahawu na Turanci yadda ya kamata a cikin sababbin yanayi. Wannan binciken mai ban mamaki yana nuna cewa tsarin zai iya gano mahimman kalmomin nahawu don waɗannan hanyoyin, yana nuna koyon nahawu na zahiri fiye da tsarin saman.

2.2 Naúrar Gano Iyaka

Binciken naúrar ɓoye na LM ya bayyana takamaiman naúrar da ke aiki a iyakokin nahawu da kalmomi. Wannan hanyar gano iyaka ta bayyana mahimmanci ga ikon tsarin na gano rukunin harshe da kaddarorinsu.

3.1 Koyon Iyakokin Nahawu

LM yana koyon iyakokin nahawu ta hanyar ƙididdige daga iyakokin kalmomi. Wannan hanyar koyo daga ƙasa zuwa sama tana ba da damar tsarin don haɓaka wakilcin matsayi na tsarin harshe ba tare da kulawa ta zahiri ba.

3.2 Rarrabe Nau'in Kalma

Bayan ilimin nahawu, LM yana ɓoye bayanan nahawu game da kalmomi, gami da nau'ikan nau'in kalma. Wannan ɓoyayyen kaddarorin nahawu da nahawu guda biyu yana ba da damar sarrafa harshe mai zurfi.

4.1 Takunkumin Zaɓi

LM yana ɗaukar takunkumin zaɓi na nahawu na kalmomin nahawu na Turanci, yana nuna sani a mahadar nahawu da nahawu. Duk da haka, tsarin yana yin wasu ƙididdiga marasa daidai, yana nuna iyakoki a cikin koyonsa.

4.2 Sakamakon Gwaji

Gwaje-gwajen sun nuna cewa tsarin haruffa na LM zai iya:

Gano rukunin harshe mafi girma (nahawu da kalmomi)
Koyon kaddarorin harshe na asali da tsarin waɗannan rukunoni
Aiwatar da hanyoyin nahawu yadda ya kamata a cikin sababbin yanayi
ɓoye bayanan nahawu da nahawu duka

5. Fahimta ta Tsakiya & Bincike

Fahimta ta Tsakiya

Tsarin harshe na matakin haruffa ba kawai suna haddace jerin haruffa ba—suna haɓaka ainihin rabe-raben harshe. Babban binciken da aka gano a nan shine fitowar wata naúrar "ganowa ta iyaka" wacce a zahiri tana aiwatar da rabe-raben nahawu mara kulawa. Wannan ba binciken tsari ba ne; tsarin yana gina ka'idar tsarin kalma daga bayanan haruffa na danye.

Tsarin Hankali

Ci gaban binciken yana da tsari kuma yana gamsarwa: 1) Lura da halayen nahawu masu samarwa, 2) Bincika hanyar sadarwa don nemo hanyoyin bayani, 3) Tabbatar da ta hanyar gwaje-gwajen gano iyaka, 4) Gwada haɗin kai na nahawu da nahawu mafi girma. Wannan yayi daidai da hanyar da ake bi a cikin takardu masu mahimmanci kamar takardar Transformer ta asali (Vaswani et al., 2017), inda aka tabbatar da sabbin abubuwan gine-gine ta hanyar bincike na tsari.

Ƙarfi & Kurakurai

Ƙarfi: Gano naúrar iyaka yana da sabon abu kuma yana da tasiri ga yadda muke fahimtar wakilcin harshe na hanyar sadarwar jijiyoyi. Ƙirar gwaji tana da kyau a cikin sauƙi—ta amfani da ayyukan kammalawa don gwada samar da nahawu. Haɗin kai zuwa takunkumin zaɓi yana nuna tsarin ba kawai yana koyon nahawu a keɓe ba.

Kurakurai: Mayar da hankali kan Turanci yana iyakance haɗawa zuwa harsuna masu wadatar nahawu. Tarin horar da haruffa miliyan 7 yana da ƙanƙanta bisa ma'auni na zamani—muna buƙatar ganin ko waɗannan binciken sun yi daidai da tarin bayanai na biliyoyin haruffa. "Ƙididdiga marasa daidai" da aka ambata amma ba a bayyana ba suna wakiltar damar da aka rasa don zurfin binciken kuskure.

Fahimta Mai Aiki

Ga masu aiki: Wannan binciken yana nuna cewa tsarin matakin haruffa sun cancanci sake dubawa don harsuna masu rikitarwa na nahawu, musamman yanayin ƙarancin albarkatu. Hanyar gano iyaka za a iya ƙirƙira ta a fili maimakon fitowa—tunani da fara naúrar iyaka ta musamman. Ga masu bincike: Wannan aikin yana haɗa da tambayoyi masu faɗi game da rabe-raben harshe a cikin hanyoyin sadarwar jijiyoyi, kama da bincike a cikin tsarin hangen nesa kamar CycleGAN (Zhu et al., 2017) waɗanda ke bincika abin da wakilcin ke fitowa yayin koyo mara kulawa. Mataki na gaba ya kamata ya zama nazarin kwatancen a cikin harsuna tare da tsarin nahawu daban-daban, watakila ta amfani da albarkatu kamar UniMorph (Kirov et al., 2018).

Mafi girman ma'ana shine cewa tsarin haruffa na iya ba da hanya zuwa ga koyon harshe mai kama da na ɗan adam—koyon nahawu daga tsarin rarraba maimakon ƙa'idodin rabe-raben zahiri. Wannan yayi daidai da ka'idodin ilimin halayyar ɗan adam na sarrafa nahawu kuma yana nuna hanyoyin sadarwar jijiyoyi na iya haɓaka wakilcin harshe mai ma'ana ba tare da kulawar alama ba.

6. Cikakkun Bayanai na Fasaha

6.1 Tsarin Lissafi

Ana iya tsara tsarin maɗaukakin haruffa kamar haka:

$\mathbf{x}_t = \mathbf{E}^\top \mathbf{v}_{c_t}$

inda $\mathbf{E} \in \mathbb{R}^{|V| \times d}$ shine matrix ɗin maɗaukaki, $\mathbf{v}_{c_t}$ shine vector mai zafi guda ɗaya don harafin $c_t$, kuma $d$ shine girman maɗaukaki.

Daidaitawar LSTM ta bi tsarin daidaitaccen tsari:

$\mathbf{f}_t = \sigma(\mathbf{W}_f [\mathbf{h}_{t-1}, \mathbf{x}_t] + \mathbf{b}_f)$

$\mathbf{i}_t = \sigma(\mathbf{W}_i [\mathbf{h}_{t-1}, \mathbf{x}_t] + \mathbf{b}_i)$

$\tilde{\mathbf{C}}_t = \tanh(\mathbf{W}_C [\mathbf{h}_{t-1}, \mathbf{x}_t] + \mathbf{b}_C)$

$\mathbf{C}_t = \mathbf{f}_t \odot \mathbf{C}_{t-1} + \mathbf{i}_t \odot \tilde{\mathbf{C}}_t$

$\mathbf{o}_t = \sigma(\mathbf{W}_o [\mathbf{h}_{t-1}, \mathbf{x}_t] + \mathbf{b}_o)$

$\mathbf{h}_t = \mathbf{o}_t \odot \tanh(\mathbf{C}_t)$

6.2 Tsarin Gwaji

Tsarin yana amfani da jihohin ɓoye na LSTM masu girma 512 da maɗaukakin haruffa da aka horar akan haruffa miliyan 7. Kimantawa ya ƙunshi duka ma'auni na ƙididdiga (rashin fahimta, daidaito) da bincike na ingancin rubutun da aka samar da ayyukan naúrar.

7. Misalin Tsarin Bincike

7.1 Hanyar Bincike

Binciken yana amfani da dabaru da yawa na bincike don bincika abin da tsarin ya koya:

Ayyukan Kammalawa: Ciyar da kalmomi na ɓangare (misali, "unhapp") kuma bincika yuwuwar da aka ba da damar kammalawa ("-y" da "-ily")
Binciken Iyaka: Sa ido kan takamaiman ayyukan naúrar ɓoye a kusa da haruffan sarari da iyakokin nahawu
Gwajin Takunkumin Zaɓi: Gabatar da tushen kalmomin nahawu da kuma kimanta hukunce-hukuncen nahawu

7.2 Nazarin Shari'a: Binciken Naúrar Iyaka

Lokacin sarrafa kalmar "unhappiness," naúrar gano iyaka tana nuna kololuwar aiki a:

Matsayi 0 (farkon kalma)
Bayan "un-" (iyakar prefix)
Bayan "happy" (iyakar tushe)
Bayan "-ness" (ƙarshen kalma)

Wannan tsari yana nuna naúrar tana koyon rabe-raben a iyakokin kalma da nahawu ta hanyar fallasa zuwa irin wannan tsarin a cikin bayanan horo.

8. Aikace-aikace na Gaba & Jagorori

8.1 Aikace-aikace Nan da Nan

Harsuna Masu Ƙarancin Albarkatu: Tsarin haruffa zai iya fi tsarin da aka gina akan kalma don harsuna masu wadatar nahawu da ƙarancin bayanan horo
Masu Nazarin Nahawu: Gano iyaka na fitowa zai iya ƙaddamar da tsarin rabe-raben nahawu mara kulawa
Kayan Aikin Ilimi: Tsarin da ke koyon nahawu a zahiri zai iya taimakawa wajen koyar da tsarin harshe

8.2 Jagororin Bincike

Nazarin Tsakanin Harsuna: Gwada ko binciken ya yi daidai da harsunan haɗaka (Turkish) ko na haɗawa (Rashanci)
Tasirin Ma'auni: Bincika yadda koyon nahawu ke canzawa tare da girman tsari da adadin bayanan horo
Sabbin Abubuwan Gine-gine: Ƙirƙira tsare-tsare tare da abubuwan nahawu na zahiri da aka sanar da waɗannan binciken
Haɗin Nau'i-nau'i: Haɗa koyon harshe na matakin haruffa tare da shigarwar gani ko ji

8.3 Tasiri na Dogon Lokaci

Wannan binciken yana nuna tsarin matakin haruffa na iya ba da hanya mafi dacewa ta fahimta ga koyon harshe, wanda zai iya haifar da:

Tsarin harshe mafi inganci na bayanai
Mafi kyawun sarrafa sababbin kalmomi da ƙirƙirar nahawu
Ingantacciyar fassara ta hanyar wakilcin harshe mai ma'ana
Gadoji tsakanin ilimin harshe na lissafi da ilimin halayyar ɗan adam

9. Nassoshi

Kementchedjhieva, Y., & Lopez, A. (2018). Alamomin da tsarin harshe na haruffa ke koyon rukunin nahawu da tsarin nahawun Turanci. arXiv preprint arXiv:1809.00066.
Sutskever, I., Martens, J., & Hinton, G. E. (2011). Samar da rubutu tare da hanyoyin sadarwar jijiyoyi masu maimaitawa. Proceedings of the 28th International Conference on Machine Learning.
Chung, J., Cho, K., & Bengio, Y. (2016). Mai fassara matakin haruffa ba tare da rabe-raben zahiri ba don fassarar inji ta jijiyoyi. Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics.
Kim, Y., Jernite, Y., Sontag, D., & Rush, A. M. (2016). Tsarin harshe na jijiyoyi masu wayo. Proceedings of the AAAI Conference on Artificial Intelligence.
Vaswani, A., et al. (2017). Hankali shine duk abin da kuke buƙata. Advances in Neural Information Processing Systems.
Zhu, J. Y., Park, T., Isola, P., & Efros, A. A. (2017). Fassarar hoto zuwa hoto mara haɗin gwiwa ta amfani da hanyoyin sadarwar adawa na zagayowar daidaitacce. Proceedings of the IEEE International Conference on Computer Vision.
Kirov, C., et al. (2018). UniMorph 2.0: Nahawu na Duniya. Proceedings of the Eleventh International Conference on Language Resources and Evaluation.
Karpathy, A. (2015). Rashin ingancin hanyoyin sadarwar jijiyoyi masu maimaitawa. Andrej Karpathy blog.