Share
Embed
Transcript
25 slides
-
֓ཁ 2 ✓ θϩγϣοτଟݴޠҙຯղੳϞσϧɺZX-parseͷఏҊ • ݴޠؒજࡏදݱͷΞϥΠϝϯτʹண͠సҠֶशͷޡࠩΛ࠷খԽ • ෳͷଛࣦؔΛಋೖ͢ΔϚϧνλεΫֶशʹΑΓҟͳΔݴޠͷ જࡏදݱ͕ྨࣅ͢ΔΑ͏ʹ •
ରݴޠͷϖΞσʔλΛඞཁͱͤͣɺӳޠϖΞͱରݴޠͷࣗવ จͷΈར༻ ✓ θϩγϣοτҙຯղੳλεΫʹ͓͍ͯෳͷݴޠͰߴ͍ੑೳ -
ҙຯղੳ (Semantic Parsing) • ࣗવݴޠͷൃΛཧܗࣜ(Logical Form)ʹม • ͞·͟·ͳλεΫͰॏཁͳΠϯϑϥ • ࣭ԠɺରγεςϜɺػցͷࢦࣔ…
4 -JTU fl JHIUTGSPN4BO'SBODJTDPUP1JUUTCVSHI 4&-&$5%*45*/$5 fl JHIU@ fl JHIU@JE'30.ʜ -
• ෳͷݴޠ͔ΒͳΔࣗવจΛಉ͡ཧܗࣜʹม[1] ଟݴޠҙຯղੳ (Cross-lingual Semantic Parsing) 5 <>.VMUJMJOHVBM4FNBOUJD1BSTJOH1BSTJOH.VMUJQMF-BOHVBHFTJOUP4FNBOUJD3FQSFTFOUBUJPOT 4&-&$5%*45*/$5 fl
JHIU@ fl JHIU@JE'30.ʜ -JTU fl JHIUTGSPN4BO 'SBODJTDPUP1JUUTCVSHI αϯϑϥϯγείൃϐοπ όʔάߦ͖ͷϑϥΠτΛ ϦετΞοϓ͍ͯͩ͘͠͞ɻ ʹ -
ઌߦݚڀ • ӳޠͷࣗવจ-ཧܗࣜͷฒྻσʔλΛ༁͠ɺ֤ରݴޠͷࣗવ จ-ཧܗࣜϖΞσʔλΛར༻ • ػց༁Λ༻͍Δ߹[2] • ػց༁ᘳͰͳ͍ • ಛʹϦιʔεݴޠʹ͓͍ͯ
ߴ࣭ͳػց༁͍͠ • ਓख༁Λ༻͍Δ߹[3] • ߴίετ 6 <>#PPUTUSBQQJOHB$SPTTMJOHVBM4FNBOUJD1BSTFS <>/FVSBM"SDIJUFDUVSFTGPS.VMUJMJOHVBM4FNBOUJD1BSTJOH ཧܗࣜ ӳޠ ରݴޠ ༁ -
• ରݴޠͷฒྻσʔλΛΘͳ͍ θϩγϣοτͱ͢Δ ➡ ӳޠͷࣗવจ-ཧܗࣜϖΞσʔλͱ ରݴޠͷࣗવจͷΈར༻ ✓ Ϟσϧߏ • ӳޠ͔Βಘͨજࡏදݱ͔Β
ཧܗࣜΛੜ͢Δσίʔμ • ݴޠԣஅతͳʢӳޠͱྨࣅͨ͠ʣ જࡏදݱΛ֫ಘ͢ΔΤϯίʔμ ఏҊख๏ 7 αϯϑϥϯγείൃϐοπ όʔάߦ͖ͷϑϥΠτΛ ϦετΞοϓ͍ͯͩ͘͠͞ɻ 4&-&$54&-&$54&-&$5 8)&3& '30.4&-&$5 Τϯίʔμ σίʔμ ❌ 4&-&$5%*45*/$5 fl JHIU@ fl JHIU@JE'30.ʜ -JTU fl JHIUTGSPN4BO 'SBODJTDPUP1JUUTCVSHI ⭕ Τϯίʔμ σίʔμ ˛ΞϥΠϝϯτΛߦΘͳ͍߹ɺݴޠ͝ͱʹ ҟͳΔજࡏදݱͱͳΓग़ྗ͕ҟͳΔ જࡏදݱ -
ϚϧνλεΫֶश • ̍ͭͷΤϯίʔμʹର͠ෳͷతؔΛ༻͍ɺಉ࣌ʹ࠷దԽ[4] • ଟݴޠλεΫʹ͓͍ͯɺʮιʔεݴޠ (ӳޠͳͲ) ͰͷඪλεΫʴ ιʔεݴޠͱରݴޠͷΞϥΠϝϯτʯΛಉ࣌ʹ࠷దԽ͢Δݚڀ ͕ଘࡏ •
ԻݴޠཧղɺςΩετ؆ུԽɺґଘੑߏจղੳɺػց༁ 8 <>.VMUJUBTL4FRVFODFUP4FRVFODF-FBSOJOH -
ఏҊϞσϧɿZX-Parse • ϚϧνλεΫSeq2seqϞσϧ ✓ తͷλεΫʹՃ͑ิॿతͳλεΫΛಋೖ • DLF ɿཧܗࣜͷੜ • DNL
ɿࣗવݴޠͷੜ • LPɿݴޠ༧ଌ 9 Τϯίʔμ z DLF LP ࣗવจ DNL ֶश(EN) ਪ જࡏදݱ Transformer x 6 (mBARTͷΤϯίʔμ) Transformer x 6 ֶश(ଞݴޠ) ཧܗࣜͷੜ ӳޠͰͷΈֶश -
DLF ɿ Generating Logical Forms 10 Τϯίʔμ z DLF LP
List fl ights from San Francisco to Pittsburgh? DNL SELECT DISTINCT fl ight_1. fl ight_id FROM … • ӳޠͷࣗવจ͔Βಉ͡ҙຯͷཧܗࣜΛੜ͢Δ ➡ཧܗࣜੜೳྗΛʹ͚ͭΔ ֶश࣌ ਪ࣌ જࡏදݱ -
• ରݴޠͷࣗવจʹϊΠζΛՃ͑ɺ࠶ߏங͘͠ӳޠ༁ ➡֤ݴޠͷࣗવจʹΤϯίʔμʔΛదԠͤ͞Δ • ݴޠݻ༗ͷಛੑΛʹ͚Δ DNL ɿ Generating Natural Language
11 Τϯίʔμ z DLF LP αϯϑϥϯγείൃϐοπ όʔάߦ͖ͷϑϥΠτΛ ϦετΞοϓ͍ͯͩ͘͠͞ɻ DNL Н ϊΠζ Н αϯϑϥϯγείൃϐοπ όʔάߦ͖ͷϑϥΠτΛ ϦετΞοϓ͍ͯͩ͘͠͞ɻ List fl ights from San Francisco to Pittsburgh? ֶश࣌ ਪ࣌ જࡏදݱ -
LPɿ Language Prediction 12 Τϯίʔμ z DLF LP List fl
ights from San Francisco to Pittsburgh? DNL English ֶश࣌ ਪ࣌ • ೖྗจͷݴޠΛྨثͰ༧ଌ͢Δ ➡ΤϯίʔμʹݴޠΛ۠ผͤ͞Δ જࡏදݱ -
LPɿ Language Prediction 13 Τϯίʔμ z DLF LP List fl
ights from San Francisco to Pittsburgh? DNL English ֶश࣌ ਪ࣌ • ೖྗจͷݴޠΛྨثͰ༧ଌ͢Δ ➡ΤϯίʔμʹݴޠΛ۠ผͤ͞Δ ✦ ٯ࣌ʹޯΛసͤ͞Δ ➡ ݴޠΛ۠ผͤ͞ͳ͍ • ݴޠʹͱΒΘΕͳ͍දݱ જࡏදݱ -
• ಉ࣌ʹ̏ͭͷతؔΛ࠷దԽ • ཧܗࣜͷੜʴݴޠ͝ͱͷಛΛֶशʴݴޠΛ۠ผ͠ͳ͍ • ରݴޠͷϖΞίʔύεΛඞཁͱ͠ͳ͍θϩγϣοτҙຯղੳ ZX-Parse 14 Τϯίʔμ z
DLF LP DNL ٯ࣌ ཧܗࣜͷੜ ࣗવݴޠͷੜ ʢ࠶ߏ/༁ʣ ݴޠ༧ଌ − ∂LLP ∂θ ∂LNL ∂θ ∂LLF ∂θ જࡏදݱ -
σʔληοτ ✦ ҙຯղੳσʔληοτ • ATIS • ཱྀߦυϝΠϯͷଟݴޠࣗવจͱཧܗࣜͷϖΞίʔύε • ӳޠ, ϑϥϯεޠ,
ϙϧτΨϧޠ, εϖΠϯޠ, υΠπޠ, தࠃޠ, ώϯσΟʔޠ, τϧίޠ • Overnight • ̔υϝΠϯͷӳޠࣗવจͱཧܗࣜͷϖΞίʔύε • ਪ࣌ͷΈ༁Ͱ࡞͞ΕͨதࠃޠͱυΠπޠσʔλΛར༻ ✦ ࣗવݴޠσʔληοτ • MKQA • ࣭จͷର༁ίʔύε • ӳޠ, ϑϥϯεޠ, ϙϧτΨϧޠ, εϖΠϯޠ, υΠπޠ, தࠃޠ • ParaCrawl • Webର༁ίʔύε 15 -
σʔληοτ ✦ ҙຯղੳσʔληοτ • ATIS • ཱྀߦυϝΠϯͷଟݴޠࣗવจͱཧܗࣜͷϖΞίʔύε • ӳޠ, ϑϥϯεޠ,
ϙϧτΨϧޠ, εϖΠϯޠ, υΠπޠ, தࠃޠ, ώϯσΟʔޠ, τϧίޠ • Overnight • ̔υϝΠϯͷӳޠࣗવจͱཧܗࣜͷϖΞίʔύε • ਪ࣌ͷΈ༁Ͱ࡞͞ΕͨதࠃޠͱυΠπޠσʔλΛར༻ ✦ ࣗવݴޠσʔληοτ • MKQA • ࣭จͷର༁ίʔύε • ӳޠ, ϑϥϯεޠ, ϙϧτΨϧޠ, εϖΠϯޠ, υΠπޠ, தࠃޠ • ParaCrawl • Webର༁ίʔύε 16 -
༁ϕʔεϥΠϯ ✦ “࠷খݶͷྗ”ϕʔεϥΠϯ • ػց༁Λ༻͍ͯ࡞ͨ͠ϕʔεϥΠϯ • Translate-Test • ςετηοτΛӳޠʹ༁ •
Translate-Train • ֶशηοτΛରݴޠʹ༁ ✦ “࠷େݶͷྗ”ϕʔεϥΠϯ • Monolingual Training • ֶशηοτΛਓखͰରݴޠʹ༁ 17 -
࣮ݧ݁Ռɿ༁ϕʔεϥΠϯͱͷൺֱ • ӳޠҎ֎ͷݴޠͷθϩγϣοτҙຯղੳλεΫͰSOTA 18 • Monolingual Training: ֶशηοτΛਓखͰରݴޠʹ༁ • Translate-Train:
ֶशηοτΛରݴޠʹ༁ • Translate-Test: ςετηοτΛӳޠʹ༁ • ATIS: ࣭จͱཧܗࣜͷϖΞσʔλ • Overnight: ෳυϝΠϯͷจͱཧܗࣜͷϖΞσʔλ ※ώϯσΟʔޠ(HI)ɺτϧίޠ(TR)ର༁ίʔύε͕ ଘࡏ͠ͳ͍ͨΊิॿతͰͷֶशʹؚ·Ε͍ͯͳ͍ -
࣮ݧ݁ՌɿαϒλεΫͷӨڹ • ୯ݴޠͷΈར༻͢Δ߹(τ = 0.0)ΑΓ༁ߦ͏߹(τ = 0.5) ͷੑೳ͕ߴ͍ • దͳจΛར༻͢Δ߹(ParaCrawl)ΑΓਪ࣌ͷೖྗͱಉ࣭͡
จΛֶशʹར༻͢Δ߹(MKQA)ͷੑೳ͕ߴ͍ 24 -
·ͱΊ ✓ θϩγϣοτଟݴޠҙຯղੳϞσϧɺZX-parseͷఏҊ • જࡏදݱͷΞϥΠϝϯτʹண͠సҠֶशͷޡࠩΛ࠷খԽ • ෳͷଛࣦؔΛಋೖ͢ΔϚϧνλεΫֶशʹΑΓҟͳΔݴޠͷ જࡏදݱ͕ྨࣅ͢ΔΑ͏ʹ • ରݴޠͷϖΞσʔλΛඞཁͱͤͣɺӳޠϖΞͱରݴޠͷࣗવ
จͷΈར༻ ✓ θϩγϣοτҙຯղੳλεΫʹ͓͍ͯෳͷݴޠͰߴ͍ੑೳ 25