修論発表.pdf

修論発表会にて使用した発表スライドです。

Transcript

1. ҟͳΔڭࢣ৴߸͔Βߏஙͨ͠
 จϕΫτϧͷൺֱͱ౷߹ ਖ਼ࢦಋڭһ: ෢ా ߒҰ ෭ࢦಋڭһ: ࡫໺ ྒྷฏ 252106192 ௩ӽ ॣ
2. จϕΫτϧ: ࣗવݴޠจͷϕΫτϧදݱ 2 จϕΫτϧۭؒ ͜Ͳ΋͕Ոʹ޲͔͍ͬͯΔɻ ͜Ͳ΋ֶ͕ߍ͔ΒՈʹ޲͔͍ͬͯΔɻ ͜Ͳ΋͕ਤॻؗʹ͍Δɻ ͜Ͳ΋͕ޕޙʹา͍͍ͯΔɻ จಉ࢜ͷҙຯ͕͍ۙ 
 ϕΫτϧಉ͕͍࢜ۙ
3. จϕΫτϧ: ࣗવݴޠจͷϕΫτϧදݱ 3 จϕΫτϧۭؒ ͜Ͳ΋͕Ոʹ޲͔͍ͬͯΔɻ ͜Ͳ΋ֶ͕ߍ͔ΒՈʹ޲͔͍ͬͯΔɻ ͜Ͳ΋͕ਤॻؗʹ͍Δɻ ͜Ͳ΋͕ޕޙʹา͍͍ͯΔɻ จಉ࢜ͷҙຯ͕͍ۙ 
 ϕΫτϧಉ͕͍࢜ۙ
4. จϕΫτϧ: ࣗવݴޠจͷϕΫτϧදݱ 4 จϕΫτϧۭؒ ͜Ͳ΋͕Ոʹ޲͔͍ͬͯΔɻ ͜Ͳ΋ֶ͕ߍ͔ΒՈʹ޲͔͍ͬͯΔɻ ͜Ͳ΋͕ਤॻؗʹ͍Δɻ ͜Ͳ΋͕ޕޙʹา͍͍ͯΔɻ จಉ࢜ͷҙຯ͕͍ۙ 
 ϕΫτϧಉ͕͍࢜ۙ • ྨࣅจݕࡧ • ΫϥελϦϯά ɹͳͲ෯޿͍Ԡ༻
5. จϕΫτϧ: ࣗવݴޠจͷϕΫτϧදݱ 5 จϕΫτϧۭؒ ͜Ͳ΋͕Ոʹ޲͔͍ͬͯΔɻ ͜Ͳ΋ֶ͕ߍ͔ΒՈʹ޲͔͍ͬͯΔɻ ͜Ͳ΋͕ਤॻؗʹ͍Δɻ ͜Ͳ΋͕ޕޙʹา͍͍ͯΔɻ จಉ࢜ͷҙຯ͕͍ۙ 
 ϕΫτϧಉ͕͍࢜ۙ • ྨࣅจݕࡧ • ΫϥελϦϯά ɹͳͲ෯޿͍Ԡ༻ • ϕΫτϧͷ඼࣭޲্ • ϕΫτϧͷੑ࣭ཧղ ɹ͕༗༻ੑ޲্ʹ௚݁ จϕΫτϧʹ͍ͭͯͷཧղΛਂΊΔ
 ͜ͱ͕ࠓޙͷൃలͷͨΊʹॏཁ
6. طଘݚڀͷ՝୊ จϕΫτϧݚڀͷݱঢ় •Ұ෦ͷϕϯνϚʔΫλεΫͰͷੑೳධՁ͕ओ •ͦΕͧΕͷख๏ͷཧղ͸ͦ͜·ͰਐΜͰ͍ͳ͍ 6
7. طଘݚڀͷ՝୊ จϕΫτϧݚڀͷݱঢ় •Ұ෦ͷϕϯνϚʔΫλεΫͰͷੑೳධՁ͕ओ •ͦΕͧΕͷख๏ͷཧղ͸ͦ͜·ͰਐΜͰ͍ͳ͍ ࣮Ԡ༻ͱͷΪϟοϓ •จϕΫτϧʹٻΊΒΕΔੑ࣭͸Ԡ༻ࣄྫ͝ͱʹมԽ͢Δ • ݕࡧͰ͸จҙΑΓ΋τϐοΫ΍ΧςΰϦ͕ॏཁ • ࣭໰Ԡ౴Ͱ͸࣭໰ͱճ౴͕ۙ͘ʹ෼෍͢Δ͜ͱ͕ॏཁ •Ԡ༻ࣄྫ͝ͱʹద੾ͳੑ࣭Λ࣋ͭจϕΫτϧΛ࢖͍͍ͨ •ʮख๏͝ͱʹͲͷΑ͏ͳੑ࣭͕͋Δ͔ʁʯ΋ۃΊͯॏཁͳ໰͍ 7
8. ڭࢣ৴߸ʹண໨ͨ͠จϕΫτϧͷ෼ੳ •ػցֶशϞσϧͷৼΔ෣͍͸܇࿅σʔλɾख๏ʹେ͖͘ґଘ •ҟͳΔڭࢣ৴߸ʹج͍ͮͯߏங͞ΕͨจϕΫτϧͷੑ࣭Λ෼ੳ •ͦΕͧΕͷख๏ʹΑΔจϕΫτϧͷੑ࣭Λ෼ੳ 8
9. ڭࢣ৴߸ʹண໨ͨ͠จϕΫτϧͷ෼ੳ •ػցֶशϞσϧͷৼΔ෣͍͸܇࿅σʔλɾख๏ʹେ͖͘ґଘ •ҟͳΔڭࢣ৴߸ʹج͍ͮͯߏங͞ΕͨจϕΫτϧͷੑ࣭Λ෼ੳ •ͦΕͧΕͷख๏ʹΑΔจϕΫτϧͷੑ࣭Λ෼ੳ ੑ࣭ൺֱʹదͨ͠จϕΫτϧख๏ •ྨࣅͨ͠ΞʔΩςΫνϟΛ͕࣋ͭҟͳΔڭࢣ৴߸Λ༻͍Δ •ϕϯνϚʔΫλεΫʹ͓͍ͯಉ౳ͷੑೳΛࣔ͢ 9
10. ڭࢣ৴߸ʹண໨ͨ͠จϕΫτϧͷ෼ੳ •ػցֶशϞσϧͷৼΔ෣͍͸܇࿅σʔλɾख๏ʹେ͖͘ґଘ •ҟͳΔڭࢣ৴߸ʹج͍ͮͯߏங͞ΕͨจϕΫτϧͷੑ࣭Λ෼ੳ •ͦΕͧΕͷख๏ʹΑΔจϕΫτϧͷੑ࣭Λ෼ੳ ੑ࣭ൺֱʹదͨ͠จϕΫτϧख๏ •ྨࣅͨ͠ΞʔΩςΫνϟΛ͕࣋ͭҟͳΔڭࢣ৴߸Λ༻͍Δ •ϕϯνϚʔΫλεΫʹ͓͍ͯಉ౳ͷੑೳΛࣔ͢ ຊݚڀͰର৅ͱ͢Δख๏ •SBERT: ࣄલֶशࡁΈݴޠϞσϧ + ࣗવݴޠਪ࿦λεΫ •DefSent: ࣄલֶशࡁΈݴޠϞσϧ + ఆٛจˠ୯ޠ༧ଌλεΫ 10
11. ࣄલֶशࡁΈݴޠϞσϧ •େن໛ͳςΩετΛ༻͍ͨࣄલֶशʹΑͬͯݴޠ஌ࣝΛ֫ಘ •୅දྫ: BERT, RoBERTa, GPT-2, GPT-3 11 BERTͷ֓ཁਤ
12. SBERT: ࣗવݴޠਪ࿦λεΫʹجͮ͘ख๏ • ࣗવݴޠਪ࿦λεΫͰ
 จϕΫτϧϞσϧΛ܇࿅͢Δख๏ • ࣗવݴޠਪ࿦λεΫ:
 จϖΞͷҙຯؔ܎Λ༧ଌ 12 จB จA BERT BERT ໃ६ ؚҙ ͦͷଞ ϥϕϧ༧ଌ૚ Pooling Pooling
13. SBERT: ࣗવݴޠਪ࿦λεΫʹجͮ͘ख๏ • ࣗવݴޠਪ࿦λεΫͰ
 จϕΫτϧϞσϧΛ܇࿅͢Δख๏ • ࣗવݴޠਪ࿦λεΫ:
 จϖΞͷҙຯؔ܎Λ༧ଌ SBERTʹΑΔfine-tuningͷखॱ 0. ࣄલֶशࡁΈݴޠϞσϧΛ༻ҙ 1. จϖΞΛͦΕͧΕจϕΫτϧʹ 2. ಘΒΕͨจϕΫτϧͷϖΞ͔Β
 จϖΞͷҙຯؔ܎Λ༧ଌ 3. ਖ਼͍͠ҙຯؔ܎Λ༧ଌͰ͖Δ
 Α͏ʹϞσϧΛ܇࿅ 13 จB จA BERT BERT ໃ६ ؚҙ ͦͷଞ ϥϕϧ༧ଌ૚ Pooling Pooling
14. DefSent: ఆٛจˠ୯ޠ༧ଌλεΫʹجͮ͘ख๏ • ఆٛจˠ୯ޠ༧ଌλεΫʹΑͬͯ
 จϕΫτϧϞσϧΛ܇࿅͢Δख๏ 14 ఆٛจ จB จA w |V| w1 w2 w3 ... BERT BERT ໃ६ ؚҙ ͦͷଞ ϥϕϧ༧ଌ૚ BERT ୯ޠ༧ଌ૚ Pooling Pooling Pooling
15. DefSent: ఆٛจˠ୯ޠ༧ଌλεΫʹجͮ͘ख๏ • ఆٛจˠ୯ޠ༧ଌλεΫʹΑͬͯ
 จϕΫτϧϞσϧΛ܇࿅͢Δख๏ DefSentʹΑΔfine-tuningͷखॱ 0. ࣄલֶशࡁΈݴޠϞσϧΛ༻ҙ 1. ఆٛจΛBERTʹೖྗͯ͠
 จϕΫτϧΛ֫ಘ 2. ಘΒΕͨϕΫτϧ͔Βఆٛจ
 ʹରԠ͢Δ୯ޠΛ༧ଌ 3. ఆٛจ͕ද͢୯ޠͷ֬཰஋Λ
 ࠷େԽ͢ΔΑ͏ʹ܇࿅ 15 ఆٛจ จB จA w |V| w1 w2 w3 ... BERT BERT ໃ६ ؚҙ ͦͷଞ ϥϕϧ༧ଌ૚ BERT ୯ޠ༧ଌ૚ Pooling Pooling Pooling
16. ຊݚڀͷ֓ཁ 16 ڭࢣ৴߸ͷҧ͍ʹ
 ண໨ͨ͠จϕΫτϧͷ
 ൺֱɾ౷߹ SBERT DefSent BERT ؚҙؔ܎ೝࣝͰ fine-tuning ఆٛจ→୯ޠ
 ༧ଌͰfine-tuning จϕΫτϧ
 Ϟσϧ
17. ຊݚڀͷ֓ཁ 17 Semantic Textual Similarity (STS) ᶃ จͷιʔε ᶄ จϖΞͷද૚తྨࣅ౓ SentEval ᶅ ײ৘ɾ੍࣌෼ྨͳͲͷԼྲྀλεΫ ᶆ ݴޠֶత৘ใͷ෼ྨλεΫ ൺֱ •SBERT→DefSent •DefSent→SBERT •ϚϧνλεΫֶश •Average •Concat ౷߹ ڭࢣ৴߸ͷҧ͍ʹ
 ண໨ͨ͠จϕΫτϧͷ
 ൺֱɾ౷߹ SBERT DefSent BERT ؚҙؔ܎ೝࣝͰ fine-tuning ఆٛจ→୯ޠ
 ༧ଌͰfine-tuning จϕΫτϧ
 Ϟσϧ
18. ҟͳΔڭࢣ৴߸͔Βߏஙͨ͠
 จϕΫτϧͷൺֱ
19. จϕΫτϧͷੑ࣭ൺֱ: STS 19 Semantic Textual Similarity (STS) ᶃ จͷιʔε ᶄ จϖΞͷද૚తྨࣅ౓ จϖΞ (จϕΫτϧͷϖΞ) ೖྗ ൺֱ؍఺ ਓؒධՁͱจϕΫτϧಉ࢜ͷ
 ྨࣅ౓ͱͷॱҐ૬ؔ܎਺ ධՁࢦඪ STSͷධՁखॱ ᶃ จϕΫτϧϞσϧΛ༻ҙ ᶄ จϖΞͦΕͧΕΛจϕΫτϧʹม׵ ᶅ จϕΫτϧͷϖΞͷྨࣅ౓Λܭࢉ ᶆ ਓؒධՁͱͷ૬ؔ܎਺Λܭࢉ จA จB จϕΫτϧϞσϧ ਓखධՁͱͷ
 ૬ؔ܎਺ͰධՁ จྨࣅ౓ ᶄ ᶃ ᶅ ᶆ
20. จϕΫτϧͷੑ࣭ൺֱ: STS 20 Semantic Textual Similarity (STS) ᶃ จͷιʔε ᶄ จϖΞͷද૚తྨࣅ౓ จϖΞ (จϕΫτϧͷϖΞ) ೖྗ ൺֱ؍఺ ਓؒධՁͱจϕΫτϧಉ࢜ͷ
 ྨࣅ౓ͱͷॱҐ૬ؔ܎਺ ධՁࢦඪ
21. จϕΫτϧͷੑ࣭ൺֱ: STS 21 Semantic Textual Similarity (STS) ᶃ จͷιʔε ᶄ จϖΞͷද૚తྨࣅ౓ จϖΞ (จϕΫτϧͷϖΞ) ೖྗ ൺֱ؍఺ ਓؒධՁͱจϕΫτϧಉ࢜ͷ
 ྨࣅ౓ͱͷॱҐ૬ؔ܎਺ ධՁࢦඪ • 2ͭͷ؍఺ͰσʔληοτΛ෼ׂ • ੑೳͷมԽΛ؍࡯
22. จϕΫτϧͷੑ࣭ൺֱ: STS 22 Semantic Textual Similarity (STS) ᶃ จͷιʔε ᶄ จϖΞͷද૚తྨࣅ౓ จϖΞ (จϕΫτϧͷϖΞ) ೖྗ ൺֱ؍఺ ਓؒධՁͱจϕΫτϧಉ࢜ͷ
 ྨࣅ౓ͱͷॱҐ૬ؔ܎਺ ධՁࢦඪ • จͷιʔεʹΑͬͯੑೳʹࠩ • ֤ख๏ͷ܇࿅σʔληοτʹ͍ۙ จͷํ͕͏·͘ྨࣅ౓ΛଌΕΔ • 2ͭͷ؍఺ͰσʔληοτΛ෼ׂ • ੑೳͷมԽΛ؍࡯
23. จϕΫτϧͷੑ࣭ൺֱ: STS 23 ද૚తྨࣅ౓ͱੑೳͷؔ܎ SBERT DefSent Semantic Textual Similarity (STS) ᶃ จͷιʔε ᶄ จϖΞͷද૚తྨࣅ౓ จϖΞ (จϕΫτϧͷϖΞ) ೖྗ ൺֱ؍఺ ਓؒධՁͱจϕΫτϧಉ࢜ͷ
 ྨࣅ౓ͱͷॱҐ૬ؔ܎਺ ධՁࢦඪ • ද૚తྨࣅ౓ʹΑͬͯੑೳࠩ • SBERT (ؚҙؔ܎)͸ද૚తྨ ࣅ౓ͷӨڹΛड͚ͮΒ͍ • DefSent (ఆٛจ)͸ද૚తʹྨ ࣅ͍ͯ͠ͳ͍จͷྨࣅ౓Λ
 ൺֱతਖ਼͘͠ਪఆͰ͖Δ • จͷιʔεʹΑͬͯੑೳʹࠩ • ֤ख๏ͷ܇࿅σʔληοτʹ͍ۙ จͷํ͕͏·͘ྨࣅ౓ΛଌΕΔ • 2ͭͷ؍఺ͰσʔληοτΛ෼ׂ • ੑೳͷมԽΛ؍࡯
24. จϕΫτϧͷੑ࣭ൺֱ: SentEval 24 SentEval ᶅ ԼྲྀλεΫ͝ͱͷੑೳ ᶆ ݴޠֶత৘ใͷ෼ྨੑೳ จϕΫτϧ ೖྗ ൺֱ؍఺ จϕΫτϧΛೖྗͱ͢Δ
 ઢܗ෼ྨثͷ෼ྨੑೳ ධՁࢦඪ จ ෼ྨੑೳ͔Β
 จຒΊࠐΈͷ඼࣭ΛධՁ จϕΫτϧϞσϧ ෼ྨث ᶄ ᶃ ᶅ ᶆ SentEvalͷධՁखॱ ᶃ จຒΊࠐΈϞσϧΛ༻ҙ ᶄ ֤จΛจϕΫτϧʹม׵ ᶅ จϕΫτϧΛೖྗͱ͢Δ෼ྨثΛ܇࿅ ᶆ ෼ྨثͷੑೳ͔ΒจϕΫτϧͷ඼࣭ΛධՁ
25. จϕΫτϧͷੑ࣭ൺֱ: SentEval 25 SentEval ᶅ ԼྲྀλεΫ͝ͱͷੑೳ ᶆ ݴޠֶత৘ใͷ෼ྨੑೳ จϕΫτϧ ೖྗ ൺֱ؍఺ จϕΫτϧΛೖྗͱ͢Δ
 ઢܗ෼ྨثͷ෼ྨੑೳ ධՁࢦඪ
26. จϕΫτϧͷੑ࣭ൺֱ: SentEval 26 SentEval ᶅ ԼྲྀλεΫ͝ͱͷੑೳ ᶆ ݴޠֶత৘ใͷ෼ྨੑೳ จϕΫτϧ ೖྗ ൺֱ؍఺ จϕΫτϧΛೖྗͱ͢Δ
 ઢܗ෼ྨثͷ෼ྨੑೳ ධՁࢦඪ • SBERT͸ҙຯతͳ৘ใΛ๛෋ ʹຒΊࠐΜͰ͍Δ • DefSent͸ද૚త৘ใ͕๛෋ • ϑϨʔζͷߏ੒΋ಘҙ
27. Length WordContent Tense SubjNumber จϕΫτϧͷੑ࣭ൺֱ: SentEval 27 SentEval ᶅ ԼྲྀλεΫ͝ͱͷੑೳ ᶆ ݴޠֶత৘ใͷ෼ྨੑೳ จϕΫτϧ ೖྗ ൺֱ؍఺ จϕΫτϧΛೖྗͱ͢Δ
 ઢܗ෼ྨثͷ෼ྨੑೳ ධՁࢦඪ • SBERT͸ҙຯతͳ৘ใΛ๛෋ ʹຒΊࠐΜͰ͍Δ • DefSent͸ද૚త৘ใ͕๛෋ • ϑϨʔζͷߏ੒΋ಘҙ • DefSent͸੍࣌΍จத୯ޠͷ৘ใ
 ͳͲද૚తͳ৘ใ͕ൺֱత๛෋ 50 60 70 80 90 Length WordContent Tense จ௕༧ଌ 50 60 70 80 90 Length WordContent Tense จத୯ޠ༧ଌ ੍࣌༧ଌ • SBERT͸จͷද૚৘ใ͕ॏཁͳλεΫ ͷੑೳ͕௿͍ • จத୯ޠͳͲͷ৘ใ͸গͳΊ
28. จϕΫτϧͷੑ࣭ൺֱ: ·ͱΊ 28 Semantic Textual Similarity (STS) ᶃ จͷιʔε ᶄ จϖΞͷද૚తྨࣅ౓ จϖΞ (จϕΫτϧͷϖΞ) ೖྗ ൺֱ؍఺ ਓؒධՁͱจϕΫτϧಉ࢜ͷ
 ྨࣅ౓ͱͷॱҐ૬ؔ܎਺ ධՁࢦඪ SentEval ᶅ ԼྲྀλεΫ͝ͱͷੑೳ ᶆ ݴޠֶత৘ใͷ෼ྨੑೳ จϕΫτϧ ೖྗ ൺֱ؍఺ จϕΫτϧΛೖྗͱ͢Δ
 ઢܗ෼ྨثͷ෼ྨੑೳ ධՁࢦඪ
29. จϕΫτϧͷੑ࣭ൺֱ: ·ͱΊ 29 ද૚తྨࣅ౓ͱੑೳͷؔ܎ SBERT DefSent Semantic Textual Similarity (STS) ᶃ จͷιʔε ᶄ จϖΞͷද૚తྨࣅ౓ จϖΞ (จϕΫτϧͷϖΞ) ೖྗ ൺֱ؍఺ ਓؒධՁͱจϕΫτϧಉ࢜ͷ
 ྨࣅ౓ͱͷॱҐ૬ؔ܎਺ ධՁࢦඪ SentEval ᶅ ԼྲྀλεΫ͝ͱͷੑೳ ᶆ ݴޠֶత৘ใͷ෼ྨੑೳ จϕΫτϧ ೖྗ ൺֱ؍఺ จϕΫτϧΛೖྗͱ͢Δ
 ઢܗ෼ྨثͷ෼ྨੑೳ ධՁࢦඪ SBERT • λεΫ: ࣗવݴޠਪ࿦ • ײ৘ۃੑͳͲҙຯత৘ใ͕๛෋ • ද૚త৘ใ͸গͳΊ DefSent • λεΫ: ఆٛจˠ୯ޠ༧ଌ • จ௕΍੍࣌ͳͲද૚త৘ใ͕๛෋ • ϑϨʔζͷߏ੒΋ಘҙ
30. ҟͳΔڭࢣ৴߸͔Βߏஙͨ͠
 จϕΫτϧͷ౷߹
31. จϕΫτϧͷ౷߹ ੑ࣭ൺֱͷ݁Ռ •ಉ͡ϞσϧΛϕʔεͱ͠ɺҟͳΔڭࢣ৴߸Λ༻͍Δख๏Λൺֱ • SBERTͱDefSent͸ҟͳΔੑ࣭Λ࣋ͭ͜ͱ͕Θ͔ͬͨ •ҟͳΔৼΔ෣͍Λ͢ΔϞσϧͷΞϯαϯϒϧ͸༗๬ͳखཱͯ • จϕΫτϧʹ͓͍ͯ΋ҟͳΔϞσϧͷ౷߹͸༗༻͔ʁ 31
32. จϕΫτϧͷ౷߹ ੑ࣭ൺֱͷ݁Ռ •ಉ͡ϞσϧΛϕʔεͱ͠ɺҟͳΔڭࢣ৴߸Λ༻͍Δख๏Λൺֱ • SBERTͱDefSent͸ҟͳΔੑ࣭Λ࣋ͭ͜ͱ͕Θ͔ͬͨ •ҟͳΔৼΔ෣͍Λ͢ΔϞσϧͷΞϯαϯϒϧ͸༗๬ͳखཱͯ • จϕΫτϧʹ͓͍ͯ΋ҟͳΔϞσϧͷ౷߹͸༗༻͔ʁ ҟͳΔੑ࣭Λ࣋ͭจϕΫτϧͷ౷߹ •5ͭͷ౷߹ख๏ʹ͍࣮ͭͯݧ • S+D, D+S • Multi • Average • Concat 32 • ౷߹ʹΑͬͯੑೳ͸޲্͢Δ͔ʁ • ࠓޙͷจϕΫτϧݚڀʹ͓͍ͯෳ਺
 ͷڭࢣ৴߸ͷ૊Έ߹Θͤ͸༗๬͔ʁ
33. จϕΫτϧͷ౷߹: ୯ҰϞσϧ΁ͷ౷߹ 33 • S+D: ୯ҰϞσϧʹSBERT, DefSentʹΑΔfine-tuningΛॱʹ࣮ࢪ • D+S: ୯ҰϞσϧʹDefSent, SBERTʹΑΔfine-tuningΛॱʹ࣮ࢪ • Multi: SBERTͱDefSentʹΑΔfine-tuningΛަޓʹ࣮ࢪ
34. จϕΫτϧͷ౷߹: ୯ҰϞσϧ΁ͷ౷߹ 34 • S+D: ୯ҰϞσϧʹSBERT, DefSentʹΑΔfine-tuningΛॱʹ࣮ࢪ • D+S: ୯ҰϞσϧʹDefSent, SBERTʹΑΔfine-tuningΛॱʹ࣮ࢪ • Multi: SBERTͱDefSentʹΑΔfine-tuningΛަޓʹ࣮ࢪ
35. จϕΫτϧͷ౷߹: ୯ҰϞσϧ΁ͷ౷߹ 35 • S+D: ୯ҰϞσϧʹSBERT, DefSentʹΑΔfine-tuningΛॱʹ࣮ࢪ • D+S: ୯ҰϞσϧʹDefSent, SBERTʹΑΔfine-tuningΛॱʹ࣮ࢪ • Multi: SBERTͱDefSentʹΑΔfine-tuningΛަޓʹ࣮ࢪ
36. จϕΫτϧͷ౷߹: ෳ਺Ϟσϧͷ౷߹ 36 • Average: SBERT, DefSentʹΑΔจϕΫτϧΛฏۉ • Concat: SBERT, DefSentʹΑΔจϕΫτϧΛ࿈݁
37. จϕΫτϧͷ౷߹: ෳ਺Ϟσϧͷ౷߹ 37 • Average: SBERT, DefSentʹΑΔจϕΫτϧΛฏۉ • Concat: SBERT, DefSentʹΑΔจϕΫτϧΛ࿈݁
38. จϕΫτϧͷ౷߹: ධՁ࣮ݧ 38 •౷߹ख๏͝ͱʹϞσϧΛ܇࿅ɾධՁ •STSͰ10ճ, SentEvalͰ3ճϞσϧΛ
 ܇࿅ͯ͠ฏۉੑೳΛใࠂ ࣮ݧઃఆ ධՁର৅ •SBERT •DefSent •S+D (SBERT→DefSent) •D+S (DefSent→SBERT) •Multi •Average •Concat ୯Ұख๏ͱੑೳΛൺֱ ධՁλεΫ •STS •SentEval
39. จϕΫτϧͷ౷߹: ධՁ࣮ݧ 39 BERT-base STS SentEval SBERT 73.19 86.49 DefSent 75.20 86.61 S+D 78.45 86.80 D+S 72.89 86.09 Multi 72.89 86.23 Average 77.82 87.47 Concat 76.03 87.93 ֤౷߹ख๏͝ͱͷSTSͱSentEvalͷฏۉੑೳ (%) SBERT→DefSentͱ Average͕ߴੑೳ • SentEvalͰ͸Concatͷੑೳ͕
 ྑ͍͕ɺจϕΫτϧͷ࣍ݩ͕ େ͖͘༗རͳͷͰ஫ҙ
40. จϕΫτϧͷ౷߹: ධՁ࣮ݧ 40 BERT-base STS SentEval SBERT 73.19 86.49 DefSent 75.20 86.61 S+D 78.45 86.80 D+S 72.89 86.09 Multi 72.89 86.23 Average 77.82 87.47 Concat 76.03 87.93 ֤౷߹ख๏͝ͱͷSTSͱSentEvalͷฏۉੑೳ (%) • ౷߹ख๏͕୯Ұख๏
 ΛԼճΔ৔߹΋ • ഁ໓త๨٫ͷӨڹ͔ SBERT→DefSentͱ Average͕ߴੑೳ DefSent→SBERT͸ ੑೳ͕ѱԽ • SentEvalͰ͸Concatͷੑೳ͕
 ྑ͍͕ɺจϕΫτϧͷ࣍ݩ͕ େ͖͘༗རͳͷͰ஫ҙ
41. จϕΫτϧͷ౷߹: ධՁ࣮ݧ 41 BERT-base STS SentEval SBERT 73.19 86.49 DefSent 75.20 86.61 S+D 78.45 86.80 D+S 72.89 86.09 Multi 72.89 86.23 Average 77.82 87.47 Concat 76.03 87.93 ֤౷߹ख๏͝ͱͷSTSͱSentEvalͷฏۉੑೳ (%) • SentEvalͰ͸Concatͷੑೳ͕
 ྑ͍͕ɺจϕΫτϧͷ࣍ݩ͕ େ͖͘༗རͳͷͰ஫ҙ • ౷߹ख๏͕୯Ұख๏
 ΛԼճΔ৔߹΋ • ഁ໓త๨٫ͷӨڹ͔ • จϕΫτϧͷ୯७ฏۉ͕Α͍ੑೳ • ౷߹ख๏ʹΑΔϕΫτϧͷੑ࣭෼ੳ ͸ࠓޙͷ՝୊ SBERT→DefSentͱ Average͕ߴੑೳ DefSent→SBERT͸ ੑೳ͕ѱԽ
42. ·ͱΊɾࠓޙͷ՝୊ 42 ౷߹ ڭࢣ৴߸ͷҧ͍ʹண໨͠จϕΫτϧͷ
 ੑ࣭Λൺֱ෼ੳɾ౷߹ ൺֱ • จͷιʔεʹΑΔख๏͝ͱͷੑೳ͕ࠩݦஶ SBERT • ײ৘ۃੑͳͲҙຯత৘ใ͕๛෋ • ද૚తྨࣅ౓ͷӨڹΛड͚ͮΒ͍ DefSent • จ௕΍੍࣌ͳͲද૚త৘ใ͕๛෋ • ϑϨʔζͷߏ੒΋ಘҙ • ද૚తྨࣅ౓͕௿͍จϖΞʹڧ͍ • ౷߹ʹΑͬͯੑೳ޲্ • SBERT→DefSent΍ Average͕ߴੑೳ • ഁ໓త๨٫ͷӨڹͰੑ ೳ͕௿Լ͢Δ৔߹΋ ෼ੳର৅ ɹSBERT: ࣗવݴޠਪ࿦ϕʔε ɹDefSent: ఆٛจ→୯ޠ༧ଌϕʔε ࠓޙͷ՝୊ 1. ΑΓ޿൚ͳϞσϧɾจϕΫτϧख๏ͷௐࠪ 2. ౷߹ख๏Ͱߏ੒͞ΕͨϕΫτϧͷੑ࣭෼ੳ 3. ΑΓΑ͍౷߹ख๏ͷ։ൃ
43. ·ͱΊɾࠓޙͷ՝୊ 43 ౷߹ ڭࢣ৴߸ͷҧ͍ʹண໨͠จϕΫτϧͷ
 ੑ࣭Λൺֱ෼ੳɾ౷߹ ൺֱ • จͷιʔεʹΑΔख๏͝ͱͷੑೳ͕ࠩݦஶ SBERT • ײ৘ۃੑͳͲҙຯత৘ใ͕๛෋ • ද૚తྨࣅ౓ͷӨڹΛड͚ͮΒ͍ DefSent • จ௕΍੍࣌ͳͲද૚త৘ใ͕๛෋ • ϑϨʔζͷߏ੒΋ಘҙ • ද૚తྨࣅ౓͕௿͍จϖΞʹڧ͍ • ౷߹ʹΑͬͯੑೳ޲্ • SBERT→DefSent΍ Average͕ߴੑೳ • ഁ໓త๨٫ͷӨڹͰੑ ೳ͕௿Լ͢Δ৔߹΋ ෼ੳର৅ ɹSBERT: ࣗવݴޠਪ࿦ϕʔε ɹDefSent: ఆٛจ→୯ޠ༧ଌϕʔε ࠓޙͷ՝୊ 1. ΑΓ޿൚ͳϞσϧɾจϕΫτϧख๏ͷௐࠪ 2. ౷߹ख๏Ͱߏ੒͞ΕͨϕΫτϧͷੑ࣭෼ੳ 3. ΑΓΑ͍౷߹ख๏ͷ։ൃ
44. ത࢜ޙظ՝ఔͷల๬
45. ത࢜ޙظ՝ఔͷల๬ •จϕΫτϧͷকདྷతͳൃలͷͨΊͷॏཁͳ՝୊ • طଘख๏͸จͷݶΒΕͨଆ໘ʹ͔͠ண໨͍ͯ͠ͳ͍ • طଘͷࣄલֶशࡁΈݴޠϞσϧ͸จϕΫτϧͷදݱྗ͕ෆ ଍ɺܭࢉྔ΋େ͖͍ •͋ΒΏΔจϕΫτϧख๏ͷج൫ͱͯ͠༻͍Δ͜ͱ͕Ͱ͖Δɺ൚ ༻ੑʹ༏ΕͨϞσϧ͕ඞཁ จϕΫτϧͷͨΊͷج൫Ϟσϧͷ։ൃ •ݚڀ1: จϕΫτϧͷੑ࣭෼ੳ •ݚڀ2: จϕΫτϧʹ͓͚Δج൫ϞσϧͷఏҊ •ݚڀ3: จϕΫτϧʹ͓͚Δج൫ϞσϧͷԠ༻ 45
46. ത࢜ޙظ՝ఔͷݚڀ 46
47. ത࢜ޙظ՝ఔͷݚڀ 47
48. ത࢜ޙظ՝ఔͷݚڀ 48
49. ݚڀܭը 49 . % % % ੑ࣭෼ੳɾ
    ৽نධՁఏҊ ج൫Ϟσϧͷ։ൃ ج൫ϞσϧͷԠ༻ ത࢜࿦จ
50. ݚڀۀ੷ ࠃ಺࿦จࢽ (ࠪಡ͋Γ) • ௩ӽॣ, ࡫໺ྒྷฏ, ෢ాߒҰ. ఆٛจΛ༻͍ͨจຒΊࠐΈߏ੒๏, ࣗવݴޠॲཧ Vol. 30 No. 1 (ൃߦ༧ఆ). ࠃࡍձٞ (ࠪಡ͋Γ) • Hayato Tsukagoshi, Ryohei Sasano, Koichi Takeda. Comparison and Combination of Sentence Embeddings Derived from Different Supervision Signals, in Proceedings of the 11th Joint Conference on Lexical and Computational Semantics (*SEM 2022). • Hayato Tsukagoshi, Ryohei Sasano, Koichi Takeda. DefSent: Sentence Embeddings using Definition Sentences, in Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (ACL-IJCNLP 2021). ࠃ಺ձٞ (ࠪಡͳ͠) • ཅాᠳฏ, ௩ӽॣ, ࡫໺ྒྷฏ, ෢ాߒҰ. Ψ΢εຒΊࠐΈʹجͮ͘จදݱੜ੒, ݴޠॲཧֶձ ୈ29ճ೥࣍େձ (NLP2023) ൃද༧ఆ. • ௩ӽॣ, ฏඌ౒, ৿Լກ, ஽ࠤࠀݾ, ࡫໺ྒྷฏ, ෢ాߒҰ. ࣗવݴޠਪ࿦ͱ࠶ݱثΛ༻͍ͨSplit and Rephraseʹ ͓͚Δੜ੒จͷ඼࣭޲্, ݴޠॲཧֶձ ୈ28ճ೥࣍େձ (NLP2022). • ௩ӽॣ, ࡫໺ྒྷฏ, ෢ాߒҰ. ఆٛจΛ༻͍ͨจຒΊࠐΈߏ੒๏, ݴޠॲཧֶձ ୈ27ճ೥࣍େձ (NLP2021). ͦͷଞ • 2023೥౓ ೔ຊֶज़ৼڵձ ಛผݚڀһ-DC1 ࠾༻಺ఆ • 2023೥౓ ໊ݹ԰େֶ༥߹ϑϩϯςΟΞϑΣϩʔೝఆ 50