Upgrade to Pro — share decks privately, control downloads, hide ads and more …

抽出的文書要約における hetero graph の応用 Heterogeneous Grap...

Avatar for uchi_k uchi_k
September 06, 2020

抽出的文書要約における hetero graph の応用 Heterogeneous Graph Neural Networks for Extractive Document Summarization

ACL 2020 に採択された Heterogeneous Graph Neural Networks for Extractive Document Summarization を読んでいます。

Avatar for uchi_k

uchi_k

September 06, 2020
Tweet

More Decks by uchi_k

Other Decks in Programming

Transcript

  1. ಺ڮ ݎࢤ uchi_k @__uchi_k__ About me yuni, inc. ୅ද nlpaper.challenge

    ӡӦ Freelance Machine Learning ɹɹɹɹɹEngineer / Researcher former ژେ৘ใӃ, ະ౿16 FreakOut Machine Learning Engineer
  2. )FUFSPHFOFPVT(SBQI/FVSBM/FUXPSLT GPS&YUSBDUJWF%PDVNFOU4VNNBSJ[BUJPO #abstract จॻཁ໿Ͱ͸ɺηϯςϯεؒͷؔ܎ੑͷϞσϧԽ͕ ඇৗʹॏཁɻैདྷ͸ɺ3//ϕʔεͷख๏ͰܥྻͰ ϞσϧԽ͍ͯͨ͠ %BORJOH8BOH 4IBOHIBJ,FZ-BCPSBUPSZPG*OUFMMJHFOU*OGPSNBUJPO1SPDFTTJOH 'VEBO6OJWFSTJUZ FUBM

    "$- நग़తจॻཁ໿Ͱηϯςϯεؒͷؔ܎ੑΛදݱ͢ΔͨΊʹ IFUFSPHFOFPVTHSBQIΛಋೖ͠ɺ4P5"Λୡ੒֦ுੑͳͲʹ͍ͭͯݕূͨ͠ɻ จॻͷҙຯߏ଄͸ܥྻΑΓάϥϑߏ଄ͷํ͕దͯ͠ ͍Δ͜ͱ͕࠷ۙͷݚڀͰΘ͔͖͍ͬͯͯΔ͕ɺྑ͍ άϥϑߏ଄͸·ͩఏҊ͞Ε͍ͯͳ͔ͬͨ ୯ޠϊʔυͱจϊʔυΛ࣋ͭIFUFSPͳHSBQIߏ ଄ΛఏҊ͠ɺ୯จॻɾଟจॻཁ໿ͦΕͧΕͰ 4P5"Λୡ੒ɻ֦ுੑʹ͍ͭͯ΋ٞ࿦ͨ͠
  3. #abstract #extractive document summarization ݩͷจॻ͔Βؔ࿈͢ΔจॻΛऔΓग़ͯ͠ɺཁ໿ ͱͯ͠࠶ߏ੒͢ΔλεΫ நग़తจॻཁ໿ ୯ޠΛܦ༝ͨ͠จͷؔ܎ੑΛදݱ͢ΔIFUFSPHSBQIΛఆٛ υΩϡϝϯτͷ֤ηϯςϯεΛ#JEJSFDUJPOBM-45.ͰϕΫτϧԽɻ͜Ε ʹΑͬͯηϯςϯεͷҙຯΛଊ͑ͨϕΫτϧ͕࡞ΒΕΔʢXPSEMBZFSʣ

    நग़ܕͱɺදݱΛந৅Խͯ͠θϩ͔Βཁ໿จΛ ࡞Δੜ੒ܕɺͦΕΒͷࠞ߹ͷύλʔϯ͕͋Δ ͞Βʹ͜ͷϕΫτϧಉ࢜ͷؔ܎ੑΛ#JEJSFDUJPOBM-45.Ͱֶश͢Δ ʢTFOUFODFMBZFSʣ ηϯςϯεΛநग़͢Δ֬཰Λग़ྗ 4VNNB3V//FS  ॳظͷݚڀ
  4. #model overview ηϯςϯεͷΈΛϊʔυͱͯ͠άϥϑΛߏங͢ ΔͷͰ͸ͳ͘ɺηϯςϯεΛͭͳ͙஥հ໾ͷΑ ͏ͳϊʔυΛ௥Ճ 1SPQPTFE(SBQI ୯ޠΛܦ༝ͨ͠จͷؔ܎ੑΛදݱ͢ΔIFUFSPHSBQIΛఆٛ จ৘ใͰ୯ޠϊʔυΛߋ৽Ͱ͖Δ ଞͷϊʔυλ ΠϓΛ௥Ճ͢ΔͳͲͷ֦ுੑ͕͋ΔɺͳͲͷར

    ఺ ͜ͷ࿦จͰ͸ɺ࠷খҙຯ୯ҐΛ୯ޠʹ͍ͯ͠ Δɻྫ͑͹ɺΑΓந৅Խͯ͠୯ޠͷҙຯ΍֓೦ ΛϊʔυλΠϓͱ͢Δ͜ͱ΋໘നͦ͏ HSBQIJOJUJBMJ[Fˠ("5Ͱߋ৽ˠηϯςϯε ಛ௃͔Βཁ໿จʹ௥Ճ͢Δ͔൱͔ͷ෼ྨ໰୊Λ ղ͘ɺͱ͍͏खॱ
  5. #model overview #graph attention network ࣗ਎ͱपғʹͦΕͧΕॏΈΛ͔͚ͨϕΫτϧ͔ΒBUUFOUJPOΛܭࢉ ͠ɺपลϊʔυ͔ΒͷBHHSFHBUJPOʹར༻ (SBQI"UUFOUJPO/FUXPSL άϥϑ্ͰͷBUUFOUJPOΛఆٛ "UUFOUJPO

    ྡ઀ϊʔυ "UUFOUJPOΛܭࢉ͢Δؔ਺ "UUFOUJPOΛߟྀͨ͠ BHHSFHBUJPO άϥϑू໿ͷڑ཭ؔ਺Λɺάϥϑߏ଄ʹґଘ͠ͳ͍BUUFOUJPOͱͯ͠ ఆֶٛ͠शϕʔεͰٻΊΔɺΈ͍ͨͳ࿩ ϊʔυಛ௃
  6. #dataset #train test split %BUBTFU ୯จॻཁ໿Ͱ͸ͭɺෳ਺จॻཁ໿Ͱ͸ͭͷσʔληοτͰ࣮ݧ • ୯จॻཁ໿Ͱ࠷΋޿͘ར༻͞Ε͍ͯΔϕϯνϚʔΫσʔληοτ • USBJO

    WBMJE UFTUσʔλ͸ͦΕͧΕ      $//%BJMZ.BJM2"σʔλ • /FX:PSL5JNFT"OOPUBUFE$PSQVT 4BOEIBVT  ͔Βऩू͞Εͨ୯จॻཁ໿ σʔληοτ • USBJO WBMJE UFTUσʔλ͸ͦΕͧΕ   ݅ /:5 .VMUJ/FXT • ෳ਺จॻཁ໿σʔληοτ • ͦΕͧΕʙͷจॻʹର͠ɺਓ͕ؒॻ͍ͨཁ໿͕͋Δ • USBJO WBMJE UFTUσʔλ͸ͦΕͧΕ     
  7. #experiment #setting #hyper-parameter #preprocessing 4FUUJOH)ZQFSQBSBNFUFST લॲཧ άϥϑ ࣮ݧ ετοϓϫʔυ΍۟ಡ఺ͷআڈ ೖྗจॻͷ࠷େ௕Λจʹ

    ઃఆ UGJEGԼҐΛআڈ ޠኮ਺Λʹ੍ݶ  ࣍ݩͷ(MP7FͰຒΊࠐΈ จϕΫτϧαΠζ͸ͰॳظԽ Τοδಛ௃ྔ ࣍ݩ͸ͰॳظԽ IFBE όοναΠζ ֶश཰F "EBN FQPDIͰMPTT ͕Լ͕Βͳ͍৔߹FBSMZTUPQQJOH ୯จॻཁ໿Ͱ͸্Ґจ  ෳ਺จॻཁ໿Ͱ͸্ҐจΛબ୒
  8. #methods #extractor • &YU#J-45. ◦$// ૚#J-45. ◦จॻΛจͷܥྻͱΈͳ͠จؔ܎Λֶश͢Δ • &YU5SBOTGPSNFS ◦5SBOTGPSNFS

    ૚USBOTGPSNFS ◦શจͷϖΞϫΠζ૬ޓ࡞༻Λֶश ◦จϨϕϧͷ׬શ࿈݁άϥϑͱΈͳͤΔ • )4( )FUFS4VN(SBQI  ◦ఏҊख๏ɻจ୯ޠจͷؔ܎ੑΛάϥϑͰϞσϧԽ ◦)4(Ͱ͸ϊʔυ෼ྨʹΑͬͯཁ໿จΛબ୒͠ɺ͞ΒʹUSJHSBN CMPDLJOHʹΑͬͯUSJHSBN͕ࣅ͍ͯΔจΛআ֎͠৑௕ੑΛ཈͑ͨόʔ δϣϯ΋࣮ݧ .FUIPET
  9. #result #CNN/DailyMail 3FTVMUʢ୯จॻཁ໿ɿ$//%BJMZ.BJMʣ $//%BJMZ.BJMͰͷ୯จॻཁ໿ͷ݁Ռɻطଘख๏͢΂ͯΛ্ճΔείΞ͕ಘΒΕͨɻ -&"%͕ϕʔεϥΠϯɺ 03"$-&͕VQQFSCPVOE MBCFM QSFWJPVTTUVEZ QSPQPTFENFUIPE จ຺όϯσΟου໰୊ͱͯ͠ఆٛ

    ͨ͠)&3ʹؔͯ͠͸ಛʹϙϦ γʔ͋Γͳ͠΋࣮ݧ͠ɺ͍ͣΕ ΋উͪ ʢ#&35Λ࢖͍ͬͯͳ͍ʣશͯͷطଘख๏ΑΓߴ͍είΞ͕ಘΒΕͨ 306(&  -ͰධՁɻͦΕ ͧΕHSBN HSBN Ұக͢Δ ࠷௕ܥྻͷྨࣅ౓ͷείΞ
  10. #result #NYT50 3FTVMUʢ୯จॻཁ໿ɿ/:5ʣ /:5Ͱͷ୯จॻཁ໿ͷ࣮ݧ݁Ռɻ$//%BJMZ.BJMͱجຊతʹಉ͡܏޲͕ݟΒΕͨɻ جຊతʹ$//%BJMZ.BJM ͱಉ͡ͰɺఏҊख๏͕طଘ ख๏Λ্ճ͍ͬͯΔ QSPQPTFENFUIPE USJHSBNCMPDLJOH͋Γ όʔδϣϯ͕ҐͰ͸ͳ͍

    ͷ͸ͳͥɾɾɾʁ ˠ$//%BJMZ.BJMͰ͸ॏෳͷ গͳ͍Օ৚ॻ͖Λ࿈݁͢Δܗࣜ ͕ͩɺ/:5Ͱ͸Ωʔϑ Ϩʔζ͕ෳ਺ճొ৔͢ΔͳͲॏ ෳ͕͋ΔɻͳͷͰɺUSJHSBN CMPDLJOHͰ͸/:5Ͱε ίΞΛग़ͮ͠Β͍ͷͰ͸
  11. #ablation #CNN/DailyMail ୯ޠϑΟϧλϦϯάͷ࡟আͰ 3 3-͸είΞݮগ 3 ͸είΞ૿Ճ "CMBUJPO $//%BJMZ.BJMͰBCMBUJPO͠Ϟδϡʔϧͷߩݙ౓Λௐ΂ͨɻ ୯ޠϑΟϧλϦϯάʹΑΓɺಛʹॏཁͳ୯ޠϊʔυʹϑΥʔΧεͰ͖Δར఺

    ͕CJHSBN৘ใΛࣦ͏σϝϦοτΛ্ճ͍ͬͯΔͷͰ͸ͳ͍͔ ("5૚ؒͷSFTJEVBM DPOOFDUJPOΛ࡟আ͢Δ͜ͱͰ είΞ͕େ͖͘ݮগ ("5૚ͷSFTJEVBMDPOOFDUJPO͸ɺIFUFSPHSBQIʹ͓͚ΔผλΠϓͷ ϊʔυ͔Βͷू໿Ͱཧ࿦తʹॏཁͳͷͰ୯ͳΔ݁߹Ͱ͸ஔ͖׵͑Ͱ͖ͳ͍
  12. #qualitative analysis #source จॻ͕૿Ճ͢Δ͜ͱͰɺϕʔεϥΠϯ ͸্ঢ͢Δ͕ఏҊख๏Ͱ͸௿Լ͠  จͰฒͿ 2VBMJUBUJWF"OBMZTJT ଟจॻཁ໿Ͱɺจॻͷ਺ͷӨڹΛௐࠪ จॻ਺ͷ૿ՃͰ)&5&346.(3"1)ͱ)&5&3%0$46.(3"1)ͷੑ

    ೳ͕֦ࠩେจॻͱจॻͷؔ܎͕ෳࡶʹͳΔ΄Ͳɺจॻϊʔυͷར఺͕Α Γେ͖͘ͳΔ 'JSTU͸ɺΧόϨοδΛ֬อͰ͖Δ จষΛ֤จॻ͔Βڧ੍తʹநग़Ͱ͖Δ จॻ਺ͷ૿Ճʹ൐͍ɺશจͷओࢫΛΧ όʔͰ͖ΔݶΒΕͨ਺ͷจΛநग़͢Δ ͜ͱ͕ࠔ೉ʹͳ͍ͬͯͨ͘Ί