Upgrade to Pro — share decks privately, control downloads, hide ads and more …

抽出的文書要約における hetero graph の応用 Heterogeneous Grap...

uchi_k
September 06, 2020

抽出的文書要約における hetero graph の応用 Heterogeneous Graph Neural Networks for Extractive Document Summarization

ACL 2020 に採択された Heterogeneous Graph Neural Networks for Extractive Document Summarization を読んでいます。

uchi_k

September 06, 2020
Tweet

More Decks by uchi_k

Other Decks in Programming

Transcript

  1. ಺ڮ ݎࢤ uchi_k @__uchi_k__ About me yuni, inc. ୅ද nlpaper.challenge

    ӡӦ Freelance Machine Learning ɹɹɹɹɹEngineer / Researcher former ژେ৘ใӃ, ະ౿16 FreakOut Machine Learning Engineer
  2. )FUFSPHFOFPVT(SBQI/FVSBM/FUXPSLT GPS&YUSBDUJWF%PDVNFOU4VNNBSJ[BUJPO #abstract จॻཁ໿Ͱ͸ɺηϯςϯεؒͷؔ܎ੑͷϞσϧԽ͕ ඇৗʹॏཁɻैདྷ͸ɺ3//ϕʔεͷख๏ͰܥྻͰ ϞσϧԽ͍ͯͨ͠ %BORJOH8BOH 4IBOHIBJ,FZ-BCPSBUPSZPG*OUFMMJHFOU*OGPSNBUJPO1SPDFTTJOH 'VEBO6OJWFSTJUZ FUBM

    "$- நग़తจॻཁ໿Ͱηϯςϯεؒͷؔ܎ੑΛදݱ͢ΔͨΊʹ IFUFSPHFOFPVTHSBQIΛಋೖ͠ɺ4P5"Λୡ੒֦ுੑͳͲʹ͍ͭͯݕূͨ͠ɻ จॻͷҙຯߏ଄͸ܥྻΑΓάϥϑߏ଄ͷํ͕దͯ͠ ͍Δ͜ͱ͕࠷ۙͷݚڀͰΘ͔͖͍ͬͯͯΔ͕ɺྑ͍ άϥϑߏ଄͸·ͩఏҊ͞Ε͍ͯͳ͔ͬͨ ୯ޠϊʔυͱจϊʔυΛ࣋ͭIFUFSPͳHSBQIߏ ଄ΛఏҊ͠ɺ୯จॻɾଟจॻཁ໿ͦΕͧΕͰ 4P5"Λୡ੒ɻ֦ுੑʹ͍ͭͯ΋ٞ࿦ͨ͠
  3. #abstract #extractive document summarization ݩͷจॻ͔Βؔ࿈͢ΔจॻΛऔΓग़ͯ͠ɺཁ໿ ͱͯ͠࠶ߏ੒͢ΔλεΫ நग़తจॻཁ໿ ୯ޠΛܦ༝ͨ͠จͷؔ܎ੑΛදݱ͢ΔIFUFSPHSBQIΛఆٛ υΩϡϝϯτͷ֤ηϯςϯεΛ#JEJSFDUJPOBM-45.ͰϕΫτϧԽɻ͜Ε ʹΑͬͯηϯςϯεͷҙຯΛଊ͑ͨϕΫτϧ͕࡞ΒΕΔʢXPSEMBZFSʣ

    நग़ܕͱɺදݱΛந৅Խͯ͠θϩ͔Βཁ໿จΛ ࡞Δੜ੒ܕɺͦΕΒͷࠞ߹ͷύλʔϯ͕͋Δ ͞Βʹ͜ͷϕΫτϧಉ࢜ͷؔ܎ੑΛ#JEJSFDUJPOBM-45.Ͱֶश͢Δ ʢTFOUFODFMBZFSʣ ηϯςϯεΛநग़͢Δ֬཰Λग़ྗ 4VNNB3V//FS  ॳظͷݚڀ
  4. #model overview ηϯςϯεͷΈΛϊʔυͱͯ͠άϥϑΛߏங͢ ΔͷͰ͸ͳ͘ɺηϯςϯεΛͭͳ͙஥հ໾ͷΑ ͏ͳϊʔυΛ௥Ճ 1SPQPTFE(SBQI ୯ޠΛܦ༝ͨ͠จͷؔ܎ੑΛදݱ͢ΔIFUFSPHSBQIΛఆٛ จ৘ใͰ୯ޠϊʔυΛߋ৽Ͱ͖Δ ଞͷϊʔυλ ΠϓΛ௥Ճ͢ΔͳͲͷ֦ுੑ͕͋ΔɺͳͲͷར

    ఺ ͜ͷ࿦จͰ͸ɺ࠷খҙຯ୯ҐΛ୯ޠʹ͍ͯ͠ Δɻྫ͑͹ɺΑΓந৅Խͯ͠୯ޠͷҙຯ΍֓೦ ΛϊʔυλΠϓͱ͢Δ͜ͱ΋໘നͦ͏ HSBQIJOJUJBMJ[Fˠ("5Ͱߋ৽ˠηϯςϯε ಛ௃͔Βཁ໿จʹ௥Ճ͢Δ͔൱͔ͷ෼ྨ໰୊Λ ղ͘ɺͱ͍͏खॱ
  5. #model overview #graph attention network ࣗ਎ͱपғʹͦΕͧΕॏΈΛ͔͚ͨϕΫτϧ͔ΒBUUFOUJPOΛܭࢉ ͠ɺपลϊʔυ͔ΒͷBHHSFHBUJPOʹར༻ (SBQI"UUFOUJPO/FUXPSL άϥϑ্ͰͷBUUFOUJPOΛఆٛ "UUFOUJPO

    ྡ઀ϊʔυ "UUFOUJPOΛܭࢉ͢Δؔ਺ "UUFOUJPOΛߟྀͨ͠ BHHSFHBUJPO άϥϑू໿ͷڑ཭ؔ਺Λɺάϥϑߏ଄ʹґଘ͠ͳ͍BUUFOUJPOͱͯ͠ ఆֶٛ͠शϕʔεͰٻΊΔɺΈ͍ͨͳ࿩ ϊʔυಛ௃
  6. #dataset #train test split %BUBTFU ୯จॻཁ໿Ͱ͸ͭɺෳ਺จॻཁ໿Ͱ͸ͭͷσʔληοτͰ࣮ݧ • ୯จॻཁ໿Ͱ࠷΋޿͘ར༻͞Ε͍ͯΔϕϯνϚʔΫσʔληοτ • USBJO

    WBMJE UFTUσʔλ͸ͦΕͧΕ      $//%BJMZ.BJM2"σʔλ • /FX:PSL5JNFT"OOPUBUFE$PSQVT 4BOEIBVT  ͔Βऩू͞Εͨ୯จॻཁ໿ σʔληοτ • USBJO WBMJE UFTUσʔλ͸ͦΕͧΕ   ݅ /:5 .VMUJ/FXT • ෳ਺จॻཁ໿σʔληοτ • ͦΕͧΕʙͷจॻʹର͠ɺਓ͕ؒॻ͍ͨཁ໿͕͋Δ • USBJO WBMJE UFTUσʔλ͸ͦΕͧΕ     
  7. #experiment #setting #hyper-parameter #preprocessing 4FUUJOH)ZQFSQBSBNFUFST લॲཧ άϥϑ ࣮ݧ ετοϓϫʔυ΍۟ಡ఺ͷআڈ ೖྗจॻͷ࠷େ௕Λจʹ

    ઃఆ UGJEGԼҐΛআڈ ޠኮ਺Λʹ੍ݶ  ࣍ݩͷ(MP7FͰຒΊࠐΈ จϕΫτϧαΠζ͸ͰॳظԽ Τοδಛ௃ྔ ࣍ݩ͸ͰॳظԽ IFBE όοναΠζ ֶश཰F "EBN FQPDIͰMPTT ͕Լ͕Βͳ͍৔߹FBSMZTUPQQJOH ୯จॻཁ໿Ͱ͸্Ґจ  ෳ਺จॻཁ໿Ͱ͸্ҐจΛબ୒
  8. #methods #extractor • &YU#J-45. ◦$// ૚#J-45. ◦จॻΛจͷܥྻͱΈͳ͠จؔ܎Λֶश͢Δ • &YU5SBOTGPSNFS ◦5SBOTGPSNFS

    ૚USBOTGPSNFS ◦શจͷϖΞϫΠζ૬ޓ࡞༻Λֶश ◦จϨϕϧͷ׬શ࿈݁άϥϑͱΈͳͤΔ • )4( )FUFS4VN(SBQI  ◦ఏҊख๏ɻจ୯ޠจͷؔ܎ੑΛάϥϑͰϞσϧԽ ◦)4(Ͱ͸ϊʔυ෼ྨʹΑͬͯཁ໿จΛબ୒͠ɺ͞ΒʹUSJHSBN CMPDLJOHʹΑͬͯUSJHSBN͕ࣅ͍ͯΔจΛআ֎͠৑௕ੑΛ཈͑ͨόʔ δϣϯ΋࣮ݧ .FUIPET
  9. #result #CNN/DailyMail 3FTVMUʢ୯จॻཁ໿ɿ$//%BJMZ.BJMʣ $//%BJMZ.BJMͰͷ୯จॻཁ໿ͷ݁Ռɻطଘख๏͢΂ͯΛ্ճΔείΞ͕ಘΒΕͨɻ -&"%͕ϕʔεϥΠϯɺ 03"$-&͕VQQFSCPVOE MBCFM QSFWJPVTTUVEZ QSPQPTFENFUIPE จ຺όϯσΟου໰୊ͱͯ͠ఆٛ

    ͨ͠)&3ʹؔͯ͠͸ಛʹϙϦ γʔ͋Γͳ͠΋࣮ݧ͠ɺ͍ͣΕ ΋উͪ ʢ#&35Λ࢖͍ͬͯͳ͍ʣશͯͷطଘख๏ΑΓߴ͍είΞ͕ಘΒΕͨ 306(&  -ͰධՁɻͦΕ ͧΕHSBN HSBN Ұக͢Δ ࠷௕ܥྻͷྨࣅ౓ͷείΞ
  10. #result #NYT50 3FTVMUʢ୯จॻཁ໿ɿ/:5ʣ /:5Ͱͷ୯จॻཁ໿ͷ࣮ݧ݁Ռɻ$//%BJMZ.BJMͱجຊతʹಉ͡܏޲͕ݟΒΕͨɻ جຊతʹ$//%BJMZ.BJM ͱಉ͡ͰɺఏҊख๏͕طଘ ख๏Λ্ճ͍ͬͯΔ QSPQPTFENFUIPE USJHSBNCMPDLJOH͋Γ όʔδϣϯ͕ҐͰ͸ͳ͍

    ͷ͸ͳͥɾɾɾʁ ˠ$//%BJMZ.BJMͰ͸ॏෳͷ গͳ͍Օ৚ॻ͖Λ࿈݁͢Δܗࣜ ͕ͩɺ/:5Ͱ͸Ωʔϑ Ϩʔζ͕ෳ਺ճొ৔͢ΔͳͲॏ ෳ͕͋ΔɻͳͷͰɺUSJHSBN CMPDLJOHͰ͸/:5Ͱε ίΞΛग़ͮ͠Β͍ͷͰ͸
  11. #ablation #CNN/DailyMail ୯ޠϑΟϧλϦϯάͷ࡟আͰ 3 3-͸είΞݮগ 3 ͸είΞ૿Ճ "CMBUJPO $//%BJMZ.BJMͰBCMBUJPO͠Ϟδϡʔϧͷߩݙ౓Λௐ΂ͨɻ ୯ޠϑΟϧλϦϯάʹΑΓɺಛʹॏཁͳ୯ޠϊʔυʹϑΥʔΧεͰ͖Δར఺

    ͕CJHSBN৘ใΛࣦ͏σϝϦοτΛ্ճ͍ͬͯΔͷͰ͸ͳ͍͔ ("5૚ؒͷSFTJEVBM DPOOFDUJPOΛ࡟আ͢Δ͜ͱͰ είΞ͕େ͖͘ݮগ ("5૚ͷSFTJEVBMDPOOFDUJPO͸ɺIFUFSPHSBQIʹ͓͚ΔผλΠϓͷ ϊʔυ͔Βͷू໿Ͱཧ࿦తʹॏཁͳͷͰ୯ͳΔ݁߹Ͱ͸ஔ͖׵͑Ͱ͖ͳ͍
  12. #qualitative analysis #source จॻ͕૿Ճ͢Δ͜ͱͰɺϕʔεϥΠϯ ͸্ঢ͢Δ͕ఏҊख๏Ͱ͸௿Լ͠  จͰฒͿ 2VBMJUBUJWF"OBMZTJT ଟจॻཁ໿Ͱɺจॻͷ਺ͷӨڹΛௐࠪ จॻ਺ͷ૿ՃͰ)&5&346.(3"1)ͱ)&5&3%0$46.(3"1)ͷੑ

    ೳ͕֦ࠩେจॻͱจॻͷؔ܎͕ෳࡶʹͳΔ΄Ͳɺจॻϊʔυͷར఺͕Α Γେ͖͘ͳΔ 'JSTU͸ɺΧόϨοδΛ֬อͰ͖Δ จষΛ֤จॻ͔Βڧ੍తʹநग़Ͱ͖Δ จॻ਺ͷ૿Ճʹ൐͍ɺશจͷओࢫΛΧ όʔͰ͖ΔݶΒΕͨ਺ͷจΛநग़͢Δ ͜ͱ͕ࠔ೉ʹͳ͍ͬͯͨ͘Ί