Upgrade to Pro — share decks privately, control downloads, hide ads and more …

第9回 全日本コンピュータビジョン勉強会 発表資料

第9回 全日本コンピュータビジョン勉強会 発表資料

ICCV 2021
DocFormer: End-to-End Transformer for Document Understanding

Yamato.OKAMOTO

December 12, 2021
Tweet

More Decks by Yamato.OKAMOTO

Other Decks in Technology

Transcript

  1. ຊ೔ɺ঺հ͢Δ࿦จ w %PDVNFOU6OEFSTUBOEJOHΛλʔήοτͱͨ͠5SBOTGPSNFSϞσϧͷఏҊ ͍ͭͰʹࢲͷ঺հ w Ԭຊେ࿨ʢ͓͔΋ͱ΍·ͱʣ w ژେඒೱݚˠΦϜϩϯˠ-*/&גࣜձࣾ ,ZPUP0GGJDF 

    w Ӊ஦Ұָ͍͠$7-BCΛژ౎ʹ࡞Δ΂͘νʔϜϏϧυத w ࠷ۙͷڵຯ͸υΩϡϝϯτղੳٕज़ → LINE DevDay2021Ͱ΋ొஃͯ͠ɺυΩϡϝϯτղੳʹ͍ͭͯ࿩͠·ͨ͠ʙ̇ ɹɹhttps://www.youtube.com/watch?v=D45wIMN6R_I
  2. 4FRVFODF-BCFMJOH )FBEFS PSBOHF 2VFTUJPO CMVF "OTXFS HSFFO ʹ෼ྨ ̍ͭ̍ͭͷςΩετΛʮ2VFTUJPOʯʮ"OTXFSʯͳͲͷΠϯςϯτΛϕʔεʹɺάϧʔϓ෼͚͢ΔλεΫɻ '6/4%

    GPSNVOEFSTUBOEJOHJOOPJTZTDBOOFEEPDVNFOUT %BUBTFU͕༗໊ɻ https://arxiv.org/abs/1905.13538 จࣈྻɺ࠲ඪΛڭࢣϥϕϧͱͯ͠෇༩ ςΩετϦϯΫͷڭࢣϥϕϧ΋͋ΔͷͰɺ ,FZ7BMVFΛநग़͢ΔΑ͏ͳλεΫ΋Մೳɻ
  3. ܏޲ɿڭࢣͳֶ͠शͰ5SBOTGPSNFSߏ଄ΛQSFUSBJO ෳ਺ͷڭࢣͳ͠λεΫͷఏҊʹΑΓ5SBOTGPSNFSߏ଄ͷֶशʹ੒ޭɻ 1.UFYUUPLFOͷҰ෦ΛӅͯ͠ɺͦΕΛ෮ݩͤ͞Δɻ 2.Ұఆׂ߹ͷςΩετΛࠇ͘ృΓͭͿ͢ɻͦͯ͠ɺృΓ௵͞Ε͍ͯΔ͔ $PWFSFE ɺృΓ௵͞Εͯͳ͍͔ /PU$PWFSFE Λ༧ଌͤ͞Δɻ 3.ͱ͖͓Γɺը૾ͱςΩετͷϖΞΛ͢Γସ͑Δɻ͜ͷ৚݅ԼͰɺ 


    υΩϡϝϯτը૾ͱςΩετͷϖΞ͕ద੾͔ʢ.BUDIʣɺ 
 ͢Γସ͑ΒΕ͍ͯΔ͔ʢ/PU.BUDIʣΛ༧ଌͤ͞Δɻ ໠ҖΛฃ͍ͬͯΔ5SBOTGPSNFSʂʂ େྔσʔλͰֶशͰ͖ΔΑ͏ʹڭࢣͳ͠λεΫΛઃఆ͢Δɻ LayoutLMv2: Multi-modal Pre-training for Visually-Rich Document Understanding (ACL'21) https://arxiv.org/pdf/2012.14740.pdf
  4. ܏޲ɿ5SBOTGPSNFSͷΞʔΩςΫνϟʔΛվળ B 7JTJPOͱ8PSEͷ5PLFOΛྻʹฒ΂ͯॱ൪ʹೖྗ͢Δɻ 
 ҟͳΔ.PEBMಉ࢜ͷσʔλͰద੾ʹ"UUFOUJPO͕͔͔Δ͔ٙ໰ɻ C 7JTJPOͱ8PSEΛҟͳΔ5SBOTGPSNFSͰॲཧ͔ͯ͠Β'VTJPO͢Δɻ 
 "UUFOUJPO͸͔͔Γͦ͏͕ͩɺ.VMUJNPEBMͷԸܙ͕ݮΓͦ͏ɻ D

    5SBOTGPSNFSʹೖྗ͢Δલͷ5PLFOஈ֊Ͱ'VTJPOͤ͞Δɻ 
 DPODBUͳͲͰγϯϓϧͳख๏Ͱ'VTJPOͤͯ͞Α͍ͷ͔ո͍͠ɻ ओு͍ͯ͠Δར఺ͱܽ఺΋ਖ਼͍͠ͷ͔ٙΘ͍͠ɺɺɺɺ ΋͸΍ͩΜͩΜͲΕ͕࠷దͳͷ͔෼͔Βͳ͘ͳ͖ͬͯͨɻɻɻɻ 7JTJPO΍-BOHVBHFͳͲੑ࣭ͷҟͳΔಛ௃ྔΛ͍͔ʹ'VTJPOͤ͞Δ͔ʁ ΞʔΩςΫνϟઃܭͰ΋ݚڀ͕ന೤தʂʂ https://arxiv.org/abs/2106.11539
  5. 7JTJPO -BOHVBHF 4QBUJBMͷछྨͷಛ௃ྔΛೖྗ 7JTJPO'FBUVSF • 3FT/FUͰ֫ಘͨ͠'FBUVSF.BQΛɺY@$POWͰ೚ҙͷDI਺ E ʹม׵ޙɺMJOFBSUSBOTGPSNFSMBZFSͰ೚ҙͷαΠζ E /

    ʹม׵ -BOHVBHF'FBUVSF • 0$3ΤϯδϯͰςΩετΛݕग़ʴೝࣝʴಛ௃ྔԽͨ͠5PLFOΛɺ&NCFEEJOHMBZFSʹ͔͚Δɻʢ˞0$3෦෼͸ֶशෆՄʣ • நग़ͨ͠ςΩετͷݸ਺͕ɺ/ݸʹຬͨͳ͍৔߹͸<1"%>Ͱ݀ຒΊ͢Δɻ/ݸΛ௒͑Δ৔߹͸5PLFOΛഁغ͢Δɻ 4QBUJBM'FBUVSF • 0$3Τϯδϯ͕ݕग़֤ͨ͠ςΩετͷCCPY࠲ඪΛՃ޻ͯ͠4QBUJBM'FBUVSFͱ͢Δɻ • CCPYͷࠨ্࠲ඪɺӈԼ࠲ඪɺ෯ɺߴ͞ɺʢը૾্Ͱࠨ্͔ΒӈԼʹ͔͚ͯCCPYΛφϯόϦϯάͨ͠ͱԾఆͯ͠ʣྡͷ#PYͱͷڑ཭Λɺ&NCFEEJOHMBZFS USBJOBCMF ʹ͔͚Δɻ • 7JTJPOઐ༻ͷ&NCFEEJOHMBZFSͱɺ-BOHVBHFઐ༻ͷ&NCFEEJOHMBZFSΛઃ͚ͯɺͦΕͧΕผʑͷ4QBUJBM'FBUVSFΛੜ੒͢Δɻ • ͜ͷͱ͖ɺ͋Θͤͯ7JTJPO΍-BOHVBHFͷ%1PTJUJPOʢԿ൪໨ͷಛ௃ྔ͔ʁʣ΋ɺ͋Θͤͯ&NCFEEJOHMBZFSʹೖྗ͢Δɻ ը૾σʔλ͔ΒҟͳΔϞμϦςΟͷಛ௃ྔΛநग़ͯ͠ɺೖྗΛΠΠײ͡ʹͨ͠ ຊ࿦จͷΞʔΩςΫνϟͰ͸ɺ֤ϞμϦςΟͰ 
 ಉ͡ݸ਺ ͜ͷਤͰ͸/ ͷಛ௃ྔΛ४උ͍ͯ͠Δɻ CCPYͰUFYUಛ௃ྔͷҐஔ৘ใΛ࣋ͨͤΔͱͱ΋ʹɺ 
 %1PTJUJPOʹΑͬͯɺWJTJPOಛ௃ྔͷҐஔ৘ใ΋࣋ͨͤΒΕΔͱ༧૝ɻ
  6. 3FDPOTUSVDUJPOΛؚΉͭͷڭࢣͳ͠λεΫΛఆٛ .BTLFE-BOHVBHF.PEFMJOH • ैདྷͷ#&35ͱಉ༷ʹɺ&ODPEF͞Εͨಛ௃ྔ͔Β.BTLFE5FYUΛ༧ଌ͢Δɻ • ͨͩ͠ɺ.VMUJNPEBMʹͨ͠ಛ௃ྔ͔Β෮ݩͤ͞Δ఺͕ಛ௃తͰ͋Δɻ 3FDPOTUSVDU • &ODPEF͞Εͨશͯͷಛ௃ྔΛར༻ͯ͠ɺ"VUP&ODPEFSͱಉ༷ʹը૾Λ3FDPOTUSVDUJPO͢Δɺ-PTT͸TNPPUI-Λ࠾༻ɻ •

    ग़ྗ͸ը૾ͷΈ͕ͩɺจࣈը૾Λਖ਼֬ʹ3FDPOTUSVDUJPO͢΂͘ɺ7JTJPOʹՃ͑ͯ-BOHVBHFͷ׆༻๏΋ֶशͯ͘͠ΕΔͱظ଴ɻ 5FYU%FTDSJCF*NBHF • Ұఆׂ߹Ͱը૾ͱ5FYUͷ૊Έ߹ΘͤΛγϟοϑϧͯ͠ɺϖΞ͕ద੾͔Ͳ͏͔ PS Λ༧ଌ͢Δɻ • ্ड़ͷͭͷλεΫ͕ओʹ-PBDM'FBUVSFʹண໨͢ΔҰํͰɺͪ͜Β͸(MPCBM'FBUVSFʹண໨͢΂͖λεΫͰ͋Δɻ • <$-4>Λ&ODͨ͠ಛ௃ྔ͔Β༧૝͍ͯ͠Δͷ͸ɺ"UUFOUJPOͰΠΠײ͡ʹશମ͔Βಛ௃ྔΛू໿͍ͯ͠Δͱظ଴Ͱ͖Δ͔Βɻ ֤छϞʔμϧɺ-PDBM'FBUVSFɺ(MPCBM'FBUVSFΛ׆༻͢ΔΑ͏ ڭࢣͳ͠λεΫΛόϥϯεΑ͘ఏҊͯ͠ɺֶशΛΠΠײ͡ʹͨ͠
  7. ఏҊΞʔΩςΫνϟͷৄࡉ w -BOHVBHFͱ7JTJPOͦΕͧΕʹઐ༻ͷMBZFSʢࢵͱ྘ʣΛઃ͚ͯɺಛ௃ྔΛม׵ͯ͠࠷ޙʹՃࢉ͢Δ w -BOHVBHF'FBUVSF͸ɺMBZFSΛॏͶΔ͝ͱʹɺҟͳΔϞʔμϧಛ௃Λࠞͥͳ͕Β܁Γฦ͠5SBOTGPSN͞ΕΔ w ҰํͰɺ7JTJPO'FBUVSFͱ4QBUJBM'FBUVSF͸ɺຖճॳظঢ়ଶͰ֤-BZFSʹԣ͔Βྲྀ͠ࠐ·ΕΔ w 4QBUJBM'FBUVSFΛ,FZ΍2VFSZʹ͢ΔMBZFS͸ڞ௨Խʢ੺ʣ͍ͯ͠Δ w

    ͨͩ͠ɺͦ΋ͦ΋4QBUJBM'FBUVSF͸-BOHVBHFઐ༻ͱ7JTJPOઐ༻Ͱผʑʹ४උͨ͠఺ʹ஫ҙ w 8SFM͸%SFMBUJWFQPTJUJPOFNCFEEJOHͱݺ͹ΕɺJ൪໨ͱK൪໨ͷؔ܎ੑΛ֫ಘ͢ΔͨΊͷֶशՄೳͳύϥϝλ w ࿦จͰ͸ɺྑ͍࣮ݧ݁Ռ͕ಘΒΕͨ͜ͱ͔Β΋ɺʮ֤Ϟʔμϧ΁ͷΞΫηεΛྑͨ͘͜͠ͱ͕ॏཁʂʯͱओு͍ͯ͠Δ https://openaccess.thecvf.com/content/ICCV2021/supplemental/Appalaraju_DocFormer_End-to-End_Transformer_ICCV_2021_supplemental.pdf ελϯμʔυͳܗ͸͜Εʂ