[読み会資料] Federated Learning for Vision-and-Language Grounding Problems

Naoki Katsura Federated Learning for Vision-and-Language Grounding Problems 2020-03-10 http://web.pkusz.edu.cn/adsp/files/2019/11/AAAI-FenglinL.1027.pdf

࿦จ৘ใ ʲஶऀʳ Fenglin Liu(Penking Univ), Xian Wu(Tencent), Shen Ge(Tencent), Wei
Fan(Tencent), Yuexian Zou(Penking Univ & Peng Cheng Laboratory) ʲग़యʳ AAAI 2020 ʲͳͥɼ͜ͷ࿦จΛબΜͩʁʳ ɹ'-ͷϑϨʔϜϫʔΫΛผͷλεΫʹҠ২ͨ͜͠ͱʹ໘നΈΛײ͔ͨ͡Βɽ ʲࡶஊʳ ୈҰஶऀ'-JVͷ೥ͷઓ੷ʢୈҰஶऀͷ࿦จʣ w /FVS*14 "DDFQUFE w *+$"* "DDFQUFE w *$%. "DDFQUFE ⟵ ⟵ ⟵

༻ޠ 7JTJPOBOE-BOHVBHF(SPVOEJOH1SPCMFN ɹࣗવݴޠॲཧͱίϯϐϡʔλϏδϣϯʹ·͕ͨΔ໰୊Λಉ࣌ʹॲཧ͢Δ໰୊ɽ ྫ image captioning, visual question answer(VQA), image
caption retrieval, etc… )PSJ[POUBM'FEFSBUFE-FBSOJOH )'- ɹTBNQMFCBTFEGFEFSBUFEMFBSOJOHͱ΋஌ΒΕ͍ͯΔɽಛ௃ۭؒ͸ಉ͕ͩ͡ɼα ϯϓϧʢϢʔβʣू߹͸ҟͳΔΑ͏ͳσʔλΛ༻͍Δ'-໰୊ɽ 7FSUJDBM'FEFSBUFE-FBSOJOH 7'- ɹGFBUVSFCBTFEGFEFSBUFEMFBSOJOHͱ΋஌ΒΕ͍ͯΔɽαϯϓϧʢϢʔβʣू߹ ͸ಉ͕ͩ͡ɼಛ௃ۭؒ͸ҟͳΔΑ͏ͳσʔλΛ༻͍Δ'-໰୊ɽ 'FEFSBUFE5SBOTGFS-FBSOJOH '5- ɹಛ௃ۭؒ΋αϯϓϧʢϢʔβʣू߹΋ҟͳΔΑ͏ͳσʔλΛ༻͍Δ'-໰୊ɽ )'- 7'- '5-ͳͲͷৄࡉ͸ɼҎԼͷ࿦จʹॻ͍ͯ͋Δɽ “Federated Learning machine learning: Concept and applications”

image caption retrieval ग़యɿDual Attention Networks for Multimodal Reasoning and
Matching

ͲΜͳ࿦จʁ ɹ7JTJPOBOE-BOHVBHF(SPVOEJOH1SPCMFNʹ͓͍ͯɼ NVMUJUBTLMFBSOJOHʢ.5-ʣΛద༻ͨ͠৔߹ɼEBUBMFBLBHF ͱ͍͏໰୊͕ੜ͡Δɽ ɹ'FEFSBUFE-FBSOJOHͷ࿮૊ΈΛ࢖͍ɼ֤λεΫΛ'-ʹ͓͚ ΔݸʑͷΫϥΠΞϯτͱΈͳ͢ͱɼλεΫ͝ͱͷσʔλΛ׬શ ʹ෼཭͠ͳ͕ΒϞσϧΛֶशͰ͖ΔΜ͡Όͳ͍͔ʁɹ ͱ͍͏ఏҊɽ

എܠ ɹJNBHFDBQUJPOJOHͱ72"͸ɼͲͪΒ΋ը૾ͱࣗવݴޠΛѻ͏ͱ͍͏఺ͰλεΫ ͕ࣅ͍ͯΔɽ

എܠ ɹJNBHFDBQUJPOJOHͱ72"͸ɼͲͪΒ΋ը૾ͱࣗવݴޠΛѻ͏ͱ͍͏఺ͰλεΫ ͕ࣅ͍ͯΔɽ ɹλεΫΛ߹ΘͤΕ͹ɼҟͳΔλεΫ͔ΒҟͳΔ஌ࣝΛ֫ಘͰ͖ΔͷͰੑೳ޲্͕ݟ ࠐΊΔΜ͡Όͳ͍ʁ .VMUJUBTLMFBSOJOHΛద༻ɹ -JFUBM ͔͠͠ɾɾɾ

എܠ ɹJNBHFDBQUJPOJOHͱ72"͸ɼͲͪΒ΋ը૾ͱࣗવݴޠΛѻ͏ͱ͍͏఺ͰλεΫ ͕ࣅ͍ͯΔɽ ɹλεΫΛ߹ΘͤΕ͹ɼҟͳΔλεΫ͔ΒҟͳΔ஌ࣝΛ֫ಘͰ͖ΔͷͰੑೳ޲্͕ݟ ࠐΊΔΜ͡Όͳ͍ʁ .VMUJUBTLMFBSOJOHΛద༻ɹ -JFUBM ͔͠͠ɾɾɾ ɹҟͳΔλεΫؒͰ΋Ϟσϧޙ൒෦·Ͱ͸ॲཧΛڞ༗͍ͯ͠Δɽ
ɹ ݁ՌɼEBUBMFBLBHF໰୊ͷൃੜ ⟹

[Li et al. , 2018] ʲ໰୊఺ʳ ɹ͜ͷ৚݅ԼͰ͸ɼλεΫؒͰσʔληοτ͕ҟͳΔͱద༻Ͱ͖ͳ͍ɽʢ7'-ʣ ɹશλεΫʹ͓͍ͯೖྗΛ౷Ұ͠ͳ͍ͱ͍͚ͳ͍ɽ .5-༻ͷಛผͳσʔληοτΛ࡞Δඞཁ͕͋Δɽ
⟹

[Nguyen & Okatani , 2019] ɹλεΫؒͰσʔληοτΛ ౷Ұ͠ͳ͍ͱ͍͚ͳ͍໰୊Λ ؇࿨ͨ͠΋ͷɽ ɹ72"ͳͲʹ͸ೖྗʹςΩετ ͕ඞਢɽ
ɹ*NBHFDBQUJPOJOHͷΑ͏ͳೖ ྗ͕ը૾ͷΈͷͱ͖ɼ͋·Γྑ͍ ݁Ռ͕ಘΒΕͳ͍ɽ ʢEBUBMFBLBHFೖྗςΩετ ͔͠͠ ͕ͨͬͯ͠

ఏҊ ɹEBUBMFBLBHFΛ๷͗ͳ͕ΒλεΫؒͰಛ௃Λڞ༗͢ΔͨΊʹɼ'-ͷ࿮૊ΈΛར༻ ͨ͠ɹ"MJHOJOH *OUFHSBUJOHBOE.BQQJOH/FUXPSLʢBJN/FUʣͷఏҊɽ ɹ༷ʑͳ৚݅Λ)'- 7'- '5-ͷ࿮ʹ౰ͯ͸ΊΔ͜ͱͰॊೈʹରԠՄೳɽ ͜ͷ࿦จʹ͓͚Δߩݙ͸ɼҎԼͷ௨Γɽ ᶃ '-ͷ࿮૊ΈΛ࢖ͬͯɼҟͳΔλεΫ΍σʔληοτΛEBUBMFBLBHFΛ๷͗ͳ͕Β
ར༻͢Δ͜ͱͰɼ୯ମͷλεΫΑΓ΋ྑ͍ಛ௃ΛಘΔ͜ͱʹ੒ޭͨ͜͠ͱɽ ᶄ '-ʹ͓͚ΔDFOUSBMJ[FENPEFMͱͯ͠ɼೖྗը૾͔ΒWJTVBMͱUFYUVBMಛ௃Λநग़ ͢ΔBJN/FUΛఏҊͨ͜͠ͱɽ ᶅ )PSJ[POUBM'FEFSBUFE-FBSOJOH 7FSUJDBM'FEFSBUFE-FBSOJOH 'FEFSBUFE 5SBOTGFS-FBSOJOHͷ৚݅Լͷ্Ͱɼ༗ޮੑ͕ࣔ͞Εͨ͜ͱɽ

ఏҊख๏ ʲఏҊख๏ͷߏ੒ʳ ᶃ UIFWJTVBMBOEUFYUVBMGFBUVSFTFYUSBDUPS ೖྗը૾͔ΒɼΑΓϦονͳදݱΛಘΔಛ௃ྗநग़ثʢ৽نੑ͸ͳ͍ʣ ᶄ "MJHOJOH *OUFHSBUJOHBOE.BQQJOH/FUXPSLBJN/FU WJTVBMUFYUVBMGFBUVSFΛΑΓϦονͳදݱʹม׵͢ΔωοτϫʔΫɽλεΫ͓ ΑͼσʔληοτؒͰڞ༗ɽ
ᶅ UIFJNQMFNFOUBUJPOJOUISFFGFEFSBUFEMFBSOJOHTFUUJOHT )'- 7'- '5-ͷঢ়گʹ౰ͯ͸ΊΔ͜ͱͰɼλεΫ΍σʔληοτʹؔ͢Δঢ়گ ʹఏҊख๏͕ࠨӈ͞Εͳ͍͜ͱΛࣔ͢ɽ

ɹWJTJPOBOEMBOHVBHFHSPVOEJOHUBTLʹ͓͍ͯɼ$//Λ༻͍ͯը૾ͷಛ௃Λநग़ ͢Δ͜ͱ͸޿͘࢖ΘΕ͍ͯΔɽຊ࿦จͰ͸ɼ'BTUFS3$//Λ࢖༻ͯ͠ɼ ಛ௃ʢWJTVBMGFBUVSFʣͷநग़Λߦͬͨɽ ͔͠͠ ɹWJTVBMGFBUVSFͷΈͩͱɼ͜ͷλεΫʹ༗ޮͳಛ௃ͷநग़ʹݶք͕͋Δɽ ɹɹɹɹɹɹɹɹɹɹɹɹɹɹɹɹɹɹɹɹɹɹɹɹɹɹɹɹʢ8VFUBM ʣ ɹUFYUVBMGFBUVSFΛ࢖͏ͱɼΑΓ༗ޮͳಛ௃Λը૾͔Β໌֬ʹநग़Ͱ͖Δɽ ɹ'BTUFS3$//ʹΑΔಛ௃ྔநग़ʹՃ͑ͯɼ<'BOHFUBM >ʹैͬͯɼ
.VMUJQMF*OTUBODF-FBSOJOHʢ;IBOHFUBM ʣΛద༻͠ɼUFYUVBMGFBUVSFΛ நग़ͨ͠ɽ Visual and Textual Feature

Visual and Textual Feature ʲΠϝʔδʳ ೖྗը૾ .VMUJQMF*OTUBODF -FBSOJOH 'BTUFS3$// WJTVBMGFBUVSFFYUSBDUPS
UFYUVBMGFBUVSFFYUSBDUPS &NCFEEJOH ⃗ I = { ⃗ i1 , ⃗ i2 , …, ⃗ iN } ∈ ℝN×d ⃗ T = { ⃗ w1 , ⃗ w2 , …, ⃗ wM } ∈ ℝM×d ɹࠨͷΑ͏ͳಘΒΕͨ୯ޠ܈ΛTFNBOUJDDPODFQU ͱ͍͍ɼ෺ମʢEPH΍GSJTCFFʣɼঢ়ଶʢP⒎΍ FMFDUSJDʣɼؔ܎ੑʢIPMEJOH qZJOHʣؚ͕·ΕΔɽ QSPWJEFECZ"OEFSTPOFUBM QSPWJEFECZ'BOHFUBM ͜ͷFNCFEEJOH૚͸ɼ DBQUJPORVFTUJPOͱڞ༗ɽ Experiment Setting ɹXPSEFNCFEEJOH͸ɼDBQUJPORVFTUJPOͱڞ༗ɽ ͸Ҏ ߱Ͱ঺հ͢Δ#BTFMJOF طଘݚڀͷϞσϧ ͷσίʔμͷೖྗ ࣍ݩʹ͢Δɽ d N = M = 36.

Aligning, Integrating and Mapping Network : aimNet ɹBJN/FU͸ɼҎԼͷͭͷϞδϡʔϧͰߏ੒͞ΕΔɽ͜Ε͸ɼ֤λεΫʹ͓͚Δը૾ ͷΤϯίʔμͷ෦෼ʹ૬౰͢Δɽ w"MJHOJOHNPEVMF
ɹWJTVBMGFBUVSFͱUFYUVBMGFBUVSFؒͷؔ܎ੑΛରԠ͚ͮΔɽ w*OUFHSBUJOHNPEVMF ɹWJTVBMGFBUVSF಺ɼ͓ΑͼUFYUVBMGFBUVSF಺Ͱͷؔ܎ੑΛରԠ͚ͮΔɽ w.BQQJOH.PEVMF ɹλεΫ͝ͱͷೖྗαΠζʹม׵͢Δɽ

Aligning, Integrating and Mapping Network : aimNet "MJHOJOH.PEVMFͱ*OUFHSBUJOH.PEVMF͸ɼ࣍ͷ#BTJD.PEVMFΛϕʔεͱ͍ͯ͠ Δɽ ʲ#BTJD.PEVMFʳ
ɹ֤ಛ௃ؒͷؔ܎ੑΛଊ͑ΔͨΊʹɼ.VMUJ)FBE"UUFOUJPOʢ.)"ʣͱ'FFE 'PSXBSE/FUXPSLʢ''/ʣΛద༻ɽ͜ΕΛ࢖͏͜ͱͰɼಛ௃ΛҰରҰͰ͸ͳ͘ɼ ଟରଟͰಛ௃ؒͷؔ܎ੑͷڧ͞ΛଌΔ͜ͱ͕Ͱ͖Δɽ ɹ ͜ͷΞΠσΞΛɼ WJTVBMGFBUVSFͱUFYUVBMGFBUVSFؒͷؔ܎ੑΛଊ͑Δ ɹ "MJHOJOH.PEVMF WJTVBMGFBUVSFಉ࢜ɼ·ͨ͸ɼUFYUVBMGFBUVSFಉ࢜ͷؔ܎ੑΛଊ͑Δ ɹ *OUFHSBUJOH.PEVMF ⟹ ⟹

Aligning, Integrating and Mapping Network : aimNet w"MJHOJOHNPEVMF ɹWJTVBMGFBUVSFͱUFYUVBMGFBUVSFؒͷؔ܎ੑΛରԠ͚ͮΔɽ w*OUFHSBUJOHNPEVMF
ɹWJTVBMGFBUVSF಺ɼ͓ΑͼUFYUVBMGFBUVSF಺Ͱͷؔ܎ੑΛରԠ͚ͮΔɽ w.BQQJOH.PEVMF ɹλεΫ͝ͱͷೖྗαΠζʹม׵͢Δɽ

Aligning, Integrating and Mapping Network : aimNet ʲ"MJHOJOH.PEVMFʳ ɹ΍͍ͬͯΔ͜ͱ͸ɼ ɹɹ4PVSDF5BSHFU"UUFOUJPOͱಉ͡ɽ
ɹɹ ɹɹ ɹ͜Ε͸72"Ͱ༗ޮͱ͍͏͜ͱ͕ࣔ͞Ε͍ͯΔɽʢ4VFUBMʣ ɹը૾ͱΞϥΠϝϯτΛऔΔ͜ͱͰɼಉ͡εϖϧͰ΋ෳ਺ͷҙຯΛ࣋ͭΑ͏ͳᐆດͳ දݱΛ཈͑ࠐΉ͜ͱ͕Ͱ͖ΔɽʢྫɿNPVTFͶͣΈɼిࢠػثͷϚ΢εʣ ⃗ Ia = FFN(MHA( ⃗ I, ⃗ T , ⃗ T )) ⃗ T a = FFN(MHA( ⃗ T , ⃗ I, ⃗ I)) Experiment Setting ɹBUUFOUJPOIFBEͷݸ਺͸ɼGFFEGPSXBSEOFUXPSLͷ࣍ ݩ਺͸

Aligning, Integrating and Mapping Network : aimNet ʲ*OUFHSBUJOH.PEVMFʳ ɹ΍͍ͬͯΔ͜ͱ͸ɼ ɹɹ4FMG"UUFOUJPOͱಉ͡ɽ
ɹɹ ɹɹ ɹ͜Ε͸*NBHF$BQUJPOJOHͰ༗ޮͱ͍͏͜ͱ͕ࣔ͞Ε͍ͯΔɽʢ:BPFUBMʣ ⃗ Ii = FFN(MHA( ⃗ Ia , ⃗ Ia , ⃗ Ia )) ⃗ T i = FFN(MHA( ⃗ T a , ⃗ T a , ⃗ T a )) Experiment Setting ɹBUUFOUJPOIFBEͷݸ਺͸ɼGFFEGPSXBSEOFUXPSLͷ࣍ ݩ਺͸

Aligning, Integrating and Mapping Network : aimNet ʲ.BQQJOH.PEVMFʳ ɹҎԼͷΑ͏ʹɼ.BQQJOHؔ਺Λ׆ੑԽؔ਺͕UBOIͷ૚ͷχϡʔϥϧωοτͱ͠ ͯఆٛ͢Δɽ
ɹɹɹɹɹɹɹɹɹɹ ɹ.BQQJOH.PEVMFͰ͸ɼλεΫ͝ͱͷσίʔμͷೖྗۭؒʹલͭͷϞδϡʔϧ͔ ΒಘΒΕͨಛ௃Λ߹ΘͤΔͨΊʹҎԼͷΑ͏ͳม׵Λߦ͏ɽ ɹɹɹɹɹɹɹɹɹɹ ɹ͜ͷϞδϡʔϧ͸ɼҰൠతʹطଘख๏ͷଟ͘Ͱ࢖༻͞Ε͍ͯΔɽ#BTFMJOFͰ͸ɼ $//ͳͲʢFH'BTUFS3$//ʣ͔Βͷಛ௃ϚοϓΛ͜ͷϞδϡʔϧʹద༻͢Δɽ Mapping(x) = tanh(xWm + bm )Wmm + bmm LayerNorm(Mapping( ⃗ It ) + Mapping( ⃗ T t ))

Implementation ɹ͞·͟·ͳλεΫ΍σʔληοτͷঢ়گԼͰ૊Έ߹ΘֶͤͨशΛߦ͏ɽ'-ʹ͓͚ Δͭͷঢ়گΛ.5-తͳղऍΛ͢Δɽ'-ʹ͓͚ΔΫϥΠΞϯτΛλεΫɼಛ௃ۭؒΛ σʔληοτͱΈͳͨ͠ɽ ɹධՁࢦඪ͸ɼ *NBHF$BQUJPOJOH41*$& $*%&S .&5&03 #-&6 72"UFTUTUBOEBSETFU
'-ʹ͓͚Δঢ়گ λεΫ σʔληοτ )'- ಉ͡ Image Captioning) ҟͳΔ MSCOCO & Flickr30k 7'- ҟͳΔ Image Captioning & VQA ಉ͡ MSCOCO & VQA v2.0 '5- ҟͳΔ Image Captioning & VQA ҟͳΔ Flickr30k & VQA v2.0 ˞72"Wͷೖྗը૾͸.4$0$0ͷσʔλͰߏ੒͞Ε͍ͯΔ

Implementation : Horizontal Federated Learning Experiment Setting Baseline(Image Captioning Decoder)
• Spatial ( Lu et al. 2017 ) • NBT ( Lu et al. 2017 )

Implementation : Vertical Federated Learning Experiment Setting Baseline(Image Captioning Decoder)
• Spatial ( Lu et al. 2017 ) • NBT ( Lu et al. 2017 ) Baseline(Visual Question Answering) • BUTB ( Anderson et al. 2018 ) • NBT ( Kim, Jun and Zhang 2018 )

Implementation : Federated Transfer Learning Experiment Setting Baseline(Image Captioning Decoder)
• Spatial ( Lu et al. 2017 ) • NBT ( Lu et al. 2017 ) Baseline(Visual Question Answering) • BUTB ( Anderson et al. 2018 ) • NBT ( Kim, Jun and Zhang 2018 )

Result : Horizontal Federated Learning

Result : Vertical Federated Learning

Result : Federated Transfer Learning

Result : Ablation study 'BTUFS3$// .BQQJOH.PEVMF 4QBUJBM#65% ೋͭͷλεΫΛ૊Έ߹Θֶͤͨश ɹ*NBHF$BQUJPOJOHʹ͸*OUFHSBUJOHɼ 72"ʹ͸"MJHOJOH͕ޮ͍͍ͯΔ͜ͱ͕Θ
͔Δɽ

·ͱΊ wEBUBMFBLBHFͷ໰୊Λ๊͍͑ͯͨWJTJPOBOEMBOHVBHFHSPVOEJOHQSPCMFNʹ͓ ͚Δ.5-ʹ'-ͷϑϨʔϜϫʔΫΛద༻͢Δ͜ͱͰɼ໰୊ͷղܾͱͭͷλεΫɼ· ͨ͸σʔληοτΑΓ΋ੑೳ޲্Λୡ੒ͨ͠ɽ w'-ͷ৚݅ )'- 7-' '5- ΛλεΫ΍σʔληοτʹରͯ͠ߟ͑Δ͜ͱʹΑΓɼॊ ೈʹఏҊख๏͸ରԠͰ͖Δɽ
wେ͖ͳσʔληοτΑΓ΋খ͞ͳσʔληοτ͸ɼΑΓޮՌతͩͱ͍͏݁Ռ͕ݱΕ ͨɽ ʲײ૝ʳ wͦ΋ͦ΋.5-ʹ͓͚ΔEBUBMFBLBHF͸WJTJPOBOEMBOHVBHFHSPVOEJOH QSPCMFNͷΑ͏ͳɼ΍΍ಛघͳλεΫʹͷΈଘࡏ͢ΔͷͰɼ͍͢͝൚༻తͳख๏Ͱ ͸ͳ͘ɼγϯϓϧͳઃఆͳΒλεΫΛ෼ׂ͠ͳ͍ํ͕͍͍ͩΖ͏ɽ w'-ͷΞϧΰϦζϜΛผͷλεΫʹԠ༻ΑΓ΋ɼ७ਮͳ'-ͷݚڀͷ΄͏͕ࣗ෼͸ڵ ຯ͕͋Δͱࢥͬͨɽ w'FEFSBUFE-FBSOJOH͕λΠτϧͷओޠ͸͓͔͍͠

[読み会資料] Federated Learning for Vision-and-Langu...

[読み会資料] Federated Learning for Vision-and-Language Grounding Problems

catla

More Decks by catla

Other Decks in Science

Featured

Transcript

Naoki Katsura Federated Learning for Vision-and-Language Grounding Problems 2020-03-10 http://web.pkusz.edu.cn/adsp/files/2019/11/AAAI-FenglinL.1027.pdf

࿦จ৘ใ ʲஶऀʳ Fenglin Liu(Penking Univ), Xian Wu(Tencent), Shen Ge(Tencent), Wei

༻ޠ 7JTJPOBOE-BOHVBHF(SPVOEJOH1SPCMFN ɹࣗવݴޠॲཧͱίϯϐϡʔλϏδϣϯʹ·͕ͨΔ໰୊Λಉ࣌ʹॲཧ͢Δ໰୊ɽ ྫ image captioning, visual question answer(VQA), image

image caption retrieval ग़యɿDual Attention Networks for Multimodal Reasoning and

എܠ ɹJNBHFDBQUJPOJOHͱ72"͸ɼͲͪΒ΋ը૾ͱࣗવݴޠΛѻ͏ͱ͍͏఺ͰλεΫ ͕ࣅ͍ͯΔɽ

എܠ ɹJNBHFDBQUJPOJOHͱ72"͸ɼͲͪΒ΋ը૾ͱࣗવݴޠΛѻ͏ͱ͍͏఺ͰλεΫ ͕ࣅ͍ͯΔɽ ɹλεΫΛ߹ΘͤΕ͹ɼҟͳΔλεΫ͔ΒҟͳΔ஌ࣝΛ֫ಘͰ͖ΔͷͰੑೳ޲্͕ݟ ࠐΊΔΜ͡Όͳ͍ʁ .VMUJUBTLMFBSOJOHΛద༻ɹ -JFUBM ͔͠͠ɾɾɾ

[Li et al. , 2018] ʲ໰୊఺ʳ ɹ͜ͷ৚݅ԼͰ͸ɼλεΫؒͰσʔληοτ͕ҟͳΔͱద༻Ͱ͖ͳ͍ɽʢ7'-ʣ ɹશλεΫʹ͓͍ͯೖྗΛ౷Ұ͠ͳ͍ͱ͍͚ͳ͍ɽ .5-༻ͷಛผͳσʔληοτΛ࡞Δඞཁ͕͋Δɽ

[Nguyen & Okatani , 2019] ɹλεΫؒͰσʔληοτΛ ౷Ұ͠ͳ͍ͱ͍͚ͳ͍໰୊Λ ؇࿨ͨ͠΋ͷɽ ɹ72"ͳͲʹ͸ೖྗʹςΩετ ͕ඞਢɽ

Visual and Textual Feature ʲΠϝʔδʳ ೖྗը૾ .VMUJQMF*OTUBODF -FBSOJOH 'BTUFS3$// WJTVBMGFBUVSFFYUSBDUPS

Aligning, Integrating and Mapping Network : aimNet ɹBJN/FU͸ɼҎԼͷͭͷϞδϡʔϧͰߏ੒͞ΕΔɽ͜Ε͸ɼ֤λεΫʹ͓͚Δը૾ ͷΤϯίʔμͷ෦෼ʹ૬౰͢Δɽ w"MJHOJOHNPEVMF

Aligning, Integrating and Mapping Network : aimNet "MJHOJOH.PEVMFͱ*OUFHSBUJOH.PEVMF͸ɼ࣍ͷ#BTJD.PEVMFΛϕʔεͱ͍ͯ͠ Δɽ ʲ#BTJD.PEVMFʳ

Aligning, Integrating and Mapping Network : aimNet w"MJHOJOHNPEVMF ɹWJTVBMGFBUVSFͱUFYUVBMGFBUVSFؒͷؔ܎ੑΛରԠ͚ͮΔɽ w*OUFHSBUJOHNPEVMF

Aligning, Integrating and Mapping Network : aimNet ʲ"MJHOJOH.PEVMFʳ ɹ΍͍ͬͯΔ͜ͱ͸ɼ ɹɹ4PVSDF5BSHFU"UUFOUJPOͱಉ͡ɽ

Aligning, Integrating and Mapping Network : aimNet w"MJHOJOHNPEVMF ɹWJTVBMGFBUVSFͱUFYUVBMGFBUVSFؒͷؔ܎ੑΛରԠ͚ͮΔɽ w*OUFHSBUJOHNPEVMF

Aligning, Integrating and Mapping Network : aimNet ʲ*OUFHSBUJOH.PEVMFʳ ɹ΍͍ͬͯΔ͜ͱ͸ɼ ɹɹ4FMG"UUFOUJPOͱಉ͡ɽ

Aligning, Integrating and Mapping Network : aimNet w"MJHOJOHNPEVMF ɹWJTVBMGFBUVSFͱUFYUVBMGFBUVSFؒͷؔ܎ੑΛରԠ͚ͮΔɽ w*OUFHSBUJOHNPEVMF

Aligning, Integrating and Mapping Network : aimNet ʲ.BQQJOH.PEVMFʳ ɹҎԼͷΑ͏ʹɼ.BQQJOHؔ਺Λ׆ੑԽؔ਺͕UBOIͷ૚ͷχϡʔϥϧωοτͱ͠ ͯఆٛ͢Δɽ

Implementation ɹ͞·͟·ͳλεΫ΍σʔληοτͷঢ়گԼͰ૊Έ߹ΘֶͤͨशΛߦ͏ɽ'-ʹ͓͚ Δͭͷঢ়گΛ.5-తͳղऍΛ͢Δɽ'-ʹ͓͚ΔΫϥΠΞϯτΛλεΫɼಛ௃ۭؒΛ σʔληοτͱΈͳͨ͠ɽ ɹධՁࢦඪ͸ɼ NBHF$BQUJPOJOH41$& $*%&S .&5&03 #-&6 72"UFTUTUBOEBSETFU

Implementation : Horizontal Federated Learning Experiment Setting Baseline(Image Captioning Decoder)

Implementation : Vertical Federated Learning Experiment Setting Baseline(Image Captioning Decoder)

Implementation : Federated Transfer Learning Experiment Setting Baseline(Image Captioning Decoder)

Result : Horizontal Federated Learning

Result : Vertical Federated Learning

Result : Federated Transfer Learning

Result : Ablation study 'BTUFS3$// .BQQJOH.PEVMF 4QBUJBM#65% ೋͭͷλεΫΛ૊Έ߹Θֶͤͨश ɹNBHF$BQUJPOJOHʹ͸OUFHSBUJOHɼ 72"ʹ͸"MJHOJOH͕ޮ͍͍ͯΔ͜ͱ͕Θ