Share
Embed
Transcript
71 slides
-
i ࣍ ୈ 1 ষ ং 1 ୈ 2
ষ ؔ࿈ݚڀ 3 2.1 ύʔηϓτϩϯ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 2.2 ଟχϡʔϥϧωοτϫʔΫ . . . . . . . . . . . . . . . . . . . . . . . 5 2.3 CNNʢConvolutional Neural NetworkɼΈࠐΈχϡʔϥϧωοτϫʔ Ϋʣ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 2.4 Attentionʢҙػߏʣ . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 2.5 GANʢGenerative Adversarial NetworkɼఢରతੜωοτϫʔΫʣ . 13 2.5.1 DCGANʢDeep Convolutional Generative Adversarial Networkɼ ΈࠐΈఢରతੜωοτϫʔΫʣ . . . . . . . . . . . . . . . 13 2.5.2 cGANʢConditional Generative Adversarial Networkɼ͖݅ ఢରతੜωοτϫʔΫʣ . . . . . . . . . . . . . . . . . . . . 15 2.6 Image to Image Translation . . . . . . . . . . . . . . . . . . . . . . . . . 16 2.6.1 pix2pix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 2.6.2 CycleGAN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 2.7 Attention-Guided Image-to-Image Translation . . . . . . . . . . . . . . . 20 2.7.1 Attention-Guided Generator Scheme I . . . . . . . . . . . . . . . 21 2.7.2 Attention-Guided Generator Scheme II . . . . . . . . . . . . . . 24 2.8 ճస֯༧ଌλεΫʢSelf-Supervised taskʣ . . . . . . . . . . . . . . 27 ୈ 3 ষ ఏҊख๏ 30 3.1 ωοτϫʔΫߏ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30 -
ii 3.2 ଛࣦؔ . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . 31 3.2.1 ఢରੑଛࣦ (Adversarial Loss) . . . . . . . . . . . . . . . . . . . 31 3.2.2 Cycle Consistency LossʢαΠΫϧҰ؏ੑଛࣦʣ . . . . . . . . 34 3.2.3 ࠷ऴతͳଛࣦؔ . . . . . . . . . . . . . . . . . . . . . . . . . 34 ୈ 4 ষ ࣮ݧ 35 4.1 ύϥϝʔλઃఆ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35 4.2 σʔληοτ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36 4.3 ධՁํ๏ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36 4.3.1 Fr´ echet Inception Distance(FID) . . . . . . . . . . . . . . . . . . 36 ୈ 5 ষ ࣮ݧ݁Ռͱߟ 38 5.1 ࣮ݧ݁Ռ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38 5.1.1 ఆྔతධՁ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38 5.1.2 ఆੑతධՁ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39 5.2 ߟ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41 5.2.1 ఆྔతධՁ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41 5.2.2 ఆੑతධՁ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41 5.2.3 ఆྔతධՁͱఆੑతධՁͷൺֱ . . . . . . . . . . . . . . . . . 42 ୈ 6 ষ ·ͱΊ 64 ँࣙ 66 ࢀߟจݙ 67 -
ୈ 1 ষ ং 1 ୈ1ষ ং ਓೳʢAIʣ 1950 ʹݚڀ͕࢝·͔ͬͯΒݱࡏʹࢸΔ·Ͱɼஶ͍͠ൃలΛ
͍͛ͯΔɽಛʹࡢࠓୈ 3 ࣍ AI ϒʔϜͷӔதʹ͋Γɼਂχϡʔϥϧωοτϫʔ ΫʢDNNʣΛத৺ͱͨ͠σΟʔϓϥʔχϯά͕ݚڀʹ༻͍ΒΕ͖ͯͨ [1]ɽσΟʔϓ ϥʔχϯάʹΑΓɼVAE GAN ͳͲͱ͍ͬͨੜϞσϧΛֶशͰ͖ΔΑ͏ʹͳͬ ͍ͯΔɽ ੜϞσϧͷҰͭͰ͋Δ GAN(Generative Adversarial Network)ɼ2014ʹ Ian J.Goodfellow ࢯʹΑͬͯ։ൃ͞ΕͨੜϞσϧͰ͋ΓɼࠓʹࢸΔ·Ͱʹ༷ʑͳԠ༻ ͕ͳ͞Ε͍ͯΔ [2] [3] [4] [5] [6] [7] [8] [9] [10] [11]ɽGAN ͷԠ༻ͷҰͭʹ CycleGAN ͕͋Γɼ ը૾தͷഅΛγϚϚʹɼGoogle Map ͷΑ͏ͳਤΛӴࣸਅʹɼࣸਅΛֆը෩ʹɼ ͱ͍ͬͨը૾ม͕Ͱ͖Δ [7]ɽCycleGAN Λ༻͍ͯͷը૾มɼ2 ͭͷσʔληο τΛ༻͍ͨڭࢣͳֶ͠शͰ͋Γɼڭࢣ͋ΓֶशΛ༻͍ͨը૾มख๏ͷ pix2pix [6] ͷ Α͏ʹσʔλෆʹؕΔ͜ͱ͕ͳ͍ͱ͍͏ͷ͕རͰ͋Δɽ ͞ΒʹCycleGANͷൃలܗͱͯ͠ɼ GeneratorʹAttentionΛ༻͍ͨAttention-Guided GAN ͕։ൃ͞Εͨ [8] [9]ɽAttention-Guided GAN Ͱɼ͍ͨ͠ը૾ͷҰ෦Λ நग़ͯ͠ Attention ͱ͠ɼAttention ͷ෦ͷը૾มΛߦ͏͜ͱ͕Ͱ͖Δ [8] [9]ɽ͠ ͔͠ը૾ͷҰ෦Λநग़͢Δࡍʹɼ͍ͨ͠෦ͱؔͷͳ͍෦Λޡม͠ ͯ͠·͏ͱ͍ͬͨ՝͕ੜͯ͡͠·͏ɽ ͜͏ͨ͠എܠ͔ΒຊݚڀͰɼը૾ͷҰ෦Λਖ਼֬ʹநग़ͯ͠ࢀরը૾ͷΑ͏ʹ Ϛοϐϯά͢Δ͜ͱΛతͱ͠ɼAttention-Guided GAN ͷ Discriminator ʹճస֯ ༧ଌλεΫʢSelf-Supervised TaskʣΛՃͨ͠ SSAttention-Guided GAN ΛఏҊ͢Δɽ Discriminator ʹճస֯༧ଌλεΫΛՃ͢Δ͜ͱʹΑͬͯɼճసෆมੑʹΑΔը ૾ͷزԿతಛΛѲͰ͖ɼࣝผਫ਼ͷ্Λࢦ͢͜ͱ͕ՄೳͱͳΔɽͦͯ͠ɼ -
ୈ 1 ষ ং 2 GAN ͷ Discriminator ͱ Generator
Λڝ͍߹ΘͤΔੑ࣭͔ΒɼDiscriminator ͷࣝผਫ਼ ͷ্͕ɼGenerator ͷֶशʹӨڹΛ༩͑ɼഎܠʹӨڹΛ༩͑ͣʹ͍ͨ͠ ෦ʹͷΈը૾ม͞Εͨը૾͕ੜͰ͖Δͱ͍͏ԾઆΛཱͯɼຊ࣮ݧΛߦͬͨɽ ຊจͰ 2 ষʹ GAN ٴͼը૾มٕज़ͷؔ࿈ݚڀʹ͍ͭͯड़Δɽ3 ষͰ Attention-Guided GAN ʹճస֯༧ଌλεΫΛՃͨ͠ SSAttention-Guided GAN ΛఏҊ͢Δɽ4 ষͰ࣮ݧʹ༻͍ΔύϥϝʔλͷઃఆɼσʔληοτɼධՁํ๏Λ հ͢Δɽͦͯ͠ 5 ষͰ࣮ݧ݁Ռͱߟʹ͍ͭͯهड़͢Δɽ࠷ޙʹ 6 ষͰ·ͱΊͱ ࠓޙͷ՝ʹ͍ͭͯड़Δɽ -
ୈ 2 ষ ؔ࿈ݚڀ 3 ୈ2ষ ؔ࿈ݚڀ ॳΊʹɼ2.1 અͰύʔηϓτϩϯɼ2.2 અͰχϡʔϥϧωοτϫʔΫͷجૅతͳ
ࣝΛड़ΔɽͦͷޙɼຊݚڀͷςʔϚͱͳ͍ͬͯΔ GAN ʹ͍ͭͯͷؔ࿈ݚڀΛड़ Δɽ 2.1 ύʔηϓτϩϯ ύʔηϓτϩϯͱɼ1957 ʹ Frank Rosenblatt ࢯʹΑͬͯߟҊ͞ΕͨΞϧΰϦ ζϜͰ͋Γɼͷਆܦճ࿏ͷҰ෦ΛࣜͰදݱͨ͠ͷͰ͋Δ [12]ɽύʔηϓτϩϯ ෳͷ৴߸Λೖྗͱͯ͠ड͚औΓɼҰͭͷ৴߸Λग़ྗ͢ΔߏͰ͋Δɽྫͱͯ͠ɼ ̎ͭͷ৴߸ x1 , x2 Λೖྗͱͯ͠ड͚औΔύʔηϓτϩϯͷߏΛਤ 2.1 ʹࣔ͢ɽ ਤ 2.1 ͷʓχϡʔϩϯ͋Δ͍ϊʔυͱݺΕΔɽx1 , x2 ೖྗ৴߸ɼy ग़ྗ ৴߸ɼw1 , w2 ॏΈΛද͢ͱ͢ΔɽᮢΛ θ ͱͯ͠ɼग़ྗ y ࣜ (2.1) ͷΑ͏ʹܭࢉ ͢Δ [12]ɽ y = 1 ɹ (w1 x1 + w2 x2 > θ) 0 ɹ (w1 x1 + w2 x2 ≤ θ) (2.1) ύʔηϓτϩϯͰɼᮢ θ ͷΘΓʹόΠΞε b = −θ Λಋೖ͢Δ͜ͱ͕͋Δɽό ΠΞεΛ 1 ͭͷχϡʔϩϯͱͯ͠ɼਤ 2.2 ͷΑ͏ͳߏͱ͢Δ߹͕͋Δɽ ύʔηϓτϩϯʹόΠΞεΛಋೖͨ͠ग़ྗ y ͷܭࢉΛࣜ (2.2) ʹࣔ͢ [12]. y = 1 ɹ (w1 x1 + w2 x2 + b > 0) 0 ɹ (w1 x1 + w2 x2 + b ≤ 0) (2.2) ࣜ (2.2) Ͱɼೖྗ x1 , x2 ͱॏΈ w1 , w2 ͷੵ w1 x1 + w2 x2 ʹόΠΞε߲ b ΛՃ͑ͨ ͕ࣜਖ਼ͷͳΒ 1ɼͦΕҎ֎ͷͳΒ 0 ΛͱΔɽग़ྗ y ɼ׆ੑԽؔ h(·) Λ ༻͍ͯܭࢉ͢Δ͜ͱͰ͖ɼࣜ (2.3) ʹࣔ͢ [12]ɽࣜ (2.3) Λࣜ (2.2) ͱಉ͡ग़ྗͱ͢ -
ୈ 2 ষ ؔ࿈ݚڀ 4 ਤ 2.1 ̎ͭͷ৴߸Λೖྗͱͯ͠ड͚औΔύʔηϓτϩϯͷྫɹ ਤ 2.2
όΠΞεΛಋೖͨ͠ύʔηϓτϩϯɹ -
ୈ 2 ষ ؔ࿈ݚڀ 5 Δ߹ɼࣜ (2.4) ʹࣔ͢εςοϓؔΛ༻͍Δ [12]ɽ y
= h(w1 x1 + w2 x2 + b) (2.3) h(u) = 1 ɹ (u > 0) 0 ɹ (u ≤ 0) (2.4) 2.2 ଟχϡʔϥϧωοτϫʔΫ ଟχϡʔϥϧωοτϫʔΫɼͷਆܦճ࿏ͷҰ෦Λ฿ͨ͠ཧϞσϧͰ͋ Γɼύʔηϓτϩϯಉ࢜Λͭͳ͗ɼͦΕΒΛଟʹͨ͠ϞσϧͰ͋Δ [12]ɽଟχϡʔ ϥϧωοτϫʔΫͷྫΛਤ 2.3 ʹࣔ͢ [13]ɽਤ 2.3 ͷࠨଆͷྻΛೖྗɼதؒͷྻΛ தؒʢӅΕʣɼӈଆͷྻΛग़ྗͱݺͿɽਤ 2.3 ʹ͓͍ͯɼN (N > 0) ݸͷೖ ྗΛද͢ม x1 , ..., xi , ..., xN ͱόΠΞε x0 ɼM (M > 0) ݸͷग़ྗΛද͢ม y1 , ..., yj , ..., yM ɼ L (L > 0) ݸͷӅΕม z1 , ..., zk , ..., zL ͱόΠΞε z0 Λࣔ͢ɽ ਤ 2.3 χϡʔϥϧωοτϫʔΫͷྫɹ -
ୈ 2 ষ ؔ࿈ݚڀ 6 χϡʔϥϧωοτϫʔΫʹ͓͚Δܭࢉʹ͍ͭͯઆ໌͢Δɽ·ͣೖྗʹͯೖྗ x1 , ..., xN
ͱόΠΞε x0 Λ༻͍֤ͯରԠ͢ΔॏΈ w(1) ki ͱͷੵΛܭࢉ͠ɼ׆ੑԽؔ h ʹೖ͕ͨ͠ӅΕؔͷͱͳΔɽ׆ੑԽؔ h ʹɺࣜ (2.5) ͰͷγάϞΠυ ؔࣜ (2.6) ͷ ReLU(Rectitled Linear Unit) ͕͋Δ [12]ɽ h(uk ) = 1 1 + exp(−uk ) (2.5) h(uk ) = max(uk , 0) (2.6) ೖྗ͔ΒதؒͷܭࢉࣜΛࣜ (2.7) ʹͯࣔ͢ɽ͜͜ͰόΠΞε x0 x0 = 1 ͱ͠ɼ ରԠ͢ΔॏΈ w(1) k0 ͱ͢Δɽ zk = h( N ∑ i=1 w(1) ki xi + w(1) k0 ) (2.7) ࣜ (2.3) ͱಉ༷ʹͯ͠ܭࢉ͠ɼग़ྗ yi Λܭࢉ͢Δɽத͔ؒΒग़ྗࢸΔͷʹ༻ ͍ΒΕΔ׆ੑԽؔ σ ɼղ͘ʹΑͬͯࣜ (2.8) ͷ߃ؔࣜ (2.9) ͷιϑτ ϚοΫε͕ؔ༻͍ΒΕΔ [12]ɽ σ(vj ) = vj (2.8) σ(vj ) = exp(vj ) ∑ M m=1 exp(vm ) (2.9) த͔ؒΒग़ྗͷܭࢉࣜΛࣜ (2.10) ʹͯࣔ͢ɽ͜͜ͰόΠΞε z0 z0 = 1 ͱ͠ɼ ରԠ͢ΔॏΈΛ w(2) j0 ͱ͢Δɽ yj = σ( L ∑ k=1 w(2) jk zk + w(2) j0 ) (2.10) ਤ 2.3 ͰͷχϡʔϥϧωοτϫʔΫͷܭࢉࣜΛࣜ (2.11) ʹͯࣔ͢ɽࠓճͷਤ 2.3 ٴͼ ࣜ (2.11) ͷΑ͏ʹೖྗ → தؒ → ग़ྗͷॱ൪Ͱ࣮ࢪ͞ΕΔܭࢉࣜɼॱൖ ʢforward propagationʣͱදݱ͢Δ [12]ɽ yj = σ( L ∑ k=1 ω(2) jk h( N ∑ i=1 w(1) ki xi + w(1) k0 ) + w(2) j0 ) (2.11) χϡʔϥϧωοτϫʔΫʹ͓͚Δσʔλɼओʹֶशσʔλͱςετσʔλͷ̎ͭ ͕͋Δɽֶशσʔλɼ࠷ॳʹͦΕࣗମͷΈͰχϡʔϥϧωοτϫʔΫΛֶश͠ɼ -
ୈ 2 ষ ؔ࿈ݚڀ 7 ॏΈͷௐΛߦ͏ͨΊʹ༻͍ΒΕΔɽςετσʔλɼχϡʔϥϧωοτϫʔΫͷ ग़ྗΛධՁ͢ΔͨΊʹ༻͍ΒΕΔɽςετσʔλͷධՁɼҰൠతʹςετσʔ λ tn (n
= 1, ..., N) ͱɼֶशσʔλΛೖྗͱͨ͠χϡʔϥϧωοτϫʔΫͷग़ྗ yn (n = 1, ..., N) ͱͷޡࠩͷେ͖͞Λࢦඪͱ͓ͯ͠ΓɼͦΕΒͷࢦඪΛଛࣦͱݺͿ [12]ɽ ଛࣦؔͷྫͱͯࣜ͠ (2.12) ͷೋޡࠩɼࣜ (2.13) ͷΫϩεΤϯτϩϐʔ͕༻ ͍ΒΕΔ [12]ɽͳ͓ɼࣜ (2.12) ࣜ (2.13) ʹ͓͍ͯɼଛࣦؔΛ E ͱ͢Δɽ E = 1 N N ∑ n=1 (yn − tn )2 (2.12) E = − 1 N N ∑ n=1 tn log yn (2.13) χϡʔϥϧωοτϫʔΫͷֶशͰɼଛࣦ͕ؔ࠷খʹۙͮ͘Α͏ʹॏΈΛߋ৽͠ ֶश͢Δ [12]ɽॏΈͷௐʹޯ๏͕༻͍ΒΕΔ [12]ɽ 2.3 CNN ʢConvolutional Neural Networkɼ ΈࠐΈχϡʔ ϥϧωοτϫʔΫʣ CNN ͱɼχϡʔϥϧωοτϫʔΫͷߏʹΈࠐΈΛಋೖͨ͠ωοτϫʔΫ Ͱɼओʹը૾ೝࣝͰ༻͍ΒΕ͍ͯΔ [12]ɽ2.2 અͰͷଟχϡʔϥϧωοτϫʔΫɼ શ݁߹ͱ׆ੑԽؔͰߏ͞Ε͍ͯΔɽ͜Εʹର͠ CNN ΈࠐΈͱϓʔϦ ϯάΛ༻͍ͯߏ͞ΕΔɽΈࠐΈͰɼྫ͑ը૾ʹ͓͚ΔϐΫηϧಉ࢜ ͷڞ௨ٴͼ૬ҧੑ RGB νϟωϧͷؔ࿈ੑͱ͍ͬͨɼۭؒతใʹج͍ͮͨσʔ λͷಛΛଊ͑Δ͜ͱ͕ՄೳͱͳΔ [12]ɽਤ 2.4 ʹ CNN ͷߏஙྫΛࣔ͢ɽ ΈࠐΈͰೖྗσʔλʹରͯ͠ɼΧʔωϧʢϑΟϧλʣͱݺΕΔॏΈͱͷ ΈࠐΈԋࢉΛߦ͏ɽਤ 2.5 ͷΈࠐΈԋࢉͷΠϝʔδྫͰɼ4x4 ͷೖྗσʔλͱ 3x3 ͷΧʔωϧͱͷΈࠐΈԋࢉʹΑͬͯɼ2x2 ͷσʔλΛग़ྗ͢Δɽ ͜͜Ͱਤ 2.5 ͷΈࠐΈԋࢉʹ͍ͭͯઆ໌͢Δɽ·ͣਤ 2.6 ͷΑ͏ʹ 4x4 ͷೖྗ σʔλͷࠨ্ͷϐΫηϧྖҬ͔Β 3x3 ͷྖҬΛऔΓɼ3x3 ͷྖҬͷ֤ཁૉͱରԠ͠ -
ୈ 2 ষ ؔ࿈ݚڀ 8 ਤ 2.4 CNNʢΈࠐΈχϡʔϥϧωοτϫʔΫʣͷߏྫɹ ਤ 2.5
ΈࠐΈʹ͓͚ΔΈࠐΈԋࢉͷΠϝʔδྫɹ -
ୈ 2 ষ ؔ࿈ݚڀ 9 ͨϑΟϧλͷཁૉͱͷੵΛٻΊɼΛࢉग़͢Δɽ࣍ʹɼਤ 2.7 ͷΑ͏ʹೖྗσʔλ ͷ 3x3
ྖҬΛ 1 ϐΫηϧӈʹͣΒ͠ɼಉ༷ʹϑΟϧλͱͷੵԋࢉΛߦ͏ɽଓ͍ͯɼ ਤ 2.8 ͷΑ͏ʹೖྗσʔλͷ 3x3 ྖҬΛ 1 ϐΫηϧԼʹͣΒ͠ɼಉ༷ʹϑΟϧλͱ ͷੵԋࢉΛߦ͏ɽ࠷ޙʹɼਤ 2.9 ͷΑ͏ʹೖྗσʔλͷ 3x3 ྖҬΛ 1 ϐΫηϧӈ ʹͣΒ͠ɼಉ༷ʹϑΟϧλͱͷੵԋࢉΛߦ͏ɽ ҰํϓʔϦϯάͰσʔλʹ͓͚ΔॎԣํͷۭؒΛখ͘͢͞ΔԋࢉΛߦ͏ɽ ਤ 2.10 ͷϓʔϦϯάԋࢉͷྫͰɼ4x4 ͷೖྗσʔλΛ̐ͭͷ 2x2 ྖҬʹׂ͠ɼ ֤ྖҬͷநग़͞Εͨ࠷େΛ 2x2 ͷσʔλʹू͍ͯ͠ΔɽϓʔϦϯάʹ͓͚Δ ԋࢉํ๏ͱͯ͠ओʹ࠷େϓʔϦϯάฏۉϓʔϦϯά͕༻͍ΒΕΔ [12]ɽ ϓʔϦϯάͰɼֶश͢ΔύϥϝʔλΛ༻͍ͣɼೖྗσʔλʹ͓͚ΔඍখͳҐ ஔมԽͷӨڹΛड͚ͳ͍͜ͱ͕ଟ͍ͱ͍͏ಛΛ࣋ͭ [12]ɽ 2.4 Attentionʢҙػߏʣ Attention ͱɼը૾จষͷಛఆͷ෦ʹ͠ɼಛΛଊ͑ΔΑ͏ʹֶश͢Δ ωοτϫʔΫͰ͋Δ [14]ɽྫ͑ਓ͕ؒը૾ΛݟΔͱ͖ɼਓؒը૾தʹ͓͚Δશͯ ͷ෦Λಉ͡Α͏ʹݟΔͷͰͳ͘ɼը૾தʹ͋ΔҰ෦ʹͦ͠ΕΛΦϒδΣΫ τͱͯ͠ೝ͍ࣝͯ͠Δɽ͜ͷΑ͏ͳਓؒͷಛੑΛػցֶशʹԠ༻ͨ͠ͷ͕ Attnetion Ͱ͋Δɽ Attention ͷΈΛਤ 2.11 ʹࣔ͢ɽਤ 2.11 Ͱɼ͍ͨ͠ಛͰ͋Δ Query ͱɼը૾ͷݩσʔλ͔ΒͦΕͧΕ KeyɼValue ͱݺΕΔಛϕΫτϧΛऔΓग़͢ɽ Query ͱ Key ͷཁૉͷ֤ੵʢྨࣅʣΛ Value ͷॏΈͱͯ͠ɼValue ͷ͍ͨ͠෦ Λڧௐ͢Δ [14]ɽ -
ୈ 2 ষ ؔ࿈ݚڀ 10 ਤ 2.6 ೖྗσʔλͷࠨ্ͷ 3x3 ྖҬͱϑΟϧλͷΈࠐΈԋࢉɹ
ਤ 2.7 ӈʹ 1 ϐΫηϧಈ͔ͨ͠ೖྗσʔλͷ 3x3 ྖҬͱϑΟϧλ ͷΈࠐΈԋࢉɹ -
ୈ 2 ষ ؔ࿈ݚڀ 11 ਤ 2.8 Լʹ 1 ϐΫηϧಈ͔ͨ͠ೖྗσʔλͷ
3x3 ྖҬͱϑΟϧλ ͷΈࠐΈԋࢉɹ ਤ 2.9 ӈͱԼʹ 1 ϐΫηϧಈ͔ͨ͠ೖྗσʔλͷ 3x3 ྖҬͱϑΟ ϧλͷΈࠐΈԋࢉɹ -
ୈ 2 ষ ؔ࿈ݚڀ 12 ਤ 2.10 ࠷େϓʔϦϯάͷྫ ਤ 2.11
Attention ͷߏɹ -
ୈ 2 ষ ؔ࿈ݚڀ 13 2.5 GANʢGenerative Adversarial Networkɼఢରతੜ ωοτϫʔΫʣ
GAN ɼ2014 ʹ Ian J. Goodfellow ࢯΒʹΑͬͯߟҊ͞ΕͨੜϞσϧʹ͓͚ ΔΞʔΩςΫνϟͰ͋Δ [3]ɽGAN ͷߏΛਤ 2.12 ʹࣔ͢ɽGAN ɼGeneratorʢੜ ثʣͱ Discriminatorʢࣝผثʣͷ 2 ͭͷωοτϫʔΫͰߏ͞ΕɼGenerator ͱ Discriminator Λఢରతʹֶशͤ͞ΔɽGenerator ϊΠζΛೖྗ͞ΕɼੜσʔλΛ ग़ྗ͢ΔɽDiscriminator ɼೖྗ͞Εͨσʔλֶ͕शઌͷσʔλͰ͋Δ͔Ͳ͏͔Λ ࣝผ͢ΔɽGAN Generator ͱ Discriminator Λఢରతʹֶश͢ΔͨΊɼGAN ͷଛ ࣦؔ V(D,G) ࣜ (2.14) ͰදͤΔ [3]ɽ min G max D V(D,G) = Ex [log(D(x))] + Ez [log(1 − D(G(z)))] (2.14) ࣜ (2.14) ʹ͓͍ͯɼೖྗσʔλ͕ຊͰ͋Δ֬Λ D(x)ɼGenerator Ͱੜ͞Εͨ σʔλΛ G(z) ͱ͢Δɽೖྗσʔλ͕ຊͰ͋Δͱஅ͞ΕΔ߹ɼD େ͖͍ ͱͳΔɽҰํͰɼೖྗσʔλِ͕ͱஅ͞ΕΔ߹ɼD খ͍͞ͱͳΔɽͨ͠ ͕ͬͯɼࣜ 2.14 ʹ͓͍ͯ log(D(x)) ͷͱ log(1 − D(G(z))) ͷ͕େ͖͘ͳΔΑ͏ʹ Discriminator Λֶश͢ΔɽҰํɼGenerator ຊσʔλʹ͍ۙੜσʔλ G(z) Λ ੜ͢ΔͨΊʹɼlog(1 − D(G(z))) ͷ͕খ͘͞ͳΔΑ͏ʹֶश͢Δɽ͜ΕΒͷֶश Λఢରతʹ܁Γฦ͢͜ͱʹΑͬͯ GAN Λֶश͢Δ [3]ɽ 2.5.1 DCGAN ʢDeep Convolutional Generative Adversarial Networkɼ ΈࠐΈఢରతੜωοτϫʔΫʣ DCGAN ͰɼGAN Ͱ༻͍ΒΕΔ̎ͭͷωοτϫʔΫ GeneratorɼDiscriminator ʹ ͦΕͧΕΈࠐΈΛ༻͍͍ͯΔ [4]ɽDCGAN Ͱ 2.3 અͷ CNN ͷߏͱҧ͍ɼ ΈࠐΈͱ׆ੑԽؔͷΈͷωοτϫʔΫߏͱͳΔɽ ਤ 2.13 ʹωοτϫʔΫͷྫΛࣔ͢ɽਤ 2.13 ͷ DCGAN ʹ͓͚Δ Generator ωοτ ϫʔΫͷߏྫͰɼϊΠζϕΫτϧ z Λ Generator ʹೖྗͯ͠ঃʑʹνϟωϧ -
ୈ 2 ষ ؔ࿈ݚڀ 14 ਤ 2.12 GAN ͷߏ ਤ
2.13 DCGAN Ͱͷ Generator ͷωοτϫʔΫߏྫ -
ୈ 2 ষ ؔ࿈ݚڀ 15 ਤ 2.14 cGAN ͷߏɹ ͷݮͱಉ࣌ʹΈࠐΈΛ༻͍ͯϐΫηϧΛΞοϓαϯϓϦϯά͠ɼࢦఆͨ͠େ͖
͞ͷੜը૾ G(z) Λग़ྗ͢Δɽਤ 2.13 ͷྫͰ 100 ࣍ݩͷϊΠζϕΫτϧΛೖྗ ͠ɼ4x4x1024ɼ8x8x512, 16x16x256, 32x32x128 ͱ 4 ճͷΞοϓαϯϓϦϯάΛհ͠ ͯɼ࠷ऴతʹνϟωϧ 3 ͷ 64x64 ϐΫηϧͷΧϥʔը૾Λग़ྗ͍ͯ͠Δ [4]ɽ 2.5.2 cGANʢConditional Generative Adversarial Networkɼ݅ ͖ఢରతੜωοτϫʔΫʣ cGAN ͱɼGAN ͷ Generator ͱ Discriminator ʹೖྗ͢ΔσʔλʹɼΫϥεʹର Ԡ͢ΔϥϕϧΛ͚Ճ͑ͨϞσϧͰ͋Δ [5]ɽcGANͷߏਤ2.14ͷ௨ΓͱͳΔɽਤ 2.14 Ͱɼਤ 2.12 Ͱࣔ͢ GAN ͷΞʔΩςΫνϟͷ͏ͪɼֶशσʔλ x ͱੜσʔλ G(z|y)ɼજࡏม z ʹͦΕͧΕϥϕϧ y ΛՃ͍ͯ͠ΔɽGenerator Ͱɼજࡏม z ͱͦΕʹରԠ͢Δϥϕϧ y Λೖྗ͠ɼϥϕϧ͚͞Εͨੜσʔλ G(z|y) Λग़ྗ ͢Δɽֶͦͯ͠शσʔλ x ͱϥϕϧ y ʹج͖ͮɼੜσʔλ G(z|y) Λ Discriminator ʹΑͬͯࣝผ͢Δɽ -
ୈ 2 ষ ؔ࿈ݚڀ 16 cGAN ͷଛࣦؔɼҎԼͷࣜ (2.15) ͰදͤΔ [5]ɽ
min G max D V(D,G) = Ex [log(D(x|y))] + Ez [log(1 − D(G(z|y)))] (2.15) GAN ͱҾ͖ଓ͖ɼࣜ (2.15) ʹ͓͍ͯ Discriminator ͷग़ྗΛ D(·)ɼGenerator ͷग़ྗ Λ G(·)ɼଛࣦؔΛ V(D,C) ͱ͓͘ɽࣜ (2.15) ͰɼରԠ͢ΔϥϕϧΛ y ͱ͓͘ɽͦ ͯ͠ɼGAN ͱಉ༷ʹ log(D(x|y)) ͷͱ log(1 − D(G(z|y))) ͷ͕େ͖͘ͳΔΑ͏ʹ Discriminator Λֶश͠ɼlog(1 − D(G(z|y))) ͷ͕খ͘͞ͳΔΑ͏ʹ Generator Λֶ श͢Δ [5]ɽ 2.6 Image to Image Translation 2.5 અͰͷ GAN ɼGenerator Λ௨ͯ͡ϊΠζ͔Βը૾Λੜ͢Δը૾ੜΛߦ ͏ϞσϧͰ͋ΔɽຊઅͰ GAN ͷը૾ੜٕज़ΛԠ༻͠ɼιʔεͱͳΔը૾͔Β λʔήοτͱͳΔը૾ͷมΛߦ͏ Image to Image Translation ͱݺΕΔը૾ม ٕज़Λઆ໌͢Δɽ2.6.1 ߲Ͱ pix2pixɼ2.6.2 ߲Ͱ CycleGAN ʹ͍ͭͯͦΕͧΕઆ໌ ͢Δɽ 2.6.1 pix2pix pix2pix ͱɼPhillip Isola ࢯΒʹΑͬͯ։ൃ͞Εͨ Image to Image Translation ख๏ Ͱ͋Δ [6]ɽpix2pix Ͱɼมઌը૾͔ΒΤοδநग़ͨ͠ը૾Λมݩը૾ͱ͢Δɽ pix2pix 2.5.2 ߲ͷ cGAN ΛԠ༻ͨ͠ϞσϧͰ͋ΓɼΤοδநग़ͨ͠มݩը૾Λ ϥϕϧͱͯ͠ը૾มΛߦ͏ɽਤ 2.15 ʹ pix2pix ͷߏΛࣔ͢ɽpix2pix Ͱ 2.5.2 ߲ ͷ cGAN Λ༻͍͓ͯΓɼਤ 2.15 ͷΤοδ͔Βը૾ͷྫͰɼϥϕϧʹΤοδͷը ૾ɼຊը૾ʹมઌͷը૾Λద༻͍ͯ͠Δɽޙ cGAN ͱಉ͘͡ɼDiscriminator ͱ Generator ͱͷఢରతֶशʹΑͬͯɼมઌͷը૾ͷΑ͏ͳը૾Λ Generator Ͱੜ ͢Δɽ ϥϕϧը૾ʹɼมݩը૾ͷΤοδ෦͚ͩͰͳ͘ɼGoogle Map ͷਤ Se- mantic Segmentation [15] Λ༻͍ͨϥϕϧը૾Λ༻͍Δ͜ͱͰ͖ΔɽGoogle Map ͷ -
ୈ 2 ষ ؔ࿈ݚڀ 17 ਤ͔ΒߤۭࣸਅΛੜͨ͠Γ Semantic Segmentation Λ༻͍ͨϥϕϧը૾͔Βࣸ ਅΛੜͨ͠Γ͢Δ͜ͱՄೳͰ͋Δɽͳ͓ɼSemantic
Segmentation ͱɼ֤ϐΫ ηϧΛपลͷϐΫηϧใʹج͖ͮɼΧςΰϦྨ͢Δํ๏Λࢦ͢ [15]ɽྫͱͯ͠ɼ Image-to-Image Demo [6] Ͱ pix2pix ʹΑΔը૾มͷ࣮ߦ݁ՌΛਤ 2.16 ʹࣔ͢ɽਤ 2.16 ͷྫͰɼۺɼϋϯυόοάͷը૾͔ΒͦΕͧΕΤοδΛऔͬͨϥϕϧը૾ ݐ֎؍ͷϥϕϧը૾ΛجʹɼςΫενϟϚοϐϯάͰࣸਅͷΑ͏ͳը૾Λੜ͠ ͍ͯΔɽ 2.6.2 CycleGAN CycleGAN ɼGAN Λ̎ͭܨ͛Δ͜ͱͰɼ͋Δσʔλ܈ͷը૾Λผͷσʔλ܈ͷ ը૾ͷΑ͏ʹ૬ޓม͢Δ Image to Image Translation ख๏Ͱ͋Δ [7]ɽਤ 2.17 ʹͯɼ pix2pix Ͱ༻͍Δมݩը૾ͱมઌը૾ͷϖΞͷྫͱ CycleGAN Ͱ༻͍Δมݩ ը૾܈ͱมઌը૾܈ͷྫΛࣔ͢ɽલऀɼۺόοάͷը૾ͱ͍ͬͨมઌը૾ ਤ 2.15 pix2pix ͷߏ -
ୈ 2 ষ ؔ࿈ݚڀ 18 ਤ 2.16 pix2pixͰͷImage to Image
Translationͷྫ ʢImage-to-Im- age Demo Ͱ࡞ [6]ʣɹ ͱ֤มઌը૾ͷΤοδը૾͕ϖΞͱͳͬͨڭࢣ͋Γֶशͱͳ͍ͬͯΔɽ͜ͷΑ͏ ʹ pix2pix ͰɼมݩͱͳΔϥϕϧͱมઌը૾ͷ̍ର̍ͷϖΞ͕ඞཁͱͳΔͨ Ίɼσʔληοτͷऩू͕͘͠ɼσʔλͷྔ͕ലେͱͳΔͱ͍ͬͨ՝͕͋Δɽ ҰํޙऀͰɼ֤มݩը૾͝ͱʹରԠͨ͠ϥϕϧΛ༻͍Δ͜ͱͳ͘ɼมݩը૾ ܈ͱมઌը૾܈ΛϖΞͱͨ͠ڭࢣͳֶ͠शͱͳ͍ͬͯΔɽCycleGANͰɼpix2pix ͱൺͯϖΞͱͳΔֶशσʔλΛඞཁͱͤͣʹը૾ม͕Ͱ͖ΔΑ͏ʹվྑ͞Εͯ ͍Δ [7]ɽ CycleGAN ͷଛࣦؔओʹɼఢରੑଛࣦʢAdversarial lossʣ ɼαΠΫϧҰ؏ੑଛ ࣦʢCycle consistency lossʣ͔Βߏ͞ΕΔ [7]ɽఢରੑଛࣦͱɼมઌͱͳΔσʔ λ܈ͷʹੜը૾ͷΛҰகͤ͞Δ͜ͱΛతͱͨ͠ଛࣦͰ͋ΓɼαΠΫϧ -
ୈ 2 ষ ؔ࿈ݚڀ 19 Ұ؏ੑଛࣦɼ̎ͭͷ Generator ಉ࢜ʹໃ६͕ੜ͡ͳ͍Α͏ʹ͢Δ͜ͱΛతͱͨ͠ ଛࣦͰ͋Δɽఢରੑଛࣦ LGAN
Λࣜ (2.16)ɼࣜ (2.17)ɼαΠΫϧҰ؏ੑଛࣦ Lcycle Λࣜ (2.18) ʹͯࣔ͢ɽ͜͜Ͱσʔλ܈Λ X, Y ͱͯ͠ɼG Λ X → Y ͷมΛߦ͏ Generatorɼ F Λ Y → X ͷมΛߦ͏ Generator ͱ͠ɼX, Y ʹରԠ͢Δ Discriminator ΛͦΕͧΕ DX , DY ͱ͢ΔɽDX ࣝผରͱͳΔσʔλ͕ը૾σʔλ܈ X தͷը૾Ͱ͋Δ͔Ͳ ͏͔ɼDY ࣝผରͱͳΔσʔλ͕ը૾σʔλ܈ Y தͷը૾Ͱ͋Δ͔Ͳ͏͔Λࣝ ผ͢Δɽ LGAN (G, DY , X, Y) = Ey [log(DY (y))] +Ex [log(1 − DY (G(x)))] (2.16) LGAN (F, DX , Y, X) = Ex [log(DX (x))] + Ey [log(1 − DX (F(y)))] (2.17) Lcycle (G, F) = Ex [∥F(G(x)) − x∥1 ] + Ey [∥G(F(y)) − y∥1 ] (2.18) ͜ΕΒͷଛࣦΛ௨ৗͷ GAN ͱಉ༷ʹఢରతʹֶशΛ܁Γฦ͢ɽαΠΫϧҰ؏ੑଛ ࣦ Lcycle ͷࣜ (2.18) ɼը૾ x ∈ X ͱɼx Λσʔλ܈ Y ʹมͨ͠ G(x) ΛͰ࠶ม ਤ 2.17 pix2pix Ͱ༻͍Δσʔλͱ CycleGAN Ͱ༻͍Δσʔλͷ ൺֱɹ -
ୈ 2 ষ ؔ࿈ݚڀ 20 ͨ͠ F(G(x)) ͱͷࠩΛܭࢉ͓ͯ͠Γɼը૾ y ∈
Y ʹରͯ͠ɼಉ༷ͷܭࢉΛͨ͠ ଛࣦͰ͋Δɽ͕ͨͬͯ͠ɼαΠΫϧҰ؏ੑଛࣦɼೖྗσʔλͱɼೖྗσʔλΛ֤ Generator Ͱ̍ճͣͭมͯ͠࠶ߏஙͨ͠σʔλ͕ಉ͡ʹͳΔΑ͏ʹ Generator ಉ࢜ Λௐ͢ΔׂΛ͍࣋ͬͯΔ [7]ɽ ͞Βʹ CycleGAN ͰɼΞΠσϯςΟςΟଛࣦʢIdentity Lossʣ Lidentity Λಋೖ͠ɼ ֤ը૾܈ʹ͓͚Δ৭߹͍ΛอͭΑ͏ͳΛ͍ͯ͠Δ [7]ɽΞΠσϯςΟςΟଛࣦɼ ҎԼͷࣜ (2.19) ͷΑ͏ʹද͞ΕΔɽ Lidentity (G, F) = Ex [∥F(x) − x∥1 ] + Ey [∥G(y) − y∥1 ] (2.19) CycleGAN ͷଛࣦؔ L ࣜ (2.20) ͷ௨ΓͱͳΔɽࣜ (2.20) தͷ λcycle , λidentity ଛ ࣦؔͷόϥϯεΛͱΔͨΊͷϋΠύʔύϥϝʔλͰ͋Δɽ L = LGAN (G, DY , X, Y) + LGAN (F, DX , Y, X) + λcycle Lcycle (G, F) + λidentity Lidentity (G, F) (2.20) CycleGANͷωοτϫʔΫΛਤ2.18ʹࣔ͢ɽCycleGANɼGANͷωοτϫʔΫΛ ̎ͭΈ߹ΘͤͨΞʔΩςΫνϟͰ͋ΓɼGenerator G, F ͱ Discriminator DX , DY Λ༻ ͍ΔɽDiscriminator DX ɼೖྗը૾͕ຊը૾ x ͔ੜը૾ F(y) ͔ɼDiscriminator DY ɼೖྗը૾͕ຊը૾ y ͔ੜը૾ G(x) ͔ΛͦΕͧΕࣝผ͢Δɽ 2.7 Attention-Guided Image-to-Image Translation Attention-Guided Image-to-Image Translation ͱɼImage to Image Translation ʹ Attentionʢҙػߏʣͷ֓೦ΛऔΓೖΕͨͷͰ͋Δ [8] [9]ɽ֤ Generator ʹ Attention -
ୈ 2 ষ ؔ࿈ݚڀ 21 ਤ 2.18 CycleGAN ͷωοτϫʔΫɹ ΛऔΓೖΕΔ͜ͱʹΑͬͯɼը૾ͷ͖͢ΦϒδΣΫτΛڧௐ͠ɼΦϒδΣ
ΫτҎ֎ͷഎܠͳͲͷ͠ͳ͍෦ʹӨڹΛ༩͑ͳ͍ը૾Λੜ͢Δ͜ͱΛՄೳ ͱ͍ͯ͠Δɽ Attention-Guided Image-to-Image Translation ͷ Generator ΞʔΩςΫνϟͷछྨ͝ ͱʹɼAttention-Guided Generatior Scheme I [9] ͱ Attention-Guided Generator Scheme II ͕͋Δ [8]ɽ 2.7.1 Attention-Guided Generator Scheme I Attention-Guided Generator Scheme IΛ༻͍ͨAttention-Guided Image-to-Image Trans- lation 2019 ʹ Hao Tang ࢯΒʹΑͬͯߟҊ͞Εͨ [8]ɽAttention-Guided Generator Scheme I ͷ Generator ͷߏΛਤ 2.19 ʹࣔ͢ɽ ਤ 2.19 ͰɼBu3dfe σʔληοτ [16] Λ༻͍ͯɼதੑతͳإͷը૾ͱޱ֯ͷ্͕ͬ ͨإͷը૾ͱͷը૾มΛߦ͍ͬͯΔɽGenerator ͦΕͧΕ G ͱ F Λ༻͍͓ͯΓɼ G தੑతͳإ͔Βޱ֯ͷ্͕ͬͨإͷը૾มΛߦ͏ GeneratorɼF ޱ֯ͷ্͕ͬ -
ୈ 2 ষ ؔ࿈ݚڀ 22 ਤ 2.19 Attention-Guided Generator Scheme
I ͷ Generator ͷߏ ɹ ͨإ͔Βதੑతͳإͷը૾มΛߦ͏ Generator Ͱ͋Δɽ ͳ͓ɼGenerator G, F ʹ ͦΕͧΕ Attention ΛՃ͓ͯ͠ΓɼAttention Mask Ay , Ax ͱ Content Mask Cy ,Cx Λग़ྗ͢Δɽ Attention Mask Ay , Ax Ͱɼը૾ͷ͍ͨ͠෦Λڧௐ͢ΔϚεΫͰ͋Δɽਤ 2.19 ͰɼAy ೖྗը૾ x தͷإͷޱ֯෦ʹ͍ͯ͠ΔɽAx ʹ͓͍ͯ Ay ͱ ಉ༷ͷํ๏Ͱɼը૾ G(x) தͷإͷޱ͕֯ڧௐ͞Ε͍ͯΔɽ Content Mask Cy ,Cx ɼมઌͷը૾܈ͷಛΛͱʹϨϯμϦϯά͞Εͨը૾ Λࣔ͢ɽɹਤ 2.19 ͰɼGenerator G Ͱޱ֯ͷ্͕ͬͨޱݩΛදݱͨ͠ը૾Λ Content Mask Cy ͱͯ͠ग़ྗ͍ͯ͠Δɽ ಉ༷ʹ Generator F ͰதੑతͳإΛදݱͨ͠ը૾ Λ Content Mask Cx ͱͯ͠ग़ྗ͍ͯ͠Δɽ ࠷ऴతʹਤ 2.19 ͷ Fusion ͷ෦ͰɼAttention Mask Ay ͱ Content Mask Cy ɼೖྗը ૾ x Λͯ͠ɼը૾มΛߦ͏ɽೖྗը૾ x Λ Generator Ͱมͨ͠ G(x) ࣜ (2.21) Ͱද͞ΕΔ [8]ɽͳ͓ɼ ⊙ ΞμϚʔϧੵͷԋࢉΛද͢ɽ G(x) = Ay ⊙ Cy + (1 − Ay ) ⊙ x (2.21) -
ୈ 2 ষ ؔ࿈ݚڀ 23 ɹಉ༷ʹɼAttention Mask Ax ͱ Content
Mask Cx ɼೖྗը૾ y Λͯ͠ɼF(y) = Ax ⊙ Cx + (1 − Ax ) ⊙ y ͱը૾มΛߦ͏ɽ Attention-Guided Generator Scheme I Λ༻͍ͨ Attention-Guided GAN ͷଛࣦؔ ɼఢରੑଛࣦɼαΠΫϧҰ؏ੑଛࣦɼAttention ఢରੑଛࣦɼAttention ଛࣦɼϐΫη ϧଛࣦʢΞΠσϯςΟςΟଛࣦʣͰߏ͞Ε͍ͯΔ [8]ɽఢରੑଛࣦɼαΠΫϧҰ؏ੑ ଛࣦɼCycleGAN Ͱ༻͍ΒΕ͍ͯΔଛࣦͱಉ͡Ͱɼఢରੑଛࣦࣜ (2.22)ɼ(2.23)ɼ αΠΫϧҰ؏ੑଛࣦࣜ (2.24) ͷ௨ΓʹදͤΔɽ LGAN (G, DY , X, Y) = Ey [log(DY (y))] +Ex [log(1 − DY (G(x)))] (2.22) LGAN (F, DX , Y, X) = Ex [log(DX (x))] + Ey [log(1 − DX (F(y)))] (2.23) Lcycle (G, F) = Ex [∥F(G(x)) − x∥1 ] + Ey [∥G(F(y)) − y∥1 ] (2.24) ϐΫηϧଛࣦɼCycleGAN ͷΞΠσϯςΟςΟଛࣦͱಉ༷ɼ֤ը૾܈ʹ͓͚Δ৭ ߹͍ΛอͭΑ͏ઃ͚ΒΕͨଛࣦͰ͋Δ [8]ɽೖྗը૾Λ Generator Ͱมͨ͠ը૾ͱ ݩͷೖྗը૾ͱͷϚϯϋολϯڑΛଛࣦͱ͓ͯ͠Γɼࣜ (2.25) ͷΑ͏ʹදͤΔɽ Lpixel (G, F) = Ex [∥G(x) − x∥1 ] + Ey [∥F(y) − y∥1 ] (2.25) ଓ͍ͯ Attention ఢରੑଛࣦɼDiscriminator Ͱ Attention ΛؚΊͯࣝผͨ͠ͱ͖ ͷఢରੑଛࣦͰ͋Δ [8]ɽAttention ఢରੑଛࣦͰ Discriminator DX , DY ͷΘΓʹ Attention-Guided Discriminator DXAttention , DYAttention Λಋೖͨ͠ͷͰ͋Γɼ ࣜ(2.26)ɼ (2.23) Ͱද͢ɽDYAttention ɼຊը૾ͱ Attention ͷϖΞ [Ax , y] ͱੜը૾ͱ Attention ͷϖ Ξ [Ax ,G(x)] Λࣝผ͠ɼDXAttention ɼຊը૾ͱ Attention ͷϖΞ [Ay , x] ͱੜը૾ͱ Attention ͷϖΞ [Ay , F(y)] Λࣝผ͢Δɽ LAGAN (G, DY , X, Y) = Ey [log(DYAttention ([Ax , y]))] + Ex [log(1 − DYAttention ([Ax ,G(x)]))] (2.26) -
ୈ 2 ষ ؔ࿈ݚڀ 24 LAGAN (F, DX , Y,
X) = Ex [log(DXAttention ([Ay , x]))] + Ey [log(1 − DXAttention ([Ax , F(y)]))] (2.27) Attention ଛࣦ [8] ɼAttention Mask ʹ Total Variation ਖ਼نԽΛߦ͍ɼత֬ͳ Attention ʹ͢Δͷʹઃ͚ΒΕͨଛࣦͰ͋ΔɽAttention Mask Ax ʹ͓͚Δ Attnetion ଛࣦ Ltv (Ax ) ࣜ (2.28) ͷ௨Γʹද͞ΕΔɽAttention Mask Ay ʹ͓͍ͯಉ༷ͷܭࢉΛߦ͏ɽͳ ͓ɼW, H ͦΕͧΕ Attention Mask Ax ͷԣ෯ɼॎ෯Λද͢ɽ Ltv (Ax ) = W,H ∑ w,h=1 |Ax (w + 1, h, c) − Ax (w, h, c)| + |Ax (w, h + 1, c) − Ax (w, h, c)| (2.28) Attention-Guided GAN ͷଛࣦؔࣜ (2.29) Ͱද͞ΕΔɽ͜͜ͰɼLGAN ࣜ (2.22)ɼ (2.23) ͷɼLAGAN ࣜ (2.26)ɼ(2.23) ͷͰ͋Δɽ͞ΒʹɼλGAN , λcycle , λpixel , λtv ଛࣦؔͷόϥϯεΛͱΔͨΊͷϋΠύʔύϥϝʔλͰ͋Δɽ L = λGAN (LGAN + LAGAN ) + λcycle Lcycle + λpixel Lpixel + λtv Ltv (2.29) 2.7.2 Attention-Guided Generator Scheme II Attention-Guided Generator Scheme I Ͱ̍ͭͷ Generator ͔Β̍ͭͣͭ Attention Mask ͱ Content Mask Λੜ͢Δ [8]ɽ͔͠͠ɼAttention-Guided Generator Scheme I ͷ Generator Λ༻͍ͨ߹ɼσʔληοτʹΑͬͯਫ਼͕མͪΔͱ͍͏ݚڀ݁Ռ͕ ͋Δ [9]ɽ͜͏͍ͬͨਫ਼্ͷΛࠀ͘͢ɼAttention-Guided Generator Scheme II ͕ߟҊ͞ΕͨɽAttention-Guided Generator Scheme II ͷ Generator ͷߏΛਤ 2.20 ʹࣔ͢ɽ ਤ 2.20 ͷ Generator G Λྫʹը૾มखॱΛઆ໌͢Δɽೖྗը૾ x Λ Parameter Sharing Encoder GE Ͱม͢ΔɽAttention-Guided Generator Scheme I Ͱ̍ͭͷ -
ୈ 2 ষ ؔ࿈ݚڀ 25 ਤ 2.20 Attention-Guided Generator Scheme
II ͷ Generator ͷߏ Generator ͔Β Attention Mask ͱ Content Mask Λಉ࣌ʹੜ͍͕ͯͨ͠ɼAttention- Guided Generator Scheme II ͷ Generator ͰɼGE Ͱͷग़ྗΛͦΕͧΕ Content Mask Generator GC ͱ Attention Mask Generator GA ʹೖྗ͠ɼ֤ Mask Generator Ͱ Content Mask ͱ Attention Mask Λग़ྗ͢Δ [9]ɽ Attention Mask Generator GA ɼ͖͢෦ͷΈΛڧௐ͢Δ Foreground At- tention Mask Af y ͱɼରը૾มʹؔͷͳ͍എܠͳͲͱ͍ͬͨ෦Λڧௐ͢Δ Background Attention Mask Ab y Λੜ͢Δɽ͜ͷΑ͏ʹվྑ͢Δ͜ͱʹΑͬͯɼFore- ground Attention Mask Af y ͱ Background Attention Mask Ab y ʹΑͬͯഎܠʹӨڹΛ༩ ͑ͣʹ͍ͨ͠෦ʹͷΈը૾มΛߦ͏͜ͱ͕Ͱ͖Δɽ Ҏ্ΑΓ Attention-Guided Generator Scheme I ͱͷΞʔΩςΫνϟͷҧ͍ɼҎԼ ͷ̏ͭʹ·ͱΊΒΕΔɽ 1 GeneratorΛ్த·Ͱֶश͠ɼֶशޙͷग़ྗΛ༻͍ͯɼAttention Mask Generator ͱ Content Mask Generator ʹׂ͢Δ 2 Attention Mask Generator ͔Β Foreground Attention Mask ͱ Background Atten- tion Mask Λੜ͢Δ 3 Attention Mask Generator ͔Βੜ͞ΕΔ Foreground Attention Mask ͱ Content -
ୈ 2 ষ ؔ࿈ݚڀ 26 Mask Generator ͔Βੜ͞ΕΔ Content Mask
ͷຕΛෳʹ૿͢ʢFore- ground Attention Mask ͱ Content Mask ͷຕಉ͡ʣ Foreground Attention Mask Af y ͱ Content Mask Cf y ͷϖΞͷཁૉͷ֤ੵͱɼҰຕͷ Background Attention Mask Af y ͱ Input x ͷཁૉͷ֤ੵΛ͠߹Θͤͯը૾Λੜ͢ ΔɽAttention Mask Generator ͔Βੜ͞ΕΔ Attention Mask ͷ૯Λ n ຕͱͨ͠ͱ ͖ɼBackground Attention Mask Ab y 1 ຕͰ͋ΔɽͦͷͨΊɼForeground Attention Mask Af y ͷ૯ n−1 ͱͳΓɼForeground Attention Mask Af y ͱϖΞͷ Content Mask Cf y n − 1 ͱͳΔɽ ͕ͨͬͯ͠ɼೖྗը૾ x Λ Generator Ͱมͨ͠ G(x) ࣜ (2.30) Ͱද͞ΕΔɽͳ ͓ɼ ⊙ ΞμϚʔϧੵͷԋࢉΛද͢ɽ G(x) = n−1 ∑ f=1 Af y ⊙ Cf y + Ab y ⊙ x (2.30) ɹ Generator F Ͱɼಉ༷ʹͯ͠ y Ͱ͋Δͱ͢ΔͱɼF(y) ࣜ (2.31) Ͱද͞ΕΔɽ F(x) = n−1 ∑ f=1 Af x ⊙ Cf x + Ab x ⊙ y (2.31) Attention-Guided GAN ͷଛࣦؔ CycleGAN ͱಉ༷ʹɼఢରੑଛࣦͱαΠΫϧ Ұ؏ੑଛࣦɼΞΠσϯςΟςΟଛࣦΛ༻͍Δ [9]ɽఢରੑଛࣦͱɼมઌͱͳΔσʔ λ܈ͷʹੜը૾ͷΛҰகͤ͞Δ͜ͱΛతͱͨ͠ଛࣦͰ͋ΓɼαΠΫϧ Ұ؏ੑଛࣦɼ̎ͭͷ Generator ಉ࢜ʹໃ६͕ੜ͡ͳ͍Α͏ʹ͢Δ͜ͱΛతͱͨ͠ ଛࣦͰ͋ΔɽͦΕʹՃ͑ͯɼΞΠσϯςΟςΟଛࣦʢIdentity Lossʣ Lidentity Λಋೖ ͠ɼ֤ը૾܈ʹ͓͚Δ৭߹͍ΛอͭΑ͏ͳΛ͍ͯ͠Δ [7]ɽఢରੑଛࣦ LGAN Λ ࣜ (2.32)ɼ(2.33)ɼαΠΫϧҰ؏ੑଛࣦ Lcycle Λࣜ (2.34)ɼΞΠσϯςΟςΟଛࣦΛࣜ (2.35) ʹͯࣔ͢ɽ͜͜Ͱσʔλ܈Λ X, Y ͱͯ͠ɼG Λ X → Y ͷมΛߦ͏ Generatorɼ F Λ Y → X ͷมΛߦ͏ Generator ͱ͠ɼX, Y ʹରԠ͢Δ Discriminator ΛͦΕͧΕ DX , DY ͱ͢ΔɽDX ࣝผରͱͳΔσʔλ͕ը૾σʔλ܈ X தͷը૾Ͱ͋Δ͔Ͳ -
ୈ 2 ষ ؔ࿈ݚڀ 27 ͏͔ɼDY ࣝผରͱͳΔσʔλ͕ը૾σʔλ܈ Y தͷը૾Ͱ͋Δ͔Ͳ͏͔Λࣝ ผ͢Δɽ
LGAN (G, DY , X, Y) = Ey [log(DY (y))] +Ex [log(1 − DY (G(x)))] (2.32) LGAN (F, DX , Y, X) = Ex [log(DX (x))] + Ey [log(1 − DX (F(y)))] (2.33) Lcycle (G, F) = Ex [∥F(G(x)) − x∥1 ] + Ey [∥G(F(y)) − y∥1 ] (2.34) Lidentity (G, F) = Ex [∥F(x) − x∥1 ] + Ey [∥G(y) − y∥1 ] (2.35) ࣜ (2.32)ɼࣜ (2.33)ɼࣜ (2.34)ɼࣜ (2.35) ΑΓɼAttention-Guided GAN ͷଛࣦؔ ࣜ (2.36) ͷ௨ΓͱͳΔɽࣜ (2.36) தͷ λcycle , λidentity ଛࣦؔͷόϥϯεΛͱΔͨ ΊͷϋΠύʔύϥϝʔλͰ͋Δ. L = LGAN (G, DY , X, Y) + LGAN (F, DX , Y, X) + λcycle Lcycle (G, F) + λidentity Lidentity (G, F) (2.36) 2.8 ճస֯༧ଌλεΫʢSelf-Supervised taskʣ ճస֯༧ଌλεΫͰɼݩͷը૾Λ͋Δ֯ճసͤͨ͞ͷΛೖྗը૾ͱ͠ɼ ͦͷೖྗը૾͕ݩͷը૾͔ΒԿճస͍ͯ͠Δ͔Λਪଌ͢Δ [17]ɽը૾ͷճస֯Λ ਪଌ͢ΔλεΫΛՃ͑Δ͜ͱʹΑͬͯɼճసෆมੑΛѲ͠ɼը૾ͷزԿతಛΛ ΑΓଊ͑ΒΕΔΑ͏ʹ͍ͯ͠Δɽ -
ୈ 2 ষ ؔ࿈ݚڀ 28 ਤ 2.21 Self-Supervised GAN ʹ͓͚Δ
Discriminator ͷߏ ͳ͓ɼGAN ʹճస֯༧ଌλεΫΛಋೖͨ͠ Self-Supervised GAN ͕ଘࡏ͢Δ [10]ɽਤ 2.21 ʹͯɼSelf-Supervised GAN ʹ͓͚Δ Discriminator ͷߏΛࣔ͢ɽSelf- Supervised GAN ͰɼDiscriminator ͷ෦Λʮೖྗը૾͕ຊը૾Ͱ͋Δ͔൱͔ʯ Λࣝผ͢Δ D ͱʮೖྗը૾͕ݩͷը૾͔ΒԿճసͨ͠ͷͰ͋Δ͔ʯΛࣝผ͢Δ Drot ʹ͚͓ͯΓɼલऀ Generator ͔Βੜ͞Εͨݩͷຊը૾ͱݩͷੜը૾ɼ ޙऀݩͷຊը૾ͱݩͷੜը૾Λ, ͦΕͧΕ 0◦, 90◦, 180◦, 270◦ ճసͤͨ͞ͷΛ ͦΕͧΕ༻͍Δɽਤ 2.21 ͷΑ͏ʹ Self-Supervised GAN ͷ Discriminator ʹճస֯ ਪଌλεΫΛՃ͑Δ͜ͱʹΑͬͯɼຊը૾ͷҐஔؔΛͱʹੜը૾ͷࣝผ͕ Ͱ͖ΔͨΊɼࣝผਫ਼্͕͕Δɽͦͯ͠ Discriminator ͱ Generator ͷఢରతֶशʹ ΑͬͯɼGenerator ֶश͢ΔͷͰɼੜը૾ͷ্࣭ͤ͞Δ͜ͱ͕ՄೳͱͳΔɽ Self-Supervised GAN ͷଛࣦؔ LD , LG ɼࣜ (2.37)ɼࣜ (2.37) ͱදͤΔɽV(D,G) ࣜ (2.14) ͱಉ༷ GAN ͷଛࣦؔͰ͋Γɼຊը૾ʹ͓͚Δճస֯༧ଌλεΫ LrotD ɼੜը૾ʹ͓͚Δճస֯༧ଌλεΫ LrotG ΛͦΕͧΕՃ͍ͯ͠Δɽ -
ୈ 2 ষ ؔ࿈ݚڀ 29 LD = V(D,G) + λd
LrotD (2.37) LG = V(D,G) − λg LrotG (2.38) ຊը૾ʹ͓͚Δճస֯༧ଌλεΫ LrotD ɼੜը૾ʹ͓͚Δճస֯༧ଌλεΫ LrotG Λࣜ(2.39)ɼ ࣜ(2.40)Ͱද͢ɽ T ճస֯ͷू߹Λද͠ɼT = {0◦, 90◦, 180◦, 270◦} ͱ͢ΔɽT ∈ T Λճస֯ͱ͠ɼຊը૾ x, y Λ T ճసͤͨ͞ը૾ΛͦΕͧΕ xT , yT ͷΑ͏ʹද͢ɽ Self-Supervised GAN Ͱ Discriminator ʹճస֯Λ༧ଌ͢Δ Drot Λಋೖ͠ɼճ స֯Λਖ਼֬ʹ༧ଌͰ͖ΔΑ͏ʹ Discriminator Λֶश͢Δɽճస͕֯ਖ਼֬ʹ༧ଌ Ͱ͖ΔΑ͏ʹͳΔʹͭΕɼDrot (x) ͷେ͖͘ͳΔɽ LrotD = Ex ET log(Drot (xT )) (2.39) LrotG = Ex ET log(Drot (xT )) (2.40) -
ୈ 3 ষ ఏҊख๏ 30 ୈ3ষ ఏҊख๏ ຊݚڀʹ͓͍ͯɼ2.7.2 ߲ͷ Attention-Guided
GAN ͷ Discriminator ʹɼճస͞ Εͨݩը૾ͷճస֯Λਪଌ͢ΔλεΫΛՃ͑ͨ Self-Supervised Attention-Guided GAN ʢSSAttention-Guided GANʣΛఏҊ͢Δɽͦͯ͠ɼઌߦݚڀͰࣔͨ͠ 2.7.2 ߲ ͷ Attnetion-Guided GAN ͱൺֱͯ͠ SSAttention-Guided GAN ͷධՁΛߦ͏ɽ SSAttention-Guided GAN Ͱɼ2 ͭͷը૾܈ X, Y ؒͰͷը૾มʹର͠ɼ2 ͭͷ Generator Λ GX→Y : X → YɼGY→X : Y → X Λ༻ҙ͢Δɽͦͯ͠ը૾܈ X, Y தͷը૾ x ∈ X ͱ y ∈ Y Λຊը૾ͱ֤ͯ͠ Discriminator DX , DY ͰࣝผΛߦ͏ɽDX ը૾܈ X ͷຊը૾ x ͱੜը૾ GY→X (y) Λࣝผ͠ɼDY ը૾܈ Y ͷຊը૾ y ͱੜը ૾ GX→Y (x) Λࣝผ͢Δɽɹ ճస֯༧ଌλεΫʹ͓͍ͯɼ T ճస֯ͷू߹Λද͠ɼT = {0◦, 90◦, 180◦, 270◦} ͱ͢ΔɽT ∈ T Λճస֯ͱ͓ͯ͠Γɼຊը૾ x, y Λ T ճసͤͨ͞ը૾ΛͦΕͧΕ xT , yT ɼੜը૾ GX→Y (x),GY→X (y) Λ T ճసͤͨ͞ը૾ΛͦΕͧΕ GT X→Y (x),GT Y→X (y) ͱද͢ɽ ճస֯༧ଌΛ͢Δ Discriminator ΛͦΕͧΕ Drot X ɼDrot Y ͱ͢ΔɽDrot X ɼຊը૾ xT ͱੜը૾GT Y→X (y)ͷճస֯Λਪଌ͠ɼDrot Y ɼຊը૾ yT ͱੜը૾GT X→Y (x) ͷճస֯Λਪଌ͢Δɽ 3.1 ωοτϫʔΫߏ ຊݚڀͰఏҊ͢Δ SSAttention-Guided GAN ɼ2.7.2 ߲ͷ Attention-Guided GAN ͱಉ༷ͷωοτϫʔΫߏΛࢀߟͱ͍ͯ͠Δ [9]ɽਤ3.1ʹͯSSAttention-Guided GAN ͷߏΛࣔ͢ɽਤ 3.1 Ͱɼਤ 2.17 Ͱࣔ͢ CycleGAN ߏͷ֤ Discriminator ෦ʹ ਤ 2.21 ͱಉ͡ߏͷճస֯༧ଌλεΫΛಋೖ͍ͯ͠ΔɽDiscriminator DX Ͱೖ -
ୈ 3 ষ ఏҊख๏ 31 ྗը૾͕ຊը૾ x ͔ੜը૾ GY→X ͔ͷࣝผΛߦ͍ɼ͔ͭ
xT ͱ GT Y→X (y) ͷճస֯ ਪఆΛߦ͍ͬͯΔɽಉ༷ʹ Discriminator DY Ͱೖྗը૾͕ຊը૾ y ͔ੜը ૾ GX→Y ͔ͷࣝผΛߦ͍ɼ͔ͭ yT ͱ GT X→Y (x) ͷճస֯ਪఆΛߦ͍ͬͯΔɽ ֤ Generator ɼAttention-Guided Generator Scheme II Λ༻͍ΔɽAttention-Guided Generator Scheme II Ͱɼn ݸͷ Attention Mask ͱ n−1 ݸͷ Content Mask Λग़ྗ͠ɼ ͜ΕΒͷ Mask ͱೖྗը૾Λͯ͠ग़ྗ͢Δ [9]ɽGenerator ͷωοτϫʔΫɼ3 ͭ ͷΈࠐΈͱɼ9 ͭͷ Residual Blocks [18]ɼ3 ͭͷٯΈࠐΈͰߏ͞ΕΔ [7] [9]ɽ Residual Blocks ɼਂχϡʔϥϧωοτϫʔΫͷֶशਫ਼্Λୡ͢ΔͨΊʹ ಋೖ͞ΕͨɼೖྗΛ 2 ࿈ଓ͢ΔΈࠐΈʹࣸ૾ͨ͠ͱ͖ͷग़ྗʹೖྗΛ͠߹ Θͤͨ ResNet ͱݺΕΔωοτϫʔΫΛܨ͗߹Θͤͯߏ͞ΕΔ [18] [12] ֤ Discriminator ɼ5 ݸͷΈࠐΈͰߏ͞Εɼ࠷ऴతʹ 512 νϟωϧͷςϯ ιϧΛग़ྗ͢Δ [7]ɽͳ͓ຊݚڀͰɼνϟωϧ 512 ͷ࠷ޙͷͷग़ྗ͔Βɼνϟ ωϧ 1 ͷग़ྗͱνϟωϧ 4 ͷग़ྗΛಘΔɽલऀɼೖྗը૾͕ຊͰ͋Δ͔Ͳ ͏͔Λࣝผ͢ΔͨΊʹ༻͍Δग़ྗͰɼޙऀɼճస֯Λ༧ଌ͢ΔͨΊʹ༻͍Δग़ ྗͰ͋Δɽ 3.2 ଛࣦؔ 2.7.2 ߲ͷ Attention-Guided GAN Ͱ CycleGAN ͱಉ͘͡ɼఢରੑଛࣦʢAdver- sarial LossʣͱαΠΫϧҰ؏ੑଛࣦʢCycle Consistency LossʣΛ༻͍ͯ͠Δɽຊ ݚڀͰհ͢Δ SSAttention-Guided GAN Ͱɼ֤ఢରੑଛࣦʹճస֯༧ଌଛࣦ ʢRotation LossʣΛՃ͢Δɽ 3.2.1 ఢରੑଛࣦ (Adversarial Loss) ఢରੑଛࣦɼDiscriminator ͷఢରੑଛࣦͱ Generator ͷఢରੑଛࣦʹ͚ΒΕ Δɽ֤ Discriminator ͷࣝผʹ͓͚ΔଛࣦΛ Discriminator ͷఢରੑଛࣦͱ͢ΔɽDis- criminator DX , DY ʹ͓͚Δఢରੑଛࣦ LGANDX , LGANDY ࣜ (3.1)ɼࣜ (3.2) ͷΑ͏ʹද -
ୈ 3 ষ ఏҊख๏ 32 ਤ 3.1 SSAttention-Guided GAN ͷߏ
͢ɽͳ͓ຊݚڀʹ͓͍ͯɼLGANDX , LGANDY ΫϩεΤϯτϩϐʔͰͳ͘࠷খೋ ๏Λ࠾༻͍ͯ͠Δɽ࠷খೋ๏Λ༻͍Δ͜ͱͰɼֶशͷ҆ఆੑΛ֬อ͢Δ͜ͱ͕Մ ೳͱͳΔ [11]ɽ LGANDX = 1 2 Ex [(DX (x) − 1)2] + 1 2 Ey [(DX (GY→X (y)))2] (3.1) LGANDY = 1 2 Ey [(DY (y) − 1)2] + 1 2 Ex [(DY (GX→Y (x)))2] (3.2) ɹ ଓ͍ͯ Generator GY→X ,GX→Y ͷఢରੑଛࣦ LGANGY→X , LGANGX→Y Λࣜ (3.3)ɼࣜ (3.4) ͷ Α͏ʹఆٛ͢Δɽ LGANGY→X = 1 2 Ey [(DX (GY→X (y)) − 1)2] (3.3) LGANGX→Y = 1 2 Ex [(DY (GX→Y (x)) − 1)2] (3.4) ຊݚڀͰఏҊ͢Δఢରੑଛࣦɼ֤ఢରੑଛࣦʹͦΕͧΕճస֯༧ଌଛࣦΛ -
ୈ 3 ষ ఏҊख๏ 33 Ճͨ͠ͷͰ͋ΔɽDiscriminator ͷఢରੑଛࣦ LGANDX , LGANDY
ʹճస֯༧ଌଛࣦ LrotDX , LrotDY ΛͦΕͧΕՃͨ͠ఢରੑଛࣦΛࣜ(3.5)ɼ ࣜ(3.6)Ͱఆٛ͢Δɽ LrotDX , LrotDY 2.8 ߲ͷࣜ (2.39) ͱಉ༷ʹࣜ (3.7), ࣜ (3.8) Ͱఆٛ͢ΔɽຊݚڀͰֶश҆ఆੑͷ ֬อͷͨΊɼLrotDX , LrotDY ɼΫϩεΤϯτϩϐʔͷΘΓʹೋޡࠩΛ༻͍͍ͯ Δɽͳ͓ LrotDX , LrotDY ଛࣦͷόϥϯεΛͱΔͨΊͷϋΠύʔύϥϝʔλ λDX , λDY ͱͷ֤ੵΛͱΔɽ LDX = LGANDX + λDX LrotDX (3.5) LDY = LGANDY + λDY LrotDY (3.6) LrotDX = Ex ET [(Drot X (xT ) − 1)2] (3.7) LrotDY = Ey ET [(Drot Y (yT ) − 1)2] (3.8) Generator ͷఢରੑଛࣦ LGANGY→X , LGANGX→Y ʹճస֯༧ଌଛࣦ LrotGY→X , LrotGX→Y Λ ͦΕͧΕՃͨ͠ఢରੑଛࣦΛࣜ (3.9)ɼࣜ (3.10) ͱఆٛ͢ΔɽLrotGY→X , LrotGX→Y 2.8 અͷࣜ (2.40) ͱಉ༷ʹࣜ (3.11)ɼ ࣜ (3.12) Ͱఆٛ͢ΔɽຊݚڀͰֶश҆ఆੑͷ֬ อͷͨΊɼLrotDX , LrotDY ɼΫϩεΤϯτϩϐʔͷΘΓʹೋޡࠩΛ༻͍͍ͯΔɽ LrotDX , LrotDY ଛࣦͷόϥϯεΛͱΔͨΊͷϋΠύʔύϥϝʔλ λGY→X , λGX→Y ͱͷ֤ ੵΛͱΔɽ LGY→X = LGANGY→X + λGY→X LrotGY→X (3.9) LGX→Y = LGANGX→Y + λGX→Y LrotGX→Y (3.10) LrotGY→X = Ey ET [(Drot X (GT Y→X (y)) − 1)2] (3.11) LrotGX→Y = Ex ET [(Drot Y (GT X→Y (x)) − 1)2] (3.12) -
ୈ 3 ষ ఏҊख๏ 34 3.2.2 Cycle Consistency LossʢαΠΫϧҰ؏ੑଛࣦʣ αΠΫϧҰ؏ੑଛࣦͰɼGenerator
GX→Y ,GY→X ؒͰໃ६͕ੜ͡ͳ͍Α͏ʹɼCy- cleGAN Attention-Guided GAN Ͱ༻͍ΒΕ͍ͯΔ [7] [8] [9]ɽຊݚڀͰઌߦݚڀͱ ಉ༷ͷαΠΫϧҰ؏ੑଛࣦ Lcycle Λѻ͏ɽαΠΫϧҰ؏ੑଛࣦ Lcycle Λࣜ (3.13) ʹͯ ද͢ɽ Lcycle = Ex [∥GY→X (GX→Y (x)) − x∥1 ] + Ey [∥GX→Y (GY→X (y)) − y∥1 ] (3.13) 3.2.3 ࠷ऴతͳଛࣦؔ Discriminator ͷଛࣦؔͦΕͧΕࣜ (3.5)ɼࣜ (3.6) ͷ௨ΓͰ͋ΔɽҰํɼGen- erator ͷଛࣦؔɼGenerator ͷ֤ఢରੑଛࣦͱαΠΫϧҰ؏ੑଛࣦΛ߹Θͤɼࣜ (3.14) ͷΑ͏ʹද͢ɽͳ͓ λcycle ଛࣦͷόϥϯεΛͱΔͨΊͷϋΠύʔύϥϝʔλ Ͱ͋Δɽ LG = LGX→Y + LGY→X + λcycle Lcycle (3.14) ͞ΒʹຊݚڀͰɼΞΠσϯςΟςΟଛࣦʢIdentity Lossʣ Lidentity Λಋೖ͢ΔɽΞ ΠσϯςΟςΟଛࣦ֤ը૾܈ʹ͓͚Δ৭߹͍ΛอͭͨΊʹઃ͚ΒΕͨଛࣦͰ͋Γɼ CycleGAN Attention-Guided GAN Ͱಉ༷ʹ༻͍ΒΕ͍ͯΔ [7] [9]ɽ ΞΠσϯςΟςΟଛࣦɼҎԼͷࣜ (3.15) ͷΑ͏ʹද͞ΕΔɽ Lidentity = Ex [∥GY→X (x) − x∥1 ] + Ey [∥GX→Y (y) − y∥1 ] (3.15) ࣜ (3.14) ʹΞΠσϯςΟςΟଛࣦΛՃ͠ɼຊݚڀͰ༻͍Δ Generator ͷ࠷ऴతͳଛ ࣦؔΛࣜ (3.16) ʹࣔ͢ɽͳ͓ λidentity ଛࣦͷόϥϯεΛͱΔͨΊͷϋΠύʔύϥ ϝʔλͰ͋Δɽ LG = LGX→Y + LGY→X + λcycle Lcycle + λidentity Lidentity (3.16) -
ୈ 4 ষ ࣮ݧ 35 ୈ4ষ ࣮ݧ ຊݚڀͰɼઌߦݚڀͷ Attention-Guided Generator
Scheme II Λ༻͍ͨ Attention- Guided CycleGAN [9] ͱఏҊख๏ͷ SSAttention-Guided GAN ͱͷൺֱΛߦ͏ɽ 4.1 ύϥϝʔλઃఆ ຊݚڀʹ͓͚Δύϥϝʔλɼઌߦݚڀͷ Attention-Guided GAN ͱಉ༷ͷͷΛ ༻͍Δ [9]ɽ·ͣόοναΠζઌߦݚڀͱಉ͡ 4, ΤϙοΫ 60, ֶश 0.0002ɼ ͦͯ͠ Attention Mask ͷΛ 10 ͱ͢Δɽ ଛࣦؔͷ֤ύϥϝʔλɼλcycle 10.0ɼλidentity 0.5 ͱ͠ɼճస֯༧ଌʹ͓ ͚Δ֤ύϥϝʔλɼຊݚڀͰද 4.1 ͷ௨ΓʹύϥϝʔλΛ Discriminator λDX , λDY : Generator λF , λG = 5 : 1 ͷൺʹͳΔΑ͏ʹݻఆ࣮ͯ͠ݧΛߦ͏ɽઌߦݚڀͰɼ Discriminator ͷճస֯ଛࣦͷόϥϯεύϥϝʔλ 1.0ɼGenerator 0.2 ͱઃఆ͞ Ε͍ͯΔ [10]ɽ ද 4.1 ճస֯༧ଌʹ͓͚Δ֤ύϥϝʔλ Discriminator ͷճస֯ଛࣦ Generator ͷճస֯ଛࣦ λDX λDY λF λG 0.5 0.5 0.1 0.1 1.0 1.0 0.2 0.2 1.5 1.5 0.3 0.3 2.0 2.0 0.4 0.4 -
ୈ 4 ষ ࣮ݧ 36 4.2 σʔληοτ ຊݚڀͰɼઌߦݚڀͰ༻͍ΒΕͨ 5 ͭͷσʔληοτΛ༻͍ΔɽຊݚڀͰ༻͍
Δσʔληοτʹ͍ͭͯද 4.2 ʹͯ·ͱΊΔɽ 4.3 ධՁํ๏ ຊݚڀͰɼFr´ echet Inception Distance(FID) Λ༻͍ͨఆྔతධՁͱ֤ੜը૾ͷ ఆੑతධՁΛߦ͏ɽ 4.3.1 Fr´ echet Inception Distance(FID) Fr´ echet Inception Distance(FID) ɼ2 ͭͷը૾ͷू߹ͷؒڑΛ༻͍ͨධՁ ํ๏Ͱ͋Δ [19]ɽ2 ͭͷը૾ͷू߹ͷؒڑΛ FID ͱ͠ɼFID ͷ͕খ͍͞΄Ͳ ੜը૾͕ຊը૾ʹ͍ۙͱ͍͏͜ͱʹͳΔɽGAN Ͱɼຊը૾σʔληοτͷ pdata ͱੜը૾σʔληοτͷ pg ؒͷ Fr´ echet ڑΛܭࢉ͠ɼFID Λࢉग़ ͢Δ [19]ɽࣜ (4.1) Ͱ GAN ͷ FID ͷܭࢉΛࣔ͢ɽpdata ͷฏۉϕΫτϧͱڞࢄߦྻ (µr , Cr )ɼpg ͷฏۉϕΫτϧͱڞࢄߦྻ (µg , Cg ) Λ༻͍ͯࣜ (4.1) ͷ௨ΓʹܭࢉΛ ߦ͏ɽ FID(pdata , pg ) = ∥µr − µg ∥ + tr(Cr + Cg − 2(Cr Cg )1/2) (4.1) ɹ ຊݚڀͰɼઌߦݚڀͱಉ༷ʹͯ͠ɼhorse2zebra ͱ apple2orange σʔληοτͰ ը૾ม͞Εͨը૾Λ༻͍ͯɼFID Ͱຊը૾ͱੜը૾ͷఆྔతධՁΛߦ͏ [9]ɽ -
ୈ 4 ষ ࣮ݧ 37 ද 4.2 ༻͢Δσʔληοτ σʔληοτ໊ σʔληοτλΠϓ
σʔληοτͷத αΠζ horse2zebra train horse 1067 ʢഅ → γϚϚʣ zebra 1334 test horse 120 zebra 140 apple2orange train apple 995 ʢΞοϓϧ → ΦϨϯδʣ orange 1019 test apple 266 orange 248 facades train facade 400 ʢݐ֎؍ → ϥϕϧʣ label 400 test facade 106 label 106 map2photo train map 1096 ʢਤ → ߤۭࣸਅʣ photo 1096 test map 1098 photo 1098 cityscapes train cityscape 2975 ʢࢢܠ؍ → ϥϕϧʣ label 2975 test cityscape 500 label 500 -
ୈ 5 ষ ࣮ݧ݁Ռͱߟ 38 ୈ5ষ ࣮ݧ݁Ռͱߟ ຊݚڀͰɼઌߦݚڀͷ Attention-Guided Generator
Scheme II ͷ Attention-Guided GAN ͷϓϩάϥϜΛ༻͍ͯɼSSAttention-Guided GAN ͷϓϩάϥϜΛ࣮͠ɼઌ ߦݚڀͱఏҊख๏ͷධՁΛߦͬͨɽ 5.1 ࣮ݧ݁Ռ 5.1.1 ఆྔతධՁ FID Ͱͷ horse2zebra ͱ apple2orange ͷఆྔతධՁΛද 5.1 ʹࣔ͢ɽද 5.1 Ͱɼ horse to zebraʢഅ → γϚϚʣͱ apple to orangeʢΞοϓϧ → ΦϨϯδʣͷ FID Λ ֤ճస֯ύϥϝʔλ͝ͱʹ͍ࣔͯ͠Δɽͳ͓ճస֯ύϥϝʔλ λD , λG , ͦΕͧΕ λDX , λDY ͱ λGX→Y , λGY→X Λ·ͱΊͨͷͰ͋Δɽճస֯ύϥϝʔλ (λD , λG ) = (0.0, 0.0) ઌߦݚڀͷ Attention-Guided GAN [9] Ͱ͋Δɽ ද5.1ͷ݁ՌΑΓɼ horse to zebraͱapple to orangeͷ྆ํʹ͓͍ͯɼ ճస֯ύϥϝʔ λ (λD , λG ) = (0.5, 0.1), (2.0, 0.4) ͷͱ͖ͷ FID ͕ઌߦݚڀͷ Attention-Guided GAN ͷ FID ΑΓখ͍͞Λͱͳ͍ͬͯΔɽҰํͰճస֯ύϥϝʔλ (λD , λG ) = (1.5, 0.3) Ͱ ɼapple to orange ͷ FID ൺֱతখ͍͞Ͱ͋Δͷʹର͠ɼhorse to zebra ͷ FID ൺֱతେ͖͍ͱͳ͍ͬͯΔɽ͞Βʹ (λD , λG ) = (1.0, 0.2) ͷͱ͖ɼhorse to zebra ͱ apple to orange ͷ྆ํʹ͓͍ͯɼFID ͕ൺֱతେ͖͍Λͱ͍ͬͯΔ͜ͱ͕ݟͯ औΕΔɽ -
ୈ 5 ষ ࣮ݧ݁Ռͱߟ 39 ද 5.1 FID ͷఆྔతධՁ ճస֯ύϥϝʔλ
(λD , λG ) horse to zebra apple to orange (0.0, 0.0) 79.64 156.88 (0.5, 0.1) 54.58 149.88 (1.0, 0.2) 117.60 158.46 (1.5, 0.3) 83.37 154.71 (2.0, 0.4) 66.87 155.53 5.1.2 ఆੑతධՁ ֤σʔληοτ horse2zebraɼapple2orangeɼmap2photoɼfacadesɼcityscapes ͷը ૾มޙͷੜը૾ͱ Attention ͷ݁Ռʹ͍ͭͯड़Δɽ ·ͣ horse to zebra มͷҰྫΛද 5.2ɼද 5.3 ʹɼapple to orange มͷҰྫΛද 5.4ɼද 5.5 ʹࣔ͢ɽhorse2zebra ͷҰྫʹ͓͍ͯɼैདྷͷ Attention-Guided GAN ͷ มը૾ͱ Attention എܠʹࣶ༷ͷม͕͞Ε͍ͯΔ͜ͱ͕ݟͯऔΕΔɽҰ ํ SSAttention-Guided GAN Λ༻͍ͨ݁Ռɼ྆ํͱഎܠͷࣶ༷͕ݮগ͠ɼͦͷ Attention ͕มରͷܗʹ͍ͬͯΔ͜ͱ͕͔Δɽද 5.2 Ͱɼճస֯ύϥϝʔ λ (λD , λG ) = (0.5, 0.1), (1.0, 0.2) Ͱɼઌߦݚڀͷ Attention-Guided GAN ͰݟΒΕͨ എܠͷ্ͷࣶ༷มͷӨڹ͕΄ͱΜͲແ͘ͳ͓ͬͯΓɼͦΕҎ߱·ͨผͷ Өڹ͕ग़͍ͯΔ͜ͱ͕؍Ͱ͖Δɽද 5.3 Ͱಉ༷ʹɼճస֯༧ଌλεΫಋೖޙɼ Attention ͕มରͷഅͷܗʹͳ͓ͬͯΓɼൺֱతγϚϚͷม͕Ͱ͖͍ͯΔ Օॴ͕ݟड͚ΒΕͨɽɹ ଓ͍ͯ apple2orange Ͱͷม݁ՌΛࣔͨ͠ද 5.4 Ͱɼ(λD , λG ) = (0.5, 0.1) ͷͱ ͖ɼઌߦݚڀͷ Attention-Guided GAN ͷͱ͖ͷ݁ՌͰݟΒΕͳ͔ͬͨԞͷ෦ ͷ Attention Mask ͕֬ೝ͞Ε͓ͯΓɼԞͷΞοϓϧ͕ը૾มͰ͖͍ͯΔ͜ͱ͕ݟ ͯऔΕΔɽҰํɼද 5.5 ͷΑ͏ʹɼճస֯༧ଌλεΫಋೖલͱಋೖޙͰ͋·Γ มԽ͕ݟΒΕͳ͔ͬͨͷ֬ೝ͞Εͨɽ Γͷσʔληοτͷը૾มʹ͍ͭͯɼmap to photo ͷҰྫΛද 5.6ɼݐ֎؍ -
ୈ 5 ষ ࣮ݧ݁Ռͱߟ 40 ͔ΒϥϕϧͷมͷҰྫΛද 5.7ɼࢢܠ؍͔ΒηάϝϯςʔγϣϯͷมͷҰ ྫΛද 5.8 ʹࣔ͢ɽ·ͣ
map2photo ͰઌߦݚڀͱൺͯɼͲͷճస֯ύϥϝʔλ Attnetion ͱੜը૾ʹ͋·ΓมԽ͕ݟΒΕͳ͔ͬͨɽଓ͘ facades ɼಛʹද 5.7 ͷΑ͏ʹɼճస֯ύϥϝʔλ͕ (λD , λG ) = (1.5, 0.3) ͷͱ͖ͷੜը૾ͷϥϕϧཱ͕ ମతʹͳ͍ͬͯΔ͜ͱ͕֬ೝͰ͖Δɽcityscapes Ͱɼද 5.8 ͷΑ͏ʹɼंݐͱ ͍ͬͨମ͔Βϥϕϧͷม͕ઌߦݚڀͷ Attention-Guided GAN ΑΓൺֱతਖ਼֬ ʹߦΘΕ͓ͯΓɼ֤ϥϕϧ͕ंݐͷܗʹͳ͍ͬͯΔ͜ͱ͕ݟͯऔΕΔɽͨͩ͠ ճస֯ύϥϝʔλ͕ (λD , λG ) = (1.0, 0.2) ͷͱ͖ɼAttention Mask ʹ̏ͭͷന͍͕ ֬ೝ͞Ε͍ͯΔɽ͜ΕΒ Attention Mask ͱରԠ͢Δੜը૾ͰมԽ͍ͯ͠ͳ͍෦ ͱͳ͍ͬͯΔɽ horse2zebra ͱ apple2orange ʹ͓͚Δੜը૾ࣗମͷఆੑతධՁʹ͍ͭͯड़Δɽ ·ͣճస֯ύϥϝʔλ͝ͱʹ horse2zebra ͷੜը૾ΛαϯϓϦϯάͨ͠ͷΛਤ 5.1, ਤ 5.2, ਤ 5.3, ਤ 5.4, ਤ 5.5 ʹͯࣔ͢ɽ͜ΕΒͷαϯϓϦϯάͨ͠ੜը૾Λൺ ֱ͢Δͱɼઌߦݚڀͷ Attention-Guided GAN Ͱͷมը૾ΑΓഎܠͷӨڹ͕վળ ͞Εͨͷ͕΄ͱΜͲͷੜը૾Ͱݟड͚ΒΕͨɽͨͩ͠Ͳͷճస֯ύϥϝʔλͰ എܠͷӨڹ͕վળ͞Ε͔ͨɼੜը૾ʹΑͬͯҟͳΔ͜ͱ͕֬ೝ͞Εͨɽͦͷ ҰํͰɼಉ༷ʹઌߦݚڀͷ Attention-Guided GAN Ͱͷมը૾ΑΓഎܠͷӨڹ͕ ֦େ͞Εͨ෦ɼࣶ༷͕ରͱͳΔ෦ʹ͍͍ͭͯͳ͍ͱ͍ͬͨ෦ݟड͚ ΒΕͨɽ ଓ͍ͯճస֯ύϥϝʔλ͝ͱʹ apple2orange ͷੜը૾ΛαϯϓϦϯάͨ͠ͷ Λਤ 5.6, ਤ 5.7, ਤ 5.8, ਤ 5.9, ਤ 5.10 ʹͯࣔ͢ɽαϯϓϦϯάͨ͠ੜը૾Λൺͯɼ தʹશ͘มԽ͠ͳ͍ੜը૾ଘࡏ͕ͨ͠ɼجຊతʹઌߦݚڀͷ Attention-Guided GAN Ͱͷੜը૾ͱൺɼճస֯ύϥϝʔλ͕ (λD , λG ) = (0.5, 0.1), (1.5, 0.3) ͷͱ͖ ʹରͱͳΔ෦શମͷม͕͞Ε͍ͯΔը૾͕ଟ͘ݟΒΕͨɽͦΕʹՃ͑ͯ ͍Ξοϓϧͱɼผͷ৭ͷΞοϓϧΞοϓϧͷதΛ۠ผͯ͠ม͢Δ͜ͱ͕Ͱ͖ ͍ͯΔ͜ͱ֬ೝͰ͖Δɽɹ ճస֯ύϥϝʔλ͝ͱʹ map2photo ٴͼ facades ٴͼ cityscapes ͷੜը૾Λαϯ ϓϦϯάͨ͠ͷΛͦΕͧΕࣔ͢ɽmap2photo ਤ 5.11, ਤ 5.12, ਤ 5.13, ਤ 5.14, ਤ -
ୈ 5 ষ ࣮ݧ݁Ռͱߟ 41 5.15 ʹɼfacades ਤ 5.16, ਤ
5.17, ਤ 5.18, ਤ 5.19, ਤ 5.20 ʹɼcityscapes ਤ 5.21, ਤ 5.22, ਤ 5.23, ਤ 5.24, ਤ 5.25 ʹͯࣔ͢ɽ·ͣ map2photo Ͱɼճస֯ύϥϝʔλ ʹΑͬͯը૾ͷ৭͕มΘ͍ͬͯΔՕॴݟΒΕ͕ͨɼҐஔؔʹؔͯ͠มԽ ݟΒΕͳ͔ͬͨɽ࣍ʹ facades Ͱɼճస֯ύϥϝʔλ͕ (λD , λG ) = (1.5, 0.3) ͷͱ ͖ϥϕϧ͕มݩͷݐ֎؍ͷΑ͏ʹཱମతʹͳ͍ͬͯΔͷ͕ଟ֬ೝ͞Εͨɽ ͳ͓ (λD , λG ) = (1.5, 0.3) Ҏ֎ͷճస֯ύϥϝʔλʹ͓͍ͯɼઌߦݚڀͷ Attention- Guided GAN Ͱͷ݁Ռͱ͋·ΓมԽ͕ݟΒΕͳ͔ͬͨɽ࠷ޙʹ cityscapes Ͱɼಛ ʹճస֯ύϥϝʔλ͕ (λD , λG ) = (1.0, 0.2), (2.0, 0.4) ͷΑ͏ʹ੨ͷंͷϥϕϧɼ ͷݐͷϥϕϧ͕ΑΓਖ਼֬ʹੜը૾ʹө͞Ε͍ͯΔ෦͕ݟΒΕͨɽͦͷҰํ Ͱɼઌߦݚڀͷ Attention-Guided GAN ͰݟΒΕ͍ͨਓӨͷ෦ൺֱతݮগͯ͠ ͍ͨɽͦΕʹՃ͑ͯɼճస֯ύϥϝʔλ͕ (λD , λG ) = (0.5, 0.1), (1.5, 0.3) ͷͱ͖ɼ ઌߦݚڀͷ Attention-Guided GAN ͷੜը૾ͱൺͯϐϯΫͷ෦͕૿Ճ͍ͯ͠Δ ͜ͱ͕֬ೝ͞Εͨɽ 5.2 ߟ 5.2.1 ఆྔతධՁ SSAttention-Guided GAN Ͱੜ͞Εͨը૾ɼઌߦݚڀͷ Attention-Guided GAN ͷੜը૾ͱൺֱͯ͠ɼύϥϝʔλʹΑͬͯ FID ͷΛվળͰ͖Δɽಛʹද 5.1 ʹ͓͍ͯɼhorse to zebraɼapple to orange ͱʹճస֯ύϥϝʔλ (λD , λG ) = (0.5, 0.1) ͷͱ͖͕࠷͍ͱͳ͍ͬͯΔɽ͕ͨͬͯ͠ɼճస֯༧ଌλεΫ FID ͷվ ળʹߩݙ͓ͯ͠Γɼ(λD , λG ) = (0.5, 0.1) ͷͱ͖͕࠷దͳճస֯ύϥϝʔλͰ͋Δͱ ߟ͑ΒΕΔɽ 5.2.2 ఆੑతධՁ horse2zebraͷఆੑతධՁͰɼද5.2ɼද5.3ͷΑ͏ʹઌߦݚڀͷAttention-Guided GAN ͰഎܠΛר͖ࠐΜͩੜը૾͔ΒɼSSAttention-Guided GAN Ͱͷੜը૾Ͱ -
ୈ 5 ষ ࣮ݧ݁Ռͱߟ 42 എܠͷӨڹ͕վળͨ͠ࣄྫ͕ݟΒΕͨɽͦΕʹ͍ɼAttention ݩͷഅͷ෦ ʹ͍ۙܗʹͳ͓ͬͯΓɼճస֯༧ଌλεΫʹΑͬͯɼݩͷഅͷزԿతಛҐஔ ؔΛ͖ͪΜͱѲͰ͖͍ͯΔͱߟ͑ΒΕΔɽ ͨͩ͠ɼճస֯ύϥϝʔλʹΑ֤ͬͯมը૾ͷมͷਫ਼͕ͦΕͧΕҧͬͯ
͍Δ͜ͱ͕֬ೝ͞Ε͓ͯΓɼੜը૾ʹΑͬͯదͳճస֯ύϥϝʔλҟͳΔ͜ ͱΛҙຯ͍ͯ͠Δͱߟ͑ΒΕΔɽ࣮ࡍʹഅ͕ॏͳ͍ͬͯͨΓɼਓؒͱ͍ͬͨഅҎ֎ ͷମ͕Ҡ͍ͬͯΔը૾ɼҰ؏ࣶ༷͕ͯ͠͏·͍͍ͭͯ͘ͳ͔ͬͨɽ apple2orangeͷఆੑతධՁʹ͓͍ͯɼ ճస֯ύϥϝʔλ (λD , λG ) = (0.5, 0.1), (1.5, 0.3) ͷͱ͖Ξοϓϧͷ৭ΛΦϨϯδʹ͢Δ texture ม҆ఆ্ͯ͠ख͘ߦΘΕ͍ͯΔɽ ͦΕʹՃ͑ͯผͷ৭ͷΞοϓϧΞοϓϧͷஅ໘ɼ༿ͳͲΛ۠ผͯ͠ը૾มΛߦͬ ͍ͯΔ͜ͱ͔Βɼճస֯ͷ༧ଌʹΑͬͯΞοϓϧͷಛ͕ѲͰ͖͍ͯΔͱߟ͑ ΒΕΔɽͳ͓มݩͷΞοϓϧͷதͰɼΦϨϯδ৭ʹ͍ۙͷ͕ଘࡏ͍ͯ͠Δ͕ɼ ͦΕΒ͋·ΓมԽ͕ݟΒΕͳ͔ͬͨɽཧ༝ͱͯ͠ɼ͍ΞοϓϧΛֶश͍ͯ͠ ΔͨΊɼผͷ৭ͷΞοϓϧͱೝࣝͯ͠͠·͍ͬͯΔ͜ͱ͕ߟ͑ΒΕΔɽ ͦͷଞͷσʔληοτʹ͍ͭͯɼ·ͣ map2photo ͰઌߦݚڀͱൺɼͲͷճ స֯ύϥϝʔλ͋·ΓมԽ͕ݟΒΕͳ͔ͬͨɽݪҼͱͯ͠ɼมݩͷਤͷํ ֯ͱมઌͷӴࣸਅͷը૾ͷํ֯Λͦͷ··ʹͯ͠ը૾ม͢ΔͨΊɼվΊͯ ճస֯༧ଌΛͯ͠ํ֯ͱ͍ͬͨಛΛѲ͢Δඞཁ͕ͳ͍ͱ͍͏͜ͱ͕ڍ͛ΒΕ Δɽfacades ಛʹද 5.7 ͷΑ͏ʹɼ(λD , λG ) = (1.5, 0.3) ͷੜը૾ͷϥϕϧཱ͕ମ తʹͳ͍ͬͯΔ͜ͱ͕ଟ֬ೝ͞Ε͍ͯͨɽcityscapes Ͱઌߦݚڀͱൺɼंݐ ϥϕϧ͕ΑΓਖ਼֬ʹ͍͍ͭͯΔՕॴ͕ଘࡏ͍ͯ͠Δɽ͜ΕΒͷ݁ՌΑΓɼfacades cityscapes σʔληοτʹ͍ͭͯճస֯༧ଌλεΫΛಋೖ͢Δ͜ͱͰը૾ ͷಛΛ͔ͭΉ͜ͱ͕Ͱ͖͍ͯΔͱߟ͑ΒΕΔɽ 5.2.3 ఆྔతධՁͱఆੑతධՁͷൺֱ horse2zebra ͷఆྔతධՁͱఆੑతධՁΛൺֱ͢Δͱɼੜը૾ͱ FID ͷʹ૬ ؔੑͳ͘ɼFID ͷͷେ͖͞ʹ͔͔ΘΒͣɼੜը૾ͷਫ਼͕ͦΕͧΕҟͳͬ -
ୈ 5 ষ ࣮ݧ݁Ռͱߟ 43 ͍ͯΔ͜ͱ͕͔Δɽྫ͑ horse2zebra ͷճస֯ύϥϝʔλ (λD ,
λG ) = (2.0, 0.4) ͷ ͱ͖ͷ FID ઌߦݚڀͱൺͯվળ͞Ε͍͕ͯͨɼද 5.2 ͱরΒ͠߹ΘͤͯݟͯΈ Δͱɼ৽ͨʹม͍ͨ͠ରҎ֎ͷ෦Λר͖ࠐΜͰ͠·͍ͬͯΔͱ͍ͬͨࣄྫ ֬ೝ͞Εͨɽ͜ͷݪҼͷҰͭͱͯ͠ɼhorse2zebra σʔληοτͷதʹਓؒݪ ͕͍ࣸͬͯͨΓɼഅ͕ॏͳ͍ͬͯͨΓͯ͠ɼ࣮ࡍʹഅ͔ΒγϚϚͷม্͕ख ͍͔͘ͳ͔ͬͨͷͷଘࡏ͕ڍ͛ΒΕΔɽhorse2zebra σʔληοτதͷਓؒݪɼ ̎ͭͷॏͳͬͨഅͷֶश͕ FID ʹӨڹΛ༩͓͑ͯΓɼੜը૾ͷਫ਼ʹ͕ࠩग़͍ͯ ΔͷͰͳ͍͔ͱߟ͑ΒΕΔɽ ରͯ͠ɼapple2orange ͰɼFID ͷ͕ྑ͍ճస֯ύϥϝʔλ΄ͲఆੑతධՁͷ ը૾ม্͕ख͘ߦΘΕ͍ͯͨɽ ྫ͑ճస֯ύϥϝʔλ (λD , λG ) = (0.5, 0.1), (1.5, 0.3), (2.0, 0.4) ͷͱ͖ͷ FID ઌߦݚڀͱൺͯྑ͍Ͱ͋ΓɼఆੑతධՁ͍Ξοϓ ϧ͔ΒΦϨϯδͷΞοϓϧͷม্͕ख͘Ͱ͖͍ͯͨɽ͕ͨͬͯ͠ɼapple2orange ʹ͓͍ͯ FID ͱੜը૾ͷ૬ؔੑ΄ͱΜͲݟΒΕΔͱ͚݁ΒΕΔɽ -
ୈ 5 ষ ࣮ݧ݁Ռͱߟ 44 ද 5.2 horse2zebra σʔληοτͰͷ horse
to zebra ม݁Ռʢ̍ʣ ɹ (λD , λG) horse zebra Attention Mask (0.0, 0.0) (0.5, 0.1) (1.0, 0.2) (1.5, 0.3) (2.0, 0.4) ’ -
ୈ 5 ষ ࣮ݧ݁Ռͱߟ 45 ද 5.3 horse2zebra σʔληοτͰͷ horse
to zebra ม݁Ռʢ̎ʣ ɹ (λD , λG) horse zebra Attention Mask (0.0, 0.0) (0.5, 0.1) (1.0, 0.2) (1.5, 0.3) (2.0, 0.4) -
ୈ 5 ষ ࣮ݧ݁Ռͱߟ 46 ද 5.4 apple2orange σʔληοτͰͷ apple
to orange ม݁Ռ ʢ̍ʣ (λD , λG) apple orange Attention Mask (0.0, 0.0) (0.5, 0.1) (1.0, 0.2) (1.5, 0.3) (2.0, 0.4) -
ୈ 5 ষ ࣮ݧ݁Ռͱߟ 47 ද 5.5 apple2orange σʔληοτͰͷ apple
to orange ม݁Ռ ʢ̎ʣɹ (λD , λG) apple orange Attention Mask (0.0, 0.0) (0.5, 0.1) (1.0, 0.2) (1.5, 0.3) (2.0, 0.4) -
ୈ 5 ষ ࣮ݧ݁Ռͱߟ 48 ද 5.6 map2photo σʔληοτͰͷ map
to photo ม݁Ռ (λD , λG) map photo Attention Mask (0.0, 0.0) (0.5, 0.1) (1.0, 0.2) (1.5, 0.3) (2.0, 0.4) -
ୈ 5 ষ ࣮ݧ݁Ռͱߟ 49 ද 5.7 facades σʔληοτΛ༻͍ͨݐ֎؍͔Βϥϕϧͷม ݁Ռ
(λD , λG) ݐ֎؍ ϥϕϧ Attention Mask (0.0, 0.0) (0.5, 0.1) (1.0, 0.2) (1.5, 0.3) (2.0, 0.4) -
ୈ 5 ষ ࣮ݧ݁Ռͱߟ 50 ද 5.8 cityscapesσʔληοτΛ༻͍ͨࢢܠ؍͔Βηάϝϯςʔ γϣϯͷม݁Ռ (λD
, λG) ࢢܠ؍ ηάϝϯςʔγϣϯ Attention Mask (0.0, 0.0) (0.5, 0.1) (1.0, 0.2) (1.5, 0.3) (2.0, 0.4) -
ୈ 5 ষ ࣮ݧ݁Ռͱߟ 51 ਤ 5.1 ճస֯ύϥϝʔλ (λD ,
λG ) = (0.0, 0.0) ͷͱ͖ͷ horse2zebra ͷੜը૾ ਤ 5.2 ճస֯ύϥϝʔλ (λD , λG ) = (0.5, 0.1) ͷͱ͖ͷ horse2zebra ͷੜը૾ -
ୈ 5 ষ ࣮ݧ݁Ռͱߟ 52 ਤ 5.3 ճస֯ύϥϝʔλ (λD ,
λG ) = (1.0, 0.2) ͷͱ͖ͷ horse2zebra ͷੜը૾ ਤ 5.4 ճస֯ύϥϝʔλ (λD , λG ) = (1.5, 0.3) ͷͱ͖ͷ horse2zebra ͷੜը૾ -
ୈ 5 ষ ࣮ݧ݁Ռͱߟ 53 ਤ 5.5 ճస֯ύϥϝʔλ (λD ,
λG ) = (2.0, 0.4) ͷͱ͖ͷ horse2zebra ͷੜը૾ ਤ 5.6 ճస֯ύϥϝʔλ (λD , λG ) = (0.0, 0.0) ͷͱ͖ͷ ap- ple2orange ͷੜը૾ -
ୈ 5 ষ ࣮ݧ݁Ռͱߟ 54 ਤ 5.7 ճస֯ύϥϝʔλ (λD ,
λG ) = (0.5, 0.1) ͷͱ͖ͷ ap- ple2orange ͷੜը૾ ਤ 5.8 ճస֯ύϥϝʔλ (λD , λG ) = (1.0, 0.2) ͷͱ͖ͷ ap- ple2orange ͷੜը૾ -
ୈ 5 ষ ࣮ݧ݁Ռͱߟ 55 ਤ 5.9 ճస֯ύϥϝʔλ (λD ,
λG ) = (1.5, 0.3) ͷͱ͖ͷ ap- ple2orange ͷੜը૾ ਤ 5.10 ճస֯ύϥϝʔλ (λD , λG ) = (2.0, 0.4) ͷͱ͖ͷ ap- ple2orange ͷੜը૾ -
ୈ 5 ষ ࣮ݧ݁Ռͱߟ 56 ਤ 5.11 ճస֯ύϥϝʔλ (λD ,
λG ) = (0.0, 0.0) ͷͱ͖ͷ map2photo ͷੜը૾ ਤ 5.12 ճస֯ύϥϝʔλ (λD , λG ) = (0.5, 0.1) ͷͱ͖ͷ map2photo ͷੜը૾ -
ୈ 5 ষ ࣮ݧ݁Ռͱߟ 57 ਤ 5.13 ճస֯ύϥϝʔλ (λD ,
λG ) = (1.0, 0.2) ͷͱ͖ͷ map2photo ͷੜը૾ ਤ 5.14 ճస֯ύϥϝʔλ (λD , λG ) = (1.5, 0.3) ͷͱ͖ͷ map2photo ͷੜը૾ -
ୈ 5 ষ ࣮ݧ݁Ռͱߟ 58 ਤ 5.15 ճస֯ύϥϝʔλ (λD ,
λG ) = (2.0, 0.4) ͷͱ͖ͷ map2photo ͷੜը૾ ਤ 5.16 ճస֯ύϥϝʔλ (λD , λG ) = (0.0, 0.0) ͷͱ͖ͷ facades ͷ ੜը૾ -
ୈ 5 ষ ࣮ݧ݁Ռͱߟ 59 ਤ 5.17 ճస֯ύϥϝʔλ (λD ,
λG ) = (0.5, 0.1) ͷͱ͖ͷ facades ͷ ੜը૾ ਤ 5.18 ճస֯ύϥϝʔλ (λD , λG ) = (1.0, 0.2) ͷͱ͖ͷ facades ͷ ੜը૾ -
ୈ 5 ষ ࣮ݧ݁Ռͱߟ 60 ਤ 5.19 ճస֯ύϥϝʔλ (λD ,
λG ) = (1.5, 0.3) ͷͱ͖ͷ facades ͷ ੜը૾ ਤ 5.20 ճస֯ύϥϝʔλ (λD , λG ) = (2.0, 0.4) ͷͱ͖ͷ facades ͷ ੜը૾ -
ୈ 5 ষ ࣮ݧ݁Ռͱߟ 61 ਤ 5.21 ճస֯ύϥϝʔλ (λD ,
λG ) = (0.0, 0.0) ͷͱ͖ͷ cityscapes ͷੜը૾ ਤ 5.22 ճస֯ύϥϝʔλ (λD , λG ) = (0.5, 0.1) ͷͱ͖ͷ cityscapes ͷੜը૾ -
ୈ 5 ষ ࣮ݧ݁Ռͱߟ 62 ਤ 5.23 ճస֯ύϥϝʔλ (λD ,
λG ) = (1.0, 0.2) ͷͱ͖ͷ cityscapes ͷੜը૾ ਤ 5.24 ճస֯ύϥϝʔλ (λD , λG ) = (1.5, 0.3) ͷͱ͖ͷ cityscapes ͷੜը૾ -
ୈ 5 ষ ࣮ݧ݁Ռͱߟ 63 ਤ 5.25 ճస֯ύϥϝʔλ (λD ,
λG ) = (2.0, 0.4) ͷͱ͖ͷ cityscapes ͷੜը૾ -
ୈ 6 ষ ·ͱΊ 64 ୈ6ষ ·ͱΊ ຊݚڀͰઌߦݚڀͷ Attention-Guided GAN
ͷ Discriminator ʹճస֯༧ଌλ εΫΛՃͨ͠ SSAttention-Guided GAN ͷϓϩάϥϜΛߏங͠ɼੜը૾ͷࣝผਫ਼ ্ΛࢼΈͨɽઌߦݚڀͷ Attention-Guided GAN ͱ SSAttention-Guided GAN ͷ ੜը૾ͱ Attention Mask Λൺֱ͠ɼมରͷಛ͕ Attention ੜը૾ʹө ͞Ε͍ͯΔ͔ʹ͍ͭͯఆྔతධՁͱఆੑతධՁΛߦͬͨɽ FID Λ༻͍ͨఆྔతධՁͰɼઌߦݚڀͷ Attention-Guided GAN ͷͱ͖ΑΓ FID ͷݮগ͕֬ೝ͞ΕͨɽຊݚڀͰɼhorse2zebra ͱ apple2orange ͷ྆ํͱճస֯ύ ϥϝʔλ (λD , λG ) = (0.5, 0.1) ͷͱ͖ʹ࠷վળ͢Δ͕͋Δ͜ͱ͕֬ೝ͞Εͨɽ ఆੑతධՁͰɼઌߦݚڀͷ Attention-Guided GAN ͷͱ͖ʹ؍͞Εͨͳ At- tention Mask ͱੜը૾͕ճస֯༧ଌλεΫͷಋೖޙʹվળͨ͠ࣄྫ͕ଟݟड ͚ΒΕͨɽhorse2zebra Ͱɼճస֯༧ଌλεΫͷಋೖޙʹରҎ֎ͷഎܠͱ͍ͬ ͨ෦ͷม͕ݮগ͠ɼͦͷͱ͖ͷ Attention Mask അͷܗʹ͍ͬͯΔͷ͕ ֬ೝ͞Εͨɽapple2orange ɼઌߦݚڀͷ Attention-Guided GAN ͷͱ͖ͷੜը૾ Ͱ্ख͘ΦϨϯδʹม͠ͳ͔͍ͬͨΞοϓϧ͕ɼճస֯༧ଌλεΫͷಋೖ ޙʹશʹΦϨϯδ৭ʹมͰ͖͍ͯΔͷݟΒΕͨɽ apple2orange Ͱ FID ͷ͕খ͍͞΄Ͳɼੜը૾ͷ͍Ξοϓϧ͕ΦϨϯδ৭ ʹ্ख͘ม͞ΕΔ͕ݟΒΕͨɽ͜ͷ݁Ռ͔Βɼapple2orange ʹ͓͚Δ FID ͷ ɼੜը૾ͷਫ਼ʹө͞ΕΔ͜ͱ͕ߟ͑ΒΕΔɽ͜Εʹରͯ͠ɼhorse2zebra ɼFID ͷʹؔΘΒͣɼੜը૾ͷਫ਼ͦΕͧΕҟͳΔ͜ͱ͕֬ೝ͞Εͨɽ͜ ͷཁҼͷҰͭͱͯ͠ɼhorse2zebra σʔληοτͷதʹਓؒݪɼ2 ͭͷॏͳͬ ͨഅ͕ࠞࡏ͓ͯ͠Γɼ͜ΕΒͷը૾ͷֶश͕ੜը૾ͷਫ਼ FID ʹӨڹΛ༩͑ͯ ͍Δͱ͍͏͜ͱ͕ڍ͛ΒΕΔɽຊݚڀͰɼճస֯ͷ༧ଌͷՃͷ༗ແʹ͔͔Θ -
ࢀߟจݙ 67 ࢀߟจݙ ʦ1ʧ ಠཱߦ๏ਓใॲཧਪਐػߏ AI നॻฤूҕһձɿAI നॻ 2019, KADOKAWA
(2019). ʦ2ʧ Gui, J., Sun, Z., Wen, Y., Tao, D. and Ye, J.: A Review on Generative Adversarial Networks: Algorithms, Theory, and Applications, CoRR, Vol. abs/2001.06937, (2020). ʦ3ʧ Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A. and Bengio, Y.: Generative Adversarial Nets, in Ghahramani, Z., Welling, M., Cortes, C., Lawrence, N. and Weinberger, K. Q. eds., Advances in Neural Information Processing Systems, Vol. 27, Curran Associates, Inc. (2014). ʦ4ʧ Radford, A., Metz, L. and Chintala, S.: Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks, in Bengio, Y. and LeCun, Y. eds., 4th International Conference on Learning Representations, ICLR 2016, San Juan, Puerto Rico, May 2-4, 2016, Conference Track Proceedings (2016). ʦ5ʧ Mirza, M. and Osindero, S.: Conditional Generative Adversarial Nets, CoRR, Vol. abs/1411.1784, (2014). ʦ6ʧ Knyaz, V. A., Kniaz, V. V. and Remondino, F.: Image-to-Voxel Model Transla- tion with Conditional Adversarial Networks, in Leal-Taix´ e, L. and Roth, S. eds., Computer Vision - ECCV 2018 Workshops - Munich, Germany, September 8-14, 2018, Proceedings, Part I, Vol. 11129 of Lecture Notes in Computer Science, pp. 601–618, Springer (2018). -
ࢀߟจݙ 68 ʦ7ʧ Zhu, J., Park, T., Isola, P. and
Efros, A. A.: Unpaired Image-to-Image Translation Using Cycle-Consistent Adversarial Networks, in IEEE International Conference on Computer Vision, ICCV 2017, Venice, Italy, October 22-29, 2017, pp. 2242– 2251, IEEE Computer Society (2017). ʦ8ʧ Tang, H., Xu, D., Sebe, N. and Yan, Y.: Attention-Guided Generative Adversarial Networks for Unsupervised Image-to-Image Translation, in International Joint Conference on Neural Networks, IJCNN 2019 Budapest, Hungary, July 14-19, 2019, pp. 1–8, IEEE (2019). ʦ9ʧ Tang, H., Liu, H., Xu, D., Torr, P. H. S. and Sebe, N.: AttentionGAN: Unpaired Image-to-Image Translation using Attention-Guided Generative Adversarial Net- works, CoRR, Vol. abs/1911.11897, (2019). ʦ10ʧ Chen, T., Zhai, X., Ritter, M., Lucic, M. and Houlsby, N.: Self-Supervised GANs via Auxiliary Rotation Loss, in IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2019, Long Beach, CA, USA, June 16-20, 2019, pp. 12154– 12163, Computer Vision Foundation / IEEE (2019). ʦ11ʧ Mao, X., Li, Q., Xie, H., Lau, R. Y. K., Wang, Z. and Smolley, S. P.: Least Squares Generative Adversarial Networks, in IEEE International Conference on Computer Vision, ICCV 2017, Venice, Italy, October 22-29, 2017, pp. 2813–2821, IEEE Computer Society (2017). ʦ12ʧ ࡈ౻߁ؽɿθϩ͔Β࡞Δ Deep Learning - Python ͰֶͿσΟʔϓϥʔχϯά ͷཧͱ࣮, ΦϥΠϦʔɾδϟύϯ (2017). ʦ13ʧ C.M. Ϗγϣ οϓɿύλʔϯೝࣝͱػցֶश ্ ϕΠζཧʹΑΔ౷ܭత༧ଌ, ؙળग़൛ (2014). ʦ14ʧ Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L. u. and Polosukhin, I.: Attention is All you Need, in Guyon, I., -
ࢀߟจݙ 69 Luxburg, U. V., Bengio, S., Wallach, H., Fergus,
R., Vishwanathan, S. and Gar- nett, R. eds., Advances in Neural Information Processing Systems, Vol. 30, Curran Associates, Inc. (2017). ʦ15ʧ Long, J., Shelhamer, E. and Darrell, T.: Fully Convolutional Networks for Se- mantic Segmentation, CoRR, Vol. abs/1411.4038, (2014). ʦ16ʧ Yin, L., Wei, X., Sun, Y., Wang, J. and Rosato, M. J.: A 3D Facial Expression Database For Facial Behavior Research, in Seventh IEEE International Confer- ence on Automatic Face and Gesture Recognition (FGR 2006), 10-12 April 2006, Southampton, UK, pp. 211–216, IEEE Computer Society (2006). ʦ17ʧ Gidaris, S., Singh, P. and Komodakis, N.: Unsupervised Representation Learn- ing by Predicting Image Rotations, in 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 - May 3, 2018, Conference Track Proceedings, OpenReview.net (2018). ʦ18ʧ He, K., Zhang, X., Ren, S. and Sun, J.: Deep Residual Learning for Image Recog- nition, CoRR, Vol. abs/1512.03385, (2015). ʦ19ʧ Heusel, M., Ramsauer, H., Unterthiner, T., Nessler, B. and Hochreiter, S.: GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilib- rium, in Guyon, I., Luxburg, von U., Bengio, S., Wallach, H. M., Fergus, R., Vishwanathan, S. V. N. and Garnett, R. eds., Advances in Neural Information Pro- cessing Systems 30: Annual Conference on Neural Information Processing Sys- tems 2017, December 4-9, 2017, Long Beach, CA, USA, pp. 6626–6637 (2017).