Share
Embed
Transcript
27 slides
-
Πϯυྉཧաఔ ʲແݶߦྻͷੜʳ ɹ·ͣɼ ͷόΠφϦߦྻ Λߟ͑ɼ ͱͳΔ߹ͷ ͷੜաఔΛߏங͢ Δɽ·ͨɼ֤ཁૉ ϕϧψʔΠ ͔Βੜ͞ΕΔͱ͢Δɽ͞Β
ʹϋΠύʔύϥϝʔλ Λ༻͍ͯɼύϥϝʔλ ͕ϕʔλ ͔Βੜ͞Ε͍ͯΔͱͨ͠Βɼߦྻ ͷ࣍ͷεϥΠυͷΑ͏ʹॻ͚Δɽ N × H M H → ∞ M mn,h ∈ {0,1} Bern(πh ) α > 0,β > 0 πh Beta(αβ/H, β) M p(πh ) = Beta(αβ/H, β) = Γ ( αβ H + β) Γ ( αβ H ) + Γ(β) π αβ H −1 h (1 − πh )β−1 p(mn,h |πh ) = Bern(πh ) = πmn,h h (1 − πh )1−mn,h πh α β mn,h n = 1,2,…, N h = 1,2,…, H -
Πϯυྉཧաఔ ʲແݶߦྻͷੜʳ ͱ͢Δͱ ͱͳΓɼશͯͷόΠφϦߦྻͷੜ͕֬ʹͳͬͯ͠· ͏ɽ p(M) =
H ∏ h=1 ∫ p(πh ) { N ∏ n=1 p(mn,h |πh ) } dπh = H ∏ h=1 ∫ Γ ( αβ H + β) Γ ( αβ H ) + Γ(β) π αβ H −1 h (1 − πh )β−1 { N ∏ n=1 πmn,h h (1 − πh )1−mn,h } dπh = H ∏ h=1 Γ ( αβ H + β) Γ ( αβ H ) + Γ(β) ∫ πNh + αβ H −1 h (1 − πh )N−Nh +β−1dπh ( ∵ Nh = N ∑ n=1 mn,h ) = H ∏ h=1 Γ ( αβ H + β) Γ ( αβ H ) + Γ(β) Γ (Nh + αβ H ) Γ(N − Nh + β) Γ ( αβ H + β + N) H → ∞ p(M) → 0 ( ∵ Beta(x, y) = Γ(x)Γ(y) Γ(x + y) < 1 where x > 1,y > 1) -
Πϯυྉཧաఔ ʲແݶߦྻͷੜʳ ɹੜ͕֬ʹͳΔ͜ͱΛ͙ͨΊʹɼ ͷྻΛฒͼସ͑Δʢlofʣ͜ͱͰಉ͡ʹͳΔ Α͏ͳߦྻͷಉྨΛ ͱ͓͘ɽ ྫɿ ͷͱ͖ɼ
ɹ ʹରͯ͠ ͱͨ͠ͱ͖ͷͷܭࢉจݙ<>ΑΓɼ࣍ͷεϥΠυͷΑ͏ ʹॻ͚Δɽ M [M] M = ( 1 0 0 0 1 0 0 0 1 ) [M] ∈ ( 1 0 0 0 1 0 0 0 1 ) , ( 0 1 0 1 0 0 0 0 1 ) , ( 0 0 1 0 1 0 1 0 0 ) , ( 1 0 0 0 0 1 0 1 0 ) p([M]) H → ∞ [1] “Infinite Latent Feature Models and the Indian Buffet Process”, 2018 -
Πϯυྉཧաఔ ʲແݶߦྻͷੜʳ ɹɹɹɹɹɹɹ ͨͩ͠ɼ ʹ͋ΔόΠφϦྻʢྻϕΫτϧ͕ಉ͡ͳΒಉ͡ʣͷݸͰɼಉ ͡όΠφϦྻͷฒͼସ͑ʹΑΔॏෳΛΩϟϯηϧ͢ΔͨΊʹׂΔɽ·ͨɼ
ͱͳΔΑ͏ͳྻ ͷݸɽ ͷظɽɹɹɹɹɹɹɹ ɹ͜ͷ ͷߦΛަͯ͠มΘΒͳ͍ͷͰɹަՄೳੑɹΛ࣋ͭɽ p([M]) = ∑ M∈[M] p(M) = H! ∏ i≥1 Hi ! H ∏ h=1 Γ ( αβ H + β) Γ ( αβ H ) + Γ(β) Γ (Nh + αβ H ) Γ(N − Nh + β) Γ ( αβ H + β + N) → (αβ)H+ ∏ i≥1 Hi ! exp(− ¯ H+ ) H+ ∏ h=1 Γ (Nh) Γ(N − Nh + β) Γ (β + N) Hi M i i i H+ Nh > 0 h ¯ H+ = α N ∑ n=1 β n + β − 1 H+ M -
Πϯυྉཧաఔ ʲແݶߦྻͷੜʳ ɹແݶྻΛ࣋ͭόΠφϦߦྻͷੜɼΠϯυྉཧաఔͷखଓ͖ʹΑͬͯɼ࣍ͷΑ͏ʹ ߦ͏͜ͱ͕Ͱ͖Δɽ ɹ࠷ॳʹྉཧళʹདྷͨ٬ɼҎԼͷϙΞιϯʹैͬͯྉཧΛͱΔɽ ɹO൪ʹདྷͨ٬ɼ֬ ʹै֤ͬͯྉཧ ΛͱΓɼ࠷ޙʹ
ʹैͬͯ৽͍͠ྉཧΛͱΔɽ Poi(α) = αx x! e−α Nh n + β − 1 h Poi ( αβ n + β − 1) ΠϯυྉཧաఔʢIBPʣͷखଓ͖ -
Πϯυྉཧաఔ ʲແݶߦྻͷੜʳ ɹੜ͞ΕΔόΠφϦߦྻ ͷಛੑ w٬ਓ͋ͨΓͷྉཧͷ ʹै͏ɽ wͱΒΕΔྉཧͷ૯ͷظ w٬ʹऔΒΕΔྉཧͷछྨͷ߹ܭ
w શһ͕ಉ͡ྉཧΛબͿ w ٬ಉ͕࢜ಉ͡ྉཧΛબͳ͘ͳΔ M ∈ {0,1}N×∞ Poi(α) Nα ¯ H+ = α N ∑ n=1 β n + β − 1 lim β→0 ¯ H+ = α lim β→∞ ¯ H+ = Nα -
Πϯυྉཧաఔ ʲແݶߦྻͷੜʳ O Poi(α) Poi
( αβ 2 + β − 1) Poi ( αβ 3 + β − 1) Poi ( αβ 4 + β − 1 ) Poi ( αβ 5 + β − 1) Nh n + β − 1 Nh n + β − 1 Nh n + β − 1 Nh n + β − 1 ग़యɿ8JLJQFEJB ϙΞιϯ -
Πϯυྉཧաఔ ʲແݶߦྻͷੜʳ O Nh n
+ β − 1 ͳͷͰɼ ֬ ͰʹͳΔɽ N2 = 3 3 4 + β − 1 h = 1 h = 2 h = 3 -
Πϯυྉཧաఔ ʲΪϒεαϯϓϦϯάʳ ɹσʔλ Λແݶ࣍ݩͷજࡏม ͱόΠφϦߦྻ ʹΑͬͯɼੜϞσϧΛϞσϧԽ ͢ΔͱҎԼͷΑ͏ʹͳΔɽ ɹ
͕ ͷΑ͏ʹղੳతʹੵআڈͰ͖ΔͱԾఆͨ͠߹ɼ ΪϒεαϯϓϦϯάʹΑͬͯࣄޙ ͔Β֤ Λ࣍ͷΑ͏ʹαϯϓϦϯάͰ ͖Δɽ ɹ ͔Β ͕αϯϓϦϯά͞ΕΔ֬ɼΠϯυྉཧաఔʹ͓͍ͯ ਓ͕ྉཧΛऔͬͨޙʹ࠷ޙͷ ൪ͷ٬͕ ൪ͷྉཧΛͱΔ͜ͱʹରԠ͍ͯ͠ Δɽ X θ M p(X, M, θ) = p(X|M, θ)p(M)p(θ) θ p(X|M) = ∫ p(X|M, θ)p(θ)dθ p(M|X) mn,h p(mn,h = 1|M\(n,h) , X) ∝ p(X|M)p(mn,h = 1|M\(n,h) ) p(mn,h |M\(n,h) ) mn,h = 1 n − 1 n h -
Πϯυྉཧաఔ ʲΪϒεαϯϓϦϯάʳ ɹ͕ͨͬͯ͠ɼ ͱͳΔ͋Δ ݸͷ ɼΪϒεαϯϓϦϯά Λ༻͍ͯɼ֬ ͱ Λܭࢉ͢Δ͜ͱʹΑΓαϯϓϦϯάͰ͖Δɽ ಉ༷ʹ
ͱͳΔΑ͏ͳ৽نόΠφϦྻͷੜɼ৽نʹੜ͞ΕΔྻͷ ͷ͕֬ ͱ ʹΑΓܭࢉͰ͖Δɽ৽نʹՃ͞ΕΔྻՃ ࢉແݶݸଘࡏ͢ΔͨΊɼݫີʹ ΛແݶճධՁ͢Δඞཁ͕͋Δ͕ɼۙࣅͯ͠༗ ݶͷީิͰܭࢉΛଧͪΔํ๏͕ΘΕΔɽʢྫɿ ͷΑ͏ʹଧͪΔɽʣ N\n,h = ∑ n′≠n mn′,h > 0 h mn,h N\n,h n + β − 1 p(X|M) N\n,h = 0 Hnew Poi( αβ N + β − 1 ) p(X|M) p(X|M) Hnew ≤ 10 p(mn,h = 1|M\(n,h) , X) ∝ p(X|M)p(mn,h = 1|M\(n,h) ) -
Πϯυྉཧաఔ ʲΪϒεαϯϓϦϯάʳ ɹ͕ͨͬͯ͠ɼ ͱͳΔ͋Δ ݸͷ ɼΪϒεαϯϓϦϯά Λ༻͍ͯɼ֬ ͱ Λܭࢉ͢Δ͜ͱʹΑΓαϯϓϦϯάͰ͖Δɽ ಉ༷ʹ
ͱͳΔΑ͏ͳ৽نόΠφϦྻͷੜɼ৽نʹੜ͞ΕΔྻͷ ͷ͕֬ ͱ ʹΑΓܭࢉͰ͖Δɽ৽نʹՃ͞ΕΔྻՃ ࢉແݶݸଘࡏ͢ΔͨΊɼݫີʹ ΛແݶճධՁ͢Δඞཁ͕͋Δ͕ɼۙࣅͯ͠༗ ݶͷީิͰܭࢉΛଧͪΔํ๏͕ΘΕΔɽʢྫɿ ͷΑ͏ʹଧͪΔɽʣ N\n,h = ∑ n′≠n mn′,h > 0 h mn,h N\n,h n + β − 1 p(X|M) N\n,h = 0 Hnew Poi( αβ N + β − 1 ) p(X|M) p(X|M) Hnew ≤ 10 p(mn,h = 1|M\(n,h) , X) ∝ p(X|M)p(mn,h = 1|M\(n,h) ) -
Πϯυྉཧաఔ ʲΪϒεαϯϓϦϯάʳ ɹ͕ͨͬͯ͠ɼ ͱͳΔ͋Δ ݸͷ ɼΪϒεαϯϓϦϯά Λ༻͍ͯɼ֬ ͱ Λܭࢉ͢Δ͜ͱʹΑΓαϯϓϦϯάͰ͖Δɽ ಉ༷ʹ
ͱͳΔΑ͏ͳ৽نόΠφϦྻͷੜɼ৽نʹੜ͞ΕΔྻͷ ͷ͕֬ ͱ ʹΑΓܭࢉͰ͖Δɽ৽نʹՃ͞ΕΔྻՃ ࢉແݶݸଘࡏ͢ΔͨΊɼݫີʹ ΛແݶճධՁ͢Δඞཁ͕͋Δ͕ɼۙࣅͯ͠༗ ݶͷީิͰܭࢉΛଧͪΔํ๏͕ΘΕΔɽʢྫɿ ͷΑ͏ʹଧͪΔɽʣ N\n,h = ∑ n′≠n mn′,h > 0 h mn,h N\n,h n + β − 1 p(X|M) N\n,h = 0 Hnew Poi( αβ N + β − 1 ) p(X|M) p(X|M) Hnew ≤ 10 ൪ʹདྷͨ٬͕ ൪ ͷྉཧΛऔΔ֬ n h ൪ʹདྷͨ٬͕৽͍͠ ྉཧΛऔΔ֬ n p(mn,h = 1|M\(n,h) , X) ∝ p(X|M)p(mn,h = 1|M\(n,h) ) -
ແݶͷχϡʔϥϧωοτϫʔΫϞσϧ ɹจݙ<>ʹج͖ͮɼඇઢܗΨε৴೦ωοτϫʔΫʢOPOMJOFBS(BVTTJBOCFMJFG OFUXPSLʣͱ͍͏ੜϞσϧΛͬͯ%//Λߏ͢Δɽ ʲඇઢܗΨε৴೦ωοτϫʔΫʳ ɹ ͷωοτϫʔΫΛߟ͑Δɽ ɹ ɿͷϢχοτɽ ɹ ɿͷ
൪ͷϢχοτɽ ɹ ɿ ྡߦྻʢʹશ݁߹ʹର͢ΔϚεΫʣɽཁૉ ɼ ͔Β ʹҹ͕ଘࡏ͢Δ͜ͱ Λҙຯ͢Δɽ ɹ ɿͷॏΈύϥϝʔλɽ ɹ ɿͷόΠΞεύϥϝʔλɽ ɹ ɿͷ׆ੑɽ ͱ͓͘ɽ L Hl l z(l) h l h M(l) ∈ ℝHl−1 ×Hl m(l) h,h′ = 1 z(l) h′ z(l−1) h W(l) ∈ ℝHl−1 ×Hl l b ∈ ℝHl l a(l) l [2] “Learning the Structure of Deep Sparse Graphical Models”, 2010 -
ແݶͷχϡʔϥϧωοτϫʔΫϞσϧ ʲඇઢܗΨε৴೦ωοτϫʔΫʳ ɹ׆ੑҎԼͷΑ͏ʹॻ͚Δɽ ͞Βʹ׆ੑ ʹΨε͔ΒͷϊΠζ͕ՃΘΔͱ͢Δɽ ӅΕϢχοτ
ҎԼͷΑ͏ʹม͞Ε͍ͯΔͱ͢Δɽ a(l) = (W(l+1) ⊙ M(l+1))z(l+1) + b(l) a(l) h ˜ a(l) h = a(l) h + ϵ ϵ ∼ (0,ν(l) h ) z(l) h z(l) h = ϕ( ˜ a(l) h ) ϕ( ⋅ ) = Tanh( ⋅ ) -
ແݶͷχϡʔϥϧωοτϫʔΫϞσϧ ʲඇઢܗΨε৴೦ωοτϫʔΫʳ ɹ͢Δͱɼ ͷҎԼͷΑ͏ʹٻΊΒΕΔɽʢ֬มͷมมʣ ɹ·ͨɼ ͱ
Ψεࣄલɼ ʹΨϯϚࣄલΛ༩͑Δɽʢڞࣄલ ʣ w ͕খ͍͞ɹˠͷۃͳΛͱΔɽ w ͕େ͖͍ɹˠ΄΅ܾఆతͳΛͱΔɽ ͞Βʹɼ ͱ͢Δɽ z(l) h p(z(l) h |a(l) h , ν(l) h ) = (ϕ−1(z(l) h )|a(l) h , ν(l) h ) ∂ ˜ a(l) h ∂z(l) h = N(ϕ−1(z(l) h )|a(l) h , ν(l) h ) ϕ′(ϕ−1(z(l) h )) ϕ′(a) = d da ϕ(a) W(l) h b(l) ν(l) h ν(l) h ν(l) h z(0) h = xh ∈ (−1,1) -
ແݶͷχϡʔϥϧωοτϫʔΫϞσϧ ʲඇઢܗΨε৴೦ωοτϫʔΫʳ ɹϞσϧશମͷಉ࣌ҎԼͷΑ͏ʹॻ͚Δ p(X, Z, M, W, b, ν)
= p(W)p(b)p(ν)p(M) N ∏ n=1 p(xn |zn , M, W, b, ν)p(zn |M, W, b, ν) -
ແݶͷχϡʔϥϧωοτϫʔΫϞσϧ ʲྻΠϯυྉཧաఔʳ ɹ͖ͬ͞ͷಉ࣌ʹؔͯ͠ɼӅΕϢχοτʹࣗ༝Λ࣋ͨͤΔΑ͏ʹ֦ு͢ Δɽ ɹ·ͣɼ Λߟ͑Δɽߦ ͷ࣍ݩ ͰݻఆͳͷͰɼΠϯυྉཧաఔͷ٬ͱΈ ͳ͢ɽ࣍ʹ Λߟ͑ΔͱɼߦΠϯυྉཧաఔʹΑͬͯಘΒΕͨ
ͷྻ ʹ ݻఆ͢Δඞཁ͕͋Γɼ͜ΕΠϯυྉཧաఔͰαϯϓϦϯάͰ͖Δɽ͜ΕΛ܁Γฦ࣮͠ ߦͯ͠ɼ ɼ ɼ ɼ ͷΑ͏ʹαϯϓϦϯά͢Δ͜ͱͰωοτϫʔΫΛߏஙͰ ͖Δɽ M(1) x H0 M(2) M(1) H1 M(1) M(2) M(3) … x ∈ ℝH0 M(1) ∈ ℝH0 ×H1 M(2) ∈ ℝH1 ×H2 M(3) ∈ ℝH2 ×H3 IBP IBP IBP IBP -
ແݶͷχϡʔϥϧωοτϫʔΫϞσϧ ʲྻΠϯυྉཧաఔʳ 2ɽͲͷΑ͏ʹಉ࣌Λਪ͢Δ͔ɽ "ɹ.$.$ͰۙࣅతʹਪͰ͖Δɽ ʢ۩ମྫʣӅΕϢχοτͷू߹ ɼόΠφϦߦྻͷू߹ ɼύϥϝʔλͷू߹ ͷͭͷϒϩοΫʹ͚ͯɼΪϒεαϯϓϦϯάʹجͮ͘ަޓαϯϓϦϯάΛߦ͑Δɽ Z
M {W, b, ν} Z ∼ p(Z|X, M, W, b, ν) M ∼ p(M|X, Z, W, b, ν) W, b, ν ∼ p(W, b, ν|X, Z, M) zn ∼ p(zn |xn , M, W, b, ν) mn,h ∼ p(mn,h |X, Z, W, b, ν, M\(n,h) ) W ∼ p(W|X, Z, M, b, ν) b ∼ p(W, b, ν|X, Z, M, W, ν) ν ∼ p(W, b, ν|X, Z, M, W, b) -
ແݶͷχϡʔϥϧωοτϫʔΫϞσϧ ʲྻΠϯυྉཧաఔʳ ɹ ͷαϯϓϧ͕༩͑ΒΕ͍ͯΔঢ়گͰɼωοτϫʔΫ͕ݻఆ͞Ε͍ͯΔͷͰɼҎલ ͬͨΑ͏ͳ௨ৗͷੜωοτϫʔΫͷਪʹؼணɽ ɹ ͷαϯϓϧ͕༩͑ΒΕ͍ͯΔঢ়گͰɼ ͷࣄલڞࣄલ͔Βબ ͍ͯ͠ΔͷͰɼ ͷαϯϓϦϯά༰қʹՄೳɽ
ɹҰํͰɼ ͷαϯϓϦϯάෳࡶɽϝτϩϙϦεɾϔΠεςΟϯάε๏Λ͏͜ ͱͰ࣮ࢪͰ͖Δɽ M Z {W, b, ν} {W, b, ν} Z M