Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
ベイズ深層学習(4.1)
Search
catla
February 07, 2020
Science
0
410
ベイズ深層学習(4.1)
catla
February 07, 2020
Tweet
Share
More Decks by catla
See All by catla
ベイズ深層学習(6.3)
catla
2
200
ベイズ深層学習(6.2)
catla
3
210
[読み会資料] Federated Learning for Vision-and-Language Grounding Problems
catla
0
250
ベイズ深層学習(5.1~5.2)
catla
0
210
ベイズ深層学習(3.3~3.4)
catla
18
11k
ベイズ深層学習(2.2~2.4)
catla
6
1.3k
23回アルゴリズムコンテスト 1位解法
catla
6
660
Learning Lightweight Lane Detection CNNs by Self Attention Distillation(ICCV2019)の紹介
catla
0
540
TGS Salt Identification Challenge 12th place solution
catla
3
11k
Other Decks in Science
See All in Science
ABEMAの効果検証事例〜効果の異質性を考える〜
s1ok69oo
4
2.1k
(2024) Livres, Femmes et Math
mansuy
0
110
JSol'Ex : traitement d'images solaires en Java
melix
0
110
The thin line between reconstruction, classification, and hallucination in brain decoding
ykamit
1
950
マテリアルズ・インフォマティクスの先端で起きていること / What's Happening at the Cutting Edge of Materials Informatics
snhryt
1
130
General Parasitology
uni_of_nomi
0
120
はじめてのバックドア基準:あるいは、重回帰分析の偏回帰係数を因果効果の推定値として解釈してよいのか問題
takehikoihayashi
2
740
WCS-LA-2024
lcolladotor
0
120
Introduction to Graph Neural Networks
joisino
PRO
4
2.1k
創薬における機械学習技術について
kanojikajino
13
4.4k
Snowflake上でRを使う: RStudioセットアップとShinyアプリケーションのデプロイ
ktatsuya
0
420
Analysis-Ready Cloud-Optimized Data for your community and the entire world with Pangeo-Forge
jbusecke
0
110
Featured
See All Featured
Docker and Python
trallard
40
3.1k
Easily Structure & Communicate Ideas using Wireframe
afnizarnur
191
16k
Unsuck your backbone
ammeep
668
57k
A Modern Web Designer's Workflow
chriscoyier
693
190k
VelocityConf: Rendering Performance Case Studies
addyosmani
325
24k
Designing the Hi-DPI Web
ddemaree
280
34k
Stop Working from a Prison Cell
hatefulcrawdad
267
20k
Reflections from 52 weeks, 52 projects
jeffersonlam
346
20k
Designing on Purpose - Digital PM Summit 2013
jponch
115
7k
The MySQL Ecosystem @ GitHub 2015
samlambert
250
12k
The Straight Up "How To Draw Better" Workshop
denniskardys
232
140k
Learning to Love Humans: Emotional Interface Design
aarron
273
40k
Transcript
ϕΠζਂֶश αϯϓϦϯάʹجͮ͘ਪख๏ ܡɹঘً
αϯϓϦϯάʹجͮ͘ਪख๏ ˞ʮύλʔϯೝࣝͱػցֶशɹԼרʯষࢀߟ
αϯϓϦϯάʹجͮ͘ਪख๏ ɹ؍ଌσʔλΛ ɼඇ؍ଌͷมͷू߹ʢFHύϥϝʔλɼજࡏม Λ ͱͨ͠ͱ ͖ɼϕΠζਪʹΑΔ౷ܭղੳͰɼ֬Ϟσϧ Λઃܭ͢Δඞཁ͕͋Δɽ ɹ࣮ࡍʹɼ֬ϞσϧΛ༻ֶ͍ͯश༧ଌɼࣄޙ Λܭࢉͯ͠ߦΘΕΔɽ X
Z p(X, Z) p(Z|X) ղܾࡦ ɹෳࡶͳϞσϧʢFHχϡʔϥϧωοτʣɼ ͕ղੳతʹٻΊΒΕͳ͍͜ͱ͕ ଟ͍ɽ p(Z|X) ɹ ΛղੳతʹٻΊΔΘΓʹɼ͜ͷ͔ΒෳͷαϯϓϧΛಘΔ͜ͱͰɼ ͷಛੑΛௐΔɽ ɹͱ͍͏͜ͱ͔ΒɼαϯϓϦϯά͢Δํ๏Λࠓճษڧ͢ΔΑʂ p(Z|X)
ຊͷ༰ ୯७ϞϯςΧϧϩ๏ غ٫αϯϓϦϯά ࣗݾਖ਼نԽॏαϯϓϦϯά ʢϚϧίϑ࿈ϞϯςΧϧϩ๏ʣ ϝτϩϙϦεɾϔΠεςΟϯάε๏ ϝτϩϙϦε๏ ϋϛϧτχΞϯϞϯςΧϧϩ๏ ϥϯδϡόϯϞϯςΧϧϩ๏ ΪϒεαϯϓϦϯά
ຊͷ༰ ୯७ϞϯςΧϧϩ๏ غ٫αϯϓϦϯά ࣗݾਖ਼نԽॏ৺αϯϓϦϯά ʢϚϧίϑ࿈ϞϯςΧϧϩ๏ʣ ϝτϩϙϦεɾϔΠεςΟϯάε๏ ϝτϩϙϦε๏ ϋϛϧτχΞϯϞϯςΧϧϩ๏ ϥϯδϡόϯϞϯςΧϧϩ๏ ΪϒεαϯϓϦϯά
.$.$
୯७ϞϯςΧϧϩ๏ ɹ ʹର͢Δؔ ͷظɹ ΛٻΊ͍ͨɽɹ p(z) f(z) p(z) [ f(z)]
= ∫ f(z)p(z)dz త ঢ়گ ɹظ ͷղੳతͳੵܭࢉ͕ࠔɽ ɹ ͔ΒͷαϯϓϦϯά༰қɽ ∫ f(z)p(z)dz p(z) ख๏ ɹ Λेେ͖ͳͱͨ͠ͱ͖ɼ T z(1), z(2), …, z(T) ∼ p(z) ∫ f(z)p(z)dz ≈ 1 T T ∑ t=1 f(z(t)) ͔Β ݸαϯϓϦϯά ⟵ p(z) T
ɹύϥϝʔλ Λ࣋ͭϞσϧ ͷपล Λܭࢉ͢Δࡍʹ༻͢ Δ߹ɼ θ p(X, θ) =
p(X|θ)p(θ) p(X) p(X) = ∫ p(X|θ)p(θ)dθ = ∫ N ∏ n=1 p(xn |θ)p(θ)dθ = p(θ) [p(X|θ)] ≈ 1 T T ∑ t=1 N ∏ n=1 p(xn |θ(t)), (θ(1), …, θ(T) ∼ p(θ)) ɹظ ʹ͓͍ͯɼ ͔Βͷαϯϓϧ ͷൣғ෯͘ͱΔඞཁ͕͋ΓɼҰํ Ͱɼ ڱ͍ ͷൣғͰ͔͠େ͖ͳΛऔΒͳ͍έʔε͕ଟ͍ɽ ɹ ൚༻త͚ͩͲɼܭࢉޮ͕ѱ͍ɽ p(z) [ f(z)] p(z) z f(z) z ⟹ ୯७ϞϯςΧϧϩ๏
غ٫αϯϓϦϯά ɹີܭࢉ͕ࠔͳ֬ ͔ΒαϯϓϧΛಘΔɽɹ p(z) z(1), z(2), … ∼ p(z) త
ঢ়گ ɹਖ਼نԽ͞Ε͍ͯͳ͍ؔ ܭࢉՄೳɽͭ·Γɼ ɽ ˜ p(z)( = Zp ⋅ p(z)) ∫ ˜ p(z)dz ≠ 1 ख๏ ɹఏҊ Λઃఆ͢Δɽҙͷ ʹରͯ͠ɼ ɹɹɹɹɹɹɹɹɹɹɹɹɹɹɹɹɹ ͱͳΔΑ͏ʹɼਖ਼ͷఆ ΛఆΊΔɽ q(z) z kq(z) > ˜ p(z) k ఏҊ ɹαϯϓϦϯά͕؆୯ʹߦ͑ΔΑ͏ͳ Ծͷɽ
غ٫αϯϓϦϯά ख๏ ͖ͭͮʣ ɹఏҊ ͔ΒαϯϓϧΛಘΔɽ ɹҰ༷ ͔ΒͷαϯϓϧΛಘΔɽ
ɹαϯϓϧ ͷड༰ʢBDDFQUʣغ٫ʢSFKFDUʣબɽ ɹɹɹ q(z) z(t) ∼ q(z) Uni(0,kq(z)) ˜ u ∼ Uni (0,kq(z(t))) z(t) if ˜ u > ˜ p(z(t)) then SFKFDU else BDDFQU ड༰ ∫ q(z) ˜ p(z) kq(z) dz = 1 k ∫ ˜ p(z)dz ߴ࣍ݩͷมͷαϯϓϦϯά͕ඞཁͳ߹ɼड༰͕ඇৗʹ͘ͳΔɽ
غ٫αϯϓϦϯά z ˜ p(z) ͷαϯϓϧΛغ٫αϯϓϧϦϯάͰ֫ಘ͢Δɽ ະɽ طɽ p(z) p(z) ˜
p(z) p(z)
غ٫αϯϓϦϯά z ˜ p(z) αϯϓϧ͕༰қͳఏҊ Λઃఆɽ p(z) q(z)
غ٫αϯϓϦϯά z ˜ p(z) kq(z) Λ෴͍͔Ϳ͞ΔΑ͏ʹ Λઃఆɽ ˜ p(z) k
kq(z) > ˜ p(z) q(z) × k
غ٫αϯϓϦϯά z ˜ p(z) kq(z) z(t) ఏҊ ͔ΒαϯϓϧΛಘΔɽ q(z) z(t)
∼ q(z) q(z)
غ٫αϯϓϦϯά z ˜ p(z) kq(z) z(t) kq(z(t)) ˜ u Ұ༷
͔ΒͷαϯϓϧΛಘΔ Uni(0,kq(z)) ˜ u ∼ Uni (0,kq(z(t)))
غ٫αϯϓϦϯά z ˜ p(z) kq(z) z(t) kq(z(t)) ड༰ غ٫ ˜
u αϯϓϧ ͷड༰ʢBDDFQUʣغ٫ʢSFKFDUʣબɽ z(t) if ˜ u > ˜ p(z(t)) then SFKFDU else BDDFQU ˜ p(z(t))
ࣗݾਖ਼نԽॏ৺αϯϓϦϯά ɹ ʹର͢Δؔ ͷظɹ Λ୯७ϞϯςΧϧϩ๏Α ΓޮతʹٻΊ͍ͨɽɹ p(z) f(z) p(z) [
f(z)] = ∫ f(z)p(z)dz త ঢ়گ ɹظ ͷղੳతͳੵܭࢉ͕ࠔɽ ɹ ͔ΒαϯϓϦϯάΛಘΒΕͳ͍ɽ ɹਖ਼نԽ͞Ε͍ͯͳ͍ؔ ܭࢉՄೳɽ ∫ f(z)p(z)dz p(z) ˜ p(z)( = Zp ⋅ p(z)) എܠ ɹغ٫αϯϓϦϯάΛ༻͍ͯɼ ΛΘͣʹαϯϓϧΛऔಘ͠ɼظ Λٻ ΊΔ͜ͱͰ͖Δ͕ɼ ͷ͕খ͞ͳྖҬʹαϯϓϧ͕ूத͢ΔՄೳੑ͕͋Δɽ ୯७ϞϯςΧϧϩ๏ͷܭࢉͷد༩͕গͳ͍ɽ ͷ͕େ͖͘ͳΔΑ͏ͳ ྖҬΛॏతʹαϯϓϧͨ͠ํ͕ޮ͕͍͍ɽ p(z) p(z) [ f(z)] f(z) ⟹ f(z)p(z)
ࣗݾਖ਼نԽॏ৺αϯϓϦϯά ɹ·ͣɼఏҊ Λઃఆ͢Δɽਖ਼نԽ͞Ε͍ͯͳ͍ؔ ɼҎԼͷΑ͏ʹද ͞ΕΔɽ ɹظͷܭࢉɼҎԼͷΑ͏ʹมܗͰ͖Δɽ q(z)
˜ p(z), ˜ q(z) p(z) = 1 Zp ˜ p(z), q(z) = 1 Zq ˜ q(z) ∫ f(z)p(z)dz = ∫ f(z) p(z) q(z) q(z)dz = q(z) [ f(z) p(z) q(z) ] = ∫ f(z) 1 Zp ˜ p(z) 1 Zq ˜ q(z) q(z)dz = Zq Zp q(z) [ f(z) ˜ p(z) ˜ q(z) ] ≈ Zq Zp 1 T T ∑ t=1 f(z(t)) ˜ p(z(t)) ˜ q(z(t)) = Zq Zp 1 T T ∑ t=1 f(z(t))w(t), w(t) = ˜ p(z(t)) ˜ q(z(t)) ख๏
ࣗݾਖ਼نԽॏ৺αϯϓϦϯά ɹ ະͷͳͷͰɼҎԼͷΑ͏ʹਖ਼نԽ߲ͷൺΛۙࣅ͢Δɽ ɹΑͬͯɼؔ ͷظ͕ۙࣅతʹಘΒΕΔɽ Zp
Zp Zq = ∫ ˜ p(z) Zq dz = ∫ ˜ p(z) ˜ q(z) q(z)dz = q(z) [ ˜ p(z) ˜ q(z) ] ≈ 1 T T ∑ t=1 w(t), z(1), …, z(T) ∼ q(z) f(z) ख๏ ͖ͭͮʣ ( ∵ p(z) = 1 Zp ˜ p(z) ) ( ∵ ∫ p(z)dz = 1 Zp ∫ ˜ p(z)dz = 1 )
ࣗݾਖ਼نԽॏ৺αϯϓϦϯά ɹ ະͷͳͷͰɼҎԼͷΑ͏ʹਖ਼نԽ߲ͷൺΛۙࣅ͢Δɽ ɹΑͬͯɼؔ ͷظ͕ۙࣅతʹಘΒΕΔɽ Zp
Zp Zq = ∫ ˜ p(z) Zq dz = ∫ ˜ p(z) ˜ q(z) q(z)dz = q(z) [ ˜ p(z) ˜ q(z) ] ≈ 1 T T ∑ t=1 w(t), z(1), …, z(T) ∼ q(z) f(z) ख๏ ͖ͭͮʣ ( ∵ p(z) = 1 Zp ˜ p(z) ) ( ∵ ∫ p(z)dz = 1 Zp ∫ ˜ p(z)dz = 1 ) ͳΜͷͨΊʹ Λ ͖࣋ͬͯͨΜͩΖ͏ʜ ˜ q(z)
ࣗݾਖ਼نԽॏαϯϓϦϯάʢޡهͷՄೳੑʣ ʮϕΠζਂֶशʯQ ɹࣗݾਖ਼نԽॏαϯϓϦϯάͷஈམͷ࠷ॳ ޡΓ ɹغ٫αϯϓϦϯάͱҟͳΔɼ ͔Βͷαϯϓϧʜʜ p(z) ɹ୯७ϞϯςΧϧϩ๏ͱҟͳΔɼ ͔Βͷαϯϓϧʜʜ p(z)
Ϛϧίϑ࿈ϞϯςΧϧϩ๏ʢ.$.$ʣ ɹغ٫αϯϓϦϯάͷɼߴ࣍ݩʹͳΔͱड༰͕ඇৗʹখ͘͞ͳΔ͜ͱɽ࣮ࡍ ʹɼ࣍ݩఔͷ؆୯ͳੵۙࣅʹ͔͠ద༻Ͱ͖ͳ͍ɽ ɹͰɼߴ࣍ݩۭؒͰޮతʹαϯϓϦϯά͢Δʹʜʜ ɹɹɹ Ϛϧίϑ࿈ϞϯςΧϧϩ๏ʢ.$.$ʣ͕ఏҊ͞Ε͍ͯΔɽ ⟹ ࣍Ϛϧίϑ࿈ ɹ֬มͷܥྻ
ʹରͯ͠ ͕Γཱͭͱ͖ͷܥྻ ͷ͜ͱɽ z(1), z(2), … p(z(t) |z(1), z(2), …, z(t−1)) = p(z(t) |z(t−1)) z(1), z(2), … άϥϑΟΧϧϞσϧ z(1) z(2) z(t−1) z(t) ⋯ ɹભҠ֬ɹΛ ͱ͓͍ͨͱ͖ɼ ͕Γཱͭͱ͖ɼ Λɹఆৗɹͱ͍͏ɽ (z(t−1), z(t)) = p(z(t) |z(t−1)) p* (z) = ∫ (z′ , z)p* (z′ )dz′ p* (z)
Ϛϧίϑ࿈ϞϯςΧϧϩ๏ʢ.$.$ʣ ɹఆৗ ʹऩଋ͢ΔΑ͏ͳભҠ֬ Λઃܭ͢Δͱɼ ͔Βαϯϓϧ ΛಘΔ͜ͱ͕Ͱ͖Δɽ ఆৗʹີܭࢉ͕ࠔͳ֬Λ͓͘ɽ p* (z) (z(t−1),
z(t)) p* (z) ⟹ ख๏ͷΩϞ ৄࡉΓ߹͍݅ p* (z)(z, z′ ) = p* (z′ )(z′ , z) ʲे݅ʳৄࡉΓ߹͍͕݅Γཱͭ ఆৗͱͳΔɽ ⟹ p* (z) p* (z)(z, z′ ) = p* (z′ )(z′ , z) ⇒ ∫ p* (z)(z, z′ )dz′ = ∫ p* (z′ )(z′ , z)dz′ ⇔ p* (z) ∫ p(z′ |z)dz′ = ∫ p* (z′ )(z′ , z)dz′ ⇔ p* (z) = ∫ p* (z′ )(z′ , z)dz′
Ϛϧίϑ࿈ϞϯςΧϧϩ๏ʢ.$.$ʣ ɹΓ߹͍݅ʹՃ͑ͯɼαϯϓϧ͕ ͱͨ͠ͱ͖ɼॳظঢ়ଶ ʹ͔͔ΘΒ ͣɼ ͕ఆৗ ʹऩଋ͢Δඞཁ͕͋Δɽ Τϧΰʔυੑ t →
∞ p(z(1)) p(z(t)) p* (z) ⟹ Τϧΰʔυੑ w طੑɹɿҙͷঢ়ଶ͔Βҙͷঢ়ଶ༗ݶճͰભҠՄೳɽ w ඇपظੑɿͯ͢ͷঢ়ଶ͕ݻఆͷपظੑΛͨͳ͍ɽ w ਖ਼࠶ؼੑɿಉ͡ঢ়ଶ͕༗ݶճͰΔ͜ͱ͕Մೳɽ
ϝτϩϙϦεɾϔΠεςΟϯάε๏ʢ.)๏ʣ త ɹະͷ֬ ͔ΒαϯϓϦϯάΛಘΔɽ p(z) લఏ ɹ ͱͳΔਖ਼نԽ͞Ε͍ͯͳ͍ؔ طͰ͋Δɽ p(z)
∝ ˜ p(z) ˜ p(z) ख๏ ɹભҠ֬ ͕ઃܭ͕͍͠߹ɼભҠͷఏҊ Λ͑Δɽ (z′ , z) q(z|z′ ) ɽఏҊ ͔Β࣍ͷαϯϓϧͷީิ ΛαϯϓϦϯά͢Δɽ ɽ࣍ͷൺ Λܭࢉ͢Δɽ ɽ Λ֬ ʹΑͬͯ ͱͯ͠ड༰͠ɼͦ͏Ͱͳ͍߹ ɼ ͱ͢Δɽ q( ⋅ |z(t)) z* r r = ˜ p(z* )q(z(t) |z* ) ˜ p(z(t))q(z* |z(t)) z* min(1,r) z(t+1) ⟵ z* z(t+1) ⟵ z(t) ΞϧΰϦζϜͷྲྀΕ
ϝτϩϙϦεɾϔΠεςΟϯάε๏ʢ.)๏ʣ ৄࡉΓ߹͍݅ͷূ໌ ɹભҠ֬ɼҎԼͷΑ͏ʹͳΔɽ (z, z′ ) = q(z′ |z) min
(1, ˜ p(z′ )q(z|z′ ) ˜ p(z)q(z′ |z) ) p(z)(z, z′ ) = p(z)q(z′ |z) min (1, ˜ p(z′ )q(z|z′ ) ˜ p(z)q(z′ |z) ) = p(z)q(z′ |z) min (1, p(z′ )q(z|z′ ) p(z)q(z′ |z) ) = min (p(z)q(z′ |z), p(z′ )q(z|z′ )) = min (p(z′ )q(z|z′ ), p(z)q(z′ |z)) = p(z′ )q(z|z′ ) min (1, p(z)q(z′ |z) p(z′ )q(z|z′ )) = p(z′ )q(z|z′ ) min (1, ˜ p(z)q(z′ |z) ˜ p(z′ )q(z|z′ )) = p(z′ )(z′ , z) ɹ ͷ߹ɼϝτϩϙϦε๏ͱݺΕΔɽ q(z′ |z) = q(z|z′ )
ϝτϩϙϦεɾϔΠεςΟϯάε๏ʢ.)๏ʣ ʮϕΠζਂֶशʯQࣜʢʣ ޡΓ (z, z′ ) = q(z′ |z) min
(1, ˜ p(z′ )q(z|z′ ) ˜ p(z)q(z′ |z) ) (z, z′ ) = q(z|z′ ) min (1, ˜ p(z′ )q(z|z′ ) ˜ p(z)q(z′ |z) )
ϝτϩϙϦεɾϔΠεςΟϯάε๏ʢ.)๏ʣ ۩ମྫͰֶͿ ඪʢະʣ ɹɹɹɹɹɹɹ ਖ਼نԽ͞Ε͍ͯͳ͍ؔʢطʣ ఏҊ
ͭ·Γ p(z) = (z|μ, Σ) = 1 (2π)D |Σ| exp {− 1 2 (z − μ)TΣ−1(z − μ)} ˜ p(z) = exp {− 1 2 (z − μ)TΣ−1(z − μ)} q(z* |z) = (z′ |z, I) z* ∼ (z, I) https://drive.google.com/open?id=1vcBZWp9HPzfCzBjj2INdkH_CJUDL-JA3
ɹαϯϓϦϯάͷલʹϋϛϧτχΞϯΛར༻ͨ͠ղੳֶతͳγϛϡϨʔγϣϯΛղ આɽຎࡲʹΑΔΤωϧΪʔͷݮগ͕ͳ͍ͱԾఆ͢ΔͱɼϋϛϧτχΞϯҎԼͷΑ͏ʹ ද͞ΕΔɽ ℋ(z, p) = (z) + (p), (p)
= 1 2m pTp, z ∈ ℝD: ମͷҐஔϕΫτϧ, p ∈ ℝD: ମͷӡಈྔϕΫτϧ, m ∈ ℝ: ମͷ࣭ྔ, ℋ(z, p): ϋϛϧτχΞϯ, (z): ϙςϯγϟϧΤωϧΪʔ, (p): ӡಈΤωϧΪʔ . ϋϛϧτχΞϯϞϯςΧϧϩ๏ʢ).$๏ʣ ).$๏ʢϋΠϒϦοτϞϯςΧϧϩ๏ʣʹମͷيಓͷझຯϨʔγϣϯ .)๏ × ɹ).$๏ɼϥϯμϜΥʔΫతͳ.)ͱൺͯɼޮతʹۭؒΛ୳ࡧՄೳɽ ϋϛϧτχΞϯͷγϛϡϨʔγϣϯ
ϋϛϧτχΞϯϞϯςΧϧϩ๏ʢ).$๏ʣ ɹ࣭ྔΛͱ͠ɼ ͱ ͷ࣌ؒ ʹؔ͢ΔڍಈɼϋϛϧτχΞϯͷภඍʹΑܾͬͯఆɽ ͜ͷඍํఔ͕ࣜղੳతʹղ͚ͳ͍ͷͱ͠ɼγϛϡϨʔγϣϯʹΑͬͯيಓΛܭ ࢉ͢Δɽ z
p τ dpi dτ = − dℋ dzi = − d dzi , dzi dτ = dℋ dpi = d dpi . ΦΠϥʔ๏ ࣌ࠁ ઌͷڍಈΛۙࣅతʹ༧ଌɽ ϵ > 0 pi (τ + ϵ) = pi (τ) + ϵ dpi dτ τ = pi (τ) − ϵ d dzi zi (τ) , zi (τ + ϵ) = zi (τ) + ϵ dzi dτ τ = zi (τ) + ϵpi (τ) ࢄԽʹΑΔޡ͕ࠩେ͖͍ɽ Ϧʔϓϑϩοά๏ ⟹
ϋϛϧτχΞϯϞϯςΧϧϩ๏ʢ).$๏ʣ Ϧʔϓϑϩοά๏ pi (τ + ϵ 2 ) = pi
(τ) − ϵ 2 d dzi zi (τ) , zi (τ + ϵ) = zi (τ) + ϵpi (τ + ϵ 2 ), pi (τ + ϵ) = pi (τ + ϵ 2 ) − ϵ 2 d dzi zi (τ + ϵ) . ͜ΕΛ ճ܁Γฦ͢͜ͱͰ࣌ࠁ ઌͷମͷҐஔ ͱӡಈྔ ΛۙࣅతʹܭࢉͰ͖Δɽ L ϵL z* p* ϋϛϧτχΞϯͷੑ࣭ ɽ ࣌ؒ ʹΑͬͯෆมɽ ɽՄٯੑɿ ͔Β ͷભҠҰରҰɽ ɽମੵอଘ ℋ τ (z, p) (z* , p* )
ϋϛϧτχΞϯϞϯςΧϧϩ๏ʢ).$๏ʣ αϯϓϦϯάΞϧΰϦζϜͷద༻ త ɹະͷ֬ ͔ΒαϯϓϦϯάΛಘΔɽ p(z) લఏ ɹ ͱͳΔਖ਼نԽ͞Ε͍ͯͳ͍ؔ طͰ͋Δɽ
ɹ ͱ֦ு͢Δͱɼ पล ͔Βαϯϓϧ͕ಘΒΕΔɽ ɹ p(z) ∝ ˜ p(z) ˜ p(z) p(z, p) = p(z)p(p) z p(z) p(p) = (p|0, I) (z) = − log (˜ p(z)) (p) = 1 2 pTp
ϋϛϧτχΞϯϞϯςΧϧϩ๏ʢ).$๏ʣ ɹಉ࣌Λܭࢉ͢ΔͱɼҎԼͷΑ͏ʹͳΔɽ ϝτϩϙϦε๏ͰΘΕΔൺ ɼҎԼͷΑ͏ʹͳΔɽ p(z, p) = p(z)p(p) =
exp (log p(z) + log p(p)) ∝ exp (log ˜ p(z) − 1 2 pTIp ) = exp (−(z) − (p)) = exp (−ℋ(z, p)) r r = p(z* , p* ) p(z, p) = exp (−ℋ(z* , p* ) + ℋ(z, p))
ϋϛϧτχΞϯϞϯςΧϧϩ๏ʢ).$๏ʣ ख๏ ɽӡಈྔΛαϯϓϦϯά ɽϦʔϓϑϩοά๏Ͱݱࡏͷ ͔Βީิ ΛಘΔɽ ɽ࣍ͷൺ Λܭࢉ͢Δɽ
ɽ Λ֬ ʹΑͬͯ ͱͯ͠ड༰͠ɼͦ͏Ͱͳ͍߹ ɼ ͱ͢Δɽ p ∼ (0, I) (z(t), p) (z* , p* ) r r = p(z* , p* ) p(z, p) z* min(1,r) z(t+1) ⟵ z* z(t+1) ⟵ z(t) ΞϧΰϦζϜͷྲྀΕ
ϋϛϧτχΞϯϞϯςΧϧϩ๏ʢ).$๏ʣ Ϧʔϓϑϩοάͷύϥϝʔλʹؔͯ͠ɼҎԼͷΑ͏ͳτϨʔυΦϑ͕͋Δɽ ɹ).$๏ɼࣄޙͷඍ͑͞ܭࢉͰ͖Εద༻Ͱ͖ɼඇৗʹ൚༻తɽҰൠతͳ χϡʔϥϧωοτϫʔΫ࿈ଓͳજࡏมͷΈͰΓཱ͍ͬͯΔ͜ͱ͕ଟ͍ͷͰɼ).$ ๏χϡʔϥϧωοτϫʔΫͷϕΠζֶशʹΘΕ͖ͯͨɽ େ͖͍ εςοϓαΠζ ϵ εςοϓ L
খ͍͞ খ͍͞ େ͖͍ ड༰ ड༰ ୳ࡧޮ ܭࢉྔ ߴ͍ ߴ͍ ͍ ͍ େ͖͍ খ͍͞ ߴ͍ ͍
ϋϛϧτχΞϯϞϯςΧϧϩ๏ʢ).$๏ʣ ϥϯδϡόϯಈྗֶ๏ ɹ ͱͨ͠߹ɼϥϯδϡόϯϞϯςΧϧϩ๏ɹ·ͨɹϥϯδϡόϯಈྗֶ๏ɹͱ ݺΕΔɽ ɹਂֶश͚ʹϛχόονֶश͕ߦ͑ΔΑ͏ʹͨ͠ɹ֬తޯϥϯδϡόϯಈྗֶ ๏ɹʹల։͞ΕΔɽ L =
1 z*i = zi (τ + ϵ) = zi (τ) + ϵ pi (τ) − ϵ 2 d dzi zi (τ) = zi (τ) − ϵ2 2 d dzi zi (τ) + ϵpi (τ)
ඪʢະʣ ɹɹɹɹɹɹɹ ਖ਼نԽ͞Ε͍ͯͳ͍ؔʢطʣ ӡಈྔͷαϯϓϦϯάɿ ӡಈΤωϧΪʔɿ
ҐஔΤωϧΪʔɿ ҐஔΤωϧΪʔͷภඍɿ p(z) = (z|μ, Σ) = 1 (2π)D |Σ| exp {− 1 2 (z − μ)TΣ−1(z − μ)} ˜ p(z) = exp {− 1 2 (z − μ)TΣ−1(z − μ)} p ∼ (0, I) (p) = 1 2 pTp (z) = − log (˜ p(z)) = − 1 2 (z − μ)TΣ−1(z − μ) ∂ ∂z = − (z − μ)TΣ−1 ϋϛϧτχΞϯϞϯςΧϧϩ๏ʢ).$๏ʣ ۩ମྫͰֶͿ https://drive.google.com/open?id=11zWctTbECXEhlHm7AqPiXC_MErAYl7hJ
ΪϒεαϯϓϦϯά త ɹ֬ ͔Β શମΛαϯϓϦϯά͢Δ͜ͱ͕͍͠ͱ͖ͷ୳ࡧɽ p(Z) Z લఏ ɹ֬ طɽ
p(Z) ख๏ ɹɽม Λ ݸͷ෦ू߹ʹ͚Δɽ ɹɽ෦ू߹Λஞ࣍తʹ୳ࡧ͢Δɽ ɹ Z M Z = {Z1 , Z2 , …, ZM } Z1 ∼ p(Z1 |Z2 , Z3 , …, ZM−1 , ZM ) Z2 ∼ p(Z2 |Z1 , Z3 , …, ZM−1 , ZM ) ⋮ ZM ∼ p(ZM |Z1 , Z2 , …, ZM−2 , ZM−1 )
ΪϒεαϯϓϦϯά ɹΪϒεαϯϓϦϯάͷଥੑɼαϯϓϦϯάͷखଓ͖͕.)๏ͷҰछͱͯ͠ղऍͰ ͖Δ͜ͱ͕อূ͞Ε͍ͯΔɽ ͷΑ͏ʹ͚ɼ Λ͚݅ͨ͠ͱͰ ͷα ϯϓϦϯάΛ͢Δ͜ͱΛߟ͑ͨ߹ɼ ͔ͭ ɹൺ
Λܭࢉ͢ΔͱҎԼͷΑ͏ʹͳΔɽ Αͬͯɼશͯड༰͞ΕΔɽ ಉ༷ɽ Z = {Z1 , Z2 } Z2 Z1 q(Z* |Z) = p(Z1* |Z2* ) Z2 = Z2* r r = p(Z* )q(Z|Z* ) p(Z)q(Z* |Z) = p(Z1* , Z2* )p(Z1 |Z2* ) p(Z1 , Z2 )p(Z1* |Z2 ) = p(Z1* |Z2* )p(Z2* )p(Z1 |Z2* ) p(Z1 |Z2 )p(Z2 )p(Z1* |Z2 ) = 1 Z2
ΪϒεαϯϓϦϯά ۩ମྫͰֶͿ ඪʢطʣ ɹɹɹɹ ͱ͢Δɽ p(z) = (z|μ,
Σ) = 1 (2π)D |Σ| exp {− 1 2 (z − μ)TΣ−1(z − μ)} z = ( z1 z2 ), μ = ( μ1 μ2 ), Σ = ( Σ11 Σ12 Σ21 Σ22 ), Λ = Σ−1 = ( Λ11 Λ12 Λ21 Λ22 ) log p(z) = log p(z1 , z2 ) = − 1 2 (z1 − μ1 )TΛ11 (z1 − μ1 ) + (z1 − μ1 )TΛ12 (z2 − μ2 )) − 1 2 (z2 − μ2 )TΛ22 (z2 − μ2 ) + (z2 − μ2 )TΛ21 (z1 − μ1 )) = − 1 2 ( zT 1 Λ11 z1 − 2z1 {Λ11 μ1 − 1 2 Λ12 (z2 − μ2 )}) + C1 − 1 2 ( zT 2 Λ22 z2 − 2z2 {Λ22 μ2 − 1 2 Λ21 (z1 − μ1 )}) + C2
ΪϒεαϯϓϦϯά ۩ମྫͰֶͿ Αͬͯɼ͖݅֬ͷରɼ ͱͳΔͷͰɼ͖݅֬ΨεͰ͋Δɼ log p(zi |zj
) = − 1 2 ( zT i Λii zi − 2zi { Λii μi − 1 2 Λij (zj − μj )}) + C p(zi |zj ) = (zi |μi , Σi ), Σ−1 i = Λii , Σ−1 i μi = Λii μi − 1 2 Λij (zj − μj ), ⇔ μi = Σi ( Λii μi − 1 2 Λij (zj − μj )) . https://drive.google.com/open?id=1ReYNvvH-NgtsuRiDDV-lz1779sps2pT0