Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
ベイズ深層学習(4.1)
Search
catla
February 07, 2020
Science
0
440
ベイズ深層学習(4.1)
catla
February 07, 2020
Tweet
Share
More Decks by catla
See All by catla
ベイズ深層学習(6.3)
catla
2
210
ベイズ深層学習(6.2)
catla
3
220
[読み会資料] Federated Learning for Vision-and-Language Grounding Problems
catla
0
270
ベイズ深層学習(5.1~5.2)
catla
0
220
ベイズ深層学習(3.3~3.4)
catla
18
11k
ベイズ深層学習(2.2~2.4)
catla
6
1.3k
23回アルゴリズムコンテスト 1位解法
catla
6
660
Learning Lightweight Lane Detection CNNs by Self Attention Distillation(ICCV2019)の紹介
catla
0
550
TGS Salt Identification Challenge 12th place solution
catla
3
11k
Other Decks in Science
See All in Science
ACL読み会2024@名大 REANO: Optimising Retrieval-Augmented Reader Models through Knowledge Graph Generation
takuma_matsubara
0
170
Causal discovery based on non-Gaussianity and nonlinearity
sshimizu2006
0
240
06_浅井雄一郎_株式会社浅井農園代表取締役社長_紹介資料.pdf
sip3ristex
0
240
メール送信サーバの集約における透過型SMTP プロキシの定量評価 / Quantitative Evaluation of Transparent SMTP Proxy in Email Sending Server Aggregation
linyows
0
750
WCS-LA-2024
lcolladotor
0
190
小杉考司(専修大学)
kosugitti
2
620
01_篠原弘道_SIPガバニングボード座長_ポスコロSIPへの期待.pdf
sip3ristex
0
240
局所保存性・相似変換対称性を満たす機械学習モデルによる数値流体力学
yellowshippo
1
190
The Incredible Machine: Developer Productivity and the Impact of AI
tomzimmermann
0
590
20分で分かる Human-in-the-Loop 機械学習におけるアノテーションとヒューマンコンピューターインタラクションの真髄
hurutoriya
5
2.8k
大規模言語モデルの論理構造の把握能力と予測モデルの生成
fuyu_quant0
0
110
ウェーブレットおきもち講座
aikiriao
1
830
Featured
See All Featured
Producing Creativity
orderedlist
PRO
344
40k
Agile that works and the tools we love
rasmusluckow
328
21k
CSS Pre-Processors: Stylus, Less & Sass
bermonpainter
356
29k
XXLCSS - How to scale CSS and keep your sanity
sugarenia
248
1.3M
VelocityConf: Rendering Performance Case Studies
addyosmani
328
24k
Principles of Awesome APIs and How to Build Them.
keavy
126
17k
How To Stay Up To Date on Web Technology
chriscoyier
790
250k
Automating Front-end Workflow
addyosmani
1369
200k
Intergalactic Javascript Robots from Outer Space
tanoku
270
27k
The Language of Interfaces
destraynor
156
24k
Chrome DevTools: State of the Union 2024 - Debugging React & Beyond
addyosmani
4
420
Building Flexible Design Systems
yeseniaperezcruz
328
38k
Transcript
ϕΠζਂֶश αϯϓϦϯάʹجͮ͘ਪख๏ ܡɹঘً
αϯϓϦϯάʹجͮ͘ਪख๏ ˞ʮύλʔϯೝࣝͱػցֶशɹԼרʯষࢀߟ
αϯϓϦϯάʹجͮ͘ਪख๏ ɹ؍ଌσʔλΛ ɼඇ؍ଌͷมͷू߹ʢFHύϥϝʔλɼજࡏม Λ ͱͨ͠ͱ ͖ɼϕΠζਪʹΑΔ౷ܭղੳͰɼ֬Ϟσϧ Λઃܭ͢Δඞཁ͕͋Δɽ ɹ࣮ࡍʹɼ֬ϞσϧΛ༻ֶ͍ͯश༧ଌɼࣄޙ Λܭࢉͯ͠ߦΘΕΔɽ X
Z p(X, Z) p(Z|X) ղܾࡦ ɹෳࡶͳϞσϧʢFHχϡʔϥϧωοτʣɼ ͕ղੳతʹٻΊΒΕͳ͍͜ͱ͕ ଟ͍ɽ p(Z|X) ɹ ΛղੳతʹٻΊΔΘΓʹɼ͜ͷ͔ΒෳͷαϯϓϧΛಘΔ͜ͱͰɼ ͷಛੑΛௐΔɽ ɹͱ͍͏͜ͱ͔ΒɼαϯϓϦϯά͢Δํ๏Λࠓճษڧ͢ΔΑʂ p(Z|X)
ຊͷ༰ ୯७ϞϯςΧϧϩ๏ غ٫αϯϓϦϯά ࣗݾਖ਼نԽॏαϯϓϦϯά ʢϚϧίϑ࿈ϞϯςΧϧϩ๏ʣ ϝτϩϙϦεɾϔΠεςΟϯάε๏ ϝτϩϙϦε๏ ϋϛϧτχΞϯϞϯςΧϧϩ๏ ϥϯδϡόϯϞϯςΧϧϩ๏ ΪϒεαϯϓϦϯά
ຊͷ༰ ୯७ϞϯςΧϧϩ๏ غ٫αϯϓϦϯά ࣗݾਖ਼نԽॏ৺αϯϓϦϯά ʢϚϧίϑ࿈ϞϯςΧϧϩ๏ʣ ϝτϩϙϦεɾϔΠεςΟϯάε๏ ϝτϩϙϦε๏ ϋϛϧτχΞϯϞϯςΧϧϩ๏ ϥϯδϡόϯϞϯςΧϧϩ๏ ΪϒεαϯϓϦϯά
.$.$
୯७ϞϯςΧϧϩ๏ ɹ ʹର͢Δؔ ͷظɹ ΛٻΊ͍ͨɽɹ p(z) f(z) p(z) [ f(z)]
= ∫ f(z)p(z)dz త ঢ়گ ɹظ ͷղੳతͳੵܭࢉ͕ࠔɽ ɹ ͔ΒͷαϯϓϦϯά༰қɽ ∫ f(z)p(z)dz p(z) ख๏ ɹ Λेେ͖ͳͱͨ͠ͱ͖ɼ T z(1), z(2), …, z(T) ∼ p(z) ∫ f(z)p(z)dz ≈ 1 T T ∑ t=1 f(z(t)) ͔Β ݸαϯϓϦϯά ⟵ p(z) T
ɹύϥϝʔλ Λ࣋ͭϞσϧ ͷपล Λܭࢉ͢Δࡍʹ༻͢ Δ߹ɼ θ p(X, θ) =
p(X|θ)p(θ) p(X) p(X) = ∫ p(X|θ)p(θ)dθ = ∫ N ∏ n=1 p(xn |θ)p(θ)dθ = p(θ) [p(X|θ)] ≈ 1 T T ∑ t=1 N ∏ n=1 p(xn |θ(t)), (θ(1), …, θ(T) ∼ p(θ)) ɹظ ʹ͓͍ͯɼ ͔Βͷαϯϓϧ ͷൣғ෯͘ͱΔඞཁ͕͋ΓɼҰํ Ͱɼ ڱ͍ ͷൣғͰ͔͠େ͖ͳΛऔΒͳ͍έʔε͕ଟ͍ɽ ɹ ൚༻త͚ͩͲɼܭࢉޮ͕ѱ͍ɽ p(z) [ f(z)] p(z) z f(z) z ⟹ ୯७ϞϯςΧϧϩ๏
غ٫αϯϓϦϯά ɹີܭࢉ͕ࠔͳ֬ ͔ΒαϯϓϧΛಘΔɽɹ p(z) z(1), z(2), … ∼ p(z) త
ঢ়گ ɹਖ਼نԽ͞Ε͍ͯͳ͍ؔ ܭࢉՄೳɽͭ·Γɼ ɽ ˜ p(z)( = Zp ⋅ p(z)) ∫ ˜ p(z)dz ≠ 1 ख๏ ɹఏҊ Λઃఆ͢Δɽҙͷ ʹରͯ͠ɼ ɹɹɹɹɹɹɹɹɹɹɹɹɹɹɹɹɹ ͱͳΔΑ͏ʹɼਖ਼ͷఆ ΛఆΊΔɽ q(z) z kq(z) > ˜ p(z) k ఏҊ ɹαϯϓϦϯά͕؆୯ʹߦ͑ΔΑ͏ͳ Ծͷɽ
غ٫αϯϓϦϯά ख๏ ͖ͭͮʣ ɹఏҊ ͔ΒαϯϓϧΛಘΔɽ ɹҰ༷ ͔ΒͷαϯϓϧΛಘΔɽ
ɹαϯϓϧ ͷड༰ʢBDDFQUʣغ٫ʢSFKFDUʣબɽ ɹɹɹ q(z) z(t) ∼ q(z) Uni(0,kq(z)) ˜ u ∼ Uni (0,kq(z(t))) z(t) if ˜ u > ˜ p(z(t)) then SFKFDU else BDDFQU ड༰ ∫ q(z) ˜ p(z) kq(z) dz = 1 k ∫ ˜ p(z)dz ߴ࣍ݩͷมͷαϯϓϦϯά͕ඞཁͳ߹ɼड༰͕ඇৗʹ͘ͳΔɽ
غ٫αϯϓϦϯά z ˜ p(z) ͷαϯϓϧΛغ٫αϯϓϧϦϯάͰ֫ಘ͢Δɽ ະɽ طɽ p(z) p(z) ˜
p(z) p(z)
غ٫αϯϓϦϯά z ˜ p(z) αϯϓϧ͕༰қͳఏҊ Λઃఆɽ p(z) q(z)
غ٫αϯϓϦϯά z ˜ p(z) kq(z) Λ෴͍͔Ϳ͞ΔΑ͏ʹ Λઃఆɽ ˜ p(z) k
kq(z) > ˜ p(z) q(z) × k
غ٫αϯϓϦϯά z ˜ p(z) kq(z) z(t) ఏҊ ͔ΒαϯϓϧΛಘΔɽ q(z) z(t)
∼ q(z) q(z)
غ٫αϯϓϦϯά z ˜ p(z) kq(z) z(t) kq(z(t)) ˜ u Ұ༷
͔ΒͷαϯϓϧΛಘΔ Uni(0,kq(z)) ˜ u ∼ Uni (0,kq(z(t)))
غ٫αϯϓϦϯά z ˜ p(z) kq(z) z(t) kq(z(t)) ड༰ غ٫ ˜
u αϯϓϧ ͷड༰ʢBDDFQUʣغ٫ʢSFKFDUʣબɽ z(t) if ˜ u > ˜ p(z(t)) then SFKFDU else BDDFQU ˜ p(z(t))
ࣗݾਖ਼نԽॏ৺αϯϓϦϯά ɹ ʹର͢Δؔ ͷظɹ Λ୯७ϞϯςΧϧϩ๏Α ΓޮతʹٻΊ͍ͨɽɹ p(z) f(z) p(z) [
f(z)] = ∫ f(z)p(z)dz త ঢ়گ ɹظ ͷղੳతͳੵܭࢉ͕ࠔɽ ɹ ͔ΒαϯϓϦϯάΛಘΒΕͳ͍ɽ ɹਖ਼نԽ͞Ε͍ͯͳ͍ؔ ܭࢉՄೳɽ ∫ f(z)p(z)dz p(z) ˜ p(z)( = Zp ⋅ p(z)) എܠ ɹغ٫αϯϓϦϯάΛ༻͍ͯɼ ΛΘͣʹαϯϓϧΛऔಘ͠ɼظ Λٻ ΊΔ͜ͱͰ͖Δ͕ɼ ͷ͕খ͞ͳྖҬʹαϯϓϧ͕ूத͢ΔՄೳੑ͕͋Δɽ ୯७ϞϯςΧϧϩ๏ͷܭࢉͷد༩͕গͳ͍ɽ ͷ͕େ͖͘ͳΔΑ͏ͳ ྖҬΛॏతʹαϯϓϧͨ͠ํ͕ޮ͕͍͍ɽ p(z) p(z) [ f(z)] f(z) ⟹ f(z)p(z)
ࣗݾਖ਼نԽॏ৺αϯϓϦϯά ɹ·ͣɼఏҊ Λઃఆ͢Δɽਖ਼نԽ͞Ε͍ͯͳ͍ؔ ɼҎԼͷΑ͏ʹද ͞ΕΔɽ ɹظͷܭࢉɼҎԼͷΑ͏ʹมܗͰ͖Δɽ q(z)
˜ p(z), ˜ q(z) p(z) = 1 Zp ˜ p(z), q(z) = 1 Zq ˜ q(z) ∫ f(z)p(z)dz = ∫ f(z) p(z) q(z) q(z)dz = q(z) [ f(z) p(z) q(z) ] = ∫ f(z) 1 Zp ˜ p(z) 1 Zq ˜ q(z) q(z)dz = Zq Zp q(z) [ f(z) ˜ p(z) ˜ q(z) ] ≈ Zq Zp 1 T T ∑ t=1 f(z(t)) ˜ p(z(t)) ˜ q(z(t)) = Zq Zp 1 T T ∑ t=1 f(z(t))w(t), w(t) = ˜ p(z(t)) ˜ q(z(t)) ख๏
ࣗݾਖ਼نԽॏ৺αϯϓϦϯά ɹ ະͷͳͷͰɼҎԼͷΑ͏ʹਖ਼نԽ߲ͷൺΛۙࣅ͢Δɽ ɹΑͬͯɼؔ ͷظ͕ۙࣅతʹಘΒΕΔɽ Zp
Zp Zq = ∫ ˜ p(z) Zq dz = ∫ ˜ p(z) ˜ q(z) q(z)dz = q(z) [ ˜ p(z) ˜ q(z) ] ≈ 1 T T ∑ t=1 w(t), z(1), …, z(T) ∼ q(z) f(z) ख๏ ͖ͭͮʣ ( ∵ p(z) = 1 Zp ˜ p(z) ) ( ∵ ∫ p(z)dz = 1 Zp ∫ ˜ p(z)dz = 1 )
ࣗݾਖ਼نԽॏ৺αϯϓϦϯά ɹ ະͷͳͷͰɼҎԼͷΑ͏ʹਖ਼نԽ߲ͷൺΛۙࣅ͢Δɽ ɹΑͬͯɼؔ ͷظ͕ۙࣅతʹಘΒΕΔɽ Zp
Zp Zq = ∫ ˜ p(z) Zq dz = ∫ ˜ p(z) ˜ q(z) q(z)dz = q(z) [ ˜ p(z) ˜ q(z) ] ≈ 1 T T ∑ t=1 w(t), z(1), …, z(T) ∼ q(z) f(z) ख๏ ͖ͭͮʣ ( ∵ p(z) = 1 Zp ˜ p(z) ) ( ∵ ∫ p(z)dz = 1 Zp ∫ ˜ p(z)dz = 1 ) ͳΜͷͨΊʹ Λ ͖࣋ͬͯͨΜͩΖ͏ʜ ˜ q(z)
ࣗݾਖ਼نԽॏαϯϓϦϯάʢޡهͷՄೳੑʣ ʮϕΠζਂֶशʯQ ɹࣗݾਖ਼نԽॏαϯϓϦϯάͷஈམͷ࠷ॳ ޡΓ ɹغ٫αϯϓϦϯάͱҟͳΔɼ ͔Βͷαϯϓϧʜʜ p(z) ɹ୯७ϞϯςΧϧϩ๏ͱҟͳΔɼ ͔Βͷαϯϓϧʜʜ p(z)
Ϛϧίϑ࿈ϞϯςΧϧϩ๏ʢ.$.$ʣ ɹغ٫αϯϓϦϯάͷɼߴ࣍ݩʹͳΔͱड༰͕ඇৗʹখ͘͞ͳΔ͜ͱɽ࣮ࡍ ʹɼ࣍ݩఔͷ؆୯ͳੵۙࣅʹ͔͠ద༻Ͱ͖ͳ͍ɽ ɹͰɼߴ࣍ݩۭؒͰޮతʹαϯϓϦϯά͢Δʹʜʜ ɹɹɹ Ϛϧίϑ࿈ϞϯςΧϧϩ๏ʢ.$.$ʣ͕ఏҊ͞Ε͍ͯΔɽ ⟹ ࣍Ϛϧίϑ࿈ ɹ֬มͷܥྻ
ʹରͯ͠ ͕Γཱͭͱ͖ͷܥྻ ͷ͜ͱɽ z(1), z(2), … p(z(t) |z(1), z(2), …, z(t−1)) = p(z(t) |z(t−1)) z(1), z(2), … άϥϑΟΧϧϞσϧ z(1) z(2) z(t−1) z(t) ⋯ ɹભҠ֬ɹΛ ͱ͓͍ͨͱ͖ɼ ͕Γཱͭͱ͖ɼ Λɹఆৗɹͱ͍͏ɽ (z(t−1), z(t)) = p(z(t) |z(t−1)) p* (z) = ∫ (z′ , z)p* (z′ )dz′ p* (z)
Ϛϧίϑ࿈ϞϯςΧϧϩ๏ʢ.$.$ʣ ɹఆৗ ʹऩଋ͢ΔΑ͏ͳભҠ֬ Λઃܭ͢Δͱɼ ͔Βαϯϓϧ ΛಘΔ͜ͱ͕Ͱ͖Δɽ ఆৗʹີܭࢉ͕ࠔͳ֬Λ͓͘ɽ p* (z) (z(t−1),
z(t)) p* (z) ⟹ ख๏ͷΩϞ ৄࡉΓ߹͍݅ p* (z)(z, z′ ) = p* (z′ )(z′ , z) ʲे݅ʳৄࡉΓ߹͍͕݅Γཱͭ ఆৗͱͳΔɽ ⟹ p* (z) p* (z)(z, z′ ) = p* (z′ )(z′ , z) ⇒ ∫ p* (z)(z, z′ )dz′ = ∫ p* (z′ )(z′ , z)dz′ ⇔ p* (z) ∫ p(z′ |z)dz′ = ∫ p* (z′ )(z′ , z)dz′ ⇔ p* (z) = ∫ p* (z′ )(z′ , z)dz′
Ϛϧίϑ࿈ϞϯςΧϧϩ๏ʢ.$.$ʣ ɹΓ߹͍݅ʹՃ͑ͯɼαϯϓϧ͕ ͱͨ͠ͱ͖ɼॳظঢ়ଶ ʹ͔͔ΘΒ ͣɼ ͕ఆৗ ʹऩଋ͢Δඞཁ͕͋Δɽ Τϧΰʔυੑ t →
∞ p(z(1)) p(z(t)) p* (z) ⟹ Τϧΰʔυੑ w طੑɹɿҙͷঢ়ଶ͔Βҙͷঢ়ଶ༗ݶճͰભҠՄೳɽ w ඇपظੑɿͯ͢ͷঢ়ଶ͕ݻఆͷपظੑΛͨͳ͍ɽ w ਖ਼࠶ؼੑɿಉ͡ঢ়ଶ͕༗ݶճͰΔ͜ͱ͕Մೳɽ
ϝτϩϙϦεɾϔΠεςΟϯάε๏ʢ.)๏ʣ త ɹະͷ֬ ͔ΒαϯϓϦϯάΛಘΔɽ p(z) લఏ ɹ ͱͳΔਖ਼نԽ͞Ε͍ͯͳ͍ؔ طͰ͋Δɽ p(z)
∝ ˜ p(z) ˜ p(z) ख๏ ɹભҠ֬ ͕ઃܭ͕͍͠߹ɼભҠͷఏҊ Λ͑Δɽ (z′ , z) q(z|z′ ) ɽఏҊ ͔Β࣍ͷαϯϓϧͷީิ ΛαϯϓϦϯά͢Δɽ ɽ࣍ͷൺ Λܭࢉ͢Δɽ ɽ Λ֬ ʹΑͬͯ ͱͯ͠ड༰͠ɼͦ͏Ͱͳ͍߹ ɼ ͱ͢Δɽ q( ⋅ |z(t)) z* r r = ˜ p(z* )q(z(t) |z* ) ˜ p(z(t))q(z* |z(t)) z* min(1,r) z(t+1) ⟵ z* z(t+1) ⟵ z(t) ΞϧΰϦζϜͷྲྀΕ
ϝτϩϙϦεɾϔΠεςΟϯάε๏ʢ.)๏ʣ ৄࡉΓ߹͍݅ͷূ໌ ɹભҠ֬ɼҎԼͷΑ͏ʹͳΔɽ (z, z′ ) = q(z′ |z) min
(1, ˜ p(z′ )q(z|z′ ) ˜ p(z)q(z′ |z) ) p(z)(z, z′ ) = p(z)q(z′ |z) min (1, ˜ p(z′ )q(z|z′ ) ˜ p(z)q(z′ |z) ) = p(z)q(z′ |z) min (1, p(z′ )q(z|z′ ) p(z)q(z′ |z) ) = min (p(z)q(z′ |z), p(z′ )q(z|z′ )) = min (p(z′ )q(z|z′ ), p(z)q(z′ |z)) = p(z′ )q(z|z′ ) min (1, p(z)q(z′ |z) p(z′ )q(z|z′ )) = p(z′ )q(z|z′ ) min (1, ˜ p(z)q(z′ |z) ˜ p(z′ )q(z|z′ )) = p(z′ )(z′ , z) ɹ ͷ߹ɼϝτϩϙϦε๏ͱݺΕΔɽ q(z′ |z) = q(z|z′ )
ϝτϩϙϦεɾϔΠεςΟϯάε๏ʢ.)๏ʣ ʮϕΠζਂֶशʯQࣜʢʣ ޡΓ (z, z′ ) = q(z′ |z) min
(1, ˜ p(z′ )q(z|z′ ) ˜ p(z)q(z′ |z) ) (z, z′ ) = q(z|z′ ) min (1, ˜ p(z′ )q(z|z′ ) ˜ p(z)q(z′ |z) )
ϝτϩϙϦεɾϔΠεςΟϯάε๏ʢ.)๏ʣ ۩ମྫͰֶͿ ඪʢະʣ ɹɹɹɹɹɹɹ ਖ਼نԽ͞Ε͍ͯͳ͍ؔʢطʣ ఏҊ
ͭ·Γ p(z) = (z|μ, Σ) = 1 (2π)D |Σ| exp {− 1 2 (z − μ)TΣ−1(z − μ)} ˜ p(z) = exp {− 1 2 (z − μ)TΣ−1(z − μ)} q(z* |z) = (z′ |z, I) z* ∼ (z, I) https://drive.google.com/open?id=1vcBZWp9HPzfCzBjj2INdkH_CJUDL-JA3
ɹαϯϓϦϯάͷલʹϋϛϧτχΞϯΛར༻ͨ͠ղੳֶతͳγϛϡϨʔγϣϯΛղ આɽຎࡲʹΑΔΤωϧΪʔͷݮগ͕ͳ͍ͱԾఆ͢ΔͱɼϋϛϧτχΞϯҎԼͷΑ͏ʹ ද͞ΕΔɽ ℋ(z, p) = (z) + (p), (p)
= 1 2m pTp, z ∈ ℝD: ମͷҐஔϕΫτϧ, p ∈ ℝD: ମͷӡಈྔϕΫτϧ, m ∈ ℝ: ମͷ࣭ྔ, ℋ(z, p): ϋϛϧτχΞϯ, (z): ϙςϯγϟϧΤωϧΪʔ, (p): ӡಈΤωϧΪʔ . ϋϛϧτχΞϯϞϯςΧϧϩ๏ʢ).$๏ʣ ).$๏ʢϋΠϒϦοτϞϯςΧϧϩ๏ʣʹମͷيಓͷझຯϨʔγϣϯ .)๏ × ɹ).$๏ɼϥϯμϜΥʔΫతͳ.)ͱൺͯɼޮతʹۭؒΛ୳ࡧՄೳɽ ϋϛϧτχΞϯͷγϛϡϨʔγϣϯ
ϋϛϧτχΞϯϞϯςΧϧϩ๏ʢ).$๏ʣ ɹ࣭ྔΛͱ͠ɼ ͱ ͷ࣌ؒ ʹؔ͢ΔڍಈɼϋϛϧτχΞϯͷภඍʹΑܾͬͯఆɽ ͜ͷඍํఔ͕ࣜղੳతʹղ͚ͳ͍ͷͱ͠ɼγϛϡϨʔγϣϯʹΑͬͯيಓΛܭ ࢉ͢Δɽ z
p τ dpi dτ = − dℋ dzi = − d dzi , dzi dτ = dℋ dpi = d dpi . ΦΠϥʔ๏ ࣌ࠁ ઌͷڍಈΛۙࣅతʹ༧ଌɽ ϵ > 0 pi (τ + ϵ) = pi (τ) + ϵ dpi dτ τ = pi (τ) − ϵ d dzi zi (τ) , zi (τ + ϵ) = zi (τ) + ϵ dzi dτ τ = zi (τ) + ϵpi (τ) ࢄԽʹΑΔޡ͕ࠩେ͖͍ɽ Ϧʔϓϑϩοά๏ ⟹
ϋϛϧτχΞϯϞϯςΧϧϩ๏ʢ).$๏ʣ Ϧʔϓϑϩοά๏ pi (τ + ϵ 2 ) = pi
(τ) − ϵ 2 d dzi zi (τ) , zi (τ + ϵ) = zi (τ) + ϵpi (τ + ϵ 2 ), pi (τ + ϵ) = pi (τ + ϵ 2 ) − ϵ 2 d dzi zi (τ + ϵ) . ͜ΕΛ ճ܁Γฦ͢͜ͱͰ࣌ࠁ ઌͷମͷҐஔ ͱӡಈྔ ΛۙࣅతʹܭࢉͰ͖Δɽ L ϵL z* p* ϋϛϧτχΞϯͷੑ࣭ ɽ ࣌ؒ ʹΑͬͯෆมɽ ɽՄٯੑɿ ͔Β ͷભҠҰରҰɽ ɽମੵอଘ ℋ τ (z, p) (z* , p* )
ϋϛϧτχΞϯϞϯςΧϧϩ๏ʢ).$๏ʣ αϯϓϦϯάΞϧΰϦζϜͷద༻ త ɹະͷ֬ ͔ΒαϯϓϦϯάΛಘΔɽ p(z) લఏ ɹ ͱͳΔਖ਼نԽ͞Ε͍ͯͳ͍ؔ طͰ͋Δɽ
ɹ ͱ֦ு͢Δͱɼ पล ͔Βαϯϓϧ͕ಘΒΕΔɽ ɹ p(z) ∝ ˜ p(z) ˜ p(z) p(z, p) = p(z)p(p) z p(z) p(p) = (p|0, I) (z) = − log (˜ p(z)) (p) = 1 2 pTp
ϋϛϧτχΞϯϞϯςΧϧϩ๏ʢ).$๏ʣ ɹಉ࣌Λܭࢉ͢ΔͱɼҎԼͷΑ͏ʹͳΔɽ ϝτϩϙϦε๏ͰΘΕΔൺ ɼҎԼͷΑ͏ʹͳΔɽ p(z, p) = p(z)p(p) =
exp (log p(z) + log p(p)) ∝ exp (log ˜ p(z) − 1 2 pTIp ) = exp (−(z) − (p)) = exp (−ℋ(z, p)) r r = p(z* , p* ) p(z, p) = exp (−ℋ(z* , p* ) + ℋ(z, p))
ϋϛϧτχΞϯϞϯςΧϧϩ๏ʢ).$๏ʣ ख๏ ɽӡಈྔΛαϯϓϦϯά ɽϦʔϓϑϩοά๏Ͱݱࡏͷ ͔Βީิ ΛಘΔɽ ɽ࣍ͷൺ Λܭࢉ͢Δɽ
ɽ Λ֬ ʹΑͬͯ ͱͯ͠ड༰͠ɼͦ͏Ͱͳ͍߹ ɼ ͱ͢Δɽ p ∼ (0, I) (z(t), p) (z* , p* ) r r = p(z* , p* ) p(z, p) z* min(1,r) z(t+1) ⟵ z* z(t+1) ⟵ z(t) ΞϧΰϦζϜͷྲྀΕ
ϋϛϧτχΞϯϞϯςΧϧϩ๏ʢ).$๏ʣ Ϧʔϓϑϩοάͷύϥϝʔλʹؔͯ͠ɼҎԼͷΑ͏ͳτϨʔυΦϑ͕͋Δɽ ɹ).$๏ɼࣄޙͷඍ͑͞ܭࢉͰ͖Εద༻Ͱ͖ɼඇৗʹ൚༻తɽҰൠతͳ χϡʔϥϧωοτϫʔΫ࿈ଓͳજࡏมͷΈͰΓཱ͍ͬͯΔ͜ͱ͕ଟ͍ͷͰɼ).$ ๏χϡʔϥϧωοτϫʔΫͷϕΠζֶशʹΘΕ͖ͯͨɽ େ͖͍ εςοϓαΠζ ϵ εςοϓ L
খ͍͞ খ͍͞ େ͖͍ ड༰ ड༰ ୳ࡧޮ ܭࢉྔ ߴ͍ ߴ͍ ͍ ͍ େ͖͍ খ͍͞ ߴ͍ ͍
ϋϛϧτχΞϯϞϯςΧϧϩ๏ʢ).$๏ʣ ϥϯδϡόϯಈྗֶ๏ ɹ ͱͨ͠߹ɼϥϯδϡόϯϞϯςΧϧϩ๏ɹ·ͨɹϥϯδϡόϯಈྗֶ๏ɹͱ ݺΕΔɽ ɹਂֶश͚ʹϛχόονֶश͕ߦ͑ΔΑ͏ʹͨ͠ɹ֬తޯϥϯδϡόϯಈྗֶ ๏ɹʹల։͞ΕΔɽ L =
1 z*i = zi (τ + ϵ) = zi (τ) + ϵ pi (τ) − ϵ 2 d dzi zi (τ) = zi (τ) − ϵ2 2 d dzi zi (τ) + ϵpi (τ)
ඪʢະʣ ɹɹɹɹɹɹɹ ਖ਼نԽ͞Ε͍ͯͳ͍ؔʢطʣ ӡಈྔͷαϯϓϦϯάɿ ӡಈΤωϧΪʔɿ
ҐஔΤωϧΪʔɿ ҐஔΤωϧΪʔͷภඍɿ p(z) = (z|μ, Σ) = 1 (2π)D |Σ| exp {− 1 2 (z − μ)TΣ−1(z − μ)} ˜ p(z) = exp {− 1 2 (z − μ)TΣ−1(z − μ)} p ∼ (0, I) (p) = 1 2 pTp (z) = − log (˜ p(z)) = − 1 2 (z − μ)TΣ−1(z − μ) ∂ ∂z = − (z − μ)TΣ−1 ϋϛϧτχΞϯϞϯςΧϧϩ๏ʢ).$๏ʣ ۩ମྫͰֶͿ https://drive.google.com/open?id=11zWctTbECXEhlHm7AqPiXC_MErAYl7hJ
ΪϒεαϯϓϦϯά త ɹ֬ ͔Β શମΛαϯϓϦϯά͢Δ͜ͱ͕͍͠ͱ͖ͷ୳ࡧɽ p(Z) Z લఏ ɹ֬ طɽ
p(Z) ख๏ ɹɽม Λ ݸͷ෦ू߹ʹ͚Δɽ ɹɽ෦ू߹Λஞ࣍తʹ୳ࡧ͢Δɽ ɹ Z M Z = {Z1 , Z2 , …, ZM } Z1 ∼ p(Z1 |Z2 , Z3 , …, ZM−1 , ZM ) Z2 ∼ p(Z2 |Z1 , Z3 , …, ZM−1 , ZM ) ⋮ ZM ∼ p(ZM |Z1 , Z2 , …, ZM−2 , ZM−1 )
ΪϒεαϯϓϦϯά ɹΪϒεαϯϓϦϯάͷଥੑɼαϯϓϦϯάͷखଓ͖͕.)๏ͷҰछͱͯ͠ղऍͰ ͖Δ͜ͱ͕อূ͞Ε͍ͯΔɽ ͷΑ͏ʹ͚ɼ Λ͚݅ͨ͠ͱͰ ͷα ϯϓϦϯάΛ͢Δ͜ͱΛߟ͑ͨ߹ɼ ͔ͭ ɹൺ
Λܭࢉ͢ΔͱҎԼͷΑ͏ʹͳΔɽ Αͬͯɼશͯड༰͞ΕΔɽ ಉ༷ɽ Z = {Z1 , Z2 } Z2 Z1 q(Z* |Z) = p(Z1* |Z2* ) Z2 = Z2* r r = p(Z* )q(Z|Z* ) p(Z)q(Z* |Z) = p(Z1* , Z2* )p(Z1 |Z2* ) p(Z1 , Z2 )p(Z1* |Z2 ) = p(Z1* |Z2* )p(Z2* )p(Z1 |Z2* ) p(Z1 |Z2 )p(Z2 )p(Z1* |Z2 ) = 1 Z2
ΪϒεαϯϓϦϯά ۩ମྫͰֶͿ ඪʢطʣ ɹɹɹɹ ͱ͢Δɽ p(z) = (z|μ,
Σ) = 1 (2π)D |Σ| exp {− 1 2 (z − μ)TΣ−1(z − μ)} z = ( z1 z2 ), μ = ( μ1 μ2 ), Σ = ( Σ11 Σ12 Σ21 Σ22 ), Λ = Σ−1 = ( Λ11 Λ12 Λ21 Λ22 ) log p(z) = log p(z1 , z2 ) = − 1 2 (z1 − μ1 )TΛ11 (z1 − μ1 ) + (z1 − μ1 )TΛ12 (z2 − μ2 )) − 1 2 (z2 − μ2 )TΛ22 (z2 − μ2 ) + (z2 − μ2 )TΛ21 (z1 − μ1 )) = − 1 2 ( zT 1 Λ11 z1 − 2z1 {Λ11 μ1 − 1 2 Λ12 (z2 − μ2 )}) + C1 − 1 2 ( zT 2 Λ22 z2 − 2z2 {Λ22 μ2 − 1 2 Λ21 (z1 − μ1 )}) + C2
ΪϒεαϯϓϦϯά ۩ମྫͰֶͿ Αͬͯɼ͖݅֬ͷରɼ ͱͳΔͷͰɼ͖݅֬ΨεͰ͋Δɼ log p(zi |zj
) = − 1 2 ( zT i Λii zi − 2zi { Λii μi − 1 2 Λij (zj − μj )}) + C p(zi |zj ) = (zi |μi , Σi ), Σ−1 i = Λii , Σ−1 i μi = Λii μi − 1 2 Λij (zj − μj ), ⇔ μi = Σi ( Λii μi − 1 2 Λij (zj − μj )) . https://drive.google.com/open?id=1ReYNvvH-NgtsuRiDDV-lz1779sps2pT0