PRML第10章

Transcript

1. ୈ 5 ճ PRML ηϛφʔ ࡔޱ ྒี 1 / 51
2. 0. ࠓճͷηϛφʔʹ͍ͭͯ ࠓճͷηϛφʔͰ͸ɺPRML ͷୈ 10 ষͷۙࣅਪ࿦๏Λ͓࿩͍ͨ͠͠ͱ ࢥ͍·͢ɻ ·ͨɺ͜ΕΒͷ࿩୊Λઆ໌͢ΔͨΊʹඞཁͳ༧උ஌ࣝΛղઆ͠·͢ɻ (PRML 9.4) ͳ͓஫ҙ఺ͱͯ͠ɺຊεϥΠυͷࣜ൪߸ͱ PRML ͷࣜ൪߸͸ҟͳΓ· ͢ͷͰɺ͝஫ҙ͍ͩ͘͞ɻ 2 / 51
3. ໨࣍ 1. ༧උ஌ࣝ (PRML 9.4) 2. ۙࣅਪ࿦๏ 2-1. ม෼ਪ࿦ (PRML 10.1) 2-2. ྫɿม෼ࠞ߹Ψ΢ε෼෍ (PRML 10.2) 2-3. ہॴతม෼ਪ࿦๏ (PRML 10.5) 2-4. ม෼ϩδεςΟοΫճؼ (PRML 10.6) 3 / 51
4. 1. ༧උ஌ࣝ EM ΞϧΰϦζϜ͸؍ଌର৅Ͱ͋Δ؍ଌม਺ X = (x1 , · · · , xN )T ͱ؍ ଌର৅Ͱͳ͍જࡏม਺ Z = (z1 , · · · , zN )T ΛؚΉܥ (۩ମྫͱͯࠞ͠߹ Ψ΢ε෼෍͕͋Δ) ͷ࠷໬ਪఆΛߦ͏ࡍͷख๏ͷҰͭͰ͋ΓɺE εςο ϓͱ M εςοϓͱ͍͏ 2 ͭͷεςοϓΛ൓෮తʹ࣮ߦ͢Δ͜ͱͰ໬౓ ؔ਺Λ࠷େԽ͢Δɻ ͜͜Ͱ͸ɺҰൠతͳ EM ΞϧΰϦζϜʹ͍ͭͯ؆୯ʹઆ໌͢Δɻ (PRML 9.4 ʹରԠ) 4 / 51
5. 1. ༧උ஌ࣝ ·ͣɺਪఆ͍ͨ͠ύϥϝʔλϕΫτϧΛ θ ͱ͠ɺN ݸͷ؍ଌม਺ X = (x1 , · · · , xN )T ʹରԠ͢Δજࡏม਺ Z = (z1 , · · · , zN )T Λ֬཰ม਺ ͱ͢Δ೚ҙͷ֬཰෼෍ؔ਺ q(Z) Λಋೖ͢Δͱɺ؍ଌσʔλ X ͷର਺໬ ౓ؔ਺ (ෆ׬શσʔλͷର਺໬౓ؔ਺)ln p(X|θ) ͸ҎԼͷΑ͏ʹͳΔɻ ln p(X|θ) = { ∑ Z q(Z) } =1 ln p(X|θ) = ∑ Z q(Z) ln { p(X, Z|θ) p(Z|X, θ) } = ∑ Z q(Z) ln { p(X, Z|θ) q(Z) · q(Z) p(Z|X, θ) } = ∑ Z q(Z) ln { p(X, Z|θ) q(Z) } − ∑ Z q(Z) ln { p(Z|X, θ) q(Z) } (1.1) 5 / 51
6. 1. ༧උ஌ࣝ ͜͜Ͱɺ৐๏ఆཧ p(X, Z|θ) = p(Z|X, θ)p(X|θ) (1.2) Λ༻͍ͨɻ ·ͨɺ L(q, θ) = ∑ Z q(Z) ln { p(X, Z|θ) q(Z) } (1.3) KL(q∥p) = − ∑ Z q(Z) ln { p(Z|X, θ) q(Z) } (1.4) ͱ͢Ε͹ɺ(1.1) ͸ ln p(X|θ) = L(q, θ) + KL(q∥p) (1.5) ͱͳΔɻ(KL(q∥p) ͸ KL μΠόʔδΣϯε) 6 / 51
7. 1. ༧උ஌ࣝ ͜͜ͰɺPRML ͷ 1.6.1 ΑΓɺKL μΠόʔδΣϯε KL(q∥p) ͸ඇෛͰ ͋Δɻ(KL(q∥p) ≥ 0 Ͱ͋Γɺ౳߸͸ q(Z) = p(Z|X, θ) ͷͱ͖ͷΈ੒ཱ ͢Δɻ) Ҏ্ΑΓɺln p(X|θ) ≥ L(q, θ) ͱͳΓɺL(q, θ) ͸ ln p(X|θ) ͷԼݶͰ͋ Δɻ(Լਤ) 7 / 51
8. 1. ༧උ஌ࣝ ෼ղ (1.5) Λར༻ͯ͠ EM ΞϧΰϦζϜΛఆٛ͠ɺͦΕ͕ର਺໬౓ ln p(X|θ) Λ࠷େԽ͢Δ͜ͱΛࣔ͢ɻ ͦͷͨΊʹ·ͣɺύϥϝʔλͷ஋Λ θold ʹॳظԽ͢Δɻ E εςοϓͰ͸ɺL(q, θold) Λ θold Λݻఆ͠ͳ͕Β q(Z) ʹ͍ͭͯ࠷େ Խ͢Δɻ(͜Ε͕ͷͪʹग़ͯ͘Δม෼๏ʹରԠ) ͜ͷ࠷େԽ͸؆୯Ͱɺର਺໬౓ ln p(X|θold) ͸ q(Z) ʹґଘ͠ͳ͍ͷͰɺ q(Z) ΛมԽͤͯ͞΋ ln p(X|θold) ͸มԽ͠ͳ͍ɻ ͜ΕΑΓɺKL(q∥p) = 0 ͭ·Γ q(Z) = p(Z|X, θold) ͷͱ͖ɺL(q, θold) ͸࠷େʹͳΔɻ ͜ͷͱ͖ɺԼਤͷΑ͏ʹ ln p(X|θold) = L(q, θold) ͱͳΔɻ 8 / 51
9. 1. ༧උ஌ࣝ ࣍ͷ M εςοϓͰ͸ɺq(Z) = p(Z|X, θold) Ͱݻఆ͠ɺL(q, θ) Λ θ ʹͭ ͍ͯ࠷େԽ͢Δɻ L(q, θ) Λ࠷େʹ͢ΔύϥϝʔλΛ θnew ͱ͢Δͱɺ͢Ͱʹ L(q, θ) ͕࠷ େ஋ʹୡ͍ͯ͠ͳ͍ݶΓ (θnew ̸= θold ͷͱ͖)ɺL(q, θ) ͸૿Ճ͠ɺ͞Β ʹ KL μΠόʔδΣϯε͸ҎԼͷΑ͏ʹਖ਼ʹͳΔɻ KL(q∥p) = − ∑ Z p(Z|X, θold) ln { p(Z|X, θnew) p(Z|X, θold) } > 0 (1.6) 9 / 51
10. 1. ༧උ஌ࣝ Αͬͯɺ͜ΕΒͷ E εςοϓͱ M εςοϓ͸ҎԼͷਤͷΑ͏ʹର਺໬ ౓ ln p(X|θ) Λ૿Ճͤ͞Δɻ 10 / 51
11. 1. ༧උ஌ࣝ ·ͨɺq(Z) = p(Z|X, θold) ͷͱ͖ͷ L(q, θ) ͸ L(q, θ) = ∑ Z p(Z|X, θold) ln { p(X, Z|θ) p(Z|X, θold) } = ∑ Z p(Z|X, θold) ln p(X, Z|θ) − ∑ Z p(Z|X, θold) ln p(Z|X, θold) =Q(θ, θold) + const. (1.7) ͜͜ͰɺQ(θ, θold) ͸ҎԼͷΑ͏ʹఆٛͨ͠ɻ Q(θ, θold) = ∑ Z p(Z|X, θold) ln p(X, Z|θ) (1.8) ͜͜Ͱͷ const. ͱ͸ɺθ ʹґଘ͠ͳ͍߲Ͱ͋Δɻ ͜ΕΑΓɺM εςοϓͷ L(q, θ) ͷ࠷େԽ͸ Q(θ, θold) ͷ࠷େԽͱ౳Ձ Ͱ͋Δɻ 11 / 51
12. 2. ۙࣅਪ࿦๏ EM ΞϧΰϦζϜͰ͸ɺ؍ଌม਺Λ X ͱ͠જࡏม਺Λ Z ͱ͠ύϥϝʔ λΛ θ ͢ΔͱɺE εςοϓͱͯ͠ࣄޙ෼෍ p(Z|X, θ) ΛٻΊɺM εςο ϓͱͯ͠׬શର਺໬౓ ln p(X, Z|θ) ͷࣄޙ෼෍ ln p(X, Z|θold) ʹΑΔظ ଴஋ Q(θ, θold) Λ࠷େʹ͢Δύϥϝʔλ θ ΛٻΊͨɻ ࠓޙͷٞ࿦ʹΑΓɺEM ΞϧΰϦζϜͷߟ͑ํΛϕΠζਪఆʹԠ༻Ͱ ͖ɺ͜ͷ࣌ࣄޙ෼෍͕ٻΊΔ͜ͱ͕Ͱ͖ͳ͍͕࣌͋Δɻ ͦͷΑ͏ͳ৔߹ʹۙࣅతʹظ଴஋ΛٻΊΔํ๏ͱͯ͠ɺ͜͜Ͱ͸ม෼ਪ ࿦๏Λಋೖ͢Δɻ(2-1) ͦͷม෼ਪ࿦๏Λࠞ߹Ψ΢εϞσϧʹదԠ͢Δɻ(2-2) ·ͨɺ෼෍ͷ෼ղͱ͸ผͷਪ࿦๏ͱͯ͠ɺہॴతม෼ਪ࿦๏Λಋೖ͠ (2-3)ɺͦͷਪ࿦๏ΛϩδεςΟοΫճؼʹద༻͢Δ (2-4)ɻ 12 / 51
13. 2-1. ม෼ਪ࿦ ม෼ਪ࿦๏͸ม෼๏ͱ͍͏਺ֶతख๏Λىݯʹ࣋ͭɻ ม෼๏Ͱ͸ɺҎԼͷΑ͏ͳ൚ؔ਺ (ؔ਺ͷܗ͕ܾ·Ε͹஋͕ܾ·Δྔ) ʹ͓͍ͯɺՄೳͳ͢΂ͯͷؔ਺ͷதͰ൚ؔ਺Λ࠷େ΋͘͠͸࠷খʹ͢Δ Α͏ͳؔ਺Λ୳͢ํ๏Ͱ͋Δɻ(෇࿥ D ࢀর) H[p] = − ∫ p(x) ln p(x) dx (2.1) (2.1) ͸ΤϯτϩϐʔͰ͋Γɺ͜Εʹม෼๏Λద༻͢Δͱɺ p(x) = const. ͕ H[p] Λ࠷େʹ͢Δ෼෍Ͱ͋Δࣄ͕Θ͔Γɺ͔ͨ͠ʹΤ ϯτϩϐʔͷ௚ײͱҰக͢Δɻ ม෼๏ࣗମ͸ɺ൚ؔ਺Λ࠷େ΋͘͠͸࠷খʹ͢ΔΑ͏ͳؔ਺ΛՄೳͳ͢ ΂ͯͷؔ਺ͷத͔Β୳͢ख๏ͳͷͰɺۙࣅతͳख๏Ͱ͸ͳ͍ɻ ม෼ਪ࿦๏Ͱ͸ɺ࠷దԽΛߦ͏ؔ਺ͷΫϥεΛ੍ݶ͠ɺۙࣅతʹ൚ؔ਺ Λ࠷େ΋͘͠͸࠷খʹ͢ΔΑ͏ͳؔ਺ΛٻΊΔɻ 13 / 51
14. 2-1. ม෼ਪ࿦ ͦΕͰ͸ɺม෼ਪ࿦๏ͷԠ༻ํ๏Λݟ͍ͯ͘ɻ ·ͣɺ͢΂ͯͷύϥϝʔλʹࣄલ෼෍͕༩͑ΒΕ͍ͯΔ׬શͳϕΠζϞ σϧ͕͋Δͱ͢Δɻ Ϟσϧʹ͸ύϥϝʔλͷଞʹજࡏม਺͕͋ΔՄೳੑ͕͋Γɺύϥϝʔλ ͱજࡏม਺Λ·ͱΊͯ Z ͱ͔͖ɺ͞Βʹ؍ଌม਺ͷू߹Λ X ͱ͢Δɻ ·ͨɺ؍ଌม਺ͱજࡏม਺ͷಉ࣌෼෍ p(X, Z) ͸༩͑ΒΕ͍ͯΔͱ ͢Δɻ ͔͠͠ɺࣄޙ෼෍ p(Z|X) = p(X, Z) ∫ p(X, Z) dZ (2.2) ͸જࡏม਺ͷ࣍ݩ͕ߴ͗͢ΔͳͲͷཧ༝Ͱੵ෼ෆՄೳͳͨΊɺٻΊΒΕ ͳ͍ͱ͢Δɻ 14 / 51
15. 2-1. ม෼ਪ࿦ ͜͜Ͱɺपล෼෍ p(X)(ϞσϧΤϏσϯε) ͷର਺͸ҎԼͷΑ͏ʹͳΔ ͜ͱΛ༻͍Δɻ ln p(X) = L[q] + KL(q∥p) (2.3) ͜͜Ͱɺ L[q] = ∫ q(Z) ln { p(X, Z) q(Z) } dZ (2.4) KL(q∥p) = − ∫ q(Z) ln { p(Z|X) q(Z) } dZ (2.5) Ͱ͋Δɻ KL μΠόʔδΣϯεͷੑ࣭ΑΓɺ࠷খʹͳΔͷ͸෼෍ q(Z) ͕ࣄޙ෼ ෍ p(Z|X) ʹҰக͢Δͱ͖Ͱ͋Δɻ ͦΕ͸ L[q] ͕࠷େʹͳΔͱ͖Ͱ͋ΓɺͦͷΑ͏ͳ෼෍ q(Z) ͸ࣄޙ෼෍ Ͱ͋Δɻ 15 / 51
16. 2-1. ม෼ਪ࿦ ͨͩ͠ԾఆʹΑΓɺࣄޙ෼෍ p(Z|X) ͸ܭࢉͰ͖ͳ͍ͷͰɺ୅ΘΓʹ q(Z) ͷऔΓ͏Δؔ਺ʹ੍ݶΛ༩͑ɺͦͷ੍ݶͷதͰ L[q] Λ࠷େʹ͢Δ Α͏ͳ q(Z) ΛٻΊΔࣄΛߟ͑Δɻ(ม෼ਪ࿦๏) ·ͣ͸੍ݶͷྫͱͯ͠ɺ෼෍ͷ෼ղΛߟ͑Δɻ 16 / 51
17. 2-1-1. ෼෍ͷ෼ղ ͜͜Ͱ͸ɺq(Z) ͕ҎԼͷΑ͏ͳܗΛऔ͍ͬͯΔ΋ͷʹ੍ݶ͢Δɻ q(Z) = M ∏ i=1 qi (Zi ) (2.6) ͜͜ͰɺZ = (z1 , z2 , · · · , zN )T Λഉ൓ͳ (ॏෳ͠ͳ͍) άϧʔϓ Zi ʹΘ ͚ͨɻ ͜͜Ͱɺqi (Zi ) ͷܗʹ͸੍ݶ͸ͳ͍ɻ (2.6) Λ (2.4) ͷӈลʹ୅ೖͯ͠ɺL[q] Λ֤ qi (Zi ) Ͱॱ൪ʹ࠷దԽ͢Δ ͜ͱΛߟ͑Δɻ ͦͷͨΊɺL[q] ͷ qj (Zj ) ʹґଘ͢Δ߲ͷΈΛऔΓग़͢ɻ 17 / 51
18. 2-1-1. ෼෍ͷ෼ղ L[q] ͷ qj (Zj ) ʹґଘ͢Δ߲͸ҎԼͷΑ͏ʹͳΔɻ L[q] = ∫ q(Z) ln { p(X, Z) q(Z) } dZ = ∫ ( ln p(X, Z) − M ∑ l=1 ln ql(Zl) ) M ∏ i=1 qi(Zi) dZi = ∫ ln p(X, Z) M ∏ i=1 qi(Zi) dZi − M ∑ l=1 ∫ ln ql(Zl) M ∏ i=1 qi(Zi) dZi = ∫ qj (Zj ) { ∫ ln p(X, Z) ∏ i̸=j qi(Zi) dZi } dZj − M ∑ l=1 ∫ ql(Zl) ln ql(Zl) dZl = ∫ qj (Zj ) { ∫ ln p(X, Z) ∏ i̸=j qi(Zi) dZi } dZj − ∫ qj (Zj ) ln qj (Zj ) dZj + const. = ∫ qj (Zj ) ln p(X, Zj ) dZj − ∫ qj (Zj ) ln qj (Zj ) dZj + const. (2.7) 18 / 51
19. 2-1-1. ෼෍ͷ෼ղ ͜͜Ͱɺ ln p(X, Zj ) = Ei̸=j [ln p(X, Z)] + const. (2.8) Ͱ͋Γɺ Ei̸=j [ln p(X, Z)] = ∫ ln p(X, Z) ∏ i̸=j qi (Zi ) dZi (2.9) ͱఆٛͨ͠ɻ 19 / 51
20. 2-1-1. ෼෍ͷ෼ղ ࣍ʹɺ(2.7) ͷ L[q] Λ qj (Zj ) ʹ͍ͭͯ࠷େԽ͢Δ͜ͱΛߟ͑Δɻ ͜Ε͸؆୯Ͱɺ(2.7) ͸ KL μΠόʔδΣϯεΛ༻͍ͯ L[q] = ∫ qj (Zj ) ln ( p(X, Zj ) qj (Zj ) ) dZj + const. = − KL(qj ∥p) + const. (2.10) ͱॻ͚ΔͷͰɺKL μΠόʔδΣϯεͷੑ࣭Λ༻͍ΔͱɺL[q] Λ࠷େʹ ͢Δ qj (Zj ) ͸ ln q⋆ j (Zj ) = ln p(X, Zj ) = Ei̸=j [ln p(X, Z)] + const. (2.11) ͱͳΔɻ ͜͜Ͱɺ(2.11) ͷ࠶ӈล͸ qi (Zi ) (i ̸= j) Ͱͷظ଴஋ʹґଘ͍ͯ͠Δͷ Ͱɺղ q⋆(Z) ΛٻΊΔʹ͸ɺ͢΂ͯͷҼࢠ qj (Zj ) Λద౰ʹॳظԽ͠ɺ (2.11) Λ༻͍ͯ͋ΔҼࢠ qj (Zj ) ΛଞͷҼࢠͷݱࡏͷ஋Ͱߋ৽͍ͯ͘͠ ͜ͱͰٻΊΔɻ(ม෼ϕΠζ๏) 20 / 51
21. 2-2. ྫɿม෼ࠞ߹Ψ΢ε෼෍ Ҏ্Ͱ঺հͨ͠ม෼ϕΠζ๏Λࠞ߹Ψ΢εϞσϧʹద༻ͯ͠ΈΔɻ ࠞ߹Ψ΢εϞσϧ͸༗޲άϥϑΛ༻͍ΔͱҎԼͷΑ͏ʹදͤΔɻ ͜͜Ͱɺθ = {π, µ, Λ} ͸ύϥϝʔλͰ͋ΓɺX = (x1 , · · · , xN )T ͸؍ ଌม਺Ͱ Z = (z1 , · · · , zN )T ͸ͦΕʹରԠ͢Δજࡏม਺Ͱ͋Δɻ 21 / 51
22. 2-2. ྫɿม෼ࠞ߹Ψ΢ε෼෍ ͜ͷάϥϑΑΓɺ֤֬཰ม਺ͷ৚݅෇͖ಠཱੑΛՃຯ͢Δͱɺಉ࣌෼෍ p(X, Z, π, µ, Λ) ͸ҎԼͷΑ͏ʹ෼ղͰ͖Δɻ(PRML 8 ষΛࢀর) p(X, Z, π, µ, Λ) = p(X|Z, µ, Λ)p(Z|π)p(π)p(µ|Λ)p(Λ) (2.12) ͜ΕΒͷӈลͷ෼෍Λಛఆͷ෼෍ʹԾఆ͍ͯ͘͠ɻ ·ͣɺ෼෍ p(Z|π) ΛҎԼͷΑ͏ʹԾఆ͢Δɻ p(Z|π) = N ∏ n=1 K ∏ k=1 πznk k (2.13) ͜͜Ͱɺࣄલ෼෍ p(π) ͸ҎԼͷΑ͏ʹσΟϦΫϨ෼෍Λ༻͍Δͱɺ͜ ͷ෼෍͸ڞ໾ࣄલ෼෍ͱͳ͓ͬͯΓɺࣄޙ෼෍΋σΟϦΫϨ෼෍ͱ ͳΔɻ p(π) = Dir(π|α0 ) = C(α0 ) K ∏ k=1 πα0+1 k (2.14) ͜͜ͰɺC(α0 ) ͸σΟϦΫϨ෼෍ͷਖ਼نԽఆ਺ ((B.23) Ͱఆٛ͞Εͯ ͍Δ) 22 / 51
23. 2-2. ྫɿม෼ࠞ߹Ψ΢ε෼෍ ࣍ʹɺ෼෍ p(X|Z, µ, Λ) ΛҎԼͷΑ͏ʹΨ΢ε෼෍ͰԾఆ͢Δɻ p(X|Z, µ, Λ) = N ∏ n=1 K ∏ k=1 N(xn |µk , Λ−1 k )znk (2.15) ͜͜ͰɺΨ΢ε෼෍ N(x|µ, Σ) ͸ҎԼͰఆٛ͞ΕΔɻ N(x|µ, Σ) = 1 (2π)D/2 1 |Σ|1/2 exp { − 1 2 (x − µ)TΣ−1(x − µ) } (2.16) ͞Βʹ (2.15) Λड͚ͯɺ෼෍ p(µ, Λ) = p(µ|Λ)p(Λ) ͸ڞ໾ࣄલ෼෍Ͱ ͋ΔΨ΢ε-΢Ογϟʔτࣄલ෼෍ (PRML 2.3.6 ͷ (2.157) ࢀর) ͱԾఆ ͢Δɻ p(µ, Λ) = K ∏ k=1 N(µk |m0 , (βΛk )−1) W(Λk |W0 , ν0 ) = K ∏ k=1 p(µk , Λk ) (2.17) ͜͜ͰɺW(Λ|W, ν) ͸΢Ογϟʔτ෼෍Ͱ͋Δɻ(PRML 2.3.6 ͷ (2.155), (2.156) ࢀর) 23 / 51
24. 2-2. ྫɿม෼ࠞ߹Ψ΢ε෼෍ ͜Ε͕ࠞ߹Ψ΢ε෼෍Ͱ͋Δ͜ͱΛ͔֬ΊΔʹ͸ɺ(2.12) ͷ྆ลΛ p(π, µ, Λ) ͰׂΓɺજࡏม਺ Z Ͱ࿨ΛͱΔ͜ͱͰ p(X|π, µ, Λ) = ∑ Z p(X, Z|π, µ, Λ) = N ∏ n=1 { K ∑ k=1 πk N(xn |µk , Λ−1 k ) } (2.18) ͱͳΓɺ͔֬ʹ͜ͷϞσϧ͸ࠞ߹Ψ΢εϞσϧͰ͋Δɻ ͦΕͰ͸ɺ͜ͷઃఆͷԼͰ໬౓ؔ਺ͷ࠷େԽΛۙࣅతʹߦ͏ͨΊʹɺҎ ԼͷΑ͏ͳ෼෍ͷ෼ղ ((2.6) ʹରԠ) Λߦ͏ɻ q(Z, π, µ, Λ) = q(Z)q(π, µ, Λ) (2.19) 24 / 51
25. 2-2. ྫɿม෼ࠞ߹Ψ΢ε෼෍ ͦΕͰ͸ɺ(2.11) Λ༻͍ͯҼࢠ q(Z) ͷߋ৽ࣜ͸ҎԼͷΑ͏ʹͳΔɻ ln q⋆(Z) = Eπ,µ,Λ [ln p(X, Z, π, µ, Λ)] + const. (2.20) (2.20) ͷӈลʹ (2.12) ͷӈลΛ୅ೖ͠ɺZ ʹґଘ͠ͳ͍߲͸ const. ʹ ٵऩͯ͠͠·͏ͱɺ ln q⋆(Z) = Eπ [ln p(Z|π)] + Eµ,Λ [ln p(X|Z, µ, Λ)] + const. (2.21) ͱͳΔɻ ͜͜Ͱɺ(2.13) Λ༻͍Δͱ Eπ [ln p(Z|π)] = Eπ [ N ∑ n=1 K ∑ k=1 znk ln πk ] = N ∑ n=1 K ∑ k=1 znk E [ ln πk ] (2.22) ͱͳΔɻ 25 / 51
26. 2-2. ྫɿม෼ࠞ߹Ψ΢ε෼෍ ·ͨɺ(2.15) Λ༻͍Δͱ Eµ,Λ [ln p(X|Z, µ, Λ)] = N ∑ n=1 K ∑ k=1 znk Eµ,Λ [ ln N(xn |µk , Λ−1 k ) ] = N ∑ n=1 K ∑ k=1 znk ( 1 2 E [ ln |Λk | ] − D 2 ln (2π) − 1 2 Eµk,Λk [ (xn − µk )TΛk (xn − µk ) ] ) (2.23) ͱͳΔɻ ͜͜Ͱɺ࠷ޙͷΠίʔϧͰ͸Ψ΢ε෼෍ͷ͋ΒΘͳࣜ (2.16) Λ༻͍ͨɻ 26 / 51
27. 2-2. ྫɿม෼ࠞ߹Ψ΢ε෼෍ (2.22) ͱ (2.23) Λ༻͍Δͱɺ(2.21) ͸ ln q⋆(Z) = N ∑ n=1 K ∑ k=1 znk ln ρnk + const. (2.24) ͱॻ͚Δɻ ͜͜Ͱɺ ln ρnk =E [ ln πk ] + 1 2 E [ ln |Λk | ] − D 2 ln (2π) − 1 2 Eµk,Λk [ (xn − µk )TΛk (xn − µk ) ] (2.25) ͱఆٛͨ͠ɻ 27 / 51
28. 2-2. ྫɿม෼ࠞ߹Ψ΢ε෼෍ Ҏ্ΑΓɺ෼෍ q⋆(Z) ͸ q⋆(Z) ∝ N ∏ n=1 K ∏ k=1 ρznk nk (2.26) ͱͳΔɻ ͜͜Ͱɺ(2.25) ʹ µ ΍ Λ ͷظ଴஋ؚ͕·ΕΔͷͰɺ෼෍ q⋆(Z) ͸෼ղ (2.19) ͷଞͷҼࢠ q(π, µ, Λ) ͷܗʹґଘ͢Δ͜ͱʹ஫ҙɻ ن֨Խ৚݅ΑΓɺq⋆(Z) ҎԼͷΑ͏ʹͳΔɻ(PRML ԋश 10.12) q⋆(Z) = N ∏ n=1 K ∏ k=1 rznk nk = N ∏ n=1 q⋆(zn ) (2.27) ͜͜Ͱ rnk ͸ҎԼͷΑ͏ʹఆٛ͞ΕΔɻ rnk = ρnk K ∑ j=1 ρnj (2.28) 28 / 51
29. 2-2. ྫɿม෼ࠞ߹Ψ΢ε෼෍ ࣍ʹɺ(2.19) ͷҼࢠ q(π, µ, Λ) ͷߋ৽ࣜʹ͍ͭͯߟ͑Δɻ q(Z) ͷ࣌ͱಉ͡Α͏ʹɺߋ৽ࣜͷҰൠతͳࣜ (2.11) Λ༻͍Δͱɺ ln q⋆(π, µ, Λ) = EZ [ln p(X, Z, π, µ, Λ)] + const. (2.29) ͱͳΓɺ(2.29) ͷӈลʹ (2.12) ͷӈลΛ୅ೖ͢Δͱɺ ln q⋆(π, µ, Λ) =EZ [ln p(X|Z, µ, Λ)p(Z|π)p(π)p(µ|Λ)p(Λ)] + const. = ln p(π) + EZ [ln p(Z|π)] + ln p(µ, Λ) + EZ [ln p(X|Z, µ, Λ)] + const. (2.30) ͱͳΔɻ 29 / 51
30. 2-2. ྫɿม෼ࠞ߹Ψ΢ε෼෍ ͜͜Ͱɺ(2.13) ͱ (2.14) ͱ (2.15) ͱ (2.17) Λ࢖͏ͱɺ(2.30) ͸ҎԼͷΑ ͏ʹͳΔɻ ln q⋆(π, µ, Λ) = K ∑ k=1 { (α0 + 1) ln πk + ln πk N ∑ n=1 E[znk ] + ln p(µk , Λk ) + N ∑ n=1 E[znk ] ln N(xn |µk , Λ−1 k ) } + const. (2.31) ͜ͷมܗͰΘ͔ͬͨ͜ͱ͸ɺ෼෍ q⋆(π, µ, Λ) ͸ҎԼͷΑ͏ʹ෼ղͰ͖ Δ͜ͱͰ͋Δɻ q⋆(π, µ, Λ) = K ∏ k=1 q⋆(πk )q⋆(µk , Λk ) (2.32) 30 / 51
31. 2-2. ྫɿม෼ࠞ߹Ψ΢ε෼෍ ·ͨɺ(2.31) ͷӈลͷதʹݱΕ͍ͯΔظ଴஋ E[znk ] ͸જࡏม਺ znk ͷ ෼෍ q⋆(Z) ͷԼͰͷظ଴஋ͳͷͰɺ(2.27) Λ༻͍Δͱ E[znk ] = ∑ Z znk q⋆(Z) = ∑ z1 · · · ∑ zN znk N ∏ n′=1 q⋆(zn′ ) = [ ∑ z1 q⋆(z1 ) ] =1 · · · [ ∑ zn znk q⋆(zn ) ] · · · [ ∑ zN q⋆(zN ) ] =1 = ∑ zn znk K ∏ k′=1 rznk′ nk′ = rnk (2.33) ͱͳΔɻ ࠷ޙͷΠίʔϧ͸ɺऔΓ͏Δ͢΂ͯͷ zn ͷ࿨ͷதͰɺznk = 1 ͱͳΔ ߲͔͠࢒Βͳ͍͜ͱΛ༻͍ͨɻ 31 / 51
32. 2-2. ྫɿม෼ࠞ߹Ψ΢ε෼෍ ͜ΕΑΓ·ͣɺ(2.32) ͷҼࢠ q⋆(π) = K ∏ k=1 q⋆(πk ) ͷର਺͸ɺ(2.31) ΑΓ ln q⋆(π) = K ∑ k=1 ln πk { (α0 + 1) + N ∑ n=1 rnk } + const. (2.34) ͱͳΔͷͰɺq⋆(π) ͸ҎԼͷΑ͏ʹσΟϦΫϨ෼෍ʹͳΔɻ q⋆(π) = Dir(π|α) = C(α) K ∏ k=1 παk+1 k (2.35) ͜͜Ͱɺ αk = α0 + Nk (2.36) ͱఆٛ͠ɺNk ͸ҎԼͰఆٛ͞ΕΔɻ Nk = N ∑ n=1 rnk (2.37) 32 / 51
33. 2-2. ྫɿม෼ࠞ߹Ψ΢ε෼෍ ࣍ʹɺ(2.31) ΑΓ (2.32) ͷҼࢠ q⋆(µk , Λk ) ͷର਺͸ҎԼͷΑ͏ʹΨ΢ ε-΢Ογϟʔτ෼෍ʹͳΔɻ(PRML ԋश 10.13 ࢀর) q⋆(µk , Λk ) = N(µk |mk , (βk Λk )−1) W(Λk |Wk , νk ) (2.38) ͜͜ͰɺҎԼͷΑ͏ʹఆٛͨ͠ɻ βk = β0 + Nk (2.39) mk = 1 βk (β0 m0 + Nk xk ) (2.40) W−1 k = W−1 0 + Nk Sk + β0 Nk β0 + Nk (xk − m0 )(xk − m0 )T (2.41) νk = ν0 + Nk (2.42) xk = 1 Nk N ∑ n=1 rnk xn (2.43) Sk = 1 Nk N ∑ n=1 rnk (xn − xk )(xn − xk )T (2.44) 33 / 51
34. 2-2. ྫɿม෼ࠞ߹Ψ΢ε෼෍ ·ͨɺྔ Nk , xk , Sk ΛٻΊΔʹ͸ɺ(2.37), (2.43), (2.44) ΑΓɺrnk Λ ٻΊΔඞཁ͕͋Δɻ (2.28) ͱ (2.25) ΑΓɺҎԼͷظ଴஋ΛٻΊΕ͹Α͍͜ͱ͕Θ͔Δɻ (PRML ԋश 10.14) Eµk,Λk [ (xn −µk )TΛk (xn − µk ) ] =Dβ−1 k + νk (xn − mk )TWk (xn − mk ) (2.45) E [ ln |Λk | ] = D ∑ i=1 ψ ( νk + 1 − i 2 ) + D ln 2 + ln |Wk | (2.46) E [ ln πk ] =ψ(αk ) − ψ(ˆ α) (2.47) ͜͜Ͱɺψ(·) ͸ PRML ͷ෇࿥ B Ͱఆٛ͞Ε͍ͯΔσΟΨϯϚؔ਺Ͱ͋ Γɺˆ α = ∑ k αk Ͱ͋Δɻ 34 / 51
35. 2-2. ྫɿม෼ࠞ߹Ψ΢ε෼෍ Ҏ্Λ·ͱΊΔͱɺࠞ߹Ψ΢εϞσϧͷม෼ਪ࿦๏͸ɺҎԼͷΑ͏ʹ EM ΞϧΰϦζϜͷ E εςοϓͱ M εςοϓʹࣅͨม෼ E εςοϓͱ ม෼ M εςοϓͷ൓෮Ͱ͋Δɻ 1. (ม෼ E εςοϓ) ύϥϝʔλͷݱࡏͷࣄޙ෼෍Λ༻͍ͯɺظ଴஋ (2.45), (2.46), (2.47) Λܭࢉ͠ɺෛ୲཰ rnk ΛٻΊΔɻ 2. (ม෼ M εςοϓ) ٻΊͨෛ୲཰Λ༻͍ͯɺࣄޙ෼෍ q⋆(π) ͱ q⋆(µk , Λk ) ΛͦΕͧΕ (2.35) ͱ (2.38) ͔Βܭࢉ͢Δɻ 35 / 51
36. 2-3. ہॴతม෼ਪ࿦๏ ͜͜·Ͱٞ࿦͖ͯͨ͠ਪ࿦๏ (෼෍ͷ෼ղ) Λ·ͱΊΔɻ ಉ࣌෼෍ p(X, Z) ͕༩͑ΒΕ͍ͯΔ࣌ɺ p(Z|X) = p(X, Z) ∫ p(X, Z) dZ (2.48) ্ͷΑ͏ʹࣄޙ෼෍ p(Z|X) ʹͯ͠ٻΊ͍͕ͨɺ෼฼ͷ Z(ύϥϝʔλ ͱજࡏม਺) ͷੵ෼͕Ͱ͖ͳ͍ͱ͢Δɻ ҰํͰɺࣄޙ෼෍ p(Z|X) ͸൚ؔ਺ L[q] L[q] = ∫ q(Z) ln { p(X, Z) q(Z) } dZ (2.49) Λ࠷େʹ͢Δ q(Z) Ͱ͋Δɻ 36 / 51
37. 2-3. ہॴతม෼ਪ࿦๏ ͦ͜Ͱɺq(Z) ΛҎԼͷΑ͏ʹ q(Z) = M ∏ i=1 qi (Zi ) (2.50) ෼ղ͠ɺ(2.49) ʹ୅ೖ͢Δͱ൚ؔ਺ L[q] ͸ҎԼͷΑ͏ʹͳΔɻ L[qi ] = ∫ M ∏ i=1 qi (Zi ) ln { p(X, Z) ∏ M i=1 qi (Zi ) } dZ (2.51) ͦͯ͠ɺL[qi ] ʹ͢Δ෼෍ͷू߹ {q⋆ i (Zi )} ΛٻΊͯɺҎԼͷ෼෍ q⋆(Z) q⋆(Z) = M ∏ i=1 q⋆ i (Zi ) (2.52) Λࣄޙ෼෍Λۙࣅ͢Δ෼෍ͱ͢Δ΋ͷͰ͋ͬͨɻ 37 / 51
38. 2-3. ہॴతม෼ਪ࿦๏ ͭ·Γɺ͜ͷ෼෍ͷ෼ղ͸ࣄޙ෼෍ p(Z|X) Λ͢΂ͯͷ֬཰ม਺ͷ׬શ ͳࣄޙ෼෍ͷۙࣅ q⋆(Z) ΛٻΊΔͱ͍͏ҙຯͰɺେҬతม෼ਪ࿦๏ͷҰ छͰ͋Δɻ ࣍ʹ঺հ͢Δਪ࿦๏͸ہॴతม෼ਪ࿦๏Ͱ͋Δɻ ۩ମతʹ͸ɺಉ࣌෼෍ p(X, Z) ͷԼքͱͳΔؔ਺Λ h(X, Z, ξ) ͱ͢Δɻ ͭ·ΓɺҎԼͷΑ͏ͳؔ਺ h(X, Z, ξ) Ͱ͋Δɻ p(X, Z) ≥ h(X, Z, ξ) (2.53) ͜ͷ h(X, Z, ξ) ͸ Z ͷੵ෼͕Ͱ͖Δ͘Β͍γϯϓϧͳؔ਺Ͱ͋Δɻ ͋ͱͰ۩ମྫͰઆ໌͢Δ͕ɺh(X, Z, ξ) ͸ ξ ʹґଘ͢ΔܗͰॻ͚ͯɺ (2.53) ʹ͓͍ͯɺ͋Δ Z⋆ ͰΠίʔϧ͕੒ཱ͢Δͱ͖ͷ ξ Λ ξ⋆ ͱ͢Δɻ 38 / 51
39. 2-3. ہॴతม෼ਪ࿦๏ (2.48) ΑΓɺࣄޙ෼෍ p(Z|X) Λؔ਺ h(X, Z, ξ⋆) Λ༻͍ͯ p(Z|X) ∼ p(X, Z) ∫ h(X, Z, ξ⋆) dZ (2.54) ͱۙࣅ͢Δํ๏Ͱ͋Δɻ ξ = ξ⋆ ͷͱ͖ɺෆ౳ࣜ (2.53) ͸ɺ͋Δ Z⋆ Ͱ͸౳ࣜʹͳΔ͕ɺଞͷ Z Ͱ͸Ұൠతʹ౳ࣜʹͳΒͳ͍ɻ ͭ·ΓɺݶΒΕͨ (ہॴతͳ)Z ͰҰக͢ΔΑ͏ͳؔ਺ h(X, Z, ξ⋆) Λ༻ ͍ͯࣄޙ෼෍Λۙࣅ͢ΔͷͰɺہॴతม෼ਪ࿦๏Ͱ͋Δɻ Ͱ͸ɺؔ਺ h(X, Z, ξ⋆) ͷٻΊํΛઆ໌͢Δɻ 39 / 51
40. 2-3. ہॴతม෼ਪ࿦๏ ·ͣɺҎԼͷը૾ͷΑ͏ͳತؔ਺ f(x) Λߟ͑Δɻ ·ͨɺݪ఺Λ௨Δ௚ઢ ηx Λߟ͑Δɻ(ը૾ͷ λ ͸ϛε) ͜ͷ௚ઢ ηx ͸ؔ਺ f(x) ͷԼքʹ͸ͳ͍ͬͯΔ͕ɺ܏͖͕ η ͷ࠷ྑͷ Լք (f(x) ͱͷ͕ࠩ࠷΋খ͍͞) Ͱ͸ͳ͍ɻ 40 / 51
41. 2-3. ہॴతม෼ਪ࿦๏ ࠷ྑͷԼք͸ҎԼͷΑ͏ʹɺf(x) ͷ઀ઢͰ͋Δɻ ͜ͷ઀ઢͷ੾ยΛ −g(η) ͱ͢Δͱɺ઀ઢͷํఔࣜ͸ ηx − g(η) Ͱ͋Δɻ ͜ͷ੾ย −g(η) ͸ҎԼͷΑ͏ʹͯ͠ٻΊΔ͜ͱ͕Ͱ͖Δɻ −g(η) = min x {f(x) − ηx} → g(η) = max x {ηx − f(x)} (2.55) ηx − f(x) Λ࠷େʹ͢Δ x ͸઀఺ͷ x ࠲ඪͰ͋Δɻ 41 / 51
42. 2-3. ہॴతม෼ਪ࿦๏ ࣍ʹɺf(x) Λ g(η) Ͱදݱ͢Δ͜ͱΛߟ͑Δɻ f(x) ͷತؔ਺ੑʹΑΓɺ͋Δ x ʹ͓͍ͯɺ max η {ηx − g(η)} (2.56) ͸ f(x) ͷ x ʹ͓͚Δ઀఺ͷ y ࠲ඪͰ͋Γɺf(x) Ͱ͋Δɻ ͭ·Γɺ f(x) = max η {ηx − g(η)} (2.57) Ͱ͋Δɻ 42 / 51
43. 2-3. ہॴతม෼ਪ࿦๏ (2.55) ͱ (2.56) ΑΓɺؔ਺ f(x) ͱԼքͱͳΔ௚ઢͷ੾ย −g(η) ʹ͸Ҏ ԼͷΑ͏ʹ૒ରؔ܎͕͋Δ͜ͱ͕Θ͔Δɻ f(x) = max η {ηx − g(η)} (2.58) g(η) = max x {ηx − f(x)} (2.59) ͦΕͰ͸ɺ͜ͷςΫχοΫΛ༻͍ͯɺϩδεςΟοΫγάϞΠυ σ(x) σ(x) = 1 1 + e−x (2.60) ͷԼքΛٻΊΔɻ 43 / 51
44. 2-3. ہॴతม෼ਪ࿦๏ ͨͩ͠ɺϩδεςΟοΫγάϞΠυ͕ತؔ਺Ͱͳ͍ͷͰɺϩδεςΟο ΫγάϞΠυͷର਺ΛҎԼͷΑ͏ʹมܗ͢Δɻ ln σ(x) = − ln (1 + e−x) = − ln {e−x/2(ex/2 + e−x/2)} = x 2 − ln (ex/2 + e−x/2) (2.61) ͜͜Ͱɺؔ਺ f(x) = − ln (ex/2 + e−x/2) ͸ x2 ͷತؔ਺Ͱ͋Δɻ (PRML ԋश 10.31) ͜Ε͔Βɺؔ਺ f(x) = − ln (ex/2 + e−x/2) ͷԼքΛ΋ͱΊΔɻ ·ͣɺx2 ͷತؔ਺Ͱ͋ΔͷͰɺ(2.59) ΑΓؔ਺ g(η) ͸ g(η) = max x2 {ηx2 − f(x)} (2.62) ͱͳΔɻ 44 / 51
45. 2-3. ہॴతม෼ਪ࿦๏ ͜͜Ͱɺηx2 − f(x) Λ x2 Ͱඍ෼͢Δͱ d dx2 [ ηx2 − f(x) ] = η − dx dx2 · d dx f(x) (2.63) ͱͳΓɺ dx dx2 = 1 2x (2.64) d dx f(x) = − d dx [ ln (ex/2 + e−x/2) ] = − 1 2 tanh x 2 (2.65) ΑΓɺ d dx2 [ ηx2 − f(x) ] = η + 1 4x tanh x 2 (2.66) ͱͳΔɻ 45 / 51
46. 2-3. ہॴతม෼ਪ࿦๏ (2.66) ͷӈล͕ 0 ͱͳΔ x Λ ξ ͱ͢Δͱɺξ ͱ η ͸ η = − 1 4ξ tanh ξ 2 = − 1 2ξ [ σ(ξ) − 1 2 ] = −λ(ξ) (2.67) ͱ͍͏ؔ܎ʹ͋Δ͜ͱ͕Θ͔Δɻ (2.62) ΑΓɺg(η) ͸ g(η) = [ ηx2 − f(x) ] x=ξ,η=−λ(ξ) = −λ(ξ)ξ2 − f(ξ) = −λ(ξ)ξ2 + ln (eξ/2 + e−ξ/2) (2.68) ͱͳΔɻ 46 / 51
47. 2-3. ہॴతม෼ਪ࿦๏ ͜ΕΑΓɺ(2.58) ͱ (2.61) ͱ (2.68) ΑΓ ln σ(x) = x 2 + max η {ηx2 − g(η)} ≥ x 2 − λ(ξ)x2 + λ(ξ)ξ2 − ln (eξ/2 + e−ξ/2) = x 2 − λ(ξ)x2 + λ(ξ)ξ2 − ξ 2 + ln σ(ξ) = 1 2 (x − ξ) − λ(ξ)(x2 − ξ2) + ln σ(ξ) (2.69) ͱͳΓɺର਺ͷ୯ௐ૿ՃੑΑΓɺσ(x) ͷԼք͸ σ(x) ≥ σ(ξ) exp { 1 2 (x − ξ) − λ(ξ)(x2 − ξ2) } (2.70) ͱͳΔɻ 47 / 51
48. 2-4. ม෼ϩδεςΟοΫճؼ ͦΕͰ͸ɺ(2.70) Λ༻͍ͯϩδεςΟοΫճؼͷม෼ਪ࿦Λߦ͏ɻ ·ͣɺϩδεςΟοΫճؼͷ໬౓ؔ਺ p(t|x, w) ͸ p(t|x, w) =σ(a)t{1 − σ(a)}1−t = ( 1 1 + e−a )t ( 1 − 1 1 + e−a )1−t = 1 (1 + e−a)t e−a(1−t) (1 + e−a)1−t =eat e−a 1 + e−a = eat 1 1 + e−(−a) = eatσ(−a) (2.71) ͱͳΔɻ ͜͜Ͱɺa = wTϕ Ͱ͋Δɻ ·ͨɺύϥϝʔλ w ͷࣄલ෼෍ΛҎԼͷΑ͏ʹΨ΢ε෼෍ͰԾఆ͢Δɻ p(w) = N(w|m0 , S0 ) (2.72) ͜͜Ͱɺm0 , S0 ͸ϋΠύʔύϥϝʔλɻ 48 / 51
49. 2-4. ม෼ϩδεςΟοΫճؼ ೖྗσʔλͷू߹ X = {x1 , x2 , · · · , xN } ͱͦΕͧΕʹରԠ͢Δ໨ඪม ਺ͷू߹ t = {t1 , t2 , · · · , tN } Λ༻ҙ͢Δɻ ڭࢣσʔλҰͭҰ͕ͭ෼෍ (2.71) ͔Βಠཱʹੜ੒͞Ε͍ͯΔͱ͢Δͱɺ ໬౓ؔ਺ p(t|X, w) ͸ҎԼͷΑ͏ʹͳΔɻ p(t|X, w) = N ∏ n=1 p(tn |xn , w) (2.73) ࠓճٻΊ͍ͨͷ͸ɺࣄޙ෼෍ p(w|t, X) Ͱ͋ΓɺҎԼͷࣜͰදͤΔɻ p(w|t, X) = p(w)p(t|X, w) ∫ p(w)p(t|X, w) dw (2.74) ͜ͷ (2.74) ͷӈลͷ෼฼͸໬౓ؔ਺͕ϩδεςΟοΫγάϞΠυͷੵ ʹͳ͍ͬͯΔͷͰɺੵ෼Ͱ͖ͳ͍ɻ 49 / 51
50. 2-4. ม෼ϩδεςΟοΫճؼ ͦ͜Ͱɺෆ౳ࣜ (2.70) Λ࢖͏ͱҎԼͷΑ͏ʹͳΔɻ p(w)p(t|X, w) =p(w) N ∏ n=1 eantn σ(−an ) ≥p(w) N ∏ n=1 eantn σ(ξn ) exp {(−an − ξn )/2 − λ(ξn )(a2 n − ξ2 n )} =p(w) N ∏ n=1 σ(ξn ) exp {an tn − (an + ξn )/2 − λ(ξn )(a2 n − ξ2 n )} (2.75) ͜͜Ͱɺan = wTϕn Ͱ͋Δɻ 50 / 51
51. 2-4. ม෼ϩδεςΟοΫճؼ ͜͜Ͱɺ(2.75) ͷ྆ลͷର਺ΛऔΔͱ ln {p(w)p(t|X, w)} ≥ ln p(w) + N ∑ n=1 [ ln σ(ξn ) + wTϕn tn − (wTϕn + ξn )/2 − λ(ξn )([wTϕn ]2 − ξ2 n ) ] (2.76) ͱͳΓɺ(2.76) ͷӈล͸ w ͷ 2 ࣍ࣜʹͳ͍ͬͯΔͷͰɺp(w)p(t|X, w) ͷԼք͸Ψ΢ε෼෍ͱͳΔɻ ln {p(w)p(t|X, w)} ≥ q(w) = N(w|mN , SN ) (2.77) ͜͜ͰɺmN , SN ͸ҎԼͰఆٛ͞ΕΔɻ mN = SN ( S−1 0 m0 + N ∑ n=1 (tn − 1/2)ϕn ) (2.78) S−1 N = S−1 0 + 2 N ∑ n=1 λ(ξn )ϕn ϕT n (2.79) Ψ΢ε෼෍ʹۙࣅͰ͖ͨͷͰɺ(2.74) ͷ෼฼͸ੵ෼͕ՄೳʹͳΓɺࣄޙ ෼෍͕ٻ·Δɻ 51 / 51