Upgrade to Pro — share decks privately, control downloads, hide ads and more …

20201102RLArch.pdf

Tatsuya Matsushima
November 02, 2020
160

 20201102RLArch.pdf

Tatsuya Matsushima

November 02, 2020
Tweet

Transcript

  1. "DUJPOBOE1FSDFQUJPOBT%JWFSHFODF.JOJNJ[BUJPO • 4VCNJUUFEPO4FQ W MBTUSFWJTFE0DU W  • ஶऀਞ͕݁ߏ͍͢͝ʢ(PPHMF#SBJO 6$-

    %FFQ.JOE 5SPOUPʣ • IUUQTBSYJWPSHBCT • *$-3VOEFSSFWJFXIUUQTPQFOSFWJFXOFUGPSVN JE+C"RTGC:T+Z ॻࢽ৘ใ  1MB/FU%SFBNFSʢम࢜Ϙε͕'SJTUPOʣ ࣗ༝ΤωϧΪʔݪཧʢ'&1ʣ -BZFS/PSN"EBNʜ %FFQ.JOEͰ͍Ζ͍Ζ
  2. ೖྗͷ֬཰ม਺ ɼͦΕҎ֎ͷʢજࡏʣม਺  ύϥϝʔλ  • ࠷దԽͷաఔͰ௚઀มߋ͢Δ͜ͱ͕Ͱ͖Δ΋ͷશͯ • ྫʣදݱʹؔ͢Δे෼౷ܭྔɼϞσϧͷύϥϝʔλɼํࡦͷύϥϝʔλ ࣮ࡍͷ෼෍

    ʢBDUVBMEJTUSJCVUJPOʣ • ੜ੒աఔ Λදݱ • ࣮ࡍͷ෼෍͸ະ஌ͷ৔߹΋͋Δɹྫʣσʔλ෼෍ • ύϥϝʔλ ͷӨڹΛड͚ͯมԽ͢Δ৔߹΋͋Δ λʔήοτͷ෼෍  • γεςϜͷؔ͢Δ޷·͍͠ઃఆʢQSFGFSFODFʣΛදݱ • ʢͪΌΜͱॻ͔Εͯͳ͍͕ʣͬͪ͜΋ύϥϝʔλ ͷӨڹΛड͚Δ x z ϕ pϕ x, z ∼ pϕ (x, z) ϕ τ(x, z) ϕ ֬཰ม਺΍෼෍ͷදه 
  3. ύϥϝʔλ ΛมԽͤͯ͞ɼશͯͷม਺ʹؔ͢Δ ࣮ࡍͷ෼෍ ΛͰ͖Δ͚ͩλʔήοτ ʹ͚ۙͮΔ໰୊  ͱͯ͠ఆࣜԽ • SFWFSTF,-΍JOGPSNBUJPOQSPKFDUJPOͱ΋ݺ͹ΕΔ •

    ʢ༨ஊʣSFWFSTF,-WTGPSXBSE,-ͷ࿩΋େࣄ • ͸Ϟσϧʢ͚ͩʣͰ͸ͳ͘ੜ੒աఔΛදݱͯ͠Δ͜ͱʹ஫ҙ • Ұ෦͸ਅͷ෼෍Ͱະ஌ɼҰ෦͸ύϥϝʔλ ʹΑΓมԽ͠͏Δ • ྫʣදݱֶशͷ৔߹ɼ ͸σʔλͱ৴೦ͷಉ࣌෼෍ɼ ͸જࡏม਺Ϟσϧ • ྫʣ੍ޚͷ৔߹ɼ ͸ݱࡏͷํࡦͰͷيಓͷ෼෍ɼ ͸يಓͷVUJMJUZ ϕ pϕ τ minϕ KL [pϕ (x, z)∥τ(x, z)] pϕ ϕ pϕ τ pϕ τ +PJOU,-.JOJNJ[BUJPO 
  4. +PJOU,-Λ෼ղ  • ୈ߲ɿજࡏม਺ͷ৴೦ Λ޷·͍͠෼෍ ʹ͚ۙͮΔ੍໿ • ୈ߲ɿ૬ޓ৘ใྔ ͷม෼Լք •

     • ૬ޓ৘ใྔͷԼքͷ࠷খԽ㱺૬ޓ৘ใྔͷ࠷େԽ • ॊೈͳ ͷ଒Λ࢖͏΄ͲλΠτͳԼք KL [pϕ (x, z)∥τ(x, z)] KPJOUEJWFSHFODF = EKL [pϕ (z ∣ x)∥τ(z)] SFBMJ[JOHMBUFOUQSFGFSFODFT − E [ln τ(x ∣ z) − ln pϕ (x)] JOGPSNBUJPOCPVOE pϕ (z|x) τ(z) I[x; z] I[x; z] ≥ I[x; z] − EKL [p(x ∣ z)∥τϕ (x ∣ z)] = E [ln τϕ (x ∣ z) − ln p(x)] τϕ (x|z) -BUFOU1SFGFSFODF  *OGPSNBUJPO#PVOET
  5. ΤʔδΣϯτ͕ೖྗ ΛมߋͰ͖ͳ͍৔߹ɼ MBUFOUQSFGFSFODFͷ ͷ߲͸ύϥϝʔλԽ͞Εͳ͍ʢఆ਺ʣ • ྫʣݻఆͷσʔληοτ͔Βͷֶश ͜ͷͱ͖ɼઌ΄Ͳͷ໨తؔ਺͸ม෼ਪ࿦ʹ͓͚Δࣗ༝ΤωϧΪʔ ʢ&-#0ʣʹରԠ  •

    ͰೖྗΛ࠶ߏ੒͠ͳ͕Βɼ Λࣄલ෼෍ ʹ͚ۙͮΔ x pϕ (x) min ϕ {EKL [pϕ (z ∣ x)∥τ(z)] − E [ln τ(x ∣ z)]} τ(x|z) pϕ (z|x) τ(z) 7BSJBUJPOBM'SFF&OFSHZ  *OGPSNBUJPO#PVOET
  6. +PJOU,-ͷଞͷ෼ղ  • ୈ߲ɿೖྗͷ৚݅෇͖෼෍ Λ޷·͍͠෼෍ ʹ͚ۙͮΔ • ୈ߲ɿ૬ޓ৘ใྔ ͷม෼Լք •

    ࠓճ͸જࡏۭؒͰදݱʢ Ͱ৚͚͖݅ͮͯͬ͞ͱಉ༷ͳٞ࿦Ͱٻ·Δʣ • ೖྗΛ؍ଌͨ͠ޙͷ৴೦ ͱ؍ଌ͢Δલͷ৴೦ ͷࠩʢ৘ใήΠϯʣΛදݱ • Լք͸ ͕࣮ࡍͷ৚݅෇͖෼෍ ΛΑۙ͘ࣅ͍ͯ͠Δ΄ͲλΠτ 㱺ม਺ͷؔ܎Λֶश͢Δ΄Ͳ૬ޓ৘ใྔΛ࠷େԽʢදݱྗ๛͔ͳ Λ࢖͏ͱ༗རʣ ૬ޓ৘ใྔͷ࠷େԽ͸ɼೖྗͷྑ͍જࡏදݱͷֶशͱɼજࡏදݱʹͱͬͯ༗ӹͳೖྗͷ ୳ࡧͷ྆ํΛઆ໌ʂ KL [pϕ (x, z)∥τ(x, z)] KPJOUEJWFSHFODF = EKL [pϕ (x ∣ z)∥τ(x)] SFBMJ[JOHJOQVUQSFGFSFODFT − E [ln τ(z ∣ x) − ln pϕ (z)] JOGPSNBUJPOCPVOE pϕ (x|z) τ(x) I[x; z] z τ(z|x) pϕ (z) τ(z|x) p(z|x) τ(z|x) *OQVU1SFGFSFODF  *OGPSNBUJPO#PVOET
  7. ͸ΤʔδΣϯτͷ޷·͍͠ೖྗͷ෼෍Λදݱ͢Δ΋ͷ • ͭ·ΓΤʔδΣϯτ͸ ʹऩଋͤ͞Α͏ͱ͢Δ • %JWFSHFODFNJOJNJ[BUJPOʹΑͬͯΤʔδΣϯτ͸ࣗ਎ͷϞσϧΛ࢖ͬ ͯ༧ଌͰ͖ΔΑ͏ͳೖྗΛݟ͚ͭΑ͏ͱ͢Δ • ࣗ਎ͷϞσϧͷ଒Ͱ͸༧ଌͰ͖ͳ͍ೖྗΛආ͚Α͏ͱ͢Δ •

    ใु ʹରͯ͠ ͱ͢Ε͹λεΫͷใुΛಋೖՄೳ • ใुͱͯ͠पล໬౓Λଧͪফ߲͢໨Λ༩͑ͯɼೖྗͷ޷·͠͞ͷ߲Λ औΓআ͘͜ͱ΋Մೳ • ͜ͷลΓ͸Ͱઆ໌ τ(x) τ(x) r(x) exp (r(x)) *OQVUQSFGFSFODF  .PEFMBT1SFGFSFODF
  8. %JWFSHFODFNJOJNJ[BUJPOͷදݱֶश΍୳ࡧʹؔ࿈͢Δ૬ޓ৘ใྔͷԼք͸දݱྗ๛͔ͳ λʔήοτΛ༻͍Δ΄ͲλΠτʢʣ • Ϟσϧ͕༩͑ΒΕͨݩͰͷೖྗͷपล໬౓ͷNBUDIJOH • ͜ͷͱ͖ɼΤʔδΣϯτ͸ೖྗΛे෼ʹ༧ଌͰ͖ͯɼ ֎ཚʹରԠͰ͖Δࣗવͳฏߧঢ়ଶʢFDPMPHJDBMOJDIFʣʹऩଋ ࣅͨΑ͏ͳ֓೦ͱͯ͠TVSQSJTFNJOJNJ[BUJPO͕͋Δ • Ϟσϧ͕༩͑ΒΕͨݩͰͷೖྗͷपล໬౓ͷ࠷େԽ

    • ͜ͷͱ͖ɼOJDIFTFFLJOH<'SJTUPO >ͱݺ͹ΕΔ • ֎ཚ͕ͳ͍ͱҰ୴୳ࡧͨ͜͠ͱͷ͋Δࣗ໌ͳղʹؕΔ • ʢদౢղऍɿͨͿΜ͜Ε͸ʮ҉͍෦԰໰୊ʯͱݴΘΕͯΔ΋ͷʣ %JWFSHFODFNJOJNJ[BUJPOͩͱɼೖྗͷपล෼෍Λ߹ΘͤΑ͏ͱ͢ΔͨΊɼ ༧ଌͰ͖ΔશͯͷೖྗΛΧόʔ͢ΔΑ͏ͳ޿͍OJDIFʹͳΔΑ͏ʹଅ͞ΕΔ • දݱྗ๛͔ͳੈքϞσϧΛ༻͍Δͱ༧ଌՄೳੑ͕ߴ·Γ޿͍OJDIFʹʢλεΫใु͸௥Ճతʣ /JDIF4FFLJOH  .PEFMBT1SFGFSFODF
  9. աڈͷೖྗ Λ؍ଌ͠ɼকདྷͷೖྗ Λ ؍ଌ͢ΔલͷKPJOU,-ͷ࠷దԽΛߟ͑Δ • ͷ૬ޓ৘ใྔͷԼքΛࣜมܗ x< x> E [ln

    τ(z ∣ x) − ln pϕ (z)] JOGPSNBUJPOCPVOE = E [ln τ(z ∣ x) − ln pϕ (z ∣ x<) + ln pϕ (z ∣ x<) − ln pϕ (z)] ≥ E [ln τ(z ∣ x) − ln pϕ (z ∣ x<)] GVUVSFJOGPSNBUJPOCPVOE + E [ln τ (z ∣ x<) − ln pϕ (z)] QBTUJOGPSNBUJPOCPVOE ࣌ܥྻ΁ͷ֦ு 
  10. ૬ޓ৘ใྔͷԼքͷࣜมܗΛར༻ͯ͠ ࠷খԽ͍ͨ͠KPJOU,- ͷ্ք͕ٻ·Δ  • +PJOU,-͸ɼաڈͷೖྗͷ༗༻ͳදݱͷֶशͱকདྷͷ༗༻ͳೖྗͷ ୳ࡧͷ྆ํΛಉ࣌ʹදݱͰ͖͍ͯΔ KL [pϕ (x,

    z)∥τ(x, z)] KL [pϕ (x, z)∥τ(x, z)] ≤ EKL [pϕ (z ∣ x<) ∥τ(z)] SFBMJ[JOHQBTUMBUFOUQSFGFSFODFT − E [ln τ (x< ∣ z) − ln pϕ (x<)] SFQSFTFOUBUJPOMFBSOJOH + EKL [pϕ (x> ∣ x< , z) ∥τ (x> ∣ x<)] SFBMJ[JOHGVUVSFJOQVUQSFGFSFODFT − E [ln τ(z ∣ x) − ln pϕ (z ∣ x<)] FYQMPSBUJPO ࣌ܥྻ΁ͷ֦ு 
  11. ʢิ଍ʣSFWJTJPO W Ͱ௥Ճ͞Εͨઆ໌ ʹ͸Ұ౓΋؍ଌ͞Εͳ͍΋ͷͱɼ࣌ؒͷܦաͱͱ΋ʹ ؍ଌ͞ΕΔ΋ͷ͕͋Δ • Ұ౓΋؍ଌ͞Εͳ͍ɿજࡏঢ়ଶͷਪఆ஋΍Ϟσϧύϥϝʔλʣ • ࣌ؒܦաͱͱ΋ʹ؍ଌɿ֬཰తͳߦಈ΍εΩϧ •

    ͜ΕΒͷม਺Λબ୒͢Δ͜ͱ͸ɼຊ౰͸ҼՌతͳհೖʹͳΔͷͰ ͸ɼະ؍ଌͷજࡏม਺ ͱɼ؍ଌ͞Εͨજࡏม਺ Ͱ৚͚݅ͮ ΒΕͨ Ͱ෼ׂ͢΂͖͕ͩɼ؆୯ͷͨΊʹলུ͍ͯ͠Δ z z z> EP(z< ) z> 
  12. ʢิ଍ʣSFWJTJPO W Ͱ௥Ճ͞Εͨઆ໌ ʹ͸Ұ౓΋؍ଌ͞Εͳ͍΋ͷͱɼ࣌ؒͷܦաͱͱ΋ʹ ؍ଌ͞ΕΔ΋ͷ͕͋Δ • Ұ౓΋؍ଌ͞Εͳ͍ɿજࡏঢ়ଶͷਪఆ஋΍Ϟσϧύϥϝʔλʣ • ࣌ؒܦաͱͱ΋ʹ؍ଌɿ֬཰తͳߦಈ΍εΩϧ •

    ͜ΕΒͷม਺Λબ୒͢Δ͜ͱ͸ɼຊ౰͸ҼՌతͳհೖʹͳΔͷͰ ͸ɼະ؍ଌͷજࡏม਺ ͱɼ؍ଌ͞Εͨજࡏม਺ Ͱ৚͚݅ͮ ΒΕͨ Ͱ෼ׂ͢΂͖͕ͩɼ؆୯ͷͨΊʹলུ͍ͯ͠Δ z z z> EP(z< ) z> 
  13. "DUVBMɿ  5BSHFUɿ  ྫʣ#//Λ༻͍ͨڭࢣ͋Γֶश • ೖྗɿը૾ ɼΫϥε  •

    σʔληοτͷHMPCBMͳදݱͱͯ͠જࡏతͳύϥϝʔλ Λਪ࿦ pϕ (x, y, w) = pϕ (w) ⏟ CFMJFG ∏ i p (xi , yi) EBUB τ(x, y, w) ∝ τ(w) ⏟ QSJPS ∏ i τ (yi ∣ xi , w) MJLFMJIPPE x = {xi } y = {yi } w ม෼ਪ࿦ʢ7BSJBUJPOBM*OGFSFODFʣ 
  14. +PJOU,-ʹͦΕͧΕ୅ೖ͢Δͱ  • ໨తؔ਺͕ࣗ༝ΤωϧΪʔʢ&-#0ʣʹͳΔ Λσϧλ෼෍ ͱ͢Δͱ  • ͭ·Γ࠷໬ਪఆʹͳΔʢλʔήοτΛ ͱͨ͠ͱ͖ͱ౳Ձʣ

    KL [pϕ ∥τ] = KL [pϕ (w)∥τ(w)] DPNQMFYJUZ − E[ln τ(y ∣ x, w)] BDDVSBDZ + E[ln p(x, y)] DPOTUBOU pϕ (w) pϕ (w) → δϕ (w) ≐ {1JGw = ϕFMTF0} KL [pϕ (w)∥τ(w)] DPNQMFYJUZ − E[ln τ(y ∣ x, w)] BDDVSBDZ → ln τ(ϕ) − E[ln τ(y ∣ x, ϕ)] DPNQMFYJUZ ≐ E [−ln τϕ (y ∣ x)] QBSBNFUFSJ[FEUBSHFU τϕ (y|x) ม෼ਪ࿦ʢ7BSJBUJPOBM*OGFSFODFʣ 
  15. "DUVBMɿ  5BSHFUɿ  ྫʣ7"& • ೖྗɿը૾  • ରԠ͢Δજࡏม਺

    Λਪ࿦ pϕ (x, z) = ∏ i p (xi) ⏟ EBUB pϕ (zi ∣ xi) FODPEFS τϕ (x, z) = ∏ i τϕ (xi |zi) EFDPEFS τ (zi) ⏟ QSJPS x = {xi } z = {zi } ঈ٫ਪ࿦ʢ"NPSUJ[FE*OGFSFODFʣ 
  16. +PJOU,-ʹͦΕͧΕ୅ೖ͢Δͱ  • ໨తؔ਺͕ࣗ༝ΤωϧΪʔʢ&-#0ʣʹͳΔ • ༧ଌͷ߲ʢୈ߲ʣ͕૬ޓ৘ใྔͷԼք • දݱ͕ೖྗʹؔͯ͠ҙຯΛ࣋ͭΑ͏ʹଅ͢ KL [pϕ

    ∥τϕ] = E [ln τϕ (x ∣ z) − ln p(x)] JOGPSNBUJPOCPVOEʢদౢෛ߸͕͍ΔͷͰ͸ʁʣ + EKL [pϕ (z ∣ x)∥τ(z)] DPNQMFYJUZ ঈ٫ਪ࿦ʢ"NPSUJ[FE*OGFSFODFʣ 
  17. "DUVBMɿ  5BSHFUɿ  ྫʣ)..Ͱকདྷͷೖྗ ͕ܽଛ͍ͯ͠Δ৔߹ • ೖྗ  •

    ରԠ͢Δજࡏม਺ Λਪ࿦ pϕ (x, z) = p (x> , x<) EBUB pϕ (z ∣ x<) CFMJFG τϕ (x, z) = τϕ (x< ∣ z) MJLFMJIPPE τϕ (x> ∣ z) QSFEJDUJPO τ(z) ⏟ QSJPS x> x = {x< , x> } z = {zi } কདྷͷೖྗʢ'VUVSF*OQVUTʣ 
  18. +PJOU,-ʹͦΕͧΕ୅ೖ͢Δͱ  • ͱͯ͠কདྷͷೖྗ ͕࠷దԽʹ ؔΘΒͳ͍Α͏ʹ͍ͯ͠ΔʢVODPOUSPMMFEGVUVSFʣ KL [pϕ ∥τϕ] =

    KL [pϕ (x< , z) ∥τϕ (x< , z)] WBSJBUJPOBMJOGFSFODF + EKL [p (x> ∣ x<) ∥τϕ (x> ∣ z)] VODPOUSPMFEGVUVSF τϕ (x> ∣ x< , z) = τϕ (x> ∣ x<) x> ঈ٫ਪ࿦ʢ"NPSUJ[FE*OGFSFODFʣ 
  19. "DUVBMɿ  5BSHFUɿ  ྫʣܾఆ࿦తͳ੍ޚΛ͢Δ৔߹ • ೖྗͷܥྻ ͸ΤʔδΣϯτ͕ύϥϝʔλ ΛมԽͤͯ͞ ෦෼తʹม͑ΒΕΔྫʣܾఆ࿦తͳํࡦɼ੍ޚଇɼϓϥϯʜ

    pϕ (x) = ∏ t pϕ (xt ∣ x1:t−1) DPOUSPMMFEEZOBNJDT τ(x) = ∏ t τ (xt ∣ x1:t−1) QSFGFSFODFT x = {xt } ϕ ʢܾఆ࿦తͳʣ੍ޚ 
  20. +PJOU,-ʹͦΕͧΕ୅ೖ͢Δͱ  • QSFGFSFODFͷର਺ͱೖྗͷΤϯτϩϐʔͷ࠷େԽ • $G .BY&OU3-  ͸ߦಈͷΤϯτϩϐʔ࠷େԽ •

    1SFGFSFODFͷઃఆ • ɿ,-DPOUSPM<,BQQFO > • ɿ,-SFHVMBSJ[FEDPOUSPM<5PEPSPW > • ɿFYQFDUFESFXBSE<4VUUPO > • ͜Ε͸DVSJPTJUZͷ߲͕ফ͑ΔΑ͏ͳઃఆʢ·ΕͳೖྗΛ໌ࣔతʹEJTDPVSBHFͯ͠୳ࡧΛ੍ݶʣ • ೖྗͷΤϯτϩϐʔͷਪఆɿٖΧ΢ϯτɼ4LFX'JUɼ3/%ɼ*$.ʜ KL [pϕ ∥τ] = − ∑ t E [ln τ (xt ∣ x1:t−1)] FYQFDUFEQSFGFSFODFT + H [pϕ (xt ∣ x1:t−1)] DVSJPTJUZ τ (xt ∣ x1:t−1) ∝ exp (r (xt)) τ (xt ∣ x1:t−1) ∝ exp (r (xt)) τ′ (xt ∣ x1:t−1) τ (xt ∣ x1:t−1) ∝ exp (r (xt)) pϕ (xt |x1:t−1 ) ʢܾఆ࿦తͳʣ੍ޚ 
  21. "DUVBMɿ  5BSHFUɿ  ྫʣ֬཰తͳ੍ޚΛ͢Δ৔߹ • NBY&OU3-ɾ3-BTJOGFSFODFͱ΋͍͏ • ೖྗͷܥྻ ɼߦಈͷܥྻ

     • աڈͷೖྗߦಈ ͕؍ଌ͞Ε͍ͯΔઃఆ pϕ (x, a) = ∏ t p (xt ∣ x1:t−1 , a1:t−1) FOWJSPONFOU pϕ (at ∣ x1:t , a1:t−1) QPMJDZ τ(x, a) ∝ ∏ t exp (r (xt)) SFXBSE p (xt ∣ x1:t−1 , a1:t−1) FOWJSPONFOU τ (at) ⏟ BDUJPOQSJPS x = {xt } a = {at } ax< .BY&OUSPQZ3- .BY&OU3- 
  22. +PJOU,-ʹͦΕͧΕ୅ೖ͢Δͱ  • ߦಈۭؒͰͷ੍໿ͱظ଴ใुͷ߲ʹͳΔ • λʔήοτʹ؀ڥͷμΠφϛΫε͕ೖΔͷͰͷDVSJPTJUZͷ߲͸ଧͪ ফ͞ΕΔ • ߦಈͷࣄલ෼෍ 

    • Ұ༷෼෍ɿ42-ɼ4"$ • աڈͷೖྗ΍ҎલͷλεΫͷ஌ࣝΛ౷߹ɿ%JTUSBM KL [pϕ ∥τ] = ∑ t EKL [pϕ (at ∣ x1:t , a1:t−1) ∥τ (at)] DPNQMFYJUZ − E [r (xt)] FYQFDUFESFXBSE τ(a) .BY&OUSPQZ3- .BY&OU3- 
  23. "DUVBMɿ  5BSHFUɿ  ྫʣ֬཰తͳ੍ޚʹ͓͚ΔHFOFSBMJ[FEFNQPXFSNFOU • աڈͷೖྗߦಈ ͕؍ଌ͞Ε͍ͯΔઃఆ • ೖྗ͔ΒߦಈΛ༧ଌ͢Δλʔήοτͷ෼෍ʢSFWFSTFQSFEJDUPSʣ

    pϕ (x, a) = ∏ t p (xt ∣ x1:t−1 , a1:t−1) FOWJSPONFOU pϕ (at ∣ x1:t , a1:t−1) QPMJDZ τ(x, a) ∝ ∏ t τϕ (at ∣ x1:T , a1:t−1) SFWFSTFQSFEJDUPS ax< &NQPXFSNFOU 
  24. +PJOU,-ʹͦΕͧΕ୅ೖ͢Δͱ  • (FOFSBMJTFEFNQPXFSNFOU͸૬ޓ৘ใྔ ͷԼք KL [pϕ ∥τϕ] = EKL[p(x

    ∣ a)∥τ(x)] DPOUSPM − E [ln τϕ (a ∣ x) − ln pϕ (a)] HFOFSBMJ[FEFNQPXFSNFOU E [ln τϕ (a ∣ x) − ln pϕ (a)] HFOFSBMJ[FEFNQPXFSNFOU ≥ ∑ t E[ln τϕ (at ∣ x, a1:t−1) EFDPEFS − ln pϕ (at ∣ a1:t−1) TPVSDF ] I[a; x] = E [ln pϕ (a|x) − ln pϕ (a)] &NQPXFSNFOU 
  25. "DUVBMɿ  5BSHFUɿ  ྫʣ࣌ؒతʹந৅Խ͞ΕͨTLJMMͰ৚͚݅ͮΒΕͨํࡦΛֶश • ೖྗͷܥྻ ɼߦಈͷܥྻ ɼεΩϧͷܥྻ 

    • εΩϧ͸ݻఆͷ෼෍ʢ͔աڈͷೖྗͷؔ਺ʣ͔Βબ୒ • ߦಈ͸աڈͷೖྗͱݱࡏͷεΩϧʹैͬͯબ୒ pϕ (x, a, z) ≐ T/K ∏ k=1 pϕ (zk) BCTUSBDUQPMJDZ T ∏ t=1 pϕ (at ∣ x1:t , a1:t−1 , z⌊t/K⌋) QPMJDZ p (xt ∣ x1:t−1 , a1:t−1) FOWJSPONFOU τϕ (x, a, z) ∝ T/K ∏ k=1 τϕ (zk ∣ x) SFWFSTFQSFEJDUPS T ∏ t=1 τ (at) ⏟ BDUJPOQSJPS x = {xt } a = {at } z = {zt } εΩϧͷൃݟʢ4LJMM%JTDPWFSZʣ 
  26. +PJOU,-ʹͦΕͧΕ୅ೖ͢Δͱ  • 4LJMMEJTDPWFSZ͸εΩϧ ͱೖྗ ͷ૬ޓ৘ใྔ ͷԼք • ΤʔδΣϯτ͸ଟ༷ͳೖྗܥྻΛ࣮ݱ͠ͳ͕ΒɼҟͳΔεΩϧͰܥ ྻ͕ॏͳΒͳ͍Α͏ʹεΩϧΛֶश͢Δ

    • ྫʣ%*:"/<&ZTFOCBDI >ɼ%"%4<4IBSNB > KL [pϕ ∥τϕ] = EKL[p(x ∣ a)∥τ(x)] DPOUSPM + EKL [pϕ (a ∣ x, z)∥τ(a)] DPNQMFYJUZ − E [ln τϕ (z ∣ x) − ln pϕ (z)] TLJMMEJTDPWFSZ z x I[z; x] εΩϧͷൃݟʢ4LJMM%JTDPWFSZʣ 
  27. "DUVBMɿ  5BSHFUɿ  ྫʣ&YQFDUFEJOGPSNBUJPOHBJOΛ࠷େԽ͢ΔߦಈΛબ୒ • ֎తͳใु͕ͳ͍৔߹ʹੈքΛޮ཰తʹֶश͢ΔͨΊ • ೖྗɿ 

    • HMPCBMͳදݱΛ࣋ͭજࡏతͳύϥϝʔλɿ  pϕ (x, w) = pϕ (w) ⏟ CFMJFG ∏ t pϕ (xt ∣ x1:t−1) DPOUSPMMFEEZOBNJDT τ(x, w) = τ(w) ⏟ QSJPS ∏ t τ (xt ∣ x1:t−1 , w) MJLFMJIPPE x = {xi } w *OGPSNBUJPO(BJO 
  28. +PJOU,-ʹͦΕͧΕ୅ೖ্ͯ͠քΛऔΔ   • ͱ͸ҟͳΓɼະདྷͷೖྗʹӨڹͰ͖Δ • աڈͷೖྗʹ͍ͭͯ༗ӹͳදݱΛֶश͠ɼະདྷͷදݱʹؔͯ͠༗ӹʹͳΔΑ͏ͳೖྗΛ୳ࡧ͢Δ • ϞσϧύϥϝʔλʹΑͬͯΤʔδΣϯτ͕؀ڥͷϧʔϧΛݟ͚ͭΔΑ͏ʹ͢Δٞ࿦ •

    BDUJWFJOGFSFODF<'SJTUPO >ɼ7*.&<)PVUIPPGU >ɼ."9<4IZBN ^ɼ1MBO&YQMPSF<4FLFS > KL [pϕ ∥τϕ] ≤ EKL [pϕ (w ∣ x<) ∥τ(w)] TJNQMJDJUZ − E [ln τϕ (x< ∣ w) − ln pϕ (x<)] SFQSFTFOUBUJPOMFBSOJOH +EKL [pϕ (x> ∣ x< , w) ∥τϕ (x> ∣ x<)] DPOUSPM − E [ln τϕ (w ∣ x) − ln pϕ (w ∣ x<)] JOGPSNBUJPOHBJO E [ln τϕ (w ∣ x) − ln pϕ (w ∣ x<)] JOGPSNBUJPOHBJO ≥ ∑ t′ >t E [ln τϕ (w ∣ x1:t′ ) − ln pϕ (w ∣ x1:t′ −1)] JOUSJOTJDSFXBSE *OGPSNBUJPO(BJO 
  29.