Upgrade to Pro — share decks privately, control downloads, hide ads and more …

時系列解析

taichi_murayama
September 27, 2022

 時系列解析

4コマ分の講義用資料。時系列解析について

taichi_murayama

September 27, 2022
Tweet

More Decks by taichi_murayama

Other Decks in Education

Transcript

  1. 3 ࣌ܥྻͱ͸ ࣌ܥྻ UJNFTFSJFT ࣌ؒͱͱ΋ʹෆنଇʹมಈ͢Δݱ৅Λɺ࿈ଓతʹ؍ଌͯ͠ಘΒΕͨ ஋ͷܥྻ 𝑡؍ଌͨ࣌͠ࠁ 𝑦! ࣌ࠁ 𝑡

    ʹ͓͚Δ؍ଌ஋ 𝑦" , … , 𝑦# ࣌ܥྻ ଟ͘ͷ࣌ؒͱͱ΋ʹมԽ͍ͯ͘͠σʔλ͸ɺ͜ͷΑ͏ͳදهͰදݱ ͢Δ͜ͱ͕Մೳ 𝑇 𝑌 ྫυϧԁ 2022年7⽉1⽇ 𝑦! = 135
  2. 5 ࣌ܥྻͱ͸ .PUJPO$BQUVSFͷྫ From: Matsubara, Yasuko, and Yasushi Sakurai. "Regime

    shifts in streams: Real-time forecasting of co- evolving time sequences." Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 2016.
  3. 6 ࣌ܥྻͱ͸ l ࣌ܥྻ༧ଌকདྷͷมಈΛਪଌ l ࣌ܥྻ෼ྨ༩͑ΒΕͨ࣌ܥྻΛछྨ͝ͱʹ෼ྨ l ҟৗݕ஌ มԽݕ஌࣌ܥྻͷதʹଘࡏ͢ΔҟৗΛൃݟ l

    ՄࢹԽ ಛ௃நग़ਓ͕ؒղऍՄೳͳܗʹ࣌ܥྻΛม׵ l ੍ޚૢ࡞Մೳͳม਺ΛมԽͤ͞ɺ໨తม਺Λ੍ޚ FUD ࣌ܥྻΛѻͬͨ՝୊ λεΫ
  4. 7 ࣌ܥྻͱ͸ l ࣌ܥྻ༧ଌকདྷͷมಈΛਪଌ l ࣌ܥྻ෼ྨ༩͑ΒΕͨ࣌ܥྻΛछྨ͝ͱʹ෼ྨ l ҟৗݕ஌ มԽݕ஌࣌ܥྻͷதʹଘࡏ͢ΔҟৗΛൃݟ l

    ՄࢹԽ ಛ௃நग़ਓ͕ؒղऍՄೳͳܗʹ࣌ܥྻΛม׵ l ੍ޚૢ࡞Մೳͳม਺ΛมԽͤ͞ɺ໨తม਺Λ੍ޚ FUD ࣌ܥྻΛѻͬͨ՝୊ λεΫ
  5. 15 ࠓճͷେ·͔ͳ֓ཁ l ౷ܭతϞσϧ l ࣌ܥྻͷੑ࣭ l ݹయత࣌ܥྻϞσϧ l Ϟσϧͷਪఆɾબ୒

    l ਂ૚ֶशϕʔεϞσϧ l ۙ೥ͷਂ૚ֶशͷൃల l ࣌ܥྻ༧ଌ ਂ૚ֶश ౷ܭతϞσϧ ਂ૚ֶशϕʔεϞσϧ
  6. 17 ࣌ܥྻͷੑ࣭ l ౳ִؒ࣌ܥྻ l αϯϓϦϯάظ͕ؒҰఆ l ݚڀͳͲͰѻ͏ݚڀͷ΄ͱΜͲ͸͜Εʹ౰ͨΔ ܽଛ஋ิ׬ͳͲͰ౳ִؒʹ͢Δ৔߹΋ l

    ෆ౳ִؒ࣌ܥྻ l Πϕϯτ࣌ܥྻͳͲ l αϯϓϦϯάظ͕ؒૄΒ ౳ִؒ࣌ܥྻ ෆ౳ִؒ࣌ܥྻ ࣌ؒ 猫かわいい 1PTU 35 35
  7. 19 ࣌ܥྻͷੑ࣭ l ૬Ճ ฏۉ l ෼ࢄ l ฏۉ͔Βͷ͹Β͖ͭ۩߹Λࣔ͢ࢦඪ l

    ฏۉ஋͔Βͷࠩ෼Ͱ͋ΔภࠩΛೋ৐ͷฏۉ l ڞ෼ࢄ l ڞ෼ࢄ͕େ͖͍ͱਖ਼ͷڧ͍ؔ܎Λ࣋ͭͳͲɺ ૊ͷରԠ͢Δσʔλͷؔ܎ Λࣔ͢ ఆৗੑඇఆৗੑ 𝐸(𝑦) = µ = 1 𝑛 ) 𝑦! 𝜎" = 𝑉𝐴𝑅 𝑦! = 1 𝑛 ) 𝑦! − µ " Cov(𝑥, 𝑦) = 𝜎#$ = E[ 𝑥 − µ# (𝑦 − µ$ )]
  8. 20 ࣌ܥྻͷੑ࣭ l ڧఆৗੑ TUSJDUTUBUJPOBSJUZ l ೚ҙͷtͱ𝑘 ೚ҙͷ࣌ؒࠩ ʹରͯ͠ɺ σʔλΛੜ੒͢Δ֬཰աఔ

    f(𝑦!, … , 𝑦!+,)͕ৗʹಉҰͷಉ࣌෼෍Λ࣋ͭ l ֬཰෼෍͕࣌ؒมԽ͠ͳ͍ l ྫαΠίϩͷग़໨ͷܥྻ΍ίΠϯτεͷ݁ՌͳͲ ఆৗੑඇఆৗੑ f 𝑦% , … , 𝑦& = f 𝑦' , … , 𝑦'(&
  9. 21 ࣌ܥྻͷੑ࣭ l ऑఆৗੑ XFBLTUBUJPOBSJUZ l ظ଴஋ͱࣗݾڞ෼ࢄ͕࣌఺ʹґଘ͠ͳ͍ l ೚ҙͷtͱ𝑘 ೚ҙͷ࣌ؒࠩ

    ʹରͯ͠ҎԼ͕੒Γཱͭ ࣌఺ʹґଘ͠ͳ͍ l ࣍ͷϞʔϝϯτ·Ͱ͕ෆม l Ұൠతʹɺʮఆৗੑʯͱݺ͹ΕΔ࣌ɺͪ͜ΒΛࢦ͢ࣄ͕ଟ͍ l ࣗݾ૬͕ؔLͱͱ΋ʹࢦ਺తʹݮਰ ఆৗੑඇఆৗੑ E 𝑦* = E 𝑦*(& , VAR 𝑦* = VAR 𝑦*(& Cov 𝑦* , 𝑦*(& = 𝛾& > 0
  10. 22 ࣌ܥྻͷੑ࣭ l ऑఆৗੑ XFBLTUBUJPOBSJUZ l ظ଴஋ͱࣗݾڞ෼ࢄ͕࣌఺ʹґଘ͠ͳ͍ l ೚ҙͷtͱ𝑘 ೚ҙͷ࣌ؒࠩ

    ʹରͯ͠ҎԼ͕੒Γཱͭ ࣌఺ʹґଘ͠ͳ͍ l ࣍ͷϞʔϝϯτ·Ͱ͕ෆม l Ұൠతʹɺʮఆৗੑʯͱݺ͹ΕΔ࣌ɺͪ͜ΒΛࢦ͢ࣄ͕ଟ͍ l ࣗݾ૬͕ؔLͱͱ΋ʹࢦ਺తʹݮਰ ఆৗੑඇఆৗੑ E 𝑦* = E 𝑦*(& , VAR 𝑦* = VAR 𝑦*(& Cov 𝑦* , 𝑦*(& = 𝛾& > 0 ଟ͘ͷ࣌ܥྻ౷ܭతϞσϧ͕ఆৗੑΛલఏͱ͍ͯ͠Δͨ Ίɺ෼ੳ͢Δ࣌ܥྻ͕ఆৗͰ͋Δͱخ͍͠ ఆৗੑΛલఏͱ͠ͳ͍ͱɺฏۉ΍෼ࢄ͕ҰఆͰແ͘ͳΔͨ Ίɺ෼ੳ͕ؤ݈Ͱͳ͘ͳΔ
  11. 23 ࣌ܥྻͷੑ࣭ l ऑఆৗ ਖ਼ن෼෍ 㱺 ڧఆৗ l ฏۉɺ෼ࢄɺڞ෼ࢄ͕ଘࡏ͢Δڧఆৗ 㱺

    ऑఆৗ ˎ ڧఆৗΛຬ͔ͨ͢Βͱ͍ͬͯɺඞͣ͠΋ऑఆৗͷ৚݅Λຬͨ͢ ͱ͸ݶΒͳ͍ FH ฏۉɺ෼ࢄ͕ଘࡏ͠ͳ͍෼෍ͳͲ ίʔγʔ෼෍ͳͲͷ੄͕ॏ͘ɾ޿͍෼෍ ఆৗੑඇఆৗੑ
  12. 26 ࣌ܥྻͷੑ࣭ l #PY$PYม׵ l σʔλͷ෼෍Λਖ਼ن෼෍ʹ͚ۙͮΔͨΊͷ΂͖৐ܕม׵ l 𝜆͸ύϥϝʔλͰɺ 𝜆ʹΑͬͯ ม׵ͷܗ͕ࣜมΘΔ

    ఆৗੑ΁ͷม׵ 𝑧* = B $!+% , (𝜆 ≠ 0) ln 𝑦 (𝜆 = 0) from: http://www.kmdatascience.com/2017/07/box-cox-transformations-in-python.html
  13. 28 ࣌ܥྻͷੑ࣭ l ֊ ֊ࠩม׵ l τϨϯυ੒෼ΛऔΓআ͖ɺฏۉʹؔͯ͠ఆৗੑΛຬͨ͢Α͏ʹม׵ ఆৗੑ΁ͷม׵ 𝑧* =

    𝑦* − 𝑦*+% from: https://datascienceplus.com/time-series-analysis-in-r-part-2-time-series-transformations/
  14. 29 ࣌ܥྻͷੑ࣭ l قઅ ֊ࠩม׵ l قઅ੒෼ʹΑΔӨڹपظੑΛऔΓআ͖ɺฏۉʹؔͯ͠ఆৗੑΛຬͨ͢Α ͏ʹม׵ l 𝑠

    ͸पظ ఆৗੑ΁ͷม׵ 𝑧* = 𝑦* − 𝑦*+- = (1 − 𝐵-)𝑦* from: https://datascienceplus.com/time-series-analysis-in-r-part-2-time-series-transformations/
  15. 30 ࣌ܥྻͷੑ࣭ l ฏ׈Խ l ϊΠζ΍୹ظతͳมಈΛআ͘ޮՌΛظ଴ͯ͠ɺ׈Β͔ʹͳΔΑ͏ʹม׵ l ୅දྫͱͯ͠ɺҠಈฏۉϑΟϧλ .PWJOH"WFSBHF'JMUFS ఆৗੑ΁ͷม׵

    𝑧* = 1 M ) ./0 1+% 𝑦. from: https://machinelearningmastery.com/moving-average-smoothing-for-time-series- forecasting-python/
  16. 32 ࣌ܥྻͷੑ࣭ l ࣗݾڞ෼ࢄؔ਺ l ऑఆৗաఔͷࣗݾڞ෼ࢄΛԾఆ l ࣗݾڞ෼ࢄؔ਺͸ҎԼͷܗͰఆٛ͞ΕΔ l ࣗݾڞ෼ࢄؔ਺͸ۮؔ਺Cov

    𝑦!, 𝑦!+, = 𝛾, = 𝛾-, = Cov 𝑦!, 𝑦!-, l ࣗݾڞ෼ࢄؔ਺ͷઈର஋ͷ࠷େ͸𝛾. 𝛾. ≥ 𝛾, l ࣗݾڞ෼ࢄؔ਺ͷਤࣔͱͯ͠ίϨϩάϥϜ͕༻͍ΒΕΔ ࣌ܥྻͷ܏޲ͷཧղ Cov 𝑦% , 𝑦& = Cov 𝑦* , 𝑦*(& = 𝛾& 𝑘͸ϥάΛද͢
  17. 33 ࣌ܥྻͷੑ࣭ l ࣗݾ૬ؔؔ਺ l ऑఆৗաఔͷࣗݾ૬ؔΛԾఆ l ҎԼͷܗͰఆٛ͞ΕΔ ࣌ܥྻͷ܏޲ͷཧղ Cor

    𝑦* , 𝑦*(& = 234 $",$"#$ 678 $" 678($"#$) = ;$ ;% = 𝑅& 𝑘͸ϥάΛද͢
  18. 37 ࣌ܥྻͷੑ࣭ l εϖΫτϧղੳ l ࣌ܥྻσʔλΛߏ੒प೾੒෼ʹ෼ղ͠ɺ֤प೾਺ͱΤωϧΪʔͷؔ܎Λऔ Γग़ͨ͢Ίͷख๏ l Α͘༻͍ΒΕΔͷ͸ɺ཭ࢄϑʔϦΤม׵ ''5

    l ෳࡶͳपظΛऔΓग़͢͜ͱΛՄೳʹ͢Δɻपظੑͱ͸𝑝͕पظΛࣔ࣌͢ʹ 𝑦! = 𝑦!-/ ͕੒Γཱͭ͜ͱ ࣌ܥྻͷ܏޲ͷཧղεϖΫτϧղੳ from: https://qiita.com/AnchorB lues/items/5497ee68c3a3 d64875d4 FFT 振動数 振幅
  19. 38 ࣌ܥྻͷੑ࣭ l ϑʔϦΤม׵ l प೾਺𝒇ɼपظؔ਺y(t ͱͨ͠ͱ͖ҎԼͷܗʹͳΔ l ෳ਺ͷ೾ͰܥྻΛઆ໌Ͱ͖ΔΑ͏ʹ෼ղ l

    ٯϑʔϦΤม׵ l ೾਺𝟐𝝅𝒇ͷਖ਼ݭ೾ʹରͯ͠ॏΈΛֻ͚ͯɺੵ෼͢Δ͜ͱͰݩͷܥྻʹ໭͢ ࣌ܥྻͷ܏޲ͷཧղεϖΫτϧղੳ 𝒀 𝒇 = 2 "# # 𝒚(𝒕)𝒆"𝟐𝝅𝒊𝒕𝒇𝒅𝒕 ∗ ∫ "# # 𝒚 𝒕 𝒅𝒕 < ∞Λຬͨ͢ 𝒚 𝒕 = 2 "# # 𝒀(𝒇)𝒆𝟐𝝅𝒊𝒕𝒇𝒅𝒇
  20. 39 ࣌ܥྻͷੑ࣭ l ϑʔϦΤม׵ l प೾਺𝒇ɼपظؔ਺y(t ͱͨ͠ͱ͖ҎԼͷܗʹͳΔ l ෳ਺ͷ೾ͰܥྻΛઆ໌Ͱ͖ΔΑ͏ʹ෼ղ l

    ٯϑʔϦΤม׵ l ೾਺𝟐𝝅𝒇ͷਖ਼ݭ೾ʹରͯ͠ॏΈΛֻ͚ͯɺੵ෼͢Δ͜ͱͰݩͷܥྻʹ໭͢ ࣌ܥྻͷ܏޲ͷཧղεϖΫτϧղੳ ∗ ∫ "# # 𝒚 𝒕 𝒅𝒕 < ∞Λຬͨ͢ ॏΈਖ਼ݭ೾ 𝒀 𝒇 = 2 "# # 𝒚(𝒕)𝒆"𝟐𝝅𝒊𝒕𝒇𝒅𝒕 𝒚 𝒕 = 2 "# # 𝒀(𝒇)𝒆𝟐𝝅𝒊𝒕𝒇𝒅𝒇
  21. 40 ࣌ܥྻͷੑ࣭ l ύϫʔεϖΫτϧ l ϑʔϦΤ੒෼𝒀(𝒇)͸೾Ͱ͋Δ𝒆𝒊𝒕𝟐𝝅𝒇ͷৼΕ෯Λද͢ 㱺 𝒀(𝒇) 𝟐͸ΤωϧΪʔ l

    ܥྻͷதʹؚ·ΕΔप೾਺ͱͦͷύϫʔΛՄࢹԽ ࣌ܥྻͷ܏޲ͷཧղεϖΫτϧղੳ 𝒚 𝒕 = 2 "# # 𝒀(𝒇)𝒆𝟐𝝅𝒊𝒕𝒇𝒅𝒇 ॏΈਖ਼ݭ೾
  22. 41 ࣌ܥྻͷੑ࣭ ࣌ܥྻͷ܏޲ͷཧղεϖΫτϧղੳ 𝑓(𝑡) = B 1, 0 < 𝑡

    < 𝜏 0, 𝑜𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒 ೖྗܥྻ 𝐹 𝜔 = 2 "# # 𝑓(𝑡)𝑒")*!𝑑𝑡 = 2 + , 𝑓(𝑡)𝑒")*!𝑑𝑡 = 1 −𝑗𝑤 𝑒")*! + , = 1 𝑗𝑤 1 − 𝑒")*, = -!"#$/& ). 𝑒)*,/0 − 𝑒")*,/0 = 𝑒")*,/0 0 * sin *, 0 ϑʔϦΤม׵ 𝐹 𝜔 = 𝑒")*,/0 2 𝜔 sin 𝜔𝜏 2 = 2 𝜔 sin 𝜔𝜏 2 ৼΕ෯εϖΫτϧ ( 𝐹 𝜔 = 𝐹 𝜔 𝑒)1(*) ) ύϫʔεϖΫτϧ 𝐹 𝜔 0 = 2 𝜔 sin 𝜔𝜏 2 0 = 4 𝜔0 sin0 𝜔𝜏 2
  23. 42 ࣌ܥྻͷੑ࣭ ࣌ܥྻͷ܏޲ͷཧղεϖΫτϧղੳ 𝑓(𝑡) = B 1, 0 < 𝑡

    < 𝜏 0, 𝑜𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒 ೖྗܥྻ 𝐹 𝜔 = 2 "# # 𝑓(𝑡)𝑒")*!𝑑𝑡 = 2 + , 𝑓(𝑡)𝑒")*!𝑑𝑡 = 1 −𝑗𝑤 𝑒")*! + , = 1 𝑗𝑤 1 − 𝑒")*, = -!"#$/& ). 𝑒)*,/0 − 𝑒")*,/0 = 𝑒")*,/0 0 * sin *, 0 ϑʔϦΤม׵ 𝐹 𝜔 = 𝑒")*,/0 2 𝜔 sin 𝜔𝜏 2 = 2 𝜔 sin 𝜔𝜏 2 ৼΕ෯εϖΫτϧ ( 𝐹 𝜔 = 𝐹 𝜔 𝑒)1(*) ) ύϫʔεϖΫτϧ 𝐹 𝜔 0 = 2 𝜔 sin 𝜔𝜏 2 0 = 4 𝜔0 sin0 𝜔𝜏 2
  24. 43 ࣌ܥྻͷੑ࣭ l ύϫʔεϖΫτϧ l ϑʔϦΤ੒෼𝒀(𝒇)͸೾Ͱ͋Δ𝒆𝒊𝒕𝟐𝝅𝒇ͷৼΕ෯Λද͢ 㱺 𝒀(𝒇) 𝟐͸ΤωϧΪʔ l

    ܥྻͷதʹؚ·ΕΔप೾਺ͱͦͷύϫʔΛՄࢹԽ l ΢Οʔφʔɾώϯνϯ 8JFOFS,IJOUDIOF ͷެࣜࣗݾڞ෼ࢄؔ਺𝛾, ͷϑʔ ϦΤม׵͸ύϫʔεϖΫτϧʹͳΔ ࣌ܥྻͷ܏޲ͷཧղεϖΫτϧղੳ 𝒚 𝒕 = 2 "# # 𝒀(𝒇)𝒆𝟐𝝅𝒊𝒕𝒇𝒅𝒇 ॏΈਖ਼ݭ೾ Cov 𝑦", 𝑦, = 𝐶ov 𝑦!, 𝑦!+, = 𝛾, 𝑘͸ϥάΛද͢ 𝑌 𝑓 = ? !"#$ $ 𝛾!𝑒#%&'!( − 1 2 ≤ 𝑓 ≤ 1 2
  25. 44 ࣌ܥྻͷੑ࣭ l ύϫʔεϖΫτϧ l ۮؔ਺ͱίαΠϯม׵ͷੑ࣭͔Β l ྫ୯ৼಈ ࣌ܥྻͷ܏޲ͷཧղεϖΫτϧղੳ 𝑌

    𝑓 = ? !"#$ $ 𝛾!𝑒#%&'!( = 𝛾) + 2 ? !"* $ 𝛾!cos (2𝜋𝑘𝑓) 𝛾! = 𝑎% 2 cos(2𝜋𝑓)𝑘) 𝑌 𝑓 = ? !"#$ $ cos(2𝜋𝑓)𝑘)cos(2𝜋𝑓𝑘) = 1 2 ? !"#$ $ cos 2𝜋𝑘 𝑓) − 𝑓 − cos 2𝜋𝑘 𝑓) + 𝑓 = M * %+, 𝑓 = 𝑓) 0, 𝑓 ≠ 𝑓)
  26. 47 ࣌ܥྻϞσϧ l ࣌ؒతʹ૬͕ؔແ͍࣌ܥྻ𝜀! l ࣗݾ૬ؔؔ਺΋ l ֬཰తͳ͹Β͖ͭΛදݱ l FH

    ಠཱʹੜ੒ͨ͠ਖ਼نཚ਺ྻͳͲ ന৭ࡶԻ ϗϫΠτϊΠζ 𝐸 𝜀* = 0, 𝑉𝐴𝑅 𝜀* = 𝜎", 𝑅. = 0 𝑗͸ϥάΛද͢ from: https://machinelearningmastery.com/white-noise-time-series-python/
  27. 48 ࣌ܥྻϞσϧ l 𝑝࣍ͷઢܗࠩ෼ํఔࣜʹϗϫΠτϊΠζΛՃ͑ͨϞσϧ"3 𝒑 l 𝑝࣍"3Ϟσϧͱݺ͹ΕΔ l աڈͷܥྻ 𝑦!-"

    , … 𝑦!-/ ͷೖྗͱϗϫΠτϊΠζͰݱࡏͷܥྻΛճؼ͢Δ Ϟσϧ ࣗݾճؼաఔ "3Ϟσϧ 𝑦* = ) !/% P 𝑎! 𝑦*+! + 𝜀*
  28. 49 ࣌ܥྻϞσϧ l ௚લͷ𝒒 ݸͷϗϫΠτϊΠζͷՃॏ࿨Ͱఆٛ͞ΕΔ𝒒 ࣍."Ϟσϧ." 𝒒 l աڈͷܥྻ 𝑦!-",

    … 𝑦!-/ ʹ༩͑ΒΕΔϗϫΠτϊΠζ ϥϯμϜγϣοΫ ͕࣌ܥྻͷকདྷͷ஋ʹ௚઀఻ൖ͢Δ͜ͱΛϞσϦϯά l ϗϫΠτϊΠζͱಉ༷ʹࣗݾڞ෼ࢄ΍ظ଴஋͸࣌఺ʹґଘͤͣɺऑఆৗੑ Λຬͨ͢ Ҡಈฏۉաఔ ."Ϟσϧ 𝑦* = 𝜇 + ) !/0 Q 𝑏! 𝜀*+!
  29. 50 ࣌ܥྻϞσϧ l ௚લͷ𝒑 ݸͷܥྻͱ௚લͷ𝒒 ݸͷϗϫΠτϊΠζͷՃॏ࿨ʹΑͬͯߏ੒͞ ΕΔϞσϧ"3." 𝒑, 𝒒) l

    "3Ϟσϧ΋."Ϟσϧ΋"3."Ϟσϧͷಛघͳྫ l "3Ϟσϧ𝒒 = 0 l ."Ϟσϧ 𝒑 = 0 l ϗϫΠτϊΠζ𝒑 = 0, 𝒒 = 0 l ϗϫΠτϊΠζ𝜀! ͸աڈͷܥྻͱແ૬ؔ𝔼 𝜀! , 𝑦!-X = 0, j > 0 ࣗݾճؼҠಈฏۉաఔ "3."Ϟσϧ 𝑦* = ) !/% P 𝑎! 𝑦*+! + ) !/% Q 𝑏! 𝜀*+! + 𝜀*
  30. 51 ࣌ܥྻϞσϧ l -BHPQFSBUPSΛ༻͍Δ͜ͱͰ"3." 𝒑, 𝒒)ϞσϧΛҎԼͷܗʹॻ͖׵͑Δ ͜ͱ͕Ͱ͖Δ ࣗݾճؼҠಈฏۉաఔ "3."Ϟσϧ 𝑦*

    = ) !/% P 𝑎! 𝑦*+! + ) !/% Q 𝑏! 𝜀*+! + 𝜀* 𝐿-BHPQFSBUPS 𝐿% 𝑦* = 𝑦*+% , 𝐿& 𝑦* = 𝑦*+& , (1 − ) !/% P 𝑎! 𝐿!)𝑦* = (1 − ) !/% Q 𝑏! 𝐿!)𝜀* = 𝑎(𝐿) = 𝑏(𝐿) "30QFSBUPS ."0QFSBUPS 𝑎(𝐿)𝑦* = 𝑏(𝐿)𝜀*
  31. 52 ࣌ܥྻϞσϧ ࣗݾճؼҠಈฏۉաఔ "3."Ϟσϧ (1 − ) !/% P 𝑎!

    𝐿!)𝑦* = (1 − ) !/% Q 𝑏! 𝐿!)𝜀* = 𝑎(𝐿) = 𝑏(𝐿) 𝑎(𝐿)𝑦* = 𝑏(𝐿)𝜀* 𝑦* = 𝑎(𝐿)+%𝑏(𝐿)𝜀* 𝑦* = ) ./0 T 𝑔. 𝐿.𝜀* = ) ./0 T 𝑔. 𝜀*+.
  32. 53 ࣌ܥྻϞσϧ ࣗݾճؼҠಈฏۉաఔ "3."Ϟσϧ (1 − ) !/% P 𝑎!

    𝐿!)𝑦* = (1 − ) !/% Q 𝑏! 𝐿!)𝜀* = 𝑎(𝐿) = 𝑏(𝐿) 𝑎(𝐿)𝑦* = 𝑏(𝐿)𝜀* 𝑦* = 𝑎(𝐿)+%𝑏(𝐿)𝜀* 𝑦* = ) ./0 T 𝑔. 𝐿.𝜀* = ) ./0 T 𝑔. 𝜀*+. "3."Ϟσϧ͸ແݶ࣍ͷ."Ϟσϧ Ͱදݱ͕Մೳ Πϯύϧε Ԡ౴ؔ਺
  33. 54 ࣌ܥྻϞσϧ ΠϯύϧεԠ౴ؔ਺ *3' 𝑦* = ) ./0 T 𝑔.

    𝜀*+. l ༩͑ΒΕͨϊΠζ Πϯύϧε ͕𝑗ظޙʹ࣌ܥྻʹͲΕ͘Β͍ӨڹΛ༩͑ Δ͔ʢӨڹྗ͕ͲΕ͚ͩ࢒Δ͔ʣΛࣔͨؔ͠਺ 𝑔0 = 1, 𝑔! = ) ./% P 𝑎. 𝑔!+. − 𝑏! *3'ͷࢉग़ from: https://alexchinco.com/impulse-response-functions-for-vars/
  34. 55 ࣌ܥྻϞσϧ l ࣗݾ૬ؔؔ਺ BVUPDPSSFMBUJPOGVODUJPO"$' l ภࣗݾ૬ؔؔ਺ BVUPDPSSFMBUJPOGVODUJPO1"$' l ϥά𝑘ͷࣗݾ૬ؔΛߟྀ͢Δࡍʹɺϥά1,

    … 𝑘 − 1·Ͱͷܥྻͷࣗݾ૬ؔͷ Өڹ΋ଘࡏ͢Δ l ͜Ε·ͰͷϥάͷӨڹΛআڈͨ͠૬ؔؔ਺ ࣗݾ૬ؔؔ਺ "$' ͱภࣗݾ૬ؔؔ਺ 1"$' Cor 𝑦* , 𝑦*(& = 234 $",$"#$ 678 $" 678($"#$) = ;$ ;% = 𝑅& 𝑘͸ϥάΛද͢ PACF 𝑦* , 𝑦*(& = Cov 𝑦*(& 𝑦*(&+% , … , 𝑦*(% , 𝑦* 𝑦*(&+% , … , 𝑦*(% 𝑉𝑎𝑟 𝑦* 𝑦*(&+% , … , 𝑦*(% 𝑉𝑎𝑟 𝑦*(& 𝑦*(&+% , … , 𝑦*(%
  35. 56 ࣌ܥྻϞσϧ l ࣗݾ૬ؔؔ਺ BVUPDPSSFMBUJPOGVODUJPO"$' l ภࣗݾ૬ؔؔ਺ BVUPDPSSFMBUJPOGVODUJPO1"$' l ϥά𝑘ͷࣗݾ૬ؔΛߟྀ͢Δࡍʹɺϥά1,

    … 𝑘 − 1·Ͱͷܥྻͷࣗݾ૬ؔͷ Өڹ΋ଘࡏ͢Δ l ͜Ε·ͰͷϥάͷӨڹΛআڈͨ͠૬ؔؔ਺ ࣗݾ૬ؔؔ਺ "$' ͱภࣗݾ૬ؔؔ਺ 1"$' Cor 𝑦* , 𝑦*(& = 234 $",$"#$ 678 $" 678($"#$) = ;$ ;% = 𝑅& 𝑘͸ϥάΛද͢ PACF 𝑦* , 𝑦*(& = Cov 𝑦*(& 𝑦*(&+% , … , 𝑦*(% , 𝑦* 𝑦*(&+% , … , 𝑦*(% 𝑉𝑎𝑟 𝑦* 𝑦*(&+% , … , 𝑦*(% 𝑉𝑎𝑟 𝑦*(& 𝑦*(&+% , … , 𝑦*(%
  36. 62 ࣌ܥྻϞσϧ l ฏۉʹؔͯ͠ඇఆৗͳ࣌ܥྻΛL֊ࠩ෼Λߦ͍ɺ"3."ϞσϧΛϑΟο ςΟϯάͨ͠Ϟσϧ"3*." 𝒑, 𝒌, 𝒒) l 𝒌

    = 𝟎ͷ࣌"3."Ϟσϧ ࣗݾճؼ࿨෼Ҡಈฏۉաఔ "3*."Ϟσϧ 𝑧* = ) !/% P 𝑎! 𝑧*+! + ) !/% Q 𝑏! 𝜀*+! + 𝜀* ∆𝑦* = 𝑦* − 𝑦*+% 𝑧* = ∆&𝑦* L֊ࠩ෼ΛऔΔ͜ͱͰ ఆৗͳ࣌ܥྻʹม׵
  37. 63 ࣌ܥྻϞσϧ l قઅੑ֊ࠩΛߟྀͨ͠"3*."Ϟσϧͱɺقઅੑ֊ࠩΛߟྀ͠ͳ͍௨ৗͷ "3*."Ϟσϧͷͭͷ੒෼Λ૊Έ߹Θͨ͠Ϟσϧ 4"3*.""3*." 𝒑, 𝒌, 𝒒) º

    𝑷, 𝑲, 𝑸)𝑺 4FBTPOBM"3*."Ϟσϧ 4"3*." 𝑧* U = ) !/% P 𝑎! U𝑧*+! U + ) !/% Q 𝑏! U𝜀*+! + 𝜀* 𝑧* U = 𝑦* − 𝑦*+U 𝑆͸Ұपظͷ௕͞
  38. 64 ࣌ܥྻϞσϧ l ྫ𝑆 = 12ͷ4"3*."Ϟσϧ 4"3*.""3*." 𝟏, 𝟎, 𝟎)

    º 𝟏, 𝟎, 𝟎)𝟏𝟐 4FBTPOBM"3*."Ϟσϧ 4"3*." 1 − 𝑎% 𝐿% 1 − 𝑎% U𝐿%" 𝑦* = 𝜀* 𝑦* = 𝑎% 𝑦*+% + 𝑎% U𝑦*+%" − 𝑎% 𝑎% U𝑦*+%V + 𝜀* 𝐿-BHPQFSBUPS 𝐿% 𝑦* = 𝑦*+% , 𝐿& 𝑦* = 𝑦*+& , قઅੑ ඇقઅੑ
  39. 65 ࣌ܥྻϞσϧ 4FBTPOBM"3*."Ϟσϧ 4"3*." Kumar Barik, Aditya, et al. "Analysis

    of GHI Forecasting Using Seasonal ARIMA." Data Management, Analytics and Innovation. Springer, Singapore, 2021. 55-69.
  40. 66 ࣌ܥྻϞσϧ l ࣌ܥྻΛτϨϯυɺقઅੑͳͲͷෳ਺ͷߏ੒ཁૉʹ෼ղ͢Δख๏ l "EEJUJWFEFDPNQPTJUJPO l .VMUJQMJDBUJWFEFDPNQPTJUJPO l ୅දతͳख๏

    l X11 decomposition l STL decopostion l SEATS decompositon ࣌ܥྻ෼ղ 5JNFTFSJFTEFDPNQPTJUJPO 𝑦! = 𝑆! ×𝑇! ×𝑅! , log 𝑦! = log 𝑆! + log 𝑇! + log 𝑅! 𝑦! = 𝑆! + 𝑇! + 𝑅!
  41. 68 ࣌ܥྻϞσϧ l ࣌఺𝑡ʹ͓͚Δ𝑛ݸͷมྔ͔ΒͳΔܥྻσʔλΛϕΫτϧͱͯ͠ଊ͑Δ l ϗϫΠτϊΠζ΋ಉ༷ʹϕΫτϧͱͯ͠ଊ͑Δ l 7"3 Q Ϟσϧ͸ҎԼͷࣜͱͳΔ

    ϕΫτϧࣗݾճؼϞσϧ 7"3Ϟσϧ 𝑌* = (𝑦%,* , 𝑦",* , … , 𝑦',* )W 𝔼 𝜺𝒕 = [0, … , 0]W, 𝔼 𝜺𝒕 𝜺𝒕+𝒌 = [ 𝚺 (𝑘 = 0) 𝟎, (𝑘 ≠ 0) 𝑌* = ) !/% P 𝑨𝒊 𝑌*+! + 𝜺𝒕 𝑨𝒊 = 𝑎! (1,1) ⋯ 𝑎! (1, 𝑛) ⋮ ⋱ ⋮ 𝑎! (𝑛, 1) ⋯ 𝑎! (𝑛, 𝑛)
  42. 69 ࣌ܥྻϞσϧ άϨϯδϟʔҼՌ (SBOHFS$BVTBMJUZ l ࣌ܥྻؒಉ࢜ͷҼՌؔ܎Λਪఆ͢ΔͨΊͷํ๏Ͱɺܥྻಉ࢜ͷ޲ ͖Λൃݟ͢Δख๏ l 𝑦%,* ͷ༧ଌʹ͓͍ͯ

    𝑦%,* ͷաڈͷ஋ʹج͍ͮͨ༧ଌΑΓ΋ɺ 𝑦%,* , 𝑦" * ͷաڈͷ஋Λ༻͍ͨ༧ଌʹΑͬͯ.4&͕খ͘͞ͳΔ৔߹ 𝒚𝟐 𝒕 ͔Β 𝒚𝟏,𝒕 ΁ͷάϨϯδϟʔҼՌੑ͕ଘࡏ͢Δͱ͢Δ 𝑦% 𝑦"
  43. 70 ࣌ܥྻϞσϧ άϨϯδϟʔҼՌ (SBOHFS$BVTBMJUZ l άϨϯδϟʔҼՌͷൃݟʹ͸'౷ܭྔΛ༻͍Δͷ͕Ұൠత 𝑦% ͷϞσϧʹΑΔ࢒ࠩฏํ࿨𝑆𝑆𝑅0 𝑦% ,

    𝑦" ͷϞσϧʹΑΔ࢒ࠩฏํ࿨𝑆𝑆𝑅% '౷ܭྔͷࢉग़𝐹 ≡ (UU]% +UU]&)/8 UU]&/(W +'P +%) '౷ܭྔͷ஋ʹج͍ͮͯάϨϯδϟʔҼՌ ͕ଘࡏ͢Δ͔Ͳ͏͔Λݕఆ
  44. 71 ࣌ܥྻϞσϧ l ඇఆৗ࣌ܥྻΛϑΟοςΟϯά͢ΔͨΊͷϞσϦϯάख๏ͷ̍ͭ ہॴఆৗ"3Ϟσϧ ϨδʔϜ෼ׂ Matsubara, Yasuko, Yasushi Sakurai,

    and Christos Faloutsos. "Autoplait: Automatic mining of co-evolving time sequences." Proceedings of the 2014 ACM SIGMOD international conference on Management of data. 2014.
  45. 74 ࣌ܥྻϞσϧ ঢ়ଶۭؒϞσϧ 𝑦! 𝑦!"# 𝑦!$# 𝑥!$# 𝑥! 𝑥!"# 𝑣!"#

    𝑣! 𝑣!$# 𝑤!$# 𝑤! 𝑤!"# 𝐹! 𝐺! 𝐻! 𝑥! = 𝐹! 𝑥!$# + 𝐺! 𝑣!$# 𝑦! = 𝐻! 𝑥! + 𝑤! γεςϜϞσϧ ؍ଌϞσϧ 𝑦* 𝑛࣍ݩ࣌ܥྻ 𝑥* 𝑚࣍ݩঢ়ଶϕΫτϧ 𝑣* 𝑘࣍ݩγεςϜϊΠζ 𝑤* 𝑛࣍ݩ؍ଌϊΠζ 𝑣* ~ 𝑁 0, 𝑄* , 𝑤* ~ 𝑁 0, 𝑅*
  46. 75 ࣌ܥྻϞσϧ "3ϞσϧΛঢ়ଶۭؒϞσϧͰදݱ 𝑥! = 𝐹! 𝑥!$# + 𝐺! 𝑣!$#

    𝑦! = 𝐻! 𝑥! + 𝑤! γεςϜϞσϧ ؍ଌϞσϧ 𝑦* = ) !/% P 𝑎! 𝑦*+! + 𝜀* 𝜀* ~ N(0, 𝜎") "3Ϟσϧ 𝑦! 𝑦!-" ⋮ 𝑦!-/+" = 𝑎4 𝑎0 1 ⋯ 𝑎5 ⋮ ⋱ ⋮ ⋯ 1 𝑦!-" 𝑦!-Z ⋮ 𝑦!-/ + 1 0 ⋮ 0 𝜀! 𝑦* = 1 0 … 0 𝑦* 𝑦*+% ⋮ 𝑦*+P(% γεςϜϞσϧ ؍ଌϞσϧ
  47. 76 ࣌ܥྻϞσϧ "3ϞσϧΛঢ়ଶۭؒϞσϧͰදݱ 𝑥! = 𝐹! 𝑥!$# + 𝐺! 𝑣!$#

    𝑦! = 𝐻! 𝑥! + 𝑤! γεςϜϞσϧ ؍ଌϞσϧ 𝑦* = ) !/% P 𝑎! 𝑦*+! + 𝜀* 𝜀* ~ N(0, 𝜎") "3Ϟσϧ 𝑦! 𝑦!-" ⋮ 𝑦!-/+" = 𝑎4 𝑎0 1 ⋯ 𝑎5 ⋮ ⋱ ⋮ ⋯ 1 𝑦!-" 𝑦!-Z ⋮ 𝑦!-/ + 1 0 ⋮ 0 𝜀! 𝑦* = 1 0 … 0 𝑦* 𝑦*+% ⋮ 𝑦*+P(% 𝐹! 𝐺! 𝐻! 𝑥!
  48. 77 ࣌ܥྻϞσϧ "3ϞσϧΛঢ়ଶۭؒϞσϧͰදݱ 𝑥! = 𝐹! 𝑥!$# + 𝐺! 𝑣!$#

    𝑦! = 𝐻! 𝑥! + 𝑤! γεςϜϞσϧ ؍ଌϞσϧ 𝑦* = ) !/% P 𝑎! 𝑦*+! + 𝜀* 𝜀* ~ N(0, 𝜎") "3Ϟσϧ 𝑦! 𝑦!-" ⋮ 𝑦!-/+" = 𝑎4 𝑎0 1 ⋯ 𝑎5 ⋮ ⋱ ⋮ ⋯ 1 𝑦!-" 𝑦!-Z ⋮ 𝑦!-/ + 1 0 ⋮ 0 𝜀! 𝑦* = 1 0 … 0 𝑦* 𝑦*+% ⋮ 𝑦*+P(% 𝐹! 𝐺! 𝐻! 𝑥! "3Ϟσϧ͸࣌ෆมͰ؍ଌϊΠζ 𝑤! = 0ͷ ঢ়ଶۭؒϞσϧͰදݱՄೳ
  49. 78 ࣌ܥྻϞσϧ "3."ϞσϧΛঢ়ଶۭؒϞσϧͰදݱ 𝜀!~ N(0, 𝜎Z) "3."Ϟσϧ 𝑦! _ 𝑦!+"|!-"

    ⋮ _ 𝑦!+,-"|!-" = 𝑎" 1 𝑎Z ⋯ ⋮ ⋱ 1 𝑎, ⋯ 𝑦!-" _ 𝑦!|!-Z ⋮ _ 𝑦!+,-Z|!-Z + 1 𝑏" ⋮ 𝑏,-" 𝜀! 𝑦* = 1 0 … 0 𝑦* m 𝑦*(%|*+% ⋮ m 𝑦*(&+%|*+% 𝐹! 𝐺! 𝐻! 𝑥! 𝑦! = b \]" / 𝑎\𝑦!-\ + b \]" ^ 𝑏\𝜀!-\ + 𝜀! _ 𝑦!+X|!-" = b \]X+" / 𝑎\ 𝑦!+X-\ + b \]X ^ 𝑏\ 𝜀!+X-\ 𝑘 = max(𝑝, 𝑞 + 1) 𝑥!-"
  50. 79 ࣌ܥྻϞσϧ ঢ়ଶۭؒϞσϧͷਪఆ 𝑦! 𝑦!"# 𝑦!$# 𝑥!$# 𝑥! 𝑥!"# 𝑣!"#

    𝑣! 𝑣!$# 𝑤!$# 𝑤! 𝑤!"# 𝐹! 𝐺! 𝐻! 𝑦% 𝑥% 𝑣% ʜ σʔλʹج͍ͮͯঢ়ଶ 𝑥* Λਪఆ͍ͨ͠ 𝑝(𝑥! |𝑦& )
  51. 80 ࣌ܥྻϞσϧ ঢ়ଶۭؒϞσϧͷਪఆ 𝑦! 𝑦!"# 𝑦!$# 𝑥!$# 𝑥! 𝑥!"# 𝑣!"#

    𝑣! 𝑣!$# 𝑤!$# 𝑤! 𝑤!"# 𝐹! 𝐺! 𝐻! 𝑦% 𝑥% 𝑣% ʜ σʔλʹج͍ͮͯঢ়ଶ 𝑥* Λਪఆ͍ͨ͠ 𝑝(𝑥! |𝑦& ) 𝑗 < 𝑡ͷ৔߹ ༧ଌ
  52. 81 ࣌ܥྻϞσϧ ঢ়ଶۭؒϞσϧͷਪఆ 𝑦! 𝑦!"# 𝑦!$# 𝑥!$# 𝑥! 𝑥!"# 𝑣!"#

    𝑣! 𝑣!$# 𝑤!$# 𝑤! 𝑤!"# 𝐹! 𝐺! 𝐻! 𝑦% 𝑥% 𝑣% ʜ σʔλʹج͍ͮͯঢ়ଶ 𝑥* Λਪఆ͍ͨ͠ 𝑝(𝑥! |𝑦& ) 𝑗 = 𝑡ͷ৔߹ ϑΟϧλ
  53. 82 ࣌ܥྻϞσϧ ঢ়ଶۭؒϞσϧͷਪఆ 𝑦! 𝑦!"# 𝑦!$# 𝑥!$# 𝑥! 𝑥!"# 𝑣!"#

    𝑣! 𝑣!$# 𝑤!$# 𝑤! 𝑤!"# 𝐹! 𝐺! 𝐻! 𝑦% 𝑥% 𝑣% ʜ σʔλʹج͍ͮͯঢ়ଶ 𝑥* Λਪఆ͍ͨ͠ 𝑝(𝑥! |𝑦& ) 𝑗 > 𝑡ͷ৔߹ ฏ׈Խ
  54. 83 ࣌ܥྻϞσϧ ঢ়ଶۭؒϞσϧͷਪఆ 𝑦! 𝑦!"# 𝑦!$# 𝑥!$# 𝑥! 𝑥!"# 𝑣!"#

    𝑣! 𝑣!$# 𝑤!$# 𝑤! 𝑤!"# 𝐹! 𝐺! 𝐻! 𝑦% 𝑥% 𝑣% ʜ σʔλʹج͍ͮͯঢ়ଶ 𝑥* Λਪఆ͍ͨ͠ 𝑝(𝑥! |𝑦& ) ࣌ܥྻϞσϧͷ໬౓ܭࢉ΍௕ظ༧ଌɺܽଛ஋ͷิ׬ ͳͲͷλεΫ͕͜ͷϑϨʔϜϫʔΫͰ࣮ݱͰ͖Δ
  55. 84 ࣌ܥྻϞσϧ ઢܗɾΨ΢εܕঢ়ଶۭؒϞσϧ 𝑦! 𝑦!"# 𝑦!$# 𝑥!$# 𝑥! 𝑥!"# 𝑣!"#

    𝑣! 𝑣!$# 𝑤!$# 𝑤! 𝑤!"# 𝐹! 𝐺! 𝐻! σʔλʹج͍ͮͯঢ়ଶ 𝑥* Λਪఆ͍ͨ͠ 𝑝 𝑥! 𝑦& ~ N(𝑥!|& , 𝑉!|& )
  56. 85 ࣌ܥྻϞσϧ ઢܗɾΨ΢εܕঢ়ଶۭؒϞσϧ 𝑦! 𝑦!"# 𝑦!$# 𝑥!$# 𝑥! 𝑥!"# 𝑣!"#

    𝑣! 𝑣!$# 𝑤!$# 𝑤! 𝑤!"# 𝐹! 𝐺! 𝐻! σʔλʹج͍ͮͯঢ়ଶ 𝑥* Λਪఆ͍ͨ͠ 𝑝 𝑥! 𝑦& ~ N(𝑥!|& , 𝑉!|& ) ΧϧϚϯϑΟϧλΛར༻
  57. 86 ࣌ܥྻϞσϧ ΧϧϚϯϑΟϧλ 𝑦! 𝑥! 𝑣! 𝑤! 𝐹! 𝐺! 𝑝

    𝑥! 𝑦& ~ N(𝑥!|& , 𝑉!|& ) l ༧ଌ 𝑥!|!$# = 𝐹! 𝑥!$#|!$# 𝑉!|!$# = 𝐹! 𝑉!$#|!$# 𝐹! ( + 𝐺! 𝑄! 𝐺! ( 𝑥!$# 𝐻!
  58. 87 ࣌ܥྻϞσϧ ΧϧϚϯϑΟϧλ 𝑦! 𝑥! 𝑣! 𝑤! 𝐹! 𝐺! 𝑝

    𝑥! 𝑦& ~ N(𝑥!|& , 𝑉!|& ) l ༧ଌ 𝑥!|!$# = 𝐹! 𝑥!$#|!$# 𝑉!|!$# = 𝐹! 𝑉!$#|!$# 𝐹! ( + 𝐺! 𝑄! 𝐺! ( 𝑥!$# 𝐻! ͭલͷঢ়ଶ 𝑡 − 1 ͔Βݱࡏͷঢ়ଶ 𝑡 Λਪఆ 𝑥!|!$# = 𝔼 𝑥! 𝑦!$# ] = 𝔼 𝐹𝑥!$# + 𝐺! 𝑣! 𝑦!$# ] = 𝐹𝔼 𝑥!$# 𝑦!$# ] = 𝐹𝑥!$#|!$#
  59. 88 ࣌ܥྻϞσϧ ΧϧϚϯϑΟϧλ 𝑦! 𝑥! 𝑣! 𝑤! 𝐹! 𝐺! 𝑝

    𝑥! 𝑦& ~ N(𝑥!|& , 𝑉!|& ) l ༧ଌ 𝑥!|!$# = 𝐹! 𝑥!$#|!$# 𝑉!|!$# = 𝐹! 𝑉!$#|!$# 𝐹! ( + 𝐺! 𝑄! 𝐺! ( 𝑥!$# 𝐻! ͭલͷঢ়ଶޡࠩ ͔Βݱࡏͷঢ়ଶޡࠩΛਪఆ 𝑉!|!$# = 𝔼[ 𝑥!$ 𝑥!|!$# ' (𝑥!$ 𝑥!|!$# )] = 𝔼[(𝐹 𝑥!$#$ 𝑥!$#|!$# + 𝐺𝑣! )' (𝐹 𝑥!$#$ 𝑥!$#|!$# + 𝐺𝑣! )] = 𝐹𝔼 𝑥!$#$ 𝑥!$#|!$# ' 𝑥!$#$ 𝑥!$#|!$# 𝐹' + 𝐺𝔼 𝑣! '𝑣! 𝐺
  60. 89 ࣌ܥྻϞσϧ ΧϧϚϯϑΟϧλ 𝑦! 𝑥! 𝑣! 𝑤! 𝐹! 𝐺! 𝑝

    𝑥! 𝑦& ~ N(𝑥!|& , 𝑉!|& ) l ϑΟϧλ 𝑥!$# K! = 𝑉!|!$# 𝐻! ((𝐻! 𝑉!|!$# 𝐻! ( + 𝑅! )$# 𝑉!|! = (𝐼 − 𝐾! 𝐻! )𝑉!|!$# 𝐻! 𝑥!|! = 𝑥!|!$# + 𝐾! (𝑦! − 𝐻! 𝑥!|!$# )
  61. 90 ࣌ܥྻϞσϧ ΧϧϚϯϑΟϧλ 𝑦! 𝑥! 𝑣! 𝑤! 𝐹! 𝐺! 𝑝

    𝑥! 𝑦& ~ N(𝑥!|& , 𝑉!|& ) l ϑΟϧλ 𝑥!$# K! = 𝑉!|!$# 𝐻! ((𝐻! 𝑉!|!$# 𝐻! ( + 𝑅! )$# 𝑉!|! = (𝐼 − 𝐾! 𝐻! )𝑉!|!$# 𝐻! 𝑥!|! = 𝑥!|!$# + 𝐾! (𝑦! − 𝐻! 𝑥!|!$# ) 𝐾* ΧϧϚϯήΠϯͱݺ͹ΕΔ΋ͷΛࢉग़ 𝑤* 𝑛࣍ݩ؍ଌϊΠζ 𝑤* ~ 𝑁 0, 𝑅*
  62. 91 ࣌ܥྻϞσϧ ΧϧϚϯϑΟϧλ 𝑦! 𝑥! 𝑣! 𝑤! 𝐹! 𝐺! 𝑝

    𝑥! 𝑦& ~ N(𝑥!|& , 𝑉!|& ) l ϑΟϧλ 𝑥!$# K! = 𝑉!|!$# 𝐻! ((𝐻! 𝑉!|!$# 𝐻! ( + 𝑅! )$# 𝑉!|! = (𝐼 − 𝐾! 𝐻! )𝑉!|!$# 𝐻! 𝑥!|! = 𝑥!|!$# + 𝐾! (𝑦! − 𝐻! 𝑥!|!$# ) ਪఆޡࠩͱ؍ଌϊΠζ 𝑹𝒕 ͷ࿨ʹ͋ͨΔ෦෼ ΧϧϚϯήΠϯ͸શମͷޡࠩʹ͓͚Δਪఆޡࠩͷׂ ߹ʹ૬౰
  63. 92 ࣌ܥྻϞσϧ ΧϧϚϯϑΟϧλ 𝑦! 𝑥! 𝑣! 𝑤! 𝐹! 𝐺! 𝑝

    𝑥! 𝑦& ~ N(𝑥!|& , 𝑉!|& ) l ϑΟϧλ 𝑥!$# K! = 𝑉!|!$# 𝐻! ((𝐻! 𝑉!|!$# 𝐻! ( + 𝑅! )$# 𝑉!|! = (𝐼 − 𝐾! 𝐻! )𝑉!|!$# 𝐻! 𝑥!|! = 𝑥!|!$# + 𝐾! (𝑦! − 𝐻! 𝑥!|!$# ) ؍ଌ஋ͱࣄલʹਪఆ͞Εͨঢ়ଶͱͷࠩ
  64. 93 ࣌ܥྻϞσϧ ΧϧϚϯϑΟϧλ 𝑦! 𝑥! 𝑣! 𝑤! 𝐹! 𝐺! 𝑝

    𝑥! 𝑦& ~ N(𝑥!|& , 𝑉!|& ) l ϑΟϧλ 𝑥!$# K! = 𝑉!|!$# 𝐻! ((𝐻! 𝑉!|!$# 𝐻! ( + 𝑅! )$# 𝑉!|! = (𝐼 − 𝐾! 𝐻! )𝑉!|!$# 𝐻! 𝑥!|! = 𝑥!|!$# + 𝐾! (𝑦! − 𝐻! 𝑥!|!$# ) ΧϧϚϯήΠϯ͸ॏΈͷ໾ׂͰ؍ଌޡࠩͱਪఆޡࠩ ͷόϥϯεΛऔΓঢ়ଶ 𝑥* Λߋ৽
  65. 94 ࣌ܥྻϞσϧ ΧϧϚϯϑΟϧλ 𝑦! 𝑥! 𝑣! 𝑤! 𝐹! 𝐺! 𝑝

    𝑥! 𝑦& ~ N(𝑥!|& , 𝑉!|& ) l ϑΟϧλ 𝑥!$# K! = 𝑉!|!$# 𝐻! ((𝐻! 𝑉!|!$# 𝐻! ( + 𝑅! )$# 𝑉!|! = (𝐼 − 𝐾! 𝐻! )𝑉!|!$# 𝐻! 𝑥!|! = 𝑥!|!$# + 𝐾! (𝑦! − 𝐻! 𝑥!|!$# ) ؍ଌޡ͕ࠩେ͖͍ͱڞ෼ࢄ͕େ͖͘ͳΔΑ͏ʹߋ৽
  66. 96 ࣌ܥྻϞσϧ ઢܗɾΨ΢εܕঢ়ଶۭؒϞσϧ 𝑦! 𝑦!"# 𝑦!$# 𝑥!$# 𝑥! 𝑥!"# 𝑣!"#

    𝑣! 𝑣!$# 𝑤!$# 𝑤! 𝑤!"# 𝐹! 𝐺! 𝐻! σʔλʹج͍ͮͯঢ়ଶ 𝑥* Λਪఆ͍ͨ͠ 𝑝 𝑥! 𝑦& ~ N(𝑥!|& , 𝑉!|& ) ΧϧϚϯϑΟϧλΛར༻
  67. 97 ࣌ܥྻϞσϧ ௕ظ༧ଌΛ͍ͨ͠৔߹ 𝑦! 𝑥! 𝑣! 𝑤! 𝐹! 𝐺! 𝑝

    𝑥! 𝑦& ~ N(𝑥!|& , 𝑉!|& ) l ௕ظ༧ଌ 𝑥!")|!$# = 𝐹! 𝑥!$#")|!$# 𝑉!")|!$# = 𝐹! 𝑉!")|!")$# 𝐹! ( + 𝐺! 𝑄! 𝐺! ( 𝑥!$# 𝐻! ϑΟϧλͷॲཧΛলུ͢Δ͜ͱͰ࣮ݱ
  68. 99 ࣌ܥྻϞσϧ ঢ়ଶۭؒϞσϧͷਪఆ 𝑦! 𝑦!"# 𝑦!$# 𝑥!$# 𝑥! 𝑥!"# 𝑣!"#

    𝑣! 𝑣!$# 𝑤!$# 𝑤! 𝑤!"# 𝐹! 𝐺! 𝐻! 𝑦% 𝑥% 𝑣% ʜ σʔλʹج͍ͮͯঢ়ଶ 𝑥* Λਪఆ͍ͨ͠ 𝑝(𝑥! |𝑦& ) 𝑗 > 𝑡ͷ৔߹ ฏ׈Խ
  69. 100 ࣌ܥྻϞσϧ ΧϧϚϯϑΟϧλ ฏ׈Խ 𝑦! 𝑥! 𝑣! 𝑤! 𝐹! 𝐺!

    𝑝 𝑥! 𝑦& ~ N(𝑥!|& , 𝑉!|& ) 𝑥!$# 𝐻! l ฏ׈Խ 𝑥!|% = 𝑥!|! + 𝐴! (𝑥!"#|( − 𝑥!"#|! ) 𝑉!|( = 𝑉!|! + 𝐴! (𝑉!"#|( − 𝑉!"#|! )𝐴! ( 𝐴! = 𝑉!|! 𝐹!"# ( 𝑉!"#|! $#
  70. 101 ࣌ܥྻϞσϧ ΧϧϚϯϑΟϧλ ฏ׈Խ 𝑦! 𝑥! 𝑣! 𝑤! 𝐹! 𝐺!

    𝑝 𝑥! 𝑦& ~ N(𝑥!|& , 𝑉!|& ) 𝑥!$# 𝐻! l ฏ׈Խ 3BVDI5VOH4USJFCFM TNPPUIFS 𝑥!|% = 𝑥!|! + 𝐴! (𝑥!"#|( − 𝑥!"#|! ) 𝑉!|( = 𝑉!|! + 𝐴! (𝑉!"#|( − 𝑉!"#|! )𝐴! ( 𝐴! = 𝑉!|! 𝐹!"# ( 𝑉!"#|! $# 𝐴* ฏۉԽརಘ
  71. 102 ࣌ܥྻϞσϧ ΧϧϚϯϑΟϧλ ฏ׈Խ 𝑦! 𝑥! 𝑣! 𝑤! 𝐹! 𝐺!

    𝑝 𝑥! 𝑦& ~ N(𝑥!|& , 𝑉!|& ) 𝑥!$# 𝐻! l ฏ׈Խ 3BVDI5VOH4USJFCFM TNPPUIFS 𝑥!|% = 𝑥!|! + 𝐴! (𝑥!"#|( − 𝑥!"#|! ) 𝑉!|( = 𝑉!|! + 𝐴! (𝑉!"#|( − 𝑉!"#|! )𝐴! ( 𝐴! = 𝑉!|! 𝐹!"# ( 𝑉!"#|! $# ฏۉԽརಘ͕ঢ়ଶߋ৽ʹ͓͚ΔॏΈͷ໾ׂ
  72. 105 ࣌ܥྻϞσϧ ඇઢܗɾඇΨ΢εܕঢ়ଶۭؒϞσϧ 𝑦! 𝑦!"# 𝑦!$# 𝑥!$# 𝑥! 𝑥!"# 𝑣!"#

    𝑣! 𝑣!$# 𝑤!$# 𝑤! 𝑤!"# 𝐹! 𝐺! 𝐻! 𝑥! = 𝐹! 𝑥!$# + 𝐺! 𝑣! 𝑦! = 𝐻! 𝑥! + 𝑤! ઢܗɾΨ΢εܕ ඇઢܗɾඇΨ΢εܕ 𝑥! = 𝑓(𝑥!$# , 𝑣! ) 𝑦! = ℎ(𝑥! ) + 𝑤! 𝑣! ~𝑞 𝑣 , 𝑤! ~𝑟(𝑤)
  73. 106 ࣌ܥྻϞσϧ ΧϧϚϯϑΟϧλ l ༧ଌ l ϑΟϧλ l ฏ׈Խ 𝑝

    𝑥! 𝑌!$# ) = ; $* * 𝑝 𝑥! 𝑥!$# )𝑝 𝑥!$# 𝑌!$# )𝑑𝑥!$# 𝑥! = 𝑓(𝑥!$# , 𝑣! ) 𝑦! = ℎ(𝑥! ) + 𝑤! 𝑝 𝑥! 𝑌! ) = 𝑝 𝑦! 𝑥! )𝑝 𝑥! 𝑌!$# ) 𝑝 𝑦! 𝑌!–# ) 𝑝 𝑥! 𝑌( ) = 𝑝 𝑥! 𝑌! ) ∫ $* * , -!"# -!), -!"# /$) , -!"# /!) 𝑑𝑥!"#
  74. 116 Ϟσϧͷਪఆɾબ୒ l ߏங͞Εͨؔ਺Ϟσϧ𝑓(ɾ)ͱ؍ଌ஋ͱͷؒͷೋ৐ޡࠩ ࢒ࠩ𝑒! ͷೋ ৐ ͕࠷খͱͳΔΑ͏ͳύϥϝʔλΛਪఆ l ֬཰ͷ֓೦͕ݱΕͳ͍

    l ճؼ෼ੳͳͲʹ͓͍ͯɺղੳతʹύϥϝʔλಋग़͕Մೳ Ϟσϧͷਪఆ࠷খೋ৐๏ 0-4 𝐸 = b \]" _ (𝑦\ − 𝑓(𝑥\))Z
  75. 117 Ϟσϧͷਪఆɾબ୒ l ྫճؼϞσϧ "3  l ͭͷ܎਺Λ࣋ͭճؼϞσϧ l ೋ৐ޡࠩ

    Ϟσϧͷਪఆ࠷খೋ৐๏ 0-4 𝐰𝐓 = (𝑤0 , 𝑤% ), 𝐱𝐢 b = (1, 𝑥)ͱఆٛ͢Δͱ 𝑓 𝑥! = v 𝑦! = 𝑤0 + 𝑤% 𝑥! = 𝐰𝐓𝐱 v 𝑦% v 𝑦c = 1, 𝑥% 1, 𝑥d 𝑤0 𝑤% v 𝐲 = 𝐗𝐰 ܭըߦྻ E = (𝐲 − 𝐗𝐰)b(𝐲 − 𝐗𝐰)
  76. 118 Ϟσϧͷਪఆɾબ୒ l ྫճؼϞσϧ "3  l ೋ৐ޡࠩEΛ𝐰Ͱඍ෼ Ϟσϧͷਪఆ࠷খೋ৐๏ 0-4

    E = 𝐲 − 𝐗𝐰 b 𝐲 − 𝐗𝐰 = 𝐲𝐓𝐲 − 𝟐𝐰𝐓 𝐗𝐓𝐲 + 𝐰𝐓𝐗𝐓𝐗𝐰 𝜕𝐸 𝜕𝐰 = −𝟐𝐗b𝐲 + 𝐗b𝐗 + 𝐗b𝐗 b 𝐰 = −2𝐗b𝐲 + 𝟐𝐗b𝐗w = 0 w = (𝐗b𝐗)+%𝐗b𝐲
  77. 119 Ϟσϧͷਪఆɾબ୒ Ϟσϧͷਪఆ࠷໬ਪఆ l ,VMMCBDL-FJCMFS ,- ৘ใྔ l ͭͷ֬཰෼෍ͷࠩҟΛଌఆ͢Δई౓ l

    ਪఆ͞ΕΔ༧ଌ෼෍͕ਅͷ෼෍Λଊ͍͑ͯΔ͔ΛධՁ l g(x)͕ਅͷ෼෍ f(x)͕Ϟσϧͷ෼෍ͱͨ࣌͠ɺ,-৘ใྔ͸ҎԼͷܗ l D`a (g| 𝑓 = 0ͷ࣌ɺ g x = f(x) l ࡾ֯ෆ౳ࣜ΍ରশੑͳͲͷڑ཭ͷެཧ͸ຬͨ͞ͳ͍ Def (g||𝑓) = ~ +T T g x log g x f x 𝑑𝑥 ∗ Def (g||𝑓) ≥ 0
  78. 120 Ϟσϧͷਪఆɾબ୒ Ϟσϧͷਪఆ࠷໬ਪఆ l ਖ਼ن෼෍ಉ࢜ͷ,-ڑ཭ D`a(g||𝑓) = m -b b

    g x log g x 𝑑𝑥 − m -b b g x log 𝑓 x 𝑑𝑥 g x ~ N 𝜇c , 𝜎c Z , f x ~ N 𝜇d , 𝜎d Z m -b b g x log 𝑓 x 𝑑𝑥 = − 1 2 log2π𝜎d Z − 𝜎c Z + 𝜇c − 𝜇d Z 2𝜎d Z m -b b g x log g x 𝑑𝑥 = − 1 2 log2π𝜎c Z − 1 2 正規分布の確率密度関数
  79. 121 Ϟσϧͷਪఆɾબ୒ Ϟσϧͷਪఆ࠷໬ਪఆ Def (g||𝑓) = ~ +T T g

    x log g x f x 𝑑𝑥 ∗ Def (g||𝑓) ≥ 0 from: https://qiita.com/ceptree/items/9a473b5163d5655420e8
  80. 123 Ϟσϧͷਪఆɾબ୒ Ϟσϧͷਪఆ࠷໬ਪఆ l ϞσϦϯάʹ͓͚ΔKL৘ใྔͷਪఆ l g(x)͕ਅͷ෼෍ f(x)͕Ϟσϧͷ෼෍ Def (g||f)

    = ~ +T T g x log g x f x 𝑑𝑥 = ~ +T T g x log g x 𝑑𝑥 − ~ +T T g x log f x 𝑑𝑥 Θ͔Βͳ͍
  81. 124 Ϟσϧͷਪఆɾબ୒ Ϟσϧͷਪఆ࠷໬ਪఆ l ϞσϦϯάʹ͓͚ΔKL৘ใྔͷਪఆ l g(x)͕ਅͷ෼෍ f(x)͕Ϟσϧͷ෼෍ Def (g||f)

    = ~ +T T g x log g x f x 𝑑𝑥 = ~ +T T g x log g x 𝑑𝑥 − ~ +T T g x log f x 𝑑𝑥 Θ͔Βͳ͍ ฏۉର਺໬౓ g(x)͕Θ͔Βͳ͍ͨΊະ஌ ͕ͩɺͲͷ f(x) Ͱ΋Ұఆ
  82. 125 Ϟσϧͷਪఆɾબ୒ Ϟσϧͷਪఆ࠷໬ਪఆ l ϞσϦϯάʹ͓͚ΔKL৘ใྔͷਪఆ l g(x)͕ਅͷ෼෍ f(x)͕Ϟσϧͷ෼෍ Def (g||f)

    = ~ +T T g x log g x f x 𝑑𝑥 = ~ +T T g x log g x 𝑑𝑥 − ~ +T T g x log f x 𝑑𝑥 Θ͔Βͳ͍ ฏۉର਺໬౓ g(x)͕Θ͔Βͳ͍ͨΊະ஌ ͕ͩɺͲͷ f(x) Ͱ΋Ұఆ ૬ରతʹɺϞσϧ෼෍ 𝐟(𝐱)ͷ ༗ޮੑʹ͍ͭͯධՁͰ͖Δʁ
  83. 126 Ϟσϧͷਪఆɾબ୒ Ϟσϧͷਪఆ࠷໬ਪఆ ~ +T T g x log f

    x 𝑑𝑥 l ฏۉର਺໬౓ l ର਺໬౓ ະ஌ 1 𝑛 b \]" W 𝐼(𝑥\) େ਺ͷ๏ଇʹΑΓɺ༩͑ΒΕͨ αϯϓϧσʔλͰۙࣅ ਅͷ໬౓ αϯϓϧσʔλ ͷ໬౓ 𝜃+ 𝜃678 ฏۉର਺໬౓ ର਺໬౓ ℓ = ) !/% ' log f x
  84. 127 Ϟσϧͷਪఆɾબ୒ Ϟσϧͷਪఆ࠷໬ਪఆ l ࠷໬ਪఆ .-& l ର਺໬౓Λ༻͍ͨύϥϝʔλਪఆ๏ l ର਺໬౓Λ࠷େԽ͢Δύϥϝʔλ

    ࠷໬ਪఆ஋ 𝜃fgh ͕ۙࣅతʹ,-৘ใྔ Λ࠷΋࠷খԽ͢ΔύϥϝʔλʹͳΔͱਪఆͰ͖Δ l ର਺໬౓Λ࠷େԽ͢Δύϥϝʔλ ࠷໬ਪఆ஋ 𝜃fgh ͸ؔ਺ℓ(𝜃)͕𝜃Ͱඍ෼ ͯ͠ ij(k) ik = 0ͱͳΔ΋ͷΛਪఆ ਅͷ໬౓ αϯϓϧσʔλ ͷ໬౓ 𝜃+ 𝜃678 ฏۉର਺໬౓ ର਺໬౓ ℓ(𝜃) = ) !/% ' log f x|θ 𝜃1gh = max i ℓ(𝜃)
  85. 128 Ϟσϧͷਪఆɾબ୒ ࠷খೋ৐๏ͱ࠷໬ਪఆ l ྫɿճؼϞσϧͷ࠷໬ਪఆɻޡࠩΛਖ਼ن෼෍ͱԾఆ v 𝑦! = 𝑓 𝑥!

    = 𝑁(𝑤0 + 𝑤% 𝑥! , 𝜎") ℓ(𝜃) = ) !/% ' log 1 2𝜋𝜎" exp(− 𝑦! − (𝑤0 + 𝑤% 𝑥! ) " 2𝜎" ) = − 𝑛 2 log 2𝜋 − 𝑛 2 log 𝜎" − ) !/% ' ( 𝑦! − (𝑤0 + 𝑤% 𝑥! ) " 2𝜎" )
  86. 129 Ϟσϧͷਪఆɾબ୒ l ߏங͞Εͨؔ਺Ϟσϧ𝑓(ɾ)ͱ؍ଌ஋ͱͷؒͷೋ৐ޡࠩ ࢒ࠩ𝑒! ͷೋ ৐ ͕࠷খͱͳΔΑ͏ͳύϥϝʔλΛਪఆ l ֬཰ͷ֓೦͕ݱΕͳ͍

    l ճؼ෼ੳͳͲʹ͓͍ͯɺղੳతʹύϥϝʔλಋग़͕Մೳ ࠷খೋ৐๏ͱ࠷໬ਪఆ 𝐸 = b \]" _ (𝑦\ − 𝑓(𝑥\))Z
  87. 130 Ϟσϧͷਪఆɾબ୒ ࠷খೋ৐๏ͱ࠷໬ਪఆ l ྫɿճؼϞσϧͷ࠷໬ਪఆɻޡࠩΛਖ਼ن෼෍ͱԾఆ v 𝑦! = 𝑓 𝑥!

    = 𝑁(𝑤0 + 𝑤% 𝑥! , 𝜎") ℓ(𝜃) = ) !/% ' log 1 2𝜋𝜎" exp(− 𝑦! − (𝑤0 + 𝑤% 𝑥! ) " 2𝜎" ) = − 𝑛 2 log 2𝜋 − 𝑛 2 log 𝜎" − ) !/% ' ( 𝑦! − (𝑤0 + 𝑤% 𝑥! ) " 2𝜎" ) ର਺໬౓ͷ࠷େԽ͸࠷খೋ৐๏ͱҰக 㱺 ࠷খೋ৐๏͸࠷໬ਪఆͷಛघྫͱҰக
  88. 132 Ϟσϧͷਪఆɾબ୒ Ϟσϧͷબఆ 𝜃+ "3  "3*."   

    ."  ͲͷϞσϧΛ࢖͏ͱ ྑ͍ʁʁʁ ର਺໬౓͕࠷΋େ͖͍ Ϟσϧ͕ྑ͍ͷͰ͸ʁ
  89. 135 Ϟσϧͷਪఆɾબ୒ Ϟσϧͷબఆ l ࠷΋ྑ͍ର਺໬౓Λ࣋ͭϞσϧΛબ୒ͨ͠৔߹ l աద߹ PWFSGJU ͷ໰୊ l

    ༩͑ΒΕͨσʔλ͸े෼આ໌Ͱ͖ͨͱͯ͠΋ɺ৽͍͠σʔλΛे෼ʹઆ໌Ͱ ͖Δͱ͸ݶΒͳ͍ from: https://datascience.foundation/sciencewhitepaper/u nderfitting-and-overfitting-in-machine-learning
  90. 136 Ϟσϧͷਪఆɾબ୒ Ϟσϧͷબఆ l ࠷΋ྑ͍ର਺໬౓Λ࣋ͭϞσϧΛબ୒ͨ͠৔߹ l աద߹ PWFSGJU ͷ໰୊ l

    ༩͑ΒΕͨσʔλ͸े෼આ໌Ͱ͖ͨͱͯ͠΋ɺ৽͍͠σʔλΛे෼ʹઆ໌Ͱ ͖Δͱ͸ݶΒͳ͍ from: https://datascience.foundation/sciencewhitepaper/u nderfitting-and-overfitting-in-machine-learning ෳࡶ͗͢ͳ͍࠷దͳϞσϧΛ બ୒͢Δ͜ͱ͕ॏཁ
  91. 137 Ϟσϧͷਪఆɾબ୒ Ϟσϧͷબఆ l ద੾ͳϞσϧબ୒ͷͨΊʹ l ର਺໬౓ʹΑͬͯਪఆ͞ΕΔ𝜃fgh ͱ ਅͷύϥϝʔλ 𝜃.

    ͷؒʹόΠΞε ͕ଘࡏ l ର਺໬౓ͱฏۉର਺໬౓ͷόΠΞε ΛධՁͯ͠ద੾ʹิਖ਼͢Δ͜ͱͰ ࠷దͳϞσϧΛબ୒͢Δඞཁ
  92. 138 Ϟσϧͷਪఆɾબ୒ Ϟσϧͷબఆ l ద੾ͳϞσϧબ୒ͷͨΊʹ l 𝜃fgh ʹ͓͚Δର਺໬౓ͱฏۉର਺ ໬౓ͷࠩ 𝐷Λิਖ਼͍ͨ͠

    1 𝑛 ? '"* , log f x|𝜃-./ ͜ͷ෦෼Λ ิਖ਼͍ͨ͠ʂ ظ଴஋ΛͱΔ 𝔼0[𝐷] = t #$ $ g x [D]𝑑𝑥 ฏۉର਺໬౓ ର਺໬౓ 𝐷 = 1 𝑛 ? '"* , log f x|𝜃-./ − t #$ $ g x log f x|𝜃-./ 𝑑𝑥
  93. 139 Ϟσϧͷਪఆɾબ୒ Ϟσϧͷબఆ l ద੾ͳϞσϧબ୒ͷͨΊʹ l 𝜃fgh ʹ͓͚Δର਺໬౓ͱฏۉର਺ ໬౓ͷࠩ 𝐷Λิਖ਼͍ͨ͠

    1 𝑛 ? '"* , log f x|𝜃-./ ͜ͷ෦෼Λ ิਖ਼͍ͨ͠ʂ ฏۉର਺໬౓ ର਺໬౓ ظ଴஋ΛͱΔ 𝔼0[𝐷] = t #$ $ g x [D]𝑑𝑥 ͜ͷิਖ਼෦෼͕ύϥϝʔλ਺Ͱ͋Δ 𝑝ʹۙࣅͰ͖Δ ิਖ਼ 𝐷 = 1 𝑛 ? '"* , log f x|𝜃-./ − t #$ $ g x log f x|𝜃-./ 𝑑𝑥
  94. 140 Ϟσϧͷਪఆɾબ୒ Ϟσϧͷબఆ"*$ l ద੾ͳϞσϧબ୒ͷͨΊʹ l ࠷େର਺໬౓ όΠΞε 𝜃fgh ʹ͓͚Δฏۉର਺໬౓

    1 𝑛 ? '"* , log f x|𝜃-./ ∑!/% ' log f x|𝜃1gh − 𝔼x [𝐷] = 𝑛 ∫ +T T g x log f x|𝜃1gh 𝑑𝑥 ∑!/% ' log f x|𝜃1gh − 𝑝= n ∫ +T T g x log f x|𝜃1gh 𝑑𝑥 𝔼0[𝐷]
  95. 141 Ϟσϧͷਪఆɾબ୒ Ϟσϧͷબఆ"*$ l ੺஑৘ใྔج४ "*$ l ฏۉର਺໬౓ͷ 𝜃fgh ʹ͓͚ΔਪఆྔͰ͋ΔҎԼͷࣜͷࠨลΛº͢Δ͜ͱͰ

    ౷ܭϞσϧͷࢦඪͱͳΔ l 𝑝͸ࣗ༝ύϥϝʔλ਺ l ͜ͷࢦඪ͕খ͍͞౷ܭϞσϧ͕ྑ͍Ϟσϧͱਪఆ͞ΕΔ l ৘ใྔج४ͷछ ∑!/% ' log f x|𝜃1gh − 𝑝= n ∫ +T T g x log f x|𝜃1gh 𝑑𝑥 AIC = −2 ) !/% ' log f x|𝜃1gh + 2𝑝
  96. 147 Ϟσϧͷਪఆɾબ୒ Ϟσϧͷબఆ"*$ l Ϟσϧͷઆ໌ม਺͕গͳ͍ͱɺ#JBT͕େ͖͘ͳΔ ֶशෆ଍ l Ϟσϧͷઆ໌ม਺͕ଟ͍ͱɺ#JBT͸খ͘͞ͳΔ͕ɺ7BSJBODF͕େ ͖͘ͳΓɺաֶशͱͳΔ l

    "*$ͳͲͷ৘ใྔج४͸ɺόΠΞεͱόϦΞϯεͷͭΛόϥϯε Α͘࠷খʹ͢Δʢظ଴༧ଌޡࠩͷ࠷খʣͷͨΊͷࢦඪ ༧ଌޡࠩ #JBT 7BSJBODF
  97. 149 Ϟσϧͷਪఆɾબ୒ Ϟσϧͷબ୒ଞͷબ୒ํ๏ l ৘ใྔج४ *$ l Ϟσϧͷ౰ͯ͸·Γͷྑ͞ͱɺϞσϧͷෳࡶ͞ʹର͢ΔϖφϧςΟ͔Βߏ ੒͞ΕΔࢦඪͷҰൠܥ l

    ର਺໬౓ʹج͍ͮͨؔ਺ͱϖφϧςΟؔ਺𝐼(𝑛)ʹΑͬͯߏ੒ l ϖφϧςΟؔ਺ΛͲͷΑ͏ʹઃܭ͢Δ͔Ͱෳ਺ͷࢦඪ͕ଘࡏ l ৘ใྔج४͕খ͍͞Ϟσϧ͕ྑ͍Ϟσϧͷࢦඪ l "*$͸𝐼 𝑛 = 2 ͷϞσϧ IC = −2 ) !/% ' log f x|𝜃1gh + 𝐼(𝑛)𝑝
  98. 150 Ϟσϧͷਪఆɾબ୒ Ϟσϧͷબ୒ଞͷબ୒ํ๏ l ҳ୤౓৘ใྔج४ %*$ l ৘ใྔج४ͷछ l ର਺໬౓ͷࣄޙฏۉΛਪఆྔͱͯ͠ར༻͢ΔϕΠζϕʔεͷࢦඪ

    l ϖφϧςΟؔ਺͸ࣄલ෼෍ʹΑΔର਺໬౓ͱࣄޙฏۉͷର਺໬౓ͷࠩ 𝐷IC = −2 ~ log f x|𝜃y1 𝑑𝜃y1 + 𝐼 ・ 2 𝐼 ・ =2 ∫ log f x|𝜃y1 𝑑𝜃y1 − 2 ∫ log f x|𝜃 ∏(𝜃|𝑥) 𝑑𝜃 𝜃y1 = ∫ 𝜃 ∏(𝜃|𝑥) 𝑑𝜃
  99. 151 Ϟσϧͷਪఆɾબ୒ Ϟσϧͷબ୒ଞͷબ୒ํ๏ l ϕΠζ৘ใྔج४ #*$ l ৘ใྔج४ͷछ l ,-৘ใྔʹجͮ͘ͷͰ͸ͳ͘पล໬౓ʹج͍ͮͨ৘ใྔ

    l 𝑛 ≥ 8ͷ࣌ʹ"*$ΑΓ΋ϖφϧςΟ͕ڧ͘ͳΔɻͭ·Γɺ"*$ΑΓ΋ύϥ ϝʔλ਺͕খ͍͞ϞσϧΛબ୒͢Δ܏޲ 𝐵IC = −2 ) !/% ' log f x|𝜃1gh + 𝑝 log 𝑛
  100. 160 ਂ૚ֶशͷൃల ϑΟʔυϑΥϫʔυχϡʔϥϧωοτϫʔΫ Input Layer Hidden Layers Output Layer X

    = A[0] a[4] A[1] A[3] X Ŷ a[1] n a[2] 1 a[2] 2 a[2] 3 a[2] n a[3] 1 a[3] 2 a[3] 3 a[3] n A[2] A[4] 𝑓% ؔ਺ 𝑓% ͷྫ 𝐴[%] = 𝑓% 𝑥 = 𝑊𝑥 + 𝑏 ઢܗϞσϧ
  101. 161 ਂ૚ֶशͷൃల ϑΟʔυϑΥϫʔυχϡʔϥϧωοτϫʔΫ Input Layer Hidden Layers Output Layer X

    = A[0] a[4] A[1] A[3] X Ŷ a[1] n a[2] 1 a[2] 2 a[2] 3 a[2] n a[3] 1 a[3] 2 a[3] 3 a[3] n A[2] A[4] 𝑓% ؔ਺ 𝑓% ͷྫ 𝐴[%] = 𝑓% 𝑥 = 𝑊𝑥 + 𝑏 ΞϑΟϯม׵ 0.6 0.2 0.8 −0.3 −0.3 0.4 0.1 0.7 ॏΈΛσʔλʹ ߹͏Α͏ʹֶश
  102. 162 ਂ૚ֶशͷൃల ϑΟʔυϑΥϫʔυχϡʔϥϧωοτϫʔΫ Input Layer Hidden Layers Output Layer X

    = A[0] a[4] A[1] A[3] X Ŷ a[1] n a[2] 1 a[2] 2 a[2] 3 a[2] n a[3] 1 a[3] 2 a[3] 3 a[3] n A[2] A[4] 𝑓% 𝐴[%] = 𝑚𝑎𝑥{𝑥, 0} ׆ੑԽؔ਺ ɾ3F-6 𝐴[%] = 1/(1 + 𝑒+#) ɾ4JHNPJE
  103. 163 ਂ૚ֶशͷൃల ਂ૚Ϟσϧͷֶश ⼊⼒ 出⼒Ŷ 𝑓% 𝑓" 𝑓V l ଛࣦؔ਺

    L 𝑊 = 4 !/% ' ℓ(𝑌! , 𝐹(𝑥! )) ೋ৐ଛࣦ NFBOTRVBSFEFSSPS ℓ 𝑦, 𝑦€ = (𝑦 − 𝑦€)" ަࠩΤϯτϩϐʔޡࠩ $SPTT&OUSPQZMPTT ℓ 𝑦, 𝑦€ = − 4 &/% • 𝑦& log 𝑦& € ͜ΕΛ࠷খԽ͢ΔΑ͏ʹֶश min l L 𝑊
  104. 164 ਂ૚ֶशͷൃల ਂ૚Ϟσϧͷֶश l ޡࠩٯ఻೻๏ l ύϥϝʔλߋ৽ͷख๏ l ଛࣦؔ਺ͷඍ෼ΛޙΖ ೖྗํ޲

    ʹͲΜͲΜ࿈࠯཯Λ༻͍ͯ఻ൖͤ͞Δ 𝜕𝐿 𝜕𝑤% 𝑥 = 𝜕𝐿 𝜕𝑓V 𝜕𝑓V 𝜕𝑓" 𝜕𝑓" 𝜕𝑓% 𝜕𝑓% 𝜕𝑤% 𝑥 ⼊⼒ 𝑓% 𝑓" 𝑓V 出⼒Ŷ L 𝑊
  105. 165 ਂ૚ֶशͷൃల ਂ૚Ϟσϧͷֶश l ޯ഑߱Լ๏ l ଛࣦؔ਺͕࠷খͱͳΔΑ͏ʹ఻ൖ͞ΕͨޡࠩʹԠͯ͡ɺύϥϝʔλΛগ͠ ͣͭߋ৽͍ͯ͘͠ख๏ l ֤ύϥϝʔλͷଛࣦؔ਺ʹର͢Δภඍ෼

    Λߦ͍ύϥϝʔλΛߋ৽ 𝑊* = 𝑊*+% − 𝜂 A 𝜕𝐿 𝜕𝑊 ‚/‚"'& From: https://axa.biopapyrus.jp/deep-learning/gradient_descent_method.htm
  106. 168 ਂ૚ֶशͷൃల ਂ૚ֶशͷಛ௃ l ଟ૚ԽͷԸܙ l χϡʔϥϧωοτϫʔΫͷදݱྗ͸૚ ͷ਺ʹΑͬͯදݱྗ͕ࢦ਺తʹ޲্ l ਂ͍χϡʔϥϧωοτϫʔΫͷදݱೳ

    ྗΛઙ͍χϡʔϥϧωοτϫʔΫͰද ݱ͠Α͏ͱ͢Δͱେ͖ͳԣ෯ χϡʔ ϩϯ ͕ඞཁ Montufar, Guido F., et al. "On the number of linear regions of deep neural networks." Advances in neural information processing systems 27 (2014). Arora, R., Basu, A., Mianjy, P., & Mukherjee, A. (2016). Understanding deep neural networks with rectified linear units. arXiv preprint arXiv:1611.01491.
  107. 169 ਂ૚ֶशͷൃల ਂ૚ֶशͷಛ௃ l εέʔϦϯάଇ l 5SBOTGPSNFSͷੑೳ͸ύϥϝʔλ ਺ɺσʔληοταΠζɺܭࢉ༧ ࢉΛม਺ͱͨ͠γϯϓϧͳ΂͖৐ ଇʹै͏

    l ͜ͷ๏ଇ͕ݴޠɾը૾ɾಈըͳͲ ͷ༷ʑͳλεΫʹద༻͞ΕΔ͜ͱ Λࣔ͢ from: Henighan, Tom, Jared Kaplan, Mor Katz, Mark Chen, Christopher Hesse, Jacob Jackson, Heewoo Jun et al. "Scaling laws for autoregressive generative modeling." arXiv preprint arXiv:2010.14701 (2020).
  108. 170 ਂ૚ֶशͷൃల ਂ૚ֶशͷಛ௃ l 0WFSQBSBNFUFSJ[BUJPO l ଟ͘ͷେن໛ϞσϧͰͷֶशͰ͸σʔλ਺ ύϥϝʔλ਺ l ύϥϝʔλ਺Λ૿΍͠΋ɺ࣮ࡍͷֶशʹ͓͍ͯաֶश͸ੜ͡ͳ͍

    l χϡʔϥϧωοτϫʔΫͰ͸൚Խޡࠩͱ܇࿅ޡࠩͷ͕ࠩখ͘͞ͳ ΔͨΊʁʢཧ࿦త෼ੳ͕ߦΘΕ͍ͯΔʣ Neyshabur, Behnam, et al. "The role of over- parametrization in generalization of neural networks." 7th International Conference on Learning Representations, ICLR 2019. 2019.
  109. 172 ࣌ܥྻσʔλͱਂ૚ֶश 'FFE'PSXBSE//ͱ࣌ܥྻ l ௨ৗͷ'FFE'PSXBE //ʹ࣌ܥྻσʔλΛ ద༻͢Δ͜ͱ͸Մೳ͕ͩɺ࣌ؒͷྲྀΕΛଊ ͑Δ͜ͱ͕Ͱ͖ͳ͍ l ྫ͑͹ɺ༧ଌͷλεΫͰ͸ɺੲͷ෦෼͸ܰ

    ͘ѻͬͯɺ࠷ۙͷ෦෼͸ॏཁͳΑ͏ʹѻ͍ ͍ͨ l ࣌఺𝑡ͷֶशʹ͓͍ͯকདྷͷ஋𝑡 + 1Λ࢖Θ ͣɺաڈͷ஋͚ͩΛ༻ֶ͍ͯश͍ͨ͠ l ೖྗ௕͕ݻఆͰͳ͘ՄมͰ͋Δ͜ͱͷ΄͏ ͕ɺ࣌ܥྻΛೖྗͷࡍʹخ͍͠
  110. 173 ࣌ܥྻσʔλͱਂ૚ֶश 3FDVSSFOU/FVSBM/FUXPSL 3// ・・・ 𝑥% 𝑥" 𝑥V 𝑥W Feature

    Vecter l ӅΕঢ়ଶΛ࣋ͬͨχϡʔϥϧωοτϫʔΫ l ࣌ؒ৘ใͷ֓೦Λ࣋ͪɺ࣍ͷ࣌ؒʹӅΕঢ়ଶͷ৘ใΛ͓͘Δ RNN Block RNN Block RNN Block RNN Block
  111. 174 ࣌ܥྻσʔλͱਂ૚ֶश 3FDVSSFOU/FVSBM/FUXPSL 3// 𝑋 Feature Vecter RNN Block from:

    https://stanford.edu/~shervine/teaching/cs-230/cheatsheet- recurrent-neural-networks 𝑎o!p = 𝑔"(𝑊 qq𝑎o!-"p + 𝑊 qr𝑥o!p + 𝑏q) 𝑦o!p = 𝑔Z(𝑊 sq𝑎o!p + 𝑏s)
  112. 175 ࣌ܥྻσʔλͱਂ૚ֶश 3FDVSSFOU/FVSBM/FUXPSL 3// l ڧΈ l Մม௕ʹରԠ l ϞσϧαΠζ͕ೖྗ௕ʹΑͬͯมԽ͠ͳ͍

    l աڈͷ৘ใΛߟྀͨ͠ܭࢉ͕Մೳ l ϞσϧͷॏΈ͕͕࣌ؒมԽͯ͠΋ڞ༗ l ऑΈ l ܭࢉͷ஗͞ l ݱࡏͷঢ়ଶʹରͯ͠কདྷͷೖྗΛߟྀͰ͖ͳ͍ l աڈͷ৘ใ͕࢒͍ͬͯͳ͍Մೳੑ
  113. 176 ࣌ܥྻσʔλͱਂ૚ֶश ଞͷ3//ܥ౷ͷϞσϧ l -POH4IPSU5FSN.FNPSZ -45. l ࣌ؒมԽ͢Δ௕ظهԱͷͨΊͷ ϝϞϦ𝑐o!pΛ࣋ͭ l

    ͭͷήʔτΛ࣋ͭ͜ͱͰաڈͷ৘ใͷ ๨٫΍ɺ৽͍͠ೖྗͷऔΓೖΕΛௐ੔ l Γ1 3FMFWBODF(BUF l Γ( 'PSHFU(BUF l Γ2: 6QEBUF(BUF l Γ3 0VUQVU(BUF 𝑐o!p = Γt ⋆ ̃ 𝑐o!p + Γu ⋆ 𝑐o!-"p ̃ 𝑐o!p = tanh(𝑊 v Γw ⋆ 𝑎o!-"p, 𝑥o!p + 𝑏v) 𝑎o!p = Γx ⋆ 𝑐o!p from: https://stanford.edu/~shervine/teaching/cs-230/cheatsheet- recurrent-neural-networks
  114. 177 ࣌ܥྻσʔλͱਂ૚ֶश ଞͷ3//ܥ౷ͷϞσϧ l (BUFE3FDVSSFOU6OJU (36 l -45.ΛҰൠԽͨ͠Ϟσϧ l -45.ΑΓ΋ܰྔͱͳ͓ͬͯΓɺͭͷ

    ήʔτͷΈͰߏ੒͠ܭࢉޮ཰ԽΛ޲্ l Γ1 3FMFWBODF(BUF l Γ2: 6QEBUF(BUF l -45.΋(36΋৽͍͠৘ใΛ֮͑ɺ աڈͷ৘ใΛద౓ʹ๨ΕΔػߏΛ උ͑ͨϞσϧ 𝑐o!p = Γt ⋆ ̃ 𝑐o!p + (1 − Γt) ⋆ 𝑐o!-"p ̃ 𝑐o!p = tanh(𝑊 v Γw ⋆ 𝑎o!-"p, 𝑥o!p + 𝑏v) 𝑎o!p = 𝑐o!p from: https://stanford.edu/~shervine/teaching/cs-230/cheatsheet-recurrent-neural-networks
  115. 178 ࣌ܥྻσʔλͱਂ૚ֶश 3FDVSSFOU/FVSBM/FUXPSL 3// l ڧΈ l Մม௕ʹରԠ l ϞσϧαΠζ͕ೖྗ௕ʹΑͬͯมԽ͠ͳ͍

    l աڈͷ৘ใΛߟྀͨ͠ܭࢉ͕Մೳ l ϞσϧͷॏΈ͕͕࣌ؒมԽͯ͠΋ڞ༗ l ऑΈ l ܭࢉͷ஗͞ l ݱࡏͷঢ়ଶʹରͯ͠কདྷͷೖྗΛߟྀͰ͖ͳ͍ l աڈͷ৘ใ͕࢒͍ͬͯͳ͍Մೳੑ
  116. 180 ࣌ܥྻσʔλͱਂ૚ֶश 3FDVSSFOU/FVSBM/FUXPSL 3// l ڧΈ l Մม௕ʹରԠ l ϞσϧαΠζ͕ೖྗ௕ʹΑͬͯมԽ͠ͳ͍

    l աڈͷ৘ใΛߟྀͨ͠ܭࢉ͕Մೳ l ϞσϧͷॏΈ͕͕࣌ؒมԽͯ͠΋ڞ༗ l ऑΈ l ܭࢉͷ஗͞ l ݱࡏͷঢ়ଶʹରͯ͠কདྷͷೖྗΛߟྀͰ͖ͳ͍ l աڈͷ৘ใ͕࢒͍ͬͯͳ͍Մೳੑ
  117. 181 ࣌ܥྻσʔλͱਂ૚ֶश ޯ഑ফࣦ 7BOJTIJOHHSBEJFOU ・・・ 𝑥% 𝑥" 𝑥V 𝑥W Feature

    Vecter RNN Block RNN Block RNN Block RNN Block 𝜕𝐿 𝜕𝑤% 𝑥% = 𝜕𝐿 𝜕𝑅𝑁𝑁W … 𝜕𝑅𝑁𝑁% 𝜕𝑤% 𝑥% ޡࠩٯ఻೻ʹඞཁͳޯ഑͕ඇৗʹ খ͘͞ͳΓɺֶश੍͕ޚͰ͖ͳ͘ ͳΔ໰୊
  118. 187 ࣌ܥྻσʔλͱਂ૚ֶश %FDPEFS΁ͷೖྗ l ֬཰ʹԠͯ͡ੜ੒σʔλ͔ڭࢣσʔλͷೖྗΛมԽͤ͞Δख๏ 4DIFEVMFE4BNQMJOH from: Bengio, Samy, et

    al. "Scheduled sampling for sequence prediction with recurrent neural networks." Advances in neural information processing systems 28 (2015).
  119. 188 ࣌ܥྻσʔλͱਂ૚ֶश %FDPEFS΁ͷೖྗ l ("/Λ༻͍ͯੜ੒σʔλͱڭࢣσʔλͷೖྗʹΑΔੜ੒ͷࠩΛখ͞ ͘͢Δ 1SPGFTTPS4BNQMJOH from: Lamb, Alex

    M., et al. "Professor forcing: A new algorithm for training recurrent networks." A neural information processing systems 29 (2016).
  120. 192 ࣌ܥྻσʔλͱਂ૚ֶश %JMBUFE$BVTBM$POWPMVUJPO from: Oord, A. V. D., Dieleman, S.,

    Zen, H., Simonyan, K., Vinyals, O., Graves, A., ... & Kavukcuoglu, K. (2016). Wavenet: A generative model for raw audio. arXiv preprint arXiv:1609.03499. l $//Λ༻͍ͨ࣌ؒ೾ܗ Ի੠ੜ੒ ʹର͢Δ֬཰తੜ੒ϞσϧͰ͋Δ 8BWF/FUͰఏҊ͞Εͨߏ੒ཁૉͷͭɻ l ೖྗ૚͔ΒॱʹͦΕͧΕɺɺɺݸͣͭεΩοϓ͠ͳ͕Β৞Έࠐ ΈΛܭࢉɻ৞ΈࠐΈ૚ͷਂ͞ʹԠͯ͡ɺೖྗͰ͖Δܥྻ௕͕ࢦ਺తʹ ૿Ճ͍ͯ͘͠ɻ l શ݁߹//ͷΑ͏ʹύϥϝʔλ͕ଟ͘ͳ͘ɺ3//ͷΑ͏ʹճؼతͳ઀ ଓ͕ແ͍ͷͰɺֶश͕3//ΑΓ଎͘ͳ͍ͬͯΔ l ௕ظͷܥྻΛೖྗͱͯ͠΋ޯ഑ফࣦ͕ੜ͡ͳ͍
  121. 193 ࣌ܥྻσʔλͱਂ૚ֶश 5SBOTGPSNFS l l"UUFOUJPOJT"MM:PVOFFEzͰఏҊ l ओʹࣗવݴޠॲཧ΍ը૾ॲཧͳͲʹ͓͍ͯߴ͍ਫ਼౓Λୡ੒͍ͯ͠ΔϞ σϧͷҰछ BERT ViT

    from: Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., ... & Polosukhin, I. (2017). Attention is all you need. Advances in neural information processing systems, 30.
  122. 199 ࣌ܥྻσʔλͱਂ૚ֶश 5SBOTGPSNFS Transformer Block l 4FMG"UUFOUJPOͱ'FFE'PSXBSEͰओʹߏ੒ l &ODPEFS%FDPEFSϞσϧ l

    $//ͱൺ΂େҬతͳؔ܎Λଊ͑Δ͜ͱ͕Մೳ l 1PTJUJPOBM&ODPEJOHͱ͍͏֤ೖྗͷ Ґஔ৘ใΛϕΫτϧͰදݱ l &ODPEFS %FDPEFSͱ΋ʹ .VMUJIFBEBUUFOUJPOͱ'FFE'PSXBSEͰ ߏ੒͞Εͨ5SBOTGPSNFS#MPDLͷੵΈॏͶ
  123. 201 4FMGBUUFOUJPO ࣌ܥྻσʔλͱਂ૚ֶश ⼊⼒系列の 潜在表現 系列⻑ × 次元数 Key :

    K Query: Q 𝑊y 𝑊z Value: V Attention Map : M 系列⻑ × 系列⻑ 𝑊{ 𝑊xt! Output 𝑠𝑜𝑓𝑡𝑚𝑎𝑥( 𝑄𝐾# 𝑑 )
  124. 202 4FMGBUUFOUJPO ࣌ܥྻσʔλͱਂ૚ֶश Value: V Attention Map : M 系列⻑

    × 系列⻑ 𝑊xt! Output 𝑠𝑜𝑓𝑡𝑚𝑎𝑥( 𝑄𝐾# 𝑑 ) ֤࣌఺ͷಛ௃͕ଞͷ࣌఺Λߟྀ ͯ͠࠶ߏ੒͞ΕΔ
  125. 205 3FTJEVBM$POOFDUJPO ࢒ࠩ઀ଓ ࣌ܥྻσʔλͱਂ૚ֶश Residual𝑂𝑢𝑡𝑝𝑢𝑡 = 𝑥 + 𝐴𝑡𝑡𝑒𝑛𝑡𝑖𝑜𝑛(𝑥) l

    ೖྗ𝑥Λ.VMUJ)FBE4FMG"UUFOUJPOͷग़ྗʹ Ճ͑Δ͜ͱͰɺֶशͷ҆ఆԽΛਤΔ l 3FT/FUͳͲͷϞσϧͰఏҊ͞Εͨख๏Ͱɺ ޯ഑ফࣦɾരൃ໰୊ΛճආͰ͖Δ
  126. 207 -BZFS/PSNBMJ[BUJPO ࣌ܥྻσʔλͱਂ૚ֶश 𝐿𝑎𝑦𝑒𝑟𝑁𝑜𝑟𝑚 𝑥 = 𝛾 𝑉𝑎𝑟 𝑥 ∗

    𝑥 − 𝜇 𝑥 + 𝛽 l ֤࣍ݩͷग़ྗΛฏۉ෼ࢄʹਖ਼نԽ͢Δख๏ l ޯ഑ফࣦɾޯ഑രൃΛ཈͑Δ໾ׂ
  127. 208 5SBOTGPSNFSº *OGMVFO[BGPSFDBTUJOH ࣌ܥྻσʔλͱਂ૚ֶश Wu, N., Green, B., Ben, X.,

    & O'Banion, S. (2020). Deep transformer models for time series forecasting: The influenza prevalence case. arXiv preprint arXiv:2001.08317.
  128. 209 ਂ૚ֶश WT౷ܭతϞσϧ ࣌ܥྻσʔλͱਂ૚ֶश Wu, N., Green, B., Ben, X.,

    & O'Banion, S. (2020). Deep transformer models for time series forecasting: The influenza prevalence case. arXiv preprint arXiv:2001.08317. .σʔληοτͷTUFQBIFBE GPSFDBTUJOHͷਫ਼౓ݕূ
  129. 213 ࣌ܥྻλεΫͱۙ೥ͷݚڀ࣌ܥྻ༧ଌ /#FBUT l 'VMMZ$POOFDUFEͰߏ੒͞Εͨϒ ϩοΫͷੵΈ্͛ʹΑΔ࣌ܥྻ༧ଌ Ϟσϧ l ϒϩοΫؒͰ͸࢒ࠩػߏʹΑΔ઀ଓ ͱɺϒϩοΫ಺෦Ͱ͸ઢܗ݁߹Ͱߏ

    ੒͞Ε͓ͯΓɺղऍੑͷߴ͍Ϟσϧ ͱͳ͍ͬͯΔʢͲͷ෦෼Ͱ൓Ԡͯ͠ ͍Δ͔͕෼͔Δʣ from: Oreshkin, Boris N., et al. "N-BEATS: Neural basis expansion analysis for interpretable time series forecasting." International Conference on Learning Representations. 2019.
  130. 214 ࣌ܥྻλεΫͱۙ೥ͷݚڀ࣌ܥྻ༧ଌ ঢ়ଶۭؒϞσϧ º ਂ૚ֶश4 l εςοϓҎ্ͷඇৗʹ௕͍γʔέϯεʹରԠͰ͖Δঢ়ଶۭؒϞσ ϧΛ׆༻ͨ͠ख๏ΛఏҊ l ঢ়ଶۭؒϞσϧΛ3FDVSSFOUͱ$POWPMVUJPOBMදݱʹஔ͖׵͑

    l )JQQPߦྻΛ3//ʹ૊ΈࠐΉ͜ͱͰ௕ظهԱੑΛ֫ಘ from: Gu, Albert, Karan Goel, and Christopher Re. "Efficiently Modeling Long Sequences with Structured State Spaces." International Conference on Learning Representations. 2021.
  131. 215 ࣌ܥྻλεΫͱۙ೥ͷݚڀ࣌ܥྻ༧ଌ ਂ૚ֶश º ෼෍ग़ྗ from: Salinas, David, et al.

    "DeepAR: Probabilistic forecasting with autoregressive recurrent networks." International Journal of Forecasting 36.3 (2020): 1181-1191. l %FFQ"3 l ֬཰తͳ༧ଌΛ࣮ݱ͢ΔͨΊʹෛͷೋ߲໬౓ؔ਺ͱϞϯςΧϧϩαϯϓ Ϧϯάͷग़ྗܗࣜͷಋೖ Training Prediction
  132. 216 ࣌ܥྻλεΫͱۙ೥ͷݚڀ࣌ܥྻ༧ଌ ਂ૚ֶश º ෼෍ग़ྗ from: Guen, Vincent Le, and

    Nicolas Thome. "Probabilistic time series forecasting with structured shape and temporal diversity." arXiv preprint arXiv:2010.07349 (2020). l 453*1& l Ψ΢ε෼෍ͳͲͷҰൠతͳ ෼෍Ͱ͸ͳ͘ɺඇఆৗͳ࣌ ܥྻʹ΋ରԠͨ࣌ؒ͠తɺ ܗঢ়త؍఺Ͱॊೈͳ෼෍༧ ଌ͕Մೳͳ&ODPEFS %FDPEFSϞσϧͷఏҊ
  133. 217 ࣌ܥྻλεΫͱۙ೥ͷݚڀ࣌ܥྻ༧ଌ ਂ૚ֶश º ෼෍ग़ྗ from: de Bézenac, E., Rangapuram,

    S. S., Benidis, K., Bohlke-Schneider, M., Kurle, R., Stella, L., ... & Januschowski, T. (2020). Normalizing kalman filters for multivariate time series analysis. Advances in Neural Information Processing Systems, 33, 2995-3007. l /PSNBMJ[JOH,BMNBO'JMJUFST l ઢܗΨ΢εঢ়ଶۭؒϞσϧΛ/PSNBMJ[JOHGMPXͰิڧͯ͠ଟมྔ࣌ܥྻ ͷϞσϦϯάΛ࣮ݱ͢Δ l 3FBM/71ͷΞʔΩςΫνϟΛར༻
  134. 218 ࣌ܥྻλεΫͱۙ೥ͷݚڀ࣌ܥྻ༧ଌ ਂ૚ֶश º ෼෍ग़ྗ from: https://lilianweng.github.io/posts/2018-10-13-flow-models/ l /PSNBMJ[JOH'MPX l

    ୯७ͳ֬཰ม਺𝑧. ʹରͯ͠ඇઢܗม׵𝑓\ ΛॏͶΔ͜ͱʹΑͬͯɺॊೈͳ෼ ෍𝑝y(𝑍y)Λ֫ಘ͢ΔͨΊͷख๏
  135. 219 ࣌ܥྻλεΫͱۙ೥ͷݚڀ࣌ܥྻ༧ଌ ෆ౳ִؒ࣌ܥྻ from: Chen, Ricky TQ, et al. "Neural

    ordinary differential equations." Advances in neural information processing systems 31 (2018). l 0%&3// l &ODPEFS%FDPEFSϞσϧʹ͓͚Δσίʔμͷજࡏม਺ग़ྗΛ0%&/FUʹ ஔ͖׵͑Δ͜ͱͰෆ౳ִؒ࣌ܥྻʹରԠ
  136. 220 ࣌ܥྻλεΫͱۙ೥ͷݚڀ࣌ܥྻ༧ଌ ෆ౳ִؒ࣌ܥྻ from: Chen, Ricky TQ, et al. "Neural

    ordinary differential equations." Advances in neural information processing systems 31 (2018). l /FVSBM0SEJOBSZ%JGGFSFOUJBM&RVBUJPO l 3FT/FUͱৗඋ෼ํఔࣜʹྨࣅ఺ʹண໨͠ɺৗඍ෼ํఔࣜͷղ๏Λχϡʔ ϥϧωοτͷදݱʹ༻͍Δख๏ l ͜ΕΛ༻͍Δ͜ͱͰϝϞϦޮ཰ͷߴ͍ ࣌ؒ࿈ଓͳϞσϧΛߏங͢Δ͜ͱ͕Մೳ ͱͳΔ l જࡏදݱ͸0%&4PMWFSʹΑͬͯܭࢉՄೳ
  137. 221 ࣌ܥྻλεΫͱۙ೥ͷݚڀ࣌ܥྻ༧ଌ 5SBOTGPSNFSº ࣌ܥྻ༧ଌ l *OGPSNFS l 5SBOTGPSNFS͕௕ظܥྻ༧ଌʹ͸ؤ݈Ͱͳ͍͜ͱ΍ɺܭࢉޮ཰͕ѱ͍͜ ͱΛղܾ͢ΔͨΊʹ4FMGBUUFOUJPOͷܭࢉޮ཰޲্ͷςΫχοΫΛఏҊ from:

    Zhou, Haoyi, et al. "Informer: Beyond efficient transformer for long sequence time-series forecasting." Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 35. No. 12. 2021. Method Complexity per Layer Convolutional 𝑂 𝐾 ~ 𝐷0 ~ 𝐿 Recurrent 𝑂 𝐿 ~ 𝐷0 Self-attention (Transformer) 𝑂 𝐿0 ~ 𝐷 K: the length of filter D: dimensionality of space L: input length N: Number of layers Computational Complexity 𝑶 𝑵×(𝑳𝟐 ‰ 𝑫)
  138. 222 ࣌ܥྻλεΫͱۙ೥ͷݚڀ࣌ܥྻ༧ଌ 5SBOTGPSNFSº ࣌ܥྻ༧ଌ l *OGPSNFS ̎ͭͷ$PNQMFYJUZ࡟ݮख๏ΛఏҊ l 1SPC4QBSTFॏཁ౓ͷߴ͍"UUFOUJPO.BQͷΈΛར༻ 𝑂

    𝐿Z ‹ 𝐷 ˠ 𝑂 𝐿 log 𝐿 ‹ 𝐷 ʹ࡟ݮ l 4FMGBUUFOUJPO%JTUJMMJOH ηϧϑΞςϯγϣϯ૚Λग़Δ౓ ܥྻͷ௕͕͞൒෼ʹͳΔΑ͏ৠཹ 𝑂 N ‹ ⋯ ˠ 𝑂 2 − 𝜖 ‹ ⋯ ʹ࡟ݮ
  139. 223 1SPC4QBSTF ࣌ܥྻλεΫͱۙ೥ͷݚڀ࣌ܥྻ༧ଌ ⼊⼒系列の 潜在表現 系列⻑(L) × 次元数(D) Key :

    K Query: • 𝑄 𝑊y 𝑊z Value: V Attention Map : M 系列⻑ × 系列⻑ 𝑊{ 𝑊xt! Output 𝑠𝑜𝑓𝑡𝑚𝑎𝑥( • 𝑄𝐾# 𝐷 ) 上位u件のQuery のみを利⽤
  140. 224 ࣌ܥྻλεΫͱۙ೥ͷݚڀ࣌ܥྻ༧ଌ 5SBOTGPSNFSº ࣌ܥྻ༧ଌ l "VUPGPSNFS l 5SBOTGPSNFS಺෦ʹ࣌ܥྻ෼ղͷػߏΛඋ͑ͨϞσϧ from: Wu,

    Haixu, et al. "Autoformer: Decomposition transformers with auto-correlation for long-term series forecasting." Advances in Neural Information Processing Systems 34 (2021): 22419-22430.
  141. 225 ࣌ܥྻλεΫͱۙ೥ͷݚڀ࣌ܥྻ༧ଌ Ґஔ৘ใ º ࣌ܥྻ༧ଌ l %JGGVTJPO$POWPMVUJPOBM3// l ଟมྔ࣌ܥྻͷؔ܎Λάϥϑͱͯ͠ଊ͑ͯ ಓ࿏ͷಓͳͲ

    ࣌ܥྻ༧ଌΛ ߦ͏ϞσϧΛఏҊ l (SBQI/FVSBM/FUXPSLͱ3//ͷ૊Έ߹Θͤ from: Li, Yaguang, et al. "Diffusion Convolutional Recurrent Neural Network: Data-Driven Traffic Forecasting." International Conference on Learning Representations. 2018.
  142. 233 ࣌ܥྻλεΫͱۙ೥ͷݚڀ࣌ܥྻ෼ྨ 30$,&5 l ཚ਺഑ྻͱ࣌ܥྻσʔλΛ͔͚߹Θͤͨಛ௃ྔΛେྔʹੜ੒͠ɺੜ੒͞ Εͨܥྻͷಛ௃ʢ࠷େ஋ͱ಺ੵ͕ਖ਼Ͱ͋ͬͨλΠϜ΢Οϯυ΢ͷׂ߹ʣ Λ༻͍Δ͜ͱͰ࣌ܥྻ෼ྨΛֶश from: Dempster, Angus,

    François Petitjean, and Geoffrey I. Webb. "ROCKET: exceptionally fast and accurate time series classification using random convolutional kernels." Data Mining and Knowledge Discovery 34.5 (2020): 1454-1495. 畳み込み Time Window 1.23 2.34 Time Window分の特徴を⽣成 ・・・ 特徴量を作成 最⼤: 2.34 正の割合: 0.21 学習
  143. 234 ࣌ܥྻλεΫͱۙ೥ͷݚڀ࣌ܥྻ෼ྨ 5-PTT l ଛࣦؔ਺Λڑ཭ֶशʹ͢Δ͜ͱͰڭࢣͳ͠Ͱ࣌ܥྻͷಛ௃ྔΛֶश l Ϟσϧࣗମ͸%JMBUFE$//Λར༻ Franceschi, Jean-Yves, Aymeric

    Dieuleveut, and Martin Jaggi. "Unsupervised scalable representation learning for multivariate time series." Advances in neural information processing systems 32 (2019).
  144. 238 ࣌ܥྻλεΫͱۙ೥ͷݚڀҟৗݕ஌ &ODPEFS%FDPEFSʹΑΔѹॖ from: Li, Shu, et al. "Fair Outlier

    Detection Based on Adversarial Representation Learning." Symmetry 14.2 (2022): 347.
  145. 243 ࣌ܥྻλεΫͱۙ೥ͷݚڀϞσϦϯά 8FCσʔλͷϞσϦϯά from: Murayama, Taichi, Yasuko Matsubara, and Sakurai

    Yasushi. "Mining Reaction and Diffusion Dynamics in Social Activities." arXiv preprint arXiv:2208.04846 (2022). l ֦ࢄ൓ԠํఔࣜͱχϡϥʔϧωοτϫʔΫͷ૊Έ߹Θͤ
  146. 244 ࣌ܥྻλεΫͱۙ೥ͷݚڀϞσϦϯά 4/4ؒͷϑϩʔ l ଟมྔ)BXLFTաఔΛ༻͍ͨ4/4ؒͷྲྀΕͷՄࢹԽ from: Zannettou, Savvas, et al.

    "The web centipede: understanding how web communities influence each other through the lens of mainstream and alternative news sources." Proceedings of the 2017 internet measurement conference. 2017.
  147. 245 ࣌ܥྻλεΫͱۙ೥ͷݚڀϞσϦϯά 1PJOU1SPDFTT ఺աఔ l ΠϕϯτσʔλͷϞσϦϯάʹ༻͍ΒΕΔ౷ܭతϞσϧ l Ұఆظؒͷۭؒɾ࣌ؒɾΠϕϯτͷੑ࣭ΛϞσϦϯά l 1PJTTPOաఔ

    l ۚ༥΍஍਒ͳͲͷ֬཰తʹൃੜ͢Δࣄ৅ͷϞσϦϯάʹ༻͍ΒΕΔ l )BXLFTաఔ l ࣗ෼ͷաڈͷΠϕϯτʹґଘ ͠ TFMGFYDJUJOH ͳੑ࣭Λ࣋ͪɺ 4/4ͷόʔετͳͲΛϞσϦϯά
  148. 247 ࣌ܥྻλεΫͱۙ೥ͷݚڀϞσϦϯά ਓؒͷձ࿩ʹ͍ͭͯͷϞσϦϯά l ʮਓͷౖΕΔஉʯʹ͓͚Δܶதձ࿩ͷϞσϦϯά from: Guo, Fangjian, et al.

    "The bayesian echo chamber: Modeling social influence via linguistic accommodation." Artificial Intelligence and Statistics. PMLR, 2015.
  149. 250 ͦͷଞ l খ੢ఃଇɼ๺઒ݯ࢛࿠ʮ৘ใྔن४ʯ l അ৔ޱ ొ தଜ ࿨ߊʮ৽͍͠৴߸ॲཧͷڭՊॻʯ l

    Ҫख ߶ ʮೖ໳ ػցֶशʹΑΔҟৗݕ஌ʕ3ʹΑΔ࣮ફΨΠυʯ l ۙߐ ਸ޺ ໺ଜ ढ़Ұʮ఺աఔͷ࣌ܥྻղੳʯ l "JMFFO/JFMTFOʮ࣮ફ ࣌ܥྻղੳ ʕ౷ܭͱػցֶशʹΑΔ༧ ଌʯ l ਺ཧख๏ᶝ ࣌ܥྻղੳ ౦ژେֶʮ਺ཧɾσʔλαΠΤϯεڭ ҭϓϩάϥϜʯ ࢀߟॻ