統計学実践ワークブック 第16章 重回帰分析 pp.125-127

統計学実践ワークブック 第16章 重回帰分析 pp.125-127について、式を導出したり図示したり行間を埋めたりしました。

Transcript

1. ౷ܭֶ࣮ફϫʔΫϒοΫ
   ୈষ
   ॏճؼ෼ੳ
   QQ
2. ه๏ ⃗ 1 ⃗ 0 X′  ͸ཁૉ͕͢΂ͯͰ͋ΔॎϕΫτϧ ྻϕΫτϧ ͸ཁૉ͕͢΂ͯͰ͋ΔॎϕΫτϧ ྻϕΫτϧ ͸ߦྻ9ͷసஔߦྻ X n×m ͸OߦNྻͷߦྻ
3. ճؼ෼ੳͷزԿֶతͳೝࣝ Im(X) Y e = Y− ̂ Y ̂ Y = PX Y = X ̂ β ℝn Im(X) = {Xv ∣ v ∈ ℝ1+d} ⊂ ℝn X n×(1+d) ̂ β (1+d)×1 ̂ Y n×1 = , e ⊥ ̂ Y :Λ*N 9 ʹࣹӨʹͨ͠Β
   ͱͳΔɻ ̂ Y ࢒ࠩϕΫτϧ e
4. 9ͷ૾ɺࣹӨɺ࢒ࠩ Y − ̂ Y ͸ Im(X) ʹࣹӨ͢Δߦྻɺ͢ͳΘࣹͪӨߦྻΛ PX ͱ͢Δͱɺ ̂ Y = PX Y Y Λ Im(X) ΁ࣹӨͨ͠ϕΫτϧͰ͋Δɻ ͳͷͰɺ ̂ Y ͱ ͸௚ަ͢Δɻ ̂ Y ͱ͜ΖͰɺ ͸ Im(X) ʹଐ͢ΔϕΫτϧͰ΋͋Δ͔Βɺ ̂ Y = X ̂ β Λຬͨ͢Α͏ͳ ̂ β ∈ ℝ1+d ͕ଘࡏ͢Δɻ X′  e = 0 ࢒ࠩϕΫτϧ e ʹ͍ͭͯɺ ͕੒Γཱͭɻ F͸*N 9 ͷ௚ަิۭؒʹؚ·ΕΔͱ͍͏͜ͱ e = Y − ̂ Y = Y − PX Y = Y − X ̂ β ࢒ࠩϕΫτϧ e Λ ͱͨ͠ͱ͖ɺ Im(X) = {Xv ∣ v ∈ ℝ1+d} ⊂ ℝn 9ͷ૾Λ*N 9 ͱද͢ɻ*N 9 ͸ू߹Ͱ͋ͬͯҎԼͷΑ͏ʹࣔ͞ΕΔɻ Y − PX Y ͱ ͕௚ަ͢ΔΑ͏ͳม׵ ʁ ͕ PX ͳͷͰ͋Δɻ PX Y ͱ X
5. 9F
   
    ∣ ∣ ∣ ⋯ ∣ ⃗ 1 x1 x2 ⋯ xd ∣ ∣ ∣ ⋯ ∣ X = − ⃗ 1 ′  − −x′  1− −x′  2− ⋮ −x′  d − X′  = X′  e = − ⃗ 1 ′  − −x′  1− −x′  2− ⋮ −x′  d − e = ⃗ 1 ′  e x1 ′  e ⋮ xd ′  e = 0 0 ⋮ 0 = ⃗ 0 n × (1 + d) (1 + d) × n (1 + d ) × n n × 1 (1 + d) × 1 9͸ Eݸͷجఈ ॎϕΫτϧ Λ
   ԣʹฒ΂ͨ΋ͷͩͱߟ͑Δɻ సஔ͢Δͱɺ
   ɾ֤ॎϕΫτϧΛసஔ
   ɾԣฒͼˠॎฒͼ
   ͱͳΔɻ جఈ͸*N 9 ʹؚ·Ε͍ͯͯɺ
   *N 9 ͷ೚ҙͷཁૉ͸࢒ࠩϕΫτϧFͱ௚ަ͢ Δɻ ͦͷΑ͏ʹߏ੒ͨ͠΋ͷ͕࢒ࠩϕΫτϧ Ͱ͋Δɺͱ΋ݴ͑Δɻ ͷجఈ͸ɺ9ͷୈྻ ୈྻ ʜ ୈ Eྻ
   Ͱ͋Δɻ Im(X) = {Xv ∣ v ∈ ℝ1+d} ⊂ ℝn ˠ
6. ࣹӨߦྻͷಋग़ X′  e = X′  (Y − ̂ Y) = X′  (Y − X ̂ β) = ⃗ 0 ΑΓɺ ⟺ X′  Y = X′  X ̂ β ⟺ ̂ β = (X′  X)−1X′  Y ̂ Y = X ̂ β = X(X′  X)−1X′  Y ·ͨɺ PX = X(X′  X)−1X′  Ͱ͋Δ͔Β ̂ Y = PX Y ͱൺֱͯ͠ X′  X͸Մٯͱ͢Δ X′  e = ⃗ 0 ΑΓɺ ͱͳΔɻ
7. ࣹӨߦྻ1Yͷੑ࣭  PX = X(X′  X)−1X′  ͱͯ͠ɺ P2 X = PX PX = (X(X′  X)−1X′  )(X(X′  X)−1X′  ) = X(X′  X)−1(X′  X)(X′  X)−1X′  = X(X′  X)−1X′  = PX P′  X = (X(X′  X)−1X′  )′  = X′  ′  ((X′  X)−1)′  X′  = X(X′  X)−1X′  = PX Im(X) = {Xv|v ∈ ℝ1+d} Im(PX ) = {PX w|w ∈ ℝn} X ͸ n × (1 + d) ߦྻͳͷͰɺ ͸ PX n × n ߦྻ O࣍ਖ਼ํߦྻ Ͱ͋Δɻ ࣹӨߦྻͷೋ৐ͱసஔ͸࠶ͼࣹӨߦྻʹ໭Δ͜ͱΛ֬ೝ͢Δɻ ͯ͞ɺ ͱ ͸౳͍͠ͱͷ͜ͱͳͷͰɺ͜ΕΛࣔͯ͠ΈΔɻ *N 1Y ΋*N 9 ΋ͲͪΒ΋ू߹Ͱ͋Δ͜ͱΑΓɺ l1 ∈ Im(X) ⟹ l1 ∈ Im(PX ) l2 ∈ Im(PX ) ⟹ l2 ∈ Im(X) Λࣔͤ͹ྑ͍ɻ
8. ࣹӨߦྻ1Yͷੑ࣭  [1]l1 ∈ Im(X) ⟹ l1 ∈ Im(PX ) [2]l2 ∈ Im(PX ) ⟹ l2 ∈ Im(X) l1 ∈ Im(X) Im(X) = {Xv|v ∈ ℝ1+d} Im(PX ) = {PX w|w ∈ ℝn} v1 ∈ ℝ1+d l1 = Xv1 ∈ ℝn w1 = Xv1 ∈ ℝn l2 ∈ Im(PX ) l2 = PX w2 = X(X′  X)−1X′  w2 ∈ ℝn l1 = PX w1 ∈ Im(PX ) l2 = X(X′  X)−1X′  w2 = Xv2 v2 = (X′  X)−1X′  w2 ∈ ℝ1+d l2 = Xv2 ∈ Im(X) ͱͳΔl1 ΛऔΔͱɺ ͱͳΔ ͕औΕΔɻ l1 = Xv1 = X(X′  X)−1(X′  X)v1 = PX Xv1 ͱදͤ͹ ͱͯ͠ ͱͳΔl2ΛऔΔͱɺ ͱͳΔw2 ∈ ℝn ͕औΕΔɻ ͱදͤ͹ ͱͯ͠ Ҏ্ΑΓɺ Im(PX ) = Im(X)
9. F 0 = ⃗ 1 ′  e Im(X) Y ̂ Y = PX Y = X ̂ β ℝn Im(X) = {Xv ∣ v ∈ ℝ1+d} ⊂ ℝn e e ⃗ 1 ˠ ͱ͸ɺ ⃗ 1 ͱe͕௚ަ͢Δɺͱݴ͍ͬͯΔɻ
10. 
    0 = ⃗ 1 ′  e = ⃗ 1 ′  (Y − X ̂ β) = n(¯ y − (1,¯ x′  ) ̂ β)ʹ͍ͭͯ෼ղͯ͠ߟ͑Δɻ ⃗ 1 ′  Y = ( 1 ⋮ 1 ) ′  y1 ⋮ yn = ∑ yi = n¯ y ⃗ 1 ′  X = ( 1 ⋮ 1 ) ′  1 , −x′  1 − ⋮ , ⋮ 1 , −x′  n − = (n, ∑ x′  i) = (n, n ¯ x′  ) 0 = ⃗ 1 ′  e = ⃗ 1 ′  (Y − X ̂ β) = (n¯ y − (n, n ¯ x′  ) ̂ β) = n(¯ y − (1, ¯ x′  ) ̂ β) ͜ͷΧϯϚ͸ྻΛ۠෼͚͢ΔͨΊͷه߸ɻ
    ಺ੵͰ͸ͳ͍ɻ −x′  i − = (xi,1 ⋯ xi,d ) ͸ԣϕΫτϧ Ҏ্ΑΓɺ ͱͳͬͯɺ ¯ y = (1, ¯ x′  ) ̂ β ΛಘΔɻ ¯ y ԣϕΫτϧͨͪͷ࿨͸
    ରԠ͢Δ੒෼͝ͱʹ࿨ΛऔΔ
11. Q
    Լஈ Y = X ̂ β + e Y − ⃗ 1 ¯ y = X ̂ β − ⃗ 1 ¯ y + e ΑΓ Y − ⃗ 1 ¯ y = X ̂ β − ⃗ 1 (1,¯ x′  ) ̂ β + e ¯ y = (1, ¯ x′  ) ̂ β ͞Βʹ ΑΓ = (X − ⃗ 1 (1,¯ x′  )) ̂ β + e = ( 1 x1 ′  ⋮ ⋮ 1 xn ′  − ⃗ 1 (1,¯ x′  )) ̂ β + e = 1 − 1 x1 ′  − x′  ⋮ ⋮ 1 − 1 xn ′  − x′  ̂ β + e = 0 (x1 − ¯ x)′  ⋮ ⋮ 0 (xn − ¯ x)′  ̂ β + e
12. Q
    தஈ Im(X) Y− ⃗ 1 ¯ y X ̂ β − ⃗ 1 ¯ y ℝn Im(X) = {Xv ∣ v ∈ ℝ1+d} ⊂ ℝn e Y X ̂ β e ⃗ 1 ¯ y͚ͩͣΒ͢ ͜ͷɹɹ্ͷࡾ֯ܗͰ
    ࡾฏํͷఆཧΛ͢Ε͹ྑ͍ɻ ℝn
13. ଓ Q
    தஈ Y − ⃗ 1 ¯ y = X ̂ β − ⃗ 1 ¯ y + e ⟺ Y − ⃗ 1 ¯ y = X ̂ β − ⃗ 1 (1,¯ x′  ) ̂ β + e ΑΓɺ ∥Y − ⃗ 1 ¯ y∥2 = ∥X ̂ β − ⃗ 1 (1,¯ x′  ) ̂ β∥2 + ∥e∥2 ⟺ ∑ (yi − ¯ y)2 = ∑ ((xi − ¯ x)′  ̂ β1:d) 2 + ∑ e2 i ͱͳΔɻ ̂ β0 ܎਺ ஫໨ϙΠϯτ ͸ɺ ͕ফ͑Δ ͷฏۉ͔Βͷภࠩ xi xi − ¯ x ͕ग़ݱ͢Δ ⃗ 1 ¯ y Λ࢖ͬͨ͜ͱͰɺ ͓ೃછΈ ͷมಈͷ෼ղͷ͕ࣜग़ݱ͢Δ ͜ͱͰ͋Δɻ