Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
高次元データに対するL1正則化の有効性
Search
Takayuki Uchiba
December 14, 2018
Technology
1
3.1k
高次元データに対するL1正則化の有効性
高次元データに対してよく用いられるL1正則化、特にLasso回帰の有効性について数理統計的にわかっている話を少しだけサマリーしました。
Takayuki Uchiba
December 14, 2018
Tweet
Share
More Decks by Takayuki Uchiba
See All by Takayuki Uchiba
statistician_ja_lt5.pdf
utaka233
0
660
縮小推定のはなし.pdf
utaka233
1
2.3k
Other Decks in Technology
See All in Technology
AWS全冠芸人が見た世界 ~資格取得より大切なこと~
masakiokuda
5
6.3k
OpenLane-V2ベンチマークと代表的な手法
kzykmyzw
0
100
システムとの会話から生まれる先手のDevOps
kakehashi
PRO
0
290
【Λ(らむだ)】最近のアプデ情報 / RPALT20250422
lambda
0
110
Linuxのパッケージ管理とアップデート基礎知識
go_nishimoto
0
380
4/17/25 - CIJUG - Java Meets AI: Build LLM-Powered Apps with LangChain4j (part 2)
edeandrea
PRO
0
120
AWSで作るセキュアな認証基盤with OAuth mTLS / Secure Authentication Infrastructure with OAuth mTLS on AWS
kaminashi
0
180
プロダクト開発におけるAI時代の開発生産性
shnjtk
2
240
Automatically generating types by running tests
sinsoku
2
3.4k
Road to Go Gem #rubykaigi
sue445
0
790
30代からでも遅くない! 内製開発の世界に飛び込み、最前線で戦うLLMアプリ開発エンジニアになろう
minorun365
PRO
11
3.2k
Dynamic Reteaming And Self Organization
miholovesq
3
570
Featured
See All Featured
Fontdeck: Realign not Redesign
paulrobertlloyd
83
5.5k
Git: the NoSQL Database
bkeepers
PRO
430
65k
Improving Core Web Vitals using Speculation Rules API
sergeychernyshev
13
680
The Cult of Friendly URLs
andyhume
78
6.3k
Fireside Chat
paigeccino
37
3.4k
Building Better People: How to give real-time feedback that sticks.
wjessup
367
19k
Cheating the UX When There Is Nothing More to Optimize - PixelPioneers
stephaniewalter
280
13k
Become a Pro
speakerdeck
PRO
27
5.3k
Documentation Writing (for coders)
carmenintech
69
4.7k
Mobile First: as difficult as doing things right
swwweet
223
9.6k
Reflections from 52 weeks, 52 projects
jeffersonlam
349
20k
Practical Tips for Bootstrapping Information Extraction Pipelines
honnibal
PRO
19
1.1k
Transcript
ߴ࣍ݩσʔλʹର͢Δ-ਖ਼ଇԽͷ༗ޮੑ !VUBLB ػցֶशͷཧ"EWFOU$BMFOEBS
എܠ ߴ࣍ݩσʔλ ɾೖྗมͷݸEαϯϓϧαΠζO ɾྫɿηϯαʔσʔλ࣍ੈγʔέϯαʔʹΑΔήϊϜྻσʔλͳͲ ߴ࣍ݩσʔλʹ͓͚Δ༧ଌ ɾදྫઢܗճؼϞσϧɿ ɹɹɾฏۉଛࣦ࠷খԽਪఆྔɿਖ਼نํఔࣜͷղ ɹɹɹߴ࣍ݩσʔλͰɺਖ਼نํఔࣜͷղͷҰҙੑΛظͰ͖ͳ͍ɻ ɹɹɹͳͥͳΒɺਖ਼نํఔࣜͷղ͕ҰҙͰ͋ΔͨΊʹ ɹɹɹཁߦྻ͕GVMMSBOLͰ͋Δඞཁ͕͋Δɻͱ͜Ζ͕ɺ
ɹɹɹͳͷͰɺߴ࣍ݩσʔλͰҰൠʹΓཱͨͣແݶʹղΛڐ͠ಘΔɻ y = Xw + ϵ, ϵ ∼ N(0,σ2En ) XT Xw = XTy rankXT X = n rankXT X = rankX ̂ w = argmin 1 2n ||y − Xw||2 2 ˠ
ઢܗճؼϞσϧʹ͓͚Δ-ਖ਼ଇԽʢ-BTTPճؼʣ ߴ࣍ݩσʔλʹ͓͚ΔઢܗճؼϞσϧ ɾूஂϞσϧʹఆ͢ΔԾઆɿճؼ͕εύʔεϕΫτϧͰ͋Δͱ͍͏ظ ɾ-BTTPճؼɿ-ਖ਼ଇԽʹΑΔεύʔεਪఆ ɹɾฏۉ̎ଛࣦ࠷খԽΛҎԼͷΑ͏ʹमਖ਼͢Δɻ ɹɹ͜ΕɺҎԼͷΑ͏ͳ੍͖࠷దԽͱಉͰ͋Δɻ ɹɹతؔͷತੑ͔Βղଘࡏͯ͠ҰҙʹͳΔɻ ɹɹ͞Βʹɺ੍݅ͷܗ͔Βղ͕εύʔεϕΫτϧʹͳΔ͜ͱ͕ظͰ͖Δɻ ̂ w
= argmin 1 2n ||y − Xw||2 2 + λn ||w|| 1 min 1 2n ||y − Xw||2 2 s . t . ||w|| 1 ≤ C
հ͢Δఆཧ ఆཧɿ</FHBICBO3BWJLVNBS8BJOXSJHIU:V $PSPMMBSZ> ूஂ͕ઢܗճؼϞσϧͰɺಛʹճؼɹ͕Lεύʔεͱ͠·͢ɻ ·ͨɺೖྗมEྻͰಠཱʹඪ४ਖ਼نʹै͍ͬͯΔͱ͠·͠ΐ͏ɻ͍· αΠζOͷඪຊΛऔͬͨ࣌ɺ ΛΈͨ͢ेେ͖ͳਖ਼ͷD͕͋Δͱ͠·͢ɻ͜ͷͱ͖ɺਖ਼ଇԽύϥϝʔλΛ ΛΈͨ͢Α͏ʹͱΕ-BTTPճؼʹΑͬͯಘΒΕΔϕΫτϧɹগͳ͘ͱ֬ ͰҎԼͷධՁΛΈͨ͢ɻ͜͜Ͱɺ$ఆͱ͢Δɻ
w* ̂ w n ≥ ck log(d) λn ≥ 8σ log(d)/n 1 − 1/d − O(exp(−n/2)) || ̂ w − w*||2 2 ≤ C kσ2 log(d) n
հ͢Δఆཧͷओு ཁ͢Δʹɺ ɾूஂ͕ઢܗճؼϞσϧͰճؼ͕ेʹεύʔεϕΫτϧͰ͋Δɻ ɾೖྗۭ͕ؒेʹߴ࣍ݩʹͳ͍ͬͯΔɻ ͷͰ͋Εɺेʹେ͖ͳਖ਼ଇԽύϥϝʔλΛΈͨ͢Α͏ʹͱΔ͜ͱͰɺ-BTTP ճؼͷਪఆྔͷฏۉޡࠩ ɾ࣍ݩʹରͯ͠ରతʹ͔͠ґଘ͠ͳ͍ɻʢ࣍ݩͷґଘ͕͍ʂʣ ɾճؼͷεύʔεੑɺޡࠩͷࢄɺαϯϓϧαΠζʹઢܗʹґଘ͢Δɻ ͱ͍͏ධՁΛ༩͍͑ͯΔɻ
ূ໌ͷͨΊͷ४උ Ωʔϫʔυɿ੍ݶڧತੑ 34$DPOEJUJPO αΠζɹɹͷߦྻ9ʹରͯ͠ɺू߹$ S Λ࣍ͷΑ͏ʹఆٛ͠·͢ɻ ਖ਼ͷఆɹ͕ଘࡏͯ͠ɺҙͷ$ S ͷݩ϶ʹରͯ͠ҎԼͷෆࣜ
ཱ͕͢Δͱ͖ɺߦྻ9$ S ʹ੍ؔͯ͠ݶڧತੑΛΈͨ͢ͱݴ͍·͢ɻ n × d C(r) = { Δ ∈ ℝd ∣ Δ ≠ 0, ||Δ|| 1 ||Δ|| 2 ≤ r } 1 n ||XΔ||2 2 ≥ κ||Δ||2 2 κ
੍ݶڧತੑͷͱͰͷ-BTTPਪఆྔͷྑ͞ ิɿ</FHBICBO3BWJLVNBS8BJOXSJHIU:V 5IFPSFN> ूஂʹର͢ΔԾఆɺఆཧͱ·ͬͨ͘ಉ͡Ͱ͋Δͱ͢Δɻ͠ਖ਼ͷఆD Λͱͬͯɺߦྻ9͕ू߹ɹɹɹɹɹɹɹʹରͯ͠ఆɹͰڧತੑΛ࣋ͭͱ͢Δɻ ͜ͷͱ͖ɺҙͷਖ਼ͷLʹରͯ͠ Ͱ͋Εɺਖ਼ଇԽύϥϝʔλ͕ɹɹɹɹɹɹɹɹͷ-BTTPճؼʹΑͬͯಘΒΕΔ ਪఆྔҎԼͷධՁΛຬͨ͠·͢ɻ C(8
n/(c log d)) κ n ≥ ck log(d) λn ≥ 2||XTϵ|| ∞ /n || ̂ w − w*||2 2 ≤ 9kλn κ2 ͜ͷධՁͩͱ͋·Γخ͕͠͞Θ͔Βͳ͍ɻ
ศརͳෆࣜ ิɿ<3BTLVUUJ8BJOXSJHIU:V 1SPQPTJUJPO> αΠζɹɹͷߦྻ9ͷ֤ߦ͕ಠཱʹଟมྔਖ਼ن/ Є ʹैͬͯಘΒΕΔͱ͖ ਖ਼ͷఆD D`͕ଘࡏͯ͠ɺҙͷE࣍ݩϕΫτϧWʹରͯ͠গͳ͘ͱ֬
ͰҎԼͷධՁ͕Γཱͪ·͢ɻͨͩ͠ɺ4ೖྗมͷඪ४ภࠩͷ࠷େͰ͢ɻ n × d 1 − c exp(−c′n) ||Xv|| 2 n ≥ 1 4 ||Σ1/2v|| 2 − 9S log(d) n ||v|| 1
ఆཧͷূ໌ 3BTLVUUJ8BJOXSJHIU:Vͷෆ͔ࣜΒ ΛಘΔɻͦ͜ͰɺɹɹɹɹɹɹɹɹͳͷͰɺఆDΛेେ͖͘ͱΕΕ ੍ݶڧತੑ͕গͳ͘ͱ֬ɹɹɹɹɹɹɹͰΓཱͭ͜ͱ͕Θ͔Γ·͢ɻ ͜͜ͰɺࠓͱͬͨఆD͕ɹɹɹɹɹɹΈͨ͢ͱԾఆͯ͠ɺ /FHBICBO3BWJLVNBS8BJOXSJHIU:VͷఆཧΛߟ͑·͢ɻਖ਼ଇԽύϥϝʔλͷ ͔݅Βɺগͳ͘ͱ֬ Ͱਪఆྔʹؔ͢ΔఆཧͷධՁΛಘΔɻҎ্ͰఆཧΛূ໌Ͱ͖ͨɻ ||Xv|| 2
n ≥ 1 4 ( 1 − 36 log(d) n ||v|| 1 ||v|| 2 ) v ∈ C(8 n/(c log d)) 1 − c exp(−c′n) n ≥ ck log(d) P [ ||XTϵ|| ∞ ≤ 8σ2n log(d)] ≥ 1 − 1 d − exp (− n 2 )
ࢀߟจݙ <>3BTLVUUJ8BJOXSJHIU:V .JOJNBYSBUFTPGFTUJNBUJPOGPSIJHI EJNFOTJPOBMMJOFBSSFHSFTTJPOPWFSMRCBMMT *&&&5SBOTBDUJPO PO*OGPSNBUJPO5IFPSZ <>/FHBICBO3BWJLVNBS8BJOXSJHIU:V "6OJpFE'SBNFXPSLGPS )JHI%JNFOTJPOBM"OBMZTJTPG.&TUJNBUPSTXJUI%FDPNQPTBCMF
3FHVMBSJ[FST 4UBUJTUJDBM4DJFODF 7PM /P <>Ԭ྄ଠ εύʔεੑʹجͮ͘ػցֶश ػցֶशϓϩϑΣογϣφϧ γϦʔζ ߨஊࣾ