Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
[読み会] Knowledge distillation: A good teacher is...
Search
mei28
October 25, 2022
0
69
[読み会] Knowledge distillation: A good teacher is patient and consistent
読み会資料
Knowledge distillation: A good teacher is patient and consistent(CVPR2022)
mei28
October 25, 2022
Tweet
Share
More Decks by mei28
See All by mei28
[読み会] CHI2025論文紹介
mei28
1
24
[読み会] “Are You Really Sure?” Understanding the Effects of Human Self-Confidence Calibration in AI-Assisted Decision Making
mei28
0
120
[JSAI'24] 人間の判断根拠は文脈によって異なるのか?〜信頼されるXAIに向けた人間の判断根拠理解〜
mei28
2
640
[CHI'24] Fair Machine Guidance to Enhance Fair Decision Making in Biased People
mei28
0
83
[DEIM2024] 卓球の得点予測における重要要素の分析
mei28
0
52
[Human-AI Decision Making勉強会] 意思決定 with AIは個人vsグループで変わるの?
mei28
0
230
[読み会] Words are All You Need? Language as an Approximation for Human Similality Judgements
mei28
0
51
[参加報告] AAAI'23
mei28
0
110
[計算機構論] Learning Models of Individual Behavior in Chess
mei28
0
85
Featured
See All Featured
Dealing with People You Can't Stand - Big Design 2015
cassininazir
367
26k
I Don’t Have Time: Getting Over the Fear to Launch Your Podcast
jcasabona
33
2.4k
Agile that works and the tools we love
rasmusluckow
329
21k
Cheating the UX When There Is Nothing More to Optimize - PixelPioneers
stephaniewalter
283
13k
個人開発の失敗を避けるイケてる考え方 / tips for indie hackers
panda_program
110
19k
KATA
mclloyd
31
14k
Designing for Performance
lara
610
69k
What's in a price? How to price your products and services
michaelherold
246
12k
How GitHub (no longer) Works
holman
314
140k
Adopting Sorbet at Scale
ufuk
77
9.5k
Why You Should Never Use an ORM
jnunemaker
PRO
58
9.5k
ピンチをチャンスに:未来をつくるプロダクトロードマップ #pmconf2020
aki_iinuma
126
53k
Transcript
ಡΈձˏ ༶໌ ,OPXMFEHFEJTUJMMBUJPO "HPPEUFBDIFSJTQBUJFOUBOEDPOTJTUFOU
จใͱબཧ༝ • จใ • બཧ༝ • ࣝৠཹͷઃఆͷڭࢣͱੜెͷؔɼ ػցڭࣔͱྨࣅ͍ͯ͠ΔͨΊ
$7ٕज़ൃల͍ͯ͠Δ͚Ͳɼ࣮ࣾձͰ͍ͮΒ͍ • എܠɿେنϞσϧੑೳ͕ߴ͍͕ɼܭࢉίετɼઃඋίε τ͕ߴ͗͢Δ • ˠݱ࣮తʹখ͞ͳϞσϧͷํ͕͍উख͕ྑ͘ॏཁ • େنϞσϧͷೳྗΛ׆༻͢ΔͨΊʹɼੑೳҡ࣋ͨ͠··
ϞσϧΛܰྔԽ͢Δ͜ͱ͕ٻΊΒΕ͍ͯΔ
,%ʮڭࢣͱੜెͷϞσϧϚονϯάʯͱղऍ • ϞσϧͷܰྔԽʹͭͷख๏͕༗໊ • .PEFMQSVOJOH • ϞσϧͷҰ෦ΛΧοτ͢Δ͜ͱͰɼܰྔԽΛ͓͜ͳ͏ɽ • ϞσϧͷߏΛมߋͰ͖ͳ͍FH3FT/FUˠ.PCJMF/FU
• ,OPXMFEHFEJTUJMMBUJPO • ڭࢣʢେ͖Ίʣͱੜెʢখ͞ΊʣΛઃఆ͠ɽੜెڭࢣͱ ಉ͡ೳྗΛࢦ͢
ࣝৠཹɼೖྗσʔλͱֶश͕࣌ؒେࣄ • ͜͜ͰɼϞσϧΛܰྔԽ͢ΔͨΊʹɼࣝৠཹΛར༻ • ࣝৠཹʹ͓͍ͯԿ͕େࣄͳͷ͔Λݟ͚ͭΔ • ੜెͱڭࢣಉ͡ೖྗΛ༻͍Δ͜ͱ͕େࣄ • ೖྗσʔλσʔλ֦ு͢Δ΄Ͳྑ͍
• ֶश࣌ؒଟ͍ํ͕ྑ͍
ઃఆɾ࣮ݧઃఆ • ධՁࢦඪʹը૾ͷΫϥεྨਫ਼Λ༻͍Δ • େنϞσϧΛڭࢣͱઃఆ͠ɼྨਫ਼Λམͱͣ͞ʹɼϞσ ϧͷܰྔԽΛࢦ͢ɽ • ڭࢣɿ#J5ͷࣄલֶशϞσϧ •
ੜెɿ3FT/FU • σʔληοτछྨ খதنͭɼେنͭ • Ϋϥε σʔλαΠζ
ڭࢣͱੜెͷग़ྗΛ͚ۙͮΔ • ࣝৠཹͷֶशʹ༻͍ΔଛࣦͰ,-μΠόʔδΣϯε༻͍Δ • ͦΕͧΕڭࢣͷ༧ଌɼੜెͷ༧ଌ • Ϟσϧͷग़ྗࣗମɼԹ͖ιϑτϚοΫε pt ,
ps KL(pt ||ps ) = ∑ i∈C [−pt,i log ps,i + pt,i log pt,i ]
ϋΠύϥઃఆͱ͔࣮ݧͷখٕͨͪ • ࠷దԽख๏ͱͯ͠"EBNΛ༻͍ΔɽϋΠύϥσϑΥϧτ • εέδϡʔϥ$PTJOFEFDBZΛ༻͍Δ • ֶश҆ఆͷͨΊɼޯͷେ͖͞Λʹ੍ݶ • όοναΠζΛʹઃఆɽͨͩ͠*NBHF/FU
.JYVQʹΑͬͯσʔλ֦ுΛ͢Δ • ͭͷ܇࿅σʔλͷϖΞΛࠞ߹ͯ͠৽͘͠σʔλΛ࡞<> • ܇࿅σʔλ͕ϛοΫε͞ΕΔ͚ͩͰͳ͘ɼϥϕϧϛοΫε • • ͔ΒҰ༷ʹαϯϓϦϯά͢Δ
ຊՈ X = λX1 + (1 − λ)X2 , y = λy1 + (1 − λ)y2 λ ∈ [0,1] λ ∼ Beta(α, α) IUUQTBSYJWPSHBCT
ը૾ͷલॲཧ • σʔλͷલॲཧͰJODFQUJPOTUZMFDSPQΛ༻͍Δ • ݩσʔλͷը૾αΠζͷ ͰϥϯμϜʹΫϦοϓ • ΫϦοϓͨ͠ޙϥϯμϜͳΞεϖΫτൺʹม
• มͨ͋͠ͱݻఆαΠζ Y ʹϦαΠζ IUUQTBSYJWPSHBCT
ओுͷ͓͞Β͍ • ੜెͱڭࢣಉ͡ೖྗΛ༻͍Δ͜ͱ͕େࣄ • ೖྗσʔλσʔλ֦ு͢Δ΄Ͳྑ͍ • ֶश࣌ؒଟ͍ํ͕ྑ͍ • ϩόετੑΛݟΔͨΊʹখதنσʔληοτͭΛར༻
• ަབྷҼࢠΛഉআ͢ΔͨΊʹ͞·͟·ͳϋΠύϥͰ࣮ݧ 1BUJFOU UFBDIFS $POTJTUFOU UFBDIFS
$POTJTUFOUUFBDIFS Ұ؏ͨ͠ڭࢣ ʹ͍ͭͯΈΔ • ڭࢣɼੜెͷઃఆΛͭ༻ҙ͢Δɽ • 'JYFEUFBDIFSɿڭࢣͷग़ྗ͕ݻఆ͞Ε͍ͯΔ • *OEFQFOEFOUOPJTFɿڭࢣɼੜెͰҟͳΔೖྗ
• $POTJTUFOUUFBDIJOHɿڭࢣͱੜెͰಉ͡ೖྗ • 'VODUJPONBUDIJOHɿ$POTJTUFOUUFBDIJOHʴσʔλ֦ு • Լೋ͕ͭҰ؏ͨ͠ڭࢣઃఆΛද͍ͯ͠Δɽ
Ұ؏ͨ͠ڭࢣਫ਼͕ߴ͍ଞաֶशΛى͜͢
1BUJFOUUFBDIFS ڧ͍ڭࢣ ʹ͍ͭͯΈΔ • Ұൠతͳڭࢣ͋Γֶशͷ߹ɼϥϕϧʹର͠ɼը૾͕େ͖͘ ΉՄೳੑ͕͋Δɽ • ͜͜ͰɼࣝৠཹڭࢣͱੜెͷؔϚονϯάͱղऍ •
Ұ؏ͯ͠ಉ͡ೖྗΛ༩͑ΔͳΒɼೖྗࣗମΜͰ͍͍ ˠσʔλ֦ு͕༗ޮͳͷͰ • σʔλ֦ுΛߦ͍ɼେ͖ͳΤϙοΫͰֶश͢ΔͱΑΓੜె Ϟσϧͷੑೳ্͕͢Δ͜ͱΛݕূ͢Δ
աֶशΛճආ͠ͳ͕Βɼੑೳ্͕͢Δ • ͍ઢʢڭࢣʣͱಉͷੑೳʹ౸ୡ • ઢʢసҠֶशʣΑΓ࠷ऴతʹੑೳ͕ྑ͍ • ΦϨϯδʢ͔ΒֶशʣΑΓੑೳ͕͍͍ˠɹࣝৠཹͷޮՌ͕͋Δ
େنσʔληοτʢ*NBHF/FUʣͰࣝৠཹ • Ұ؏ͨ͠ڭࢣաֶश͍ͯ͠ͳ͍ • ·ͨσʔλ֦ு͢Δͱগͳ͍Τϙο ΫͰߴ͍ਫ਼Λୡ • ˠJUFSBUJPOಉ͡
ҟͳΔղ૾Ͱࣝৠཹ͕͏·͍͘͘ • ڭࢣ͕ߴ͍ղ૾ˠੜె͕͍ղ૾ͷํ͕ੑೳ͕ྑ͘ͳΔ
4IBNQPPΛֶͬͯशͨ͠Βߴʹͳͬͨ • σʔλ֦ுʹΑͬͯੑೳ্ͨ͠ ͕ɼͦͷֶश͕࣌ؒ৳ͼͯ͠·͏ • "EBN͔ΒΑΓڧྗͳ࠷దԽख๏Ͱ ͋Δ4IBNQPPΛͬͯΈΔɽ • ˠֶश͕ഒʹͳͬͨʂ
IUUQTBSYJWPSHBCT
సҠֶश͔ͭͬͯࣝৠཹ͍͍Μ͡Όͳ͍ • 1BUJFOUUFBDIFSͷͱ͖ɼసҠֶ श༗ޮ͔ͩͬͨΒࣝৠཹ స Ҡֶश͍͍ͷͰ • ֶशॳظసҠֶशΑ͔͚ͬͨ
Ͳɼ࠷ऴతʹٯస
ࣝৠཹͬͺΓ༗ޮ • ࣝৠཹͬͯແ͍͍ͯͬͯ͘͜ ͱͳ͍ʁ • ࣝৠཹ͕ͳ͍ͱաֶशΛى͜͢
·ͱΊͱײ • ࣝৠཹͰॏཁͳཁૉͱͯ͠ɼ • ɽڭࢣͱੜెͷೖྗ͕ಉ͡ • ɽσʔλ֦ுΛͨ͘͞Μ͢Δ • ɽֶशΤϙοΫΛ૿͢ɹɹɹɹ͜ͱʹ͍ͭͯࣔͨ͠ɽ
• ࣝৠཹͷ৽ख๏ΛఏҊͨ͠Θ͚Ͱͳ͍͕ɼطଘͷϞσϧΛ༻ ͍ͯɼܰྔϞσϧ͕4P5"ΛऔΕΔ͔͠Εͳ͍ϩϚϯΛײ͡Εͨ