Upgrade to Pro — share decks privately, control downloads, hide ads and more …

[読み会] Knowledge distillation: A good teacher is...

mei28
October 25, 2022
43

[読み会] Knowledge distillation: A good teacher is patient and consistent

読み会資料
Knowledge distillation: A good teacher is patient and consistent(CVPR2022)

mei28

October 25, 2022
Tweet

More Decks by mei28

Transcript

  1. $POTJTUFOUUFBDIFS Ұ؏ͨ͠ڭࢣ ʹ͍ͭͯΈΔ  • ڭࢣɼੜెͷઃఆΛͭ༻ҙ͢Δɽ • 'JYFEUFBDIFSɿڭࢣͷग़ྗ͕ݻఆ͞Ε͍ͯΔ • *OEFQFOEFOUOPJTFɿڭࢣɼੜెͰҟͳΔೖྗ

    • $POTJTUFOUUFBDIJOHɿڭࢣͱੜెͰಉ͡ೖྗ • 'VODUJPONBUDIJOHɿ$POTJTUFOUUFBDIJOHʴσʔλ֦ு • Լೋ͕ͭҰ؏ͨ͠ڭࢣઃఆΛද͍ͯ͠Δɽ
  2. 1BUJFOUUFBDIFS ೜଱ڧ͍ڭࢣ ʹ͍ͭͯΈΔ  • Ұൠతͳڭࢣ͋Γֶशͷ৔߹ɼϥϕϧʹର͠ɼը૾͕େ͖͘ ࿪ΉՄೳੑ͕͋Δɽ • ͜͜Ͱ͸ɼ஌ࣝৠཹڭࢣͱੜెͷؔ਺Ϛονϯάͱղऍ •

    Ұ؏ͯ͠ಉ͡ೖྗΛ༩͑ΔͳΒɼೖྗࣗମ͸࿪ΜͰ΋͍͍ ˠσʔλ֦ு͕༗ޮͳͷͰ͸  • σʔλ֦ுΛߦ͍ɼେ͖ͳΤϙοΫ਺Ͱֶश͢ΔͱΑΓੜె Ϟσϧͷੑೳ͕޲্͢Δ͜ͱΛݕূ͢Δ
  3. ·ͱΊͱײ૝  • ஌ࣝৠཹͰॏཁͳཁૉͱͯ͠ɼ • ɽڭࢣͱੜెͷೖྗ͕ಉ͡ • ɽσʔλ֦ுΛͨ͘͞Μ͢Δ • ɽֶशΤϙοΫ਺Λ૿΍͢ɹɹɹɹ͜ͱʹ͍ͭͯࣔͨ͠ɽ

    • ஌ࣝৠཹͷ৽ख๏ΛఏҊͨ͠Θ͚Ͱ͸ͳ͍͕ɼطଘͷϞσϧΛ༻ ͍ͯɼܰྔϞσϧ͕4P5"ΛऔΕΔ͔΋͠Εͳ͍ϩϚϯΛײ͡Εͨ