CVPR2020読み会 Proxy Anchor Loss for Deep Metric Learning

CVPR2020読み会(後編)の資料です

なるべくMetric Learningに良い感じに入門できるようになってます

Transcript

1. CVPR2020ಡΈձ (2020/7/18)
 sandglass Proxy Anchor Loss for
 Deep Metric Learning
2. ·ͣ͸ࣗݾ঺հ !2 • ໊લ: ࠤ౻ ܒथ Sato Keiju • Twitter: @TodayInsane
3. ·ͣ͸ࣗݾ঺հ !3 • ໊લ: ࠤ౻ ܒथ Sato Keiju • Twitter: @TodayInsane K-POPେ޷͖ͳͷͰ Metric LearningͰ
 PyTorchͱTWICEΛ஌Δ
 ABEJAΞυϕϯτΧϨϯμʔ ॻ͖·ͨ͠
4. ·ͣ͸ࣗݾ঺հ !4 • ໊લ: ࠤ౻ ܒथ Sato Keiju • Twitter: @TodayInsane • ॴଐ: ౦ژେֶిؾܥ޻ֶઐ߈ͷ৘ใܥ म࢜2೥ • ֶ෦: CMOSΞφϩάిࢠճ࿏ in ਆށେֶిి • גࣜձࣾABEJAͰ௕ظΠϯλʔϯ 2019/4 ʙ • Vue.jsͱ͔Firebaseͱ͔Goͱ͔Kubernetesͱ͔AWSͱ͔Deep Learningͱ͔Λϝϯλʔ͞Μํʹڭ͑ͯ΋Β͍·͍ͬͯ͘·͢
5. ·ͣ͸ࣗݾ঺հ !5 • ໊લ: ࠤ౻ ܒथ Sato Keiju • Twitter: @TodayInsane • ॴଐ: ౦ژେֶిؾܥ޻ֶઐ߈ͷ৘ใܥ म࢜2೥ • ֶ෦: CMOSΞφϩάిࢠճ࿏ in ਆށେֶిి • גࣜձࣾABEJAͰ௕ظΠϯλʔϯ 2019/4 ʙ • Vue.jsͱ͔Firebaseͱ͔Goͱ͔Kubernetesͱ͔AWSͱ͔Deep Learningͱ͔Λϝϯλʔ͞Μํʹڭ͑ͯ΋Β͍·͍ͬͯ͘·͢
6. • ໊લ: ࠤ౻ ܒथ Sato Keiju • Twitter: @TodayInsane • ॴଐ: ౦ژେֶిؾܥ޻ֶઐ߈ͷ৘ใܥ म࢜2೥ • ֶ෦: CMOSΞφϩάిࢠճ࿏ in ਆށେֶిి • גࣜձࣾABEJAͰ௕ظΠϯλʔϯ 2019/4 ʙ • Vue.jsͱ͔Firebaseͱ͔Goͱ͔Kubernetesͱ͔AWSͱ͔Deep Learningͱ͔Λϝϯλʔ͞Μํʹڭ͑ͯ΋Β͍·͍ͬͯ͘·͢ ·ͣ͸ࣗݾ঺հ !6 ΤϯδχΞ: @xecus͞Μ ͕։ൃͨ͠ɺإೝূٕज़Λ࢖ͬͨ ΊͪΌͪ͘Ό໘ന͍ϓϩμΫτͷ ஀ੜൿ࿩ ݱࡏΠϯλʔϯͰ͓ख఻͍ͨ͠࿩ Λଓฤͱͯࣥ͠චதͰ͢
7. • ໊લ: ࠤ౻ ܒथ Sato Keiju • Twitter: @TodayInsane • ॴଐ: ౦ژେֶిؾܥ޻ֶઐ߈ͷ৘ใܥ म࢜2೥ • ֶ෦: CMOSΞφϩάిࢠճ࿏ in ਆށେֶిి • גࣜձࣾABEJAͰ௕ظΠϯλʔϯ 2019/4 ʙ • Vue.jsͱ͔Firebaseͱ͔Goͱ͔Kubernetesͱ͔AWSͱ͔Deep Learningͱ͔Λϝϯλʔ͞Μํʹڭ͑ͯ΋Β͍·͍ͬͯ͘·͢ ·ͣ͸ࣗݾ঺հ !7 Ϧαʔν: @peisuke͞Μ ʹ༏͘͠എதΛԡ͍͖ͯͨͩ͠ ຊ೔ਓੜॳొஃͰ͢
8. ΑΖ͓͘͠ئ͍͠·͢ !8
9. ·ͣ͸͜ͷൃදͷΰʔϧΛ֬ೝʂ !9
10. ࠓ೔ͷΰʔϧ !10 ͷ͓ؾ࣋ͪΛཧղ͢Δ ঺հ࿦จΑΓҾ༻
11. ࠓ೔ͷΰʔϧ !11 ͷ͓ؾ࣋ͪΛཧղ͢Δ ԿͷֆͱԿͷࣜʁʁʁ
12. ͓͠ͳ͕͖ • Metric Learningͬͯʁ • CVʹ͓͚ΔMetric LearningͷྲྀΕ • ຊ೔ͷ࿦จ঺հ !12 ͷ3ຊͰɺMetric Learningͷੈքͷུ஍ਤΛ͓౉͠Ͱ͖Ε͹ͱࢥ͍ͬͯ·͢ ͍ΖΜͳํͱ͍ΖΜͳ͓࿩͍ͨ͠ͷͰTwitterͳͲ͓ؾܰʹʂ͓ئ͍͠·͢ʂ
13. ͓͠ͳ͕͖ • Metric Learningͬͯʁ • CVʹ͓͚ΔMetric LearningͷྲྀΕ • ຊ೔ͷ࿦จ঺հ !13
14. Metric Learning: σʔλͷڑ཭ࢦඪΛ֫ಘ͢Δ !14 Black footed Albatross Sooty Albatross
15. Metric Learning: σʔλͷڑ཭ࢦඪΛ֫ಘ͢Δ !15 Black footed Albatross Sooty Albatross Black footed Albatross Sooty Albatross • ಉ͡Ϋϥεͷσʔλಉ࢜͸͍ۙ • ҟͳΔΫϥεؒ͸཭Ε͍ͯΔ
16. Metric Learning: σʔλͷڑ཭ࢦඪΛ֫ಘ͢Δ !16 Black footed Albatross Sooty Albatross Black footed Albatross Sooty Albatross • ಉ͡Ϋϥεͷσʔλಉ࢜͸͍ۙ • ҟͳΔΫϥεؒ͸཭Ε͍ͯΔ σʔλ΍ ͦͷಛ௃ྔ ࣅͯΔͱ ۙ͘ͳΔ ۭؒ
17. Metric Learning: σʔλͷڑ཭ࢦඪΛ֫ಘ͢Δ !17 Black footed Albatross Sooty Albatross Black footed Albatross Sooty Albatross • ಉ͡Ϋϥεͷσʔλಉ࢜͸͍ۙ • ҟͳΔΫϥεؒ͸཭Ε͍ͯΔ σʔλ΍ ͦͷಛ௃ྔ ࣅͯΔͱ ۙ͘ͳΔ ۭؒ ͜ΕΒΛ֫ಘ͢Δͷ͕Metric(ࢦඪ)Learning(ֶश)
18. ͸͡·Γ͸ม׵ߦྻ(ϚϋϥϊϏεڑ཭)͔Β… !18
19. ͸͡·Γ͸ม׵ߦྻ(ϚϋϥϊϏεڑ཭)͔Β… !19 ม਺ಉ࢜ͷڞ෼ࢄߦྻ • ֤࣍ݩͷ෼ࢄΛߟྀͨ͠Ұൠతͳڑ཭ • ൒ਖ਼ఆ஋ߦྻͰ͋Ε͹͞Βʹɹɹɹɹɹɹɹɹɹɹɹɹɹɹɹͱม׵
 ͭ·ΓσʔλؒͷϢʔΫϦουڑ཭͕ద੾ʹͳΔΑ͏ͳม׵LΛֶश
 →ྨࣅσʔλಉ࢜͸ۙ͘ɺඇྨࣅσʔλಉ࢜ͷڑ཭͸ԕ͘͢Δ
 ɹ࠷దԽ໰୊
20. Deep LearningͷൃలͰMetric Learning΋Ξπ͍ !20 Channel Width Height ը૾: H × W × C࣍ݩσʔλ
21. Deep LearningͷൃలͰMetric Learning΋Ξπ͍ !21 Channel Width Height ը૾: H × W × C࣍ݩσʔλ Hand-craftedͳಛ௃ྔઃܭ x1 x2 : : : : : : : :
 : xN N࣍ݩಛ௃ϕΫτϧ = N࣍ݩۭؒ಺ͷ఺
22. Deep LearningͷൃలͰMetric Learning΋Ξπ͍ !22 Channel Width Height ը૾: H × W × C࣍ݩσʔλ Hand-craftedͳಛ௃ྔઃܭ x1 x2 : : : : : : : :
 : xN N࣍ݩಛ௃ϕΫτϧ = N࣍ݩۭؒ಺ͷ఺ Deep CNN
23. Deep LearningͷൃలͰMetric Learning΋Ξπ͍ !23 Channel Width Height ը૾: H × W × C࣍ݩσʔλ Hand-craftedͳಛ௃ྔઃܭ x1 x2 : : : : : : : :
 : xN N࣍ݩಛ௃ϕΫτϧ = N࣍ݩۭؒ಺ͷ఺ Deep CNN ࣮ݱ͍ۭͨؒ͠ͷڑ཭తͳ ͓ؾ͕࣋ͪ࿅Γ͜·ΕͨLossؔ਺ ֶश
24. ը૾ೝࣝ: ֤Ϋϥε͕·ͱ·Δಛ௃ྔۭؒͰࣝผ !24 ಉ͡Ϋϥεϥϕϧͷ͍ͭͨը૾͸ۙ͘ ҧ͏Ϋϥε͸཭ΕͯຒΊࠐ·ΕΔ
25. ը૾ೝࣝ: ֤Ϋϥε͕·ͱ·Δಛ௃ྔۭؒͰࣝผ !25 https://twitter.com/sanacottoncandy/status/1131140431775801345 https://twitter.com/WinterFrame/status/1200973640197492736 https://twitter.com/JYPETWICE/status/1272053602790502400 ಉ͡Ϋϥεϥϕϧͷ͍ͭͨը૾͸ۙ͘ ҧ͏Ϋϥε͸཭ΕͯຒΊࠐ·ΕΔ
26. ը૾ೝࣝ: ֤Ϋϥε͕·ͱ·Δಛ௃ྔۭؒͰࣝผ !26 ಉ͡Ϋϥεϥϕϧͷ͍ͭͨը૾͸ۙ͘ ҧ͏Ϋϥε͸཭ΕͯຒΊࠐ·ΕΔ ʮಉ͡Ϋϥεͱ൑அ͢Δڑ཭ͷᮢ஋ʯΛ ͍͍ײ͡ʹઃఆ͢Δ ը૾෼ྨ αφͪΌΜ π΢ΟͪΌΜ
27. ը૾ೝࣝ: ֤Ϋϥε͕·ͱ·Δಛ௃ྔۭؒͰࣝผ !27 ಉ͡Ϋϥεϥϕϧͷ͍ͭͨը૾͸ۙ͘ ҧ͏Ϋϥε͸཭ΕͯຒΊࠐ·ΕΔ ʮಉ͡Ϋϥεͱ൑அ͢Δڑ཭ͷᮢ஋ʯΛ ͍͍ײ͡ʹઃఆ͢Δ ը૾෼ྨ (ओʹϢʔΫϦουڑ཭Ͱ)ΫΤϦը૾ͷ
 ຒΊࠐΈ఺ʹ͍ۙॱʹྨࣅը૾ͱ͢Δ ը૾ݕࡧ ΫΤϦ Top-1(࠷ۙ๣) https://twitter.com/JYPETWICE_JAPAN/status/1156761954024734721
28. ը૾ೝࣝ: ֤Ϋϥε͕·ͱ·Δಛ௃ྔۭؒͰࣝผ !28 ಉ͡Ϋϥεϥϕϧͷ͍ͭͨը૾͸ۙ͘ ҧ͏Ϋϥε͸཭ΕͯຒΊࠐ·ΕΔ ʮಉ͡Ϋϥεͱ൑அ͢Δڑ཭ͷᮢ஋ʯΛ ͍͍ײ͡ʹઃఆ͢Δ ը૾෼ྨ (ओʹϢʔΫϦουڑ཭Ͱ)ΫΤϦը૾ͷ
 ຒΊࠐΈ఺ʹ͍ۙॱʹྨࣅը૾ͱ͢Δ ը૾ݕࡧ ෼ྨʹՃ͑ͯະ஌ͷΫϥε΋ ʮະ஌ͷΫϥεʯͱͯ͠ରԠͰ͖Δ ը૾ࣝผ https://twitter.com/JYPETWICE_JAPAN/status/1156761692547629056 New?
29. ը૾ೝࣝ: ֤Ϋϥε͕·ͱ·Δಛ௃ྔۭؒͰࣝผ !29 ະ஌ͷΫϥε΋ʮະ஌ͷΫϥεʯͱͯ͠ରԠͰ͖Δ
30. ը૾ೝࣝ: ֤Ϋϥε͕·ͱ·Δಛ௃ྔۭؒͰࣝผ !30 ະ஌ͷΫϥε΋ʮະ஌ͷΫϥεʯͱͯ͠ରԠͰ͖Δ ʮֶश࣌ʹແ͍Ϋϥε͕ਪ࿦ʹग़ͯ͘Δʯ՝୊ઃఆʹ˕ • Face Recognition / Verification • Person Re-Identification • Image Retrieval ͳͲͳͲ...
31. !31 ଞʹ΋ Metric Learning for…
32. Adversarial Attack(ݸਓతʹ໘ന͔ͬͨͷͰ͝঺հ) !32 “Metric Learning for Adversarial Robustness”(NeurIPS’19) Adversarial Example͸ ಛ௃ྔۭؒʹ͓͍ͯ Ϋϥεͷڥք෇ۙʹҐஔ͍ͯ͠Δʁ(͔Βؒҧ͑Δʁ) Adversarial ExampleΛ ຊ౰ͷΫϥεʹ͍ۙํ޲ʹ ಈ͔͢Loss ը૾ͱAdversarialͳը૾Λಛ௃ྔۭؒʹ౤Ө͢Δͱ…ʁ
33. !33 “OpenGAN: Open Set Generative Adversarial Networks”(arXiv 2020/5/18) ҟৗݕ஌ “Unsupervised Anomaly Detection with Generative Adversarial Networks
 to Guide Marker Discovery”(IPMI’17) GAN Distillation “Relational Knowledge Distillation”(CVPR’19)
34. ౳ʑ… ͋͘·Ͱ͘͝Ұ෦ͷ࿩͔͍ͭܰ঺հͰ͢ͷͰ ͍Ζ͍Ζ୳ͨ͠Γࢼͨ͠Γͯ͠Έ͍ͯͩ͘͞ʂ !34
35. ͓͠ͳ͕͖ • Metric Learningͬͯʁ • CVʹ͓͚ΔMetric LearningͷྲྀΕ • ຊ೔ͷ࿦จ঺հ !35
36. ࢝૆: 2ͭͷσʔλ఺Λ͚ۙͮΔ/཭͢Contrastive Loss !36
37. ࢝૆: 2ͭͷσʔλ఺Λ͚ۙͮΔ/཭͢Contrastive Loss !37 Deep CNN x1 x2 : : : : : : : :
 : xN x1 x2 : : : : : : : :
 : xN ಉ͡Ϋϥεͷը૾
38. ࢝૆: 2ͭͷσʔλ఺Λ͚ۙͮΔ/཭͢Contrastive Loss !38 Deep CNN x1 x2 : : : : : : : :
 : xN x1 x2 : : : : : : : :
 : xN ಉ͡Ϋϥεͷը૾
39. ࢝૆: 2ͭͷσʔλ఺Λ͚ۙͮΔ/཭͢Contrastive Loss !39 Deep CNN x1 x2 : : : : : : : :
 : xN x1 x2 : : : : : : : :
 : xN ΑΓ ۙͮ͘ ಉ͡Ϋϥεͷը૾
40. ࢝૆: 2ͭͷσʔλ఺Λ͚ۙͮΔ/཭͢Contrastive Loss !40 Deep CNN x1 x2 : : : : : : : :
 : xN x1 x2 : : : : : : : :
 : xN ҧ͏Ϋϥεͷը૾
41. ࢝૆: 2ͭͷσʔλ఺Λ͚ۙͮΔ/཭͢Contrastive Loss !41 Deep CNN x1 x2 : : : : : : : :
 : xN x1 x2 : : : : : : : :
 : xN ҧ͏Ϋϥεͷը૾
42. ࢝૆: 2ͭͷσʔλ఺Λ͚ۙͮΔ/཭͢Contrastive Loss !42 Deep CNN x1 x2 : : : : : : : :
 : xN x1 x2 : : : : : : : :
 : xN ΑΓ ཭ΕΔ ҧ͏Ϋϥεͷը૾
43. ࢝૆: 2ͭͷσʔλ఺Λ͚ۙͮΔ/཭͢Contrastive Loss !43 Deep CNN x1 x2 : : : : : : : :
 : xN x1 x2 : : : : : : : :
 : xN ҧ͏Ϋϥεͷը૾ “Dimensionality Reduction by Learning an Invariant Mapping” (CVPR’06)
44. ࢝૆ͷࢠ(ʁ): 3ͭͷσʔλ఺Λ͚ۙͮΔ/཭͢Triplet Loss !44 [~]+: ώϯδؔ਺(= max(~, 0))
45. ࢝૆ͷࢠ(ʁ): 3ͭͷσʔλ఺Λ͚ۙͮΔ/཭͢Triplet Loss !45 Deep CNN anchor: ͋Δσʔλ positive: anchorͱ ಉΫϥε negative: ผͷΫϥε [~]+: ώϯδؔ਺(= max(~, 0)) negative anchor positive
46. ࢝૆ͷࢠ(ʁ): 3ͭͷσʔλ఺Λ͚ۙͮΔ/཭͢Triplet Loss !46 Deep CNN anchor: ͋Δσʔλ positive: anchorͱ ಉΫϥε negative: ผͷΫϥε [~]+: ώϯδؔ਺(= max(~, 0)) anchor positive negative dn dp dp ʹϚʔδϯαΛ଍ͯ͠΋ ·ͩdn ͷํ͕σΧ͘ͳͬͯ΄͍͠ • จ຺(ϥϕϧͱλεΫ)ʹؔ܎ͳ͘
 ૬ରతʹۙ͘/ԕ͘͢Ε͹Α͍ • ૊Έ߹Θͤ਺͕๲େͰ
 ֶश͕ऩଋ͠ͳ͍…
47. ࢝૆ͷࢠ(ʁ): 3ͭͷσʔλ఺Λ͚ۙͮΔ/཭͢Triplet Loss !47 Deep CNN anchor: ͋Δσʔλ positive: anchorͱ ಉΫϥε negative: ผͷΫϥε [~]+: ώϯδؔ਺(= max(~, 0)) anchor positive negative dn dp dpʹϚʔδϯαΛ଍ͯ͠΋ dnΑΓখ͘͢͞Δ • จ຺(ϥϕϧͱλεΫ)ʹؔ܎ͳ͘
 ૬ରతʹۙ͘/ԕ͘͢Ε͹Α͍ • ૊Έ߹Θͤ਺͕๲େͰ
 ֶश͕ऩଋ͠ͳ͍… “Learning Fine-grained Image Similarity with Deep Ranking” (CVPR’14)
48. Contrastive / Triplet Loss͔Β೿ੜ→੝Γ্͕͖ͬͯͨ • ֶशʹ༗ޮͳϖΞΛ୳͢/࡞Δ • ͞Βʹߟྀ͢ΔΫϥε਺Λ֦ு • ۭؒΛ௒ٿ໘্ʹ੍໿ • ڑ཭ࢦඪΛ֯౓Ͱ • ୅ද఺ͰMetric Learning • Ϋϥε෼ྨܕωοτϫʔΫߏ଄ !48 ͳͲͳͲ…
49. Contrastive / Triplet Loss͔Β೿ੜ→੝Γ্͕͖ͬͯͨ !49 “FaceNet: A Unified Embedding for Face Recognition and Clustering”(CVPR’15) !49 !49 • ֶशʹ༗ޮͳϖΞΛ୳͢/࡞Δ • ͞Βʹߟྀ͢ΔΫϥε਺Λ֦ு • ۭؒΛ௒ٿ໘্ʹ੍໿ • ڑ཭ࢦඪΛ֯౓Ͱ • ୅ද఺ͰMetric Learning • Ϋϥε෼ྨܕωοτϫʔΫߏ଄ Tripletͷ૊Έ߹Θͤʹ͍ͭͯͷʮ೉͠͞ʯΛఆٛ Loss͕େ͖͘ͳΔSemi-hard NegativeϖΞΛޮ཰Α͘࡞੒
50. Contrastive / Triplet Loss͔Β೿ੜ→੝Γ্͕͖ͬͯͨ !50 “Improved Deep Metric Learning with Multi-class N-pair Loss Objective”(NIPS’16) NΫϥε͔Β2ը૾ͣͭ
 औΓग़ͯ͠ಛ௃ྔʹม׵ N૊ͷηοτΛ࡞Γ
 LossΛܭࢉ • ֶशʹ༗ޮͳϖΞΛ୳͢/࡞Δ • ͞Βʹߟྀ͢ΔΫϥε਺Λ֦ு • ۭؒΛ௒ٿ໘্ʹ੍໿ • ڑ཭ࢦඪΛ֯౓Ͱ • ୅ද఺ͰMetric Learning • Ϋϥε෼ྨܕωοτϫʔΫߏ଄
51. Contrastive / Triplet Loss͔Β೿ੜ→੝Γ্͕͖ͬͯͨ • ֶशʹ༗ޮͳϖΞΛ୳͢/࡞Δ • ͞Βʹߟྀ͢ΔΫϥε਺Λ֦ு • ۭؒΛ௒ٿ໘্ʹ੍໿ • ڑ཭ࢦඪΛ֯౓Ͱ • ୅ද఺ͰMetric Learning • Ϋϥε෼ྨܕωοτϫʔΫߏ଄ !51 “L2-constrained Softmax Loss for Discriminative Face Verification”(arXiv 2017/3) ໌ࣔతͳڑ཭࠷దԽ͸͠ͳ͍͕ ಛ௃ྔϕΫτϧΛϊϧϜͰׂͬͯ ಛ௃ྔ࣍ݩۭؒͷٿ໘্ʹ ಺ੵ(cosྨࣅ౓)Ͱදݱྗͷௐ੔ ֶशͷ೉қ౓΋εέʔϦϯάύϥϝʔλαͰௐ੔
52. Contrastive / Triplet Loss͔Β೿ੜ→੝Γ্͕͖ͬͯͨ !52 “Deep Metric Learning with Angular Loss”(ICCV’17) anchorɺpositiveɺ negativeΛಈ͔͢ ޲͖(ޯ഑)Λ ֯౓͔Βߟ͑Δ • ֶशʹ༗ޮͳϖΞΛ୳͢/࡞Δ • ͞Βʹߟྀ͢ΔΫϥε਺Λ֦ு • ۭؒΛ௒ٿ໘্ʹ੍໿ • ڑ཭ࢦඪΛ֯౓Ͱ • ୅ද఺ͰMetric Learning • Ϋϥε෼ྨܕωοτϫʔΫߏ଄
53. Contrastive / Triplet Loss͔Β೿ੜ→੝Γ্͕͖ͬͯͨ !53 • ֶशʹ༗ޮͳϖΞΛ୳͢/࡞Δ • ͞Βʹߟྀ͢ΔΫϥε਺Λ֦ு • ۭؒΛ௒ٿ໘্ʹ੍໿ • ڑ཭ࢦඪΛ֯౓Ͱ • ୅ද఺ͰMetric Learning • Ϋϥε෼ྨܕωοτϫʔΫߏ଄ “No Fuss Distance Metric Learning using proxies”(ICCV’17) positiveͱnegativeʹ ݸผͷσʔλ఺Ͱ͸ͳ͘ ֤Ϋϥεͷ୅ද஋ͷ ຒΊࠐΈϕΫτϧ(proxy) Λ༻͍Δ ѹ౗తऩଋͷ଎͞ ຊ೔ͷ࿦จͷϥΠόϧ
54. Contrastive / Triplet Loss͔Β೿ੜ→੝Γ্͕͖ͬͯͨ !54 • ֶशʹ༗ޮͳϖΞΛ୳͢/࡞Δ • ͞Βʹߟྀ͢ΔΫϥε਺Λ֦ு • ۭؒΛ௒ٿ໘্ʹ੍໿ • ڑ཭ࢦඪΛ֯౓Ͱ • ୅ද఺ͰMetric Learning • Ϋϥε෼ྨܕωοτϫʔΫߏ଄ “ArcFace: Additive Angular Margin Loss for Deep Face Recognition”(CVPR’19) ಛ௃ྔϕΫτϧͷޙஈʹશ݁߹૚Λ઀ଓ Softmaxؔ਺ͱCross Entropy LossͰ Metric LearningΛߦ͏(ਪ࿦࣌͸શ݁߹૚Λ֎͢) ຊ೔ͷ࿦จͷϥΠόϧ
 Ͱ͸ͳ͍͕ඞಡʂ
55. ͓·͚: CVPR’20ͰMetric Learningͷ࿩ͯ͠Δ࿦จ(ͷҰ෦) !55 •Deep Metric Learning via Adaptive Learnable Assessment •End-to-End Illuminant Estimation Based on Deep Metric Learning •Cross-Batch Memory for Embedding Learning •Optimizing Rank-Based Metrics With Blackbox Differentiation •Universal Weighting Metric Learning for Cross-Modal Matching •Fast(er) Reconstruction of Shredded Text Documents via
 Self-Supervised Deep Asymmetric Metric Learning •Moving in the Right Direction: A Regularization for Deep Metric Learning •CurricularFace: Adaptive Curriculum Learning Loss for Deep Face Recognition •Circle Loss: A Unified Perspective of Pair Similarity Optimization
56. ౳ʑ… ͋͘·Ͱ͘͝Ұ෦ͷ࿩͔͍ͭܰ঺հͰ͢ͷͰ ͍Ζ͍Ζ୳ͨ͠Γࢼͨ͠Γͯ͠Έ͍ͯͩ͘͞ʂ (࠶ܝ) !56
57. ࠓ೔ͷΰʔϧ(࠶ܝ) !57 ͷ͓ؾ࣋ͪΛཧղ͢Δ
58. ࠓ೔ͷΰʔϧ(࠶ܝ) !58 Triplet Loss • anchor: ͋Δσʔλ఺ • positive: anchorͱಉ͡Ϋϥεʹ ଐ͢Δผͷσʔλ఺ • negative: anchorɺpositiveͱ͸ ҟͳΔΫϥεͷσʔλ఺ ↓ • anchorͱpositive͕͖ۙͮɺ anchorͱnegative͕཭ΕΔLoss • σʔλͷ૊Έ߹Θͤ૯਺ଟ͗͢ • ֶशʹޮ͘ϖΞʹ౰ͨΓʹ͍͘
59. ࠓ೔ͷΰʔϧ(࠶ܝ) !59 N-pair Loss • anchor: ͋Δσʔλ఺ • positive: anchorͱಉ͡Ϋϥεʹ ଐ͢Δผͷσʔλ఺ • negatives: anchorɺpositiveͰ ͳ͍Ϋϥεͷσʔλ఺(1ͭͣͭ) • ۙ͞Λߟྀ ↓ • TripletΑΓ΋Ұ౓ʹશମΛݟΕΔ • ·ͩσʔλશମͱ͸ݴ͑ͳ͍
 (ϛχόονͷதͷ͞ΒʹҰ෦)
60. ࠓ೔ͷΰʔϧ(࠶ܝ) !60 Lifted Structure Loss • anchor: ͋Δσʔλ఺ • positive: anchorͱಉ͡Ϋϥεʹ ଐ͢Δผͷσʔλ఺ • negatives: anchorɺpositiveͰ ͳ͍Ϋϥεͷશσʔλ఺ ↓ • N-pairΑΓ΋Ұ౓ʹશମΛݟΕΔ • ·ͩσʔλશମͱ͸ݴ͑ͳ͍
 (anchorΫϥεͷҰ෦σʔλ΍ϛ χόον಺ͷΈͰݶఆత)
61. ࠓ೔ͷΰʔϧ(࠶ܝ) !61 Proxy-NCA Loss • anchor: ͋Δσʔλ఺ • proxy: ֤Ϋϥεʹ͖ͭ1ͭ͋Δ ʮ୅ද఺ʯ ↓ • anchorͱશproxyͰ
 N-pair Lossతͳֶश • ऩଋ͕ΊͪΌͪ͘Ό଎͍ • σʔλશମͷۭؒత৘ใ͸
 ׆͔͖͠Εͯͳ͍
62. ࠓ೔ͷΰʔϧ(࠶ܝ) !62 ࢒͢͸ຊ೔ͷ࿦จ Proxy Anchor Loss ͷΈ
63. ࠓ೔ͷΰʔϧ(࠶ܝ) !63 Proxy Anchor Loss • anchor: ͋ΔΫϥεͷproxy • positive: anchorͷproxyʹଐ͢ ΔΫϥεͷϛχόον಺શσʔλ ఺ • negative: anchorͱҟͳΔΫϥε ͷϛχόον಺શσʔλ఺ ↓ • ऩଋ͕͞ΒʹΊͪΌͪ͘Ό଎͍ • σʔλશମͷۭؒత৘ใΛ׆͔͠ ·͍ͬͯ͘Δ
64. ࠓ೔ͷΰʔϧ(࠶ܝ) !64 ͍͟ղઆ΁
65. ͓͠ͳ͕͖ • Metric Learningͬͯʁ • CVʹ͓͚ΔMetric LearningͷྲྀΕ • ຊ೔ͷ࿦จ঺հ !65
66. ఏҊͯ͠Δ಺༰: Proxy Anchor Loss !66
67. ఏҊͯ͠Δ಺༰: Proxy Anchor Loss α: εέʔϦϯάύϥϝʔλ δ: Ϛʔδϯ ϛχόον಺ͷσʔλ શΫϥε෼ͷproxyͨͪͱ ͦͷݸ਺ proxy pͷΫϥεʹଐ͢Δ ϛχόον಺ͷશembedding (σʔλಛ௃ྔ)ϕΫτϧ embeddingϕΫτϧxͱ ରԠ͢ΔΫϥεͷproxyϕΫτϧpͷ ྨࣅ౓(ຊ࿦จͰ͸cosྨࣅ౓) શproxyͱ ͦͷݸ਺(=Ϋϥε਺) proxy pͷΫϥεʹଐ͞ͳ͍ ϛχόον಺ͷશembedding (σʔλಛ௃ྔ)ϕΫτϧ
68. ఏҊͯ͠Δ಺༰: Proxy Anchor Loss α: εέʔϦϯάύϥϝʔλ δ: Ϛʔδϯ ϛχόον಺ͷσʔλ શΫϥε෼ͷproxyͨͪͱ ͦͷݸ਺ proxy pͷΫϥεʹଐ͢Δ શembedding(σʔλಛ௃ྔ) ϕΫτϧ embeddingϕΫτϧxͱ ରԠ͢ΔΫϥεͷproxyϕΫτϧpͷ ྨࣅ౓(ຊ࿦จͰ͸cosྨࣅ౓) શproxyͱ ͦͷݸ਺(=Ϋϥε਺) proxy pͷΫϥεʹଐ͞ͳ͍ શembedding(σʔλಛ௃ྔ)ϕΫτϧ ʁ
69. Proxy Anchor Lossͷ͓ؾ࣋ͪΛཧղ͍ͨ͠ !69
70. Proxy Anchor Lossͷ͓ؾ࣋ͪΛཧղ͍ͨ͠ !70 “easier-to-interpret form”
71. “easier-to-interpret form” Proxy Anchor Lossͷ͓ؾ࣋ͪΛཧղ͍ͨ͠ !71 “smooth approximation of ReLU” ͭ·Γඍ෼Մೳ࿈ଓͳmax(0, z)
72. “easier-to-interpret form” Proxy Anchor Lossͷ͓ؾ࣋ͪΛཧղ͍ͨ͠ !72 “smooth approximation of max function” LogSumExp
73. “easier-to-interpret form” Proxy Anchor Lossͷ͓ؾ࣋ͪ !73 ͋Δproxy pͱhardest positive (p͔ΒҰ൪ԕ͍positive)͕ۙͮ͘ ͋Δproxy pͱhardest negative (p͔ΒҰ൪͍ۙnegative)͕཭ΕΔ “smooth approximation of max function”
74. Proxy Anchor Lossͷਅͷ͓ؾ࣋ͪ !74
75. Proxy Anchor Lossͷਅͷ͓ؾ࣋ͪ !75 proxyΛanchor(ج४)ʹͯ͠ positive΋negative΋ શͯͷྨࣅ౓ݟͯΔɺͷਤ
76. Proxy Anchor Lossͷਅͷ͓ؾ࣋ͪ !76 anchorͱ֤positiveͱͷྨࣅ౓͕ ߴ͍΄Ͳখ͘͞ͳͬͯخ͍߲͠ proxyΛanchor(ج४)ʹͯ͠ positive΋negative΋ શͯͷྨࣅ౓ݟͯΔɺͷਤ anchorͱ֤negativeͱͷྨࣅ౓͕ ௿͍΄Ͳখ͘͞ͳͬͯخ͍߲͠
77. !77 ͲΜͳͱ͜Ζ͕ྑ͘ͳͬͯΔʁ
78. ࿦จᐌ͘1: ੑೳྑͯ͘ऩଋ଎͍ʂ !78
79. ࿦จᐌ͘1: ੑೳྑͯ͘ऩଋ଎͍ʂ !79 Recall@1είΞ͕
 Ұ൪ߴ͍ʂ ऩଋ͕଎͍ʂ
80. ࿦จᐌ͘1: ੑೳྑͯ͘ऩଋ଎͍ʂ !80 σʔλͷۭؒతߏ଄΍෼෍Λ
 Α͘ଊֶ͑ͨश͕Ͱ͖͍ͯΔ Recall@1είΞ͕
 Ұ൪ߴ͍ʂ ऩଋ͕଎͍ʂ
81. ࿦จᐌ͘1: ੑೳྑͯ͘ऩଋ଎͍ʂ !81 ୅ද఺Λ࢖͍ͳ͕Β σʔλશମ΋ݟΕ͍ͯΔ Recall@1είΞ͕
 Ұ൪ߴ͍ʂ ऩଋ͕଎͍ʂ
82. ࿦จᐌ͘1: ੑೳྑͯ͘ऩଋ଎͍ʂ !82 Recall@1είΞ͕
 Ұ൪ߴ͍ʂ ऩଋ͕଎͍ʂ Recall@KείΞ • Retrieval(ݕࡧ)λεΫ΍Metric LearningͷੑೳΛଌΔͷʹΑ͘࢖ΘΕΔ • ֶशͨ͠ม׵(ωοτϫʔΫ)ʹςετσʔλΛશͯೖྗͯ͠embeddingԽͨ࣌͠ʹ
 ֤σʔλ఺ʹ͍ͭͯɺͦͷۭؒ಺Ͱ͍ۙॱϥϯΩϯάTop-KҐͷதʹ
 ಉ͡Ϋϥεͷσʔλ఺͕͋ΔΑ͏ͳ఺ͷׂ߹ • ྫ͑͹Recall@2ͳΒɺRecall@1(࠷ۙ๣఺͕ಉ͡ΫϥεͰ͋ΔΑ͏ͳ఺ͷׂ߹)ʹ
 ʮ࠷دΓ͸ผΫϥεͷσʔλ఺͕ͩ2൪໨ʹ͍ۙ఺ͳΒಉ͡ΫϥεʯͰ͋ΔϠπΒΛ଍ͨ͠
 είΞͱͳΔ • ʮະ஌ΫϥεΛ͖ͪΜͱ෼཭Ͱ͖Δ͔ʯΛଌΔͨΊɺσʔληοτΛ2෼ׂͯ͠
 ֶशͱςετΛ׬શʹผͷΫϥεͰߦ͏(ͳ͔ͳ͔ϋʔυ)
83. ࿦จᐌ͘2: ྨࣅ౓ʹԠͯ͡஋͕ͪΌΜͱมΘΔʂ !83 Proxy-NCA Loss Proxy Anchor Loss positive proxyʹ͍ͭͯ มԽ͕ݻఆ શσʔλ఺ʹ͍ͭͯ hard͞ʹԠֶͨ͡श
84. ࿦จᐌ͘3: ૊Έ߹Θͤ਺͕(طଘݚڀͷதͰ)࠷খʂ !84 M: ֶशσʔλͷ૯਺ C: ֶशσʔλͷΫϥε਺ B: 1epochͷόον਺ U: 1Ϋϥε͋ͨΓͷproxy਺
85. Ϋϥε਺C << σʔλ૯਺M ࿦จᐌ͘3: ૊Έ߹Θͤ਺͕(طଘݚڀͷதͰ)࠷খʂ !85 M: ֶशσʔλͷ૯਺ C: ֶशσʔλͷΫϥε਺ B: 1epochͷόον਺ U: 1Ϋϥε͋ͨΓͷproxy਺ ϙΠϯτ
86. ࿦จᐌ͘3: ૊Έ߹Θͤ਺͕(طଘݚڀͷதͰ)࠷খʂ !86 ૊Έ߹Θͤ૯਺͕ଟ͗ͯ͢
 ֶशʹ༗ޮͳσʔλϖΞΛ Ҿ͖౰ͯΔͷʹ͔͔࣌ؒΔ Ϋϥε਺C << σʔλ૯਺M M: ֶशσʔλͷ૯਺ C: ֶशσʔλͷΫϥε਺ B: 1epochͷόον਺ U: 1Ϋϥε͋ͨΓͷproxy਺ ϙΠϯτ
87. !87 ࣮ݧͨ͠ΒSOTAͳΜͰ͠ΐ…ʁ
88. ࣮ݧ݁Ռ: SOTAʂ !88
89. ࣮ݧ݁Ռ: SOTAʂ !89 ௗ ं
90. ࣮ݧ݁Ռ: SOTAʂ !90 ಛ௃ྔͷ ࣍ݩ਺Λ ͪΌΜͱ ߹Θͤͯ ൺֱʂ (݁ߏޮ͍ͯ͘Δ)
91. ࣮ݧ݁Ռ: SOTAʂ !91 σΧ͍ը૾࢖ͬͯΔͷʹ΋߹ΘͤͯΔʂ(݁ߏޮ͍ͯ͘Δ)
92. ࣮ݧ݁Ռ: SOTAʂ !92 backbone networkهࡌͯ͠Δ͠߹ΘͤͯΔʂ(݁ߏޮ͍ͯ͘Δ)
93. ࣮ݧ݁Ռ: SOTAʂ !93 ஸೡʂ
94. ࣮ݧ݁Ռ: SOTAʂ !94 Stanford Online Products (ΊͪΌͪ͘Ό͍ΖΜͳ෺) In-shop(ϑΝογϣϯΞΠςϜ)
95. ࣮ݧ݁Ռ: SOTAʂ !95 ஸೡʂ ஸೡʂ
96. !96 ͚ͩͰ͸ͳ͘…ʂ
97. ΫΤϦը૾ͷྨࣅ౓ݕࡧ݁ՌTop4 !97 CUB-200 SOP In-shop Cars-196 ΋͸΍ ؒҧͬͯΔϠπ ਓ͕΍ͬͯ΋ Ϝζ͍͡ΌΜ…
98. ϋΠύϥߟ࡯ͦͷ1: όοναΠζ !98 ʮ1όον಺ͷσʔλ਺͕ଟ͍ํ͕
 σʔλશମͷߏ଄ΛΑΓଊ͑ΒΕΔʯ ҙ֎ͱ ͔ࠩͭͳ͍ͳ…
99. ϋΠύϥߟ࡯ͦͷ2: ಛ௃ྔۭؒͷ࣍ݩ਺ !99 ʮେ͖͍ํ͕΋ͪΖΜੑೳྑ͍ ͕ ݕࡧ଎౓ͱ͔͸τϨʔυΦϑʯ σΧ͗͢Δͱ ੑೳԼ͕ΔΜͩ…
100. ϋΠύϥߟ࡯ͦͷ3: εέʔϦϯάύϥϝʔλαͱϚʔδϯδ !100 ʮεέʔϦϯάύϥϝʔλ͸ 16Ҏ্Ͱ҆ఆͯ͠Δɺ ͕ͦͬͪे෼ͳେ͖ͩ͞ͱ ϚʔδϯͰ͸͋Μ·Γ͕ࠩग़ͳ͍ʯ εέʔϦϯά ύϥϝʔλ ࢮ׆໰୊͡ΌΜ…
101. ϋΠύϥߟ࡯ͦͷ3: εέʔϦϯάύϥϝʔλαͱϚʔδϯδ !101 ʮεέʔϦϯάύϥϝʔλ͸ 16Ҏ্Ͱ҆ఆͯ͠Δɺ ͕ͦͬͪे෼ͳେ͖ͩ͞ͱ ϚʔδϯͰ͸͋Μ·Γ͕ࠩग़ͳ͍ʯ εέʔϦϯά ύϥϝʔλ ࢮ׆໰୊͡ΌΜ… • ෳ਺ͷೖྗಉ࢜ͷʮࠩΛڧௐ͢Δ౓߹͍ʯ • cosྨࣅ౓Λ࢖͏Metric LearningͰΑ͘ಋೖ͞ΕΔ
 (ͦ΋ͦ΋ͷ஋Ҭ͕-1 ≦ cosx ≦ 1ͳͷͰ͕͖ࠩͭʹ͍ͨ͘Ί) • DistillationΛఏҊͨ͠࿦จͰొ৔ͨ͠softmaxԹ౓ύϥϝʔλΛٯ਺ʹͨ͠΋ͷ ※͍͔ͭ͘ͷ࿦จΛݟ͖ͯͨײ͡20~60͙Β͍͕ଥ౰ͳΠϝʔδͩͬͨͷͰೲಘ εέʔϦϯάύϥϝʔλ “Distilling the Knowledge in a Neural Network”(NIPS’14 Workshop)
102. ͞ΒʹarXiv൛Ͱ͸… !102
103. Appendix1: backbone networkͱը૾αΠζ !103 ʮ࣮ݧ͸Inception-BNͰ΍͚ͬͨͲResNetͩͱ΋ͬͱੌ͍ɺ GPUϝϞϦ࣍ୈͰը૾͸σΧ͍ํ͕ྑ͍ʯ ͦΓΌͦ͏ͩ…
104. Appendix2: ΫΤϦը૾ͷྨࣅ౓ݕࡧ݁Ռ(vs Proxy-NCA) !104 CUB-200 SOP In-shop Cars-196
105. Appendix3: ςετσʔλͷ෼෍ͲΜͳײ͡ʁ(t-SNEՄࢹԽ) !105 CUB-200 SOP In-shop Cars-196
106. ͞Βʹ͞Βʹ… !106
107. ͔ͳΓ៉ྷͳஶऀPyTorch࣮૷ʂ !107 https://github.com/tjddus9597/Proxy-Anchor-CVPR2020 • LossΫϥε͔Βtrain/testίʔυͷ࣮૷͸΋ͪΖΜ • σʔληοτ4छͷDatasetΫϥε࣮૷΋͋Γ • backbone networkͨͪͷ࣮૷΋͋Γ • طଘݚڀͷLossͨͪ΋ࢼͤΔΑ͏ʹͳͬͯΔ
 (Pytorch-Metric-Learningύοέʔδ͕ඞཁ) ※͜ͷൃද·Ͱʹಈ͔͓͖͔ͯͨͬͨ͠ͷͰ͕࣌ؒ͢଍Γͣʀʀ
108. ౳ʑ Deep Metric Learning in CVͷݱঢ়͕ ͱͯ΋ྑ͍ײ͡ʹ·ͱ·͍ͬͯΔҹ৅Ͱ͢ ͜ͷࢿྉยखʹ͝ҰಡΛʂ !108
109. ·ͱΊ !109 •Metric Learning(ಛʹCV෼໺)ͷ࿩ •ຊ೔ͷ࿦จ: Proxy Anchor Loss ͋Γ͕ͱ͏͍͟͝·ͨ͠ʂ
110. !110