Turing × atmaCup #18 - 1st Place Solution

Turing × atmaCup #18 の表彰式での登壇資料です
https://turing.connpass.com/event/338583/

Transcript

1. © 2024 Wantedly, Inc. 1st Place Solution Dec. 14 2024 - Shuhei Goda Turing × atmaCup #18 දজࣜ&ৼΓฦΓձ
2. © 2024 Wantedly, Inc. ໊લɿ 
 ߹ా पฏ Shuhei Goda ॴଐͱ໾ׂɿ 
 ΢ΥϯςουϦʔגࣜձࣾ 
 ɾData Team Manager 
 ɾMachine Learning Tech Lead 
 ɾProduct Manager 
 
 Kaggle Tierɿ 
 Kaggle Competitions Grandmaster @jy_msc ࣗݾ঺հ https://www.kaggle.com/shuheigoda
3. © 2024 Wantedly, Inc. Turing × atmaCup #18 ʹ͍ͭͯ • ։࠵ظؒɿ2024/11/15 17:30 ʙ 2024/11/24 18:00 • ࣗಈंͷ૸ߦγʔϯͷΧϝϥը૾΍ं྆ͷঢ়ଶσʔλͳͲ͔Βɺ0.5 ~ 3s ޙͷ ࣗंͷҐஔΛਪఆ͢ΔλεΫʢي੻༧ଌʣ
4. © 2024 Wantedly, Inc. ࠓճͷ੒੷ Public LeaderBoard : 1st Place 🎉 Private LeaderBoard : 1st Place 🎉
5. © 2024 Wantedly, Inc. λΠϜϥΠϯʢ11/19͔ΒࢀՃʣ
6. © 2024 Wantedly, Inc. ϓϩηεʢͬ͘͟Γͱʣ 1.෼ੳͱํ਑ܾΊ
7. © 2024 Wantedly, Inc. ϓϩηεʢͬ͘͟Γͱʣ 2. 1st-stageͷվળ
8. © 2024 Wantedly, Inc. ϓϩηεʢͬ͘͟Γͱʣ 3. 2nd-stageͷվળ
9. © 2024 Wantedly, Inc. ϓϩηεʢͬ͘͟Γͱʣ 4. Ξϯαϯϒϧͷ४උͱ࣮ࢪ
10. © 2024 Wantedly, Inc. 1. ෼ੳͱํ਑ܾΊ
11. © 2024 Wantedly, Inc. 1. ෼ੳͱํ਑ܾΊ EDA΍ϕʔεϥΠϯΞϓϩʔν͔Βɺ܏޲Λ೺Ѳͯ͠ํ਑Λߟ͑Δ • σʔλ਺͸ଟ͘ͳ͘ɺγϯϓϧͳ͕Βςʔϒϧಛ௃ྔ͕ڧͦ͏ • ͱ͸͍ͬͯ΋ɺ༧ଌʹ͓͍ͯը૾ͷ৘ใ͸༗ޮͦ͏ʹݟ͑Δʢৄࡉ͸ޙड़ʣ → ଞࢀՃऀͱͷେ͖ͳࠩҟͱͳΓಘΔͷ͸ʮը૾৘ใͷѻ͍ํʯͩͱߟ͑ͨ
12. © 2024 Wantedly, Inc. 1. ෼ੳͱํ਑ܾΊ ਐΊํͱ࣌ؒͷ࢖͍ํΛܾΊΔ • 1st-stage Ϟσϧʢը૾Λѻ͏ϞσϧʣͷվળΛॏ఺తʹ΍Δ • 2nd-stage Ϟσϧʢςʔϒϧಛ௃ྔϝΠϯͷϞσϧʣͷվળΛগ͠΍Δ • ͋ͱ͸Ϟσϧ਺ΛՔ͙࡞ۀΛߦ͍Ξϯαϯϒϧ͢Δ
13. © 2024 Wantedly, Inc. 1. ෼ੳͱํ਑ܾΊ ࣮ݧʹ࢖༻ͨ͠ϕʔεϥΠϯ • tk@tnkcoder ͞Μ͕ެ։ͨ͠ϕʔεϥΠϯϞσϧΛར༻ • [CV 0.2008/LB 0.2017] LightGBM + CNN stacking baseline (LightGBM + CNN) 1st-stage: 
 CNN 2nd-stage: GBDT Image Tabular 1st-stage Predictions Submission
14. © 2024 Wantedly, Inc. 2. 1st-stageͷվળ
15. © 2024 Wantedly, Inc. 2-1. Target ͷվળ ֤ Target ͷ࠷େ஋Ͱׂͬͨ Target Λֶशɾ༧ଌ͢Δ • ݁ՌɿX ͱ Y ͷ༧ଌੑೳ͕վળɻt ͕খ͍͞΄ͲޮՌ͕େ͖͍܏޲ • ղऍɿ༧ଌ࣌఺Ͱλʔήοτͷεέʔϧ͕େ͖͘ҟͳΔɻεέʔϧΛ߹ΘͤࠐΉ͜ͱͰɺ ֤༧ଌ࣌఺ͷ৘ใΛ·ͱΊͯޮՌతʹֶशͰ͖ΔͷͰ͸ͳ͍͔
16. © 2024 Wantedly, Inc. 2-1. Target ͷվળ ֤༧ଌ࣌఺ͷ଎౓΍Ճ଎౓Λ Auxiliary Target ͱֶͯ͠शɾ༧ଌ͢Δ • ྫ͑͹ Target ͷ x_0, x_1 ͔Β vx_1 Λࢉग़͢Δ͜ͱ͕Ͱ͖Δ • ݁Ռͱͯ͠ɺ1st-stage CV: 0.2312 → 0.2288 (-0.0024) ʹվળ • ·ͨɺAuxiliary Target ʹର͢Δ༧ଌ஋Λޙஈͷಛ௃ྔͱͯ͠௥Ճ͢Δ͜ͱͰ ΋ɺ2nd-stage ͷείΞ͕վળʢ2nd-stage CV: 0.1963 → 0.1933ʣ
17. © 2024 Wantedly, Inc. 2-2. HorizontalFlip ࢥͬͨ͜ͱɿࣗಈं͔ΒࡱӨ͞Εͨը૾͸ɺਫฏ൓సͤͯ͞΋ҧ࿨ײ͕গͳ͍ Ͳ͕ͬͪΦϦδφϧʁ
18. © 2024 Wantedly, Inc. 2-2. HorizontalFlip ֶश࣌ɾਪ࿦࣌ʹ HorizontalFlip ΛՃ͑Δ • ֶश࣌ɿp=0.5 Ͱ HorizontalFlip • ਪ࿦࣌ɿΦϦδφϧը૾ͷਪ࿦݁Ռͱਫฏ൓సͨ͠ਪ࿦݁ՌΛฏۉ͢Δ • ͜ΕΒʹΑͬͯείΞ͸৳ͼΔ͕ɺ1st-stage Ϟσϧʹೖྗ͢Δςʔϒϧಛ௃ྔ ͷ൓సΛ๨Εͯ͠·͏ͱείΞ͕ٯʹԼ͕ͬͯ͠·͏ͷͰ஫ҙ • 1st-stageͰѻ͏ςʔϒϧಛ௃ྔ͸ۃྗγϯϓϧʹ཈͑Δඞཁ͕͋Δɻ୅Θ Γʹ 2nd-stage ʹෳࡶͳFEΛدͤΔ͜ͱ͕Ͱ͖Δ {“steeringAngleDeg”: 15, “leftBlinker”: True, “rightBlinker”: False} 
 → {“steeringAngleDeg”: -15, “leftBlinker”: False, “rightBlinker”: True}
19. © 2024 Wantedly, Inc. 2-3. Scene୯Ґ ࢥͬͨ͜ͱɿಉҰγʔϯͷલޙͷࢹ֮త৘ใΛ࢖͏͜ͱͰɺ୯ҰID(t-1.0 ~ t)͚ͩͩ ͱࠔ೉ͳ༧ଌ΋Ͱ͖ΔΑ͏ʹͳΔͷͰ͸ʁΑΓ௕ظతͳӡసঢ়گͷ೺Ѳ͕ॏཁ ྫ1ɿ sec=2.0, t-0.5 sec=2.0, t-1.0 sec=2.0 sec=12.0 12secޙͷ৘ใ͔Βɺͦͷ··௚ਐ͢Ε͹ 
 ྑ͔ͬͨ͜ͱ͕Θ͔Δ
20. © 2024 Wantedly, Inc. 2-3. Scene୯Ґ ࢥͬͨ͜ͱɿಉҰγʔϯͷલޙͷࢹ֮త৘ใΛ࢖͏͜ͱͰɺ୯ҰID(t-1.0 ~ t)͚ͩͩ ͱࠔ೉ͳ༧ଌ΋Ͱ͖ΔΑ͏ʹͳΔͷͰ͸ʁΑΓ௕ظతͳӡసঢ়گͷ೺Ѳ͕ॏཁ ྫ2ɿ sec=2.0 sec=12.0 12secޙͷ৘ใ͔ΒɺࣼΊʹਐΊ͹ 
 ྑ͔ͬͨ͜ͱ͕Θ͔Δ
21. © 2024 Wantedly, Inc. 2-3. Scene୯Ґ ୹ظͷมԽʢ1ඵ୯Ґʣ ɾٸͳૢ࡞มԽ ɾՃݮ଎ ɾंઢมߋ ɾӈࠨં ௕ظͷมԽʢ෼୯Ґʣ ɾ૸ߦ໨త ɾӡసελΠϧ ɾ໨త஍΁ͷܦ࿏
22. © 2024 Wantedly, Inc. 2-3. Scene୯Ґ ϘτϧωοΫͱͳ͍ͬͯΔʮ௕ظมԽʯΛޮՌతʹϞσϧԽ͢Δʹ͸ʁ • ϕʔεϥΠϯͰ͸2nd-stageʹ͓͍ͯલޙͷ௕ظ৘ใΛߟྀͨ͠༧ଌ͕Մೳͩ ͕ɺ1st-stageʹ͓͍ͯ͸֤ID͕ಠཱͳ΋ͷͱͯ͠ѻ͏&ܦ࿏༧ଌͷ݁Ռͱͯ͠ͷ ৘ใΛ఻ൖͤ͞ΔܗʹͳΔͷͰඇޮ཰ʹݟ͑Δ • 1st-stage ͷNNͷஈ֊Ͱɺ௕ظͷมԽʹجͮ͘܏޲ΛֶशͰ͖ΔΑ͏ʹ͢Δ 1st-stage: 
 CNN 2nd-stage: GBDT sceneA,ID1 1st-stage Predictions 1st-stage: 
 CNN 1st-stage Predictions FE(e.g. shift features) sceneA,ID2 Shared
23. © 2024 Wantedly, Inc. 2-3. Scene୯Ґ ۩ମతͳΞϓϩʔνɿScene୯ҐͰ 2.5D-CNN + LSTM CNN 1st-stage Predictions (B×S×N) BiLSTM … Tabular sec=20 MLP sec=120 Scene
24. © 2024 Wantedly, Inc. 2-3. Scene୯Ґ sec=20 sec=120 Pad Pad Pad Pad sec=220 sec=320 sec=520 Pad Pad Pad sec=20 sec=120 sec=220 sec=320 sec=420 sec=520 scene=A scene=B scene=C όονͷ࡞Γํ • Scene͝ͱʹ௕͕͞ҟͳΔͷͰɺ٧ΊͯPadding • αϯϓϧؒͰ࣌ܥྻతͳҐஔؔ܎͕ҟͳΔͷͰɺscene_sec ΍ scene_num ʢsceneͷத ͰԿ൪໨ʹొ৔ͨ͠ID͔ʣΛಛ௃ྔͱͯ͠ೖྗ
25. © 2024 Wantedly, Inc. 2-3. Scene୯Ґ ۩ମతͳΞϓϩʔνɿ2.5D-CNN + LSTM Λ࠾༻͢Δ • ͜ͷΞʔΩςΫνϟʹมߋ͢Δ͜ͱͰɺCNN୯ମͰ Private 4Ґ૬౰ͷείΞʹ ྫ1ɿ ྫ2ɿ
26. © 2024 Wantedly, Inc. 3. 2nd-stageͷվળ
27. © 2024 Wantedly, Inc. ͍͔ͭ͘ͷಛ௃ྔͷ௥Ճ ͍ͣΕ΋ͦͦ͜͜ͷվળʹد༩ͨ͠ • 1st stage ͷ target (x_0 ~ z_5) ͷ༧ଌ஋ʹՃ͑ͯɺaux target ͷ༧ଌ஋΋ಛ௃ྔ ͱͯ͠ར༻͢Δ • 2छྨͷं྆ϞσϧʢϢχαΠΫϧϞσϧͱಈྗֶతόΠγΫϧϞσϧʣͷ༧ଌ ݁ՌΛಛ௃ྔͱͯ͠ར༻͢Δ
28. © 2024 Wantedly, Inc. 4. Ξϯαϯϒϧ
29. © 2024 Wantedly, Inc. Ξϯαϯϒϧ ༷ʑͳόοΫϘʔϯͰϞσϧΛ࡞ͬͯ Weighted Average • جຊతʹϞσϧΛ૿΍͢΄ͲείΞ͕େ͖͘৳ͼΔɻ࠷ऴతʹ͸11ݸࠞͥͨɻ • ࢖ͬͨόοΫϘʔϯɿresnext, efficientnet, resnet, swin-transformer ͳͲ
30. © 2024 Wantedly, Inc. ࠷ऴ݁Ռ
31. © 2024 Wantedly, Inc. ֤ϞσϧͷύϑΥʔϚϯε model cv public private private ॱҐ ɹsingle 1st stage 0.1906 0.1958 0.1808 4Ґ ɹsingle 2nd stage 0.1883 0.1928 0.1785 1Ґ ɹensemble 0.1792 0.1885 0.1754 1Ґ
32. © 2024 Wantedly, Inc. ϕʔεϥΠϯʹൺ΂ͯ͏·͍͘͘Α͏ʹͳͬͨྫ - ௕ظతͳঢ়گ೺Ѳ͕ޮ͍͍ͯΔ าಓʹಥͬࠐ 
 ·ͳ͘ͳͬͨ นʹಥͬࠐ 
 ·ͳ͘ͳͬͨ ର޲ंઢʹ 
 ৵ೖ͠ͳ͘ͳͬͨ ΨʔυϨʔϧ 
 ʹಥͬࠐ·ͳ 
 ͘ͳͬͨ
33. © 2024 Wantedly, Inc. ૬มΘΒͣ͏·͍͔͘ͳ͍ྫ - ͦͷ৔ʹ͓͚Δঢ়گ೺Ѳ͕ͳ͔ͳ͔೉͍͠ ࠨ͔Β 
 ं͕ग़͖ͯͨ τϥοΫͰ 
 ৴߸͕ݟ͑ͳ͍ ETCϨʔϯ͸ 
 ঃߦ͠ͳ͍ͱ 
 ͍͚ͳ͍ ԣஅาಓۙ͘ʹ 
 ௨ߦਓ͸͍ͳ͍
34. © 2024 Wantedly, Inc. ຊίϯϖʹର͢ΔऔΓ૊Έํʹ͍ͭͯ
35. © 2024 Wantedly, Inc. എܠ ࢠͲ΋͕ੜ·Ε͔ͯΒɺॳΊͯͷσʔλ෼ੳίϯϖͷࢀՃ ύύKagglerʹͳΓ·ͨ͠
36. © 2024 Wantedly, Inc. എܠ ͔ͤͬ͘ࢀՃ͢ΔͳΒPrizeݍ಺ʹೖΓ͍ͨ… Ͱ΋ • ͕ͬͭΓίʔυΛॻ͚Δͷ͸ɺൺֱత௕͘৸ͯ͘ΕΔਂ໷ͷΈ • ։࠵ظؒͷલ൒ͱޙ൒͸Ոఉͷ༻ࣄͰ1೔த͕࣌ؒऔΕͳ͍ ͋Μ·Γ࣌ؒऔΕͳ͍ɺͲ͏͠Α͏
37. © 2024 Wantedly, Inc. Ͳ͏औΓ૊Ή΂͖͔ Do everything
38. © 2024 Wantedly, Inc. Ͳ͏औΓ૊Ή΂͖͔ Do everything ΍Δ͜ͱɾ΍Βͳ͍ ͜ͱΛܾΊΔ
39. © 2024 Wantedly, Inc. ελϯε ΍Δ͜ͱ • ڝ૪༏ҐͱͳΔٕज़՝୊ʢղܾ͢΂͖໰͍ʣΛਪఆ͠ɺͦΕʹṌ͚ͯऔΓ૊Ή • ֎ΕͨΒૉ௚ʹఘΊΔɺΘΜͪΌΜϗʔϜϥϯͶΒ͍ ΍Βͳ͍͜ͱ • ࡉ͔͍վળɺϋΠύϥνϡʔχϯάͳͲ • ܭࢉϦιʔε͕ۭ͍͍ͯͯ΋ɺͳΜͱͳ͘Ͱ࣮ݧΛճ͞ͳ͍Α͏ʹ͢Δ
40. © 2024 Wantedly, Inc. Ͳ͏͍͏՝୊Λղ͘΂͖͔Λઃఆ͢Δ Ͳ͏͍͏໰͍ʢnot Ξϓϩʔνʣ͕ࠩผԽϙΠϯτʹͳΔͷ͔ߟ͑Δ • ΞΠσΞΛεϙοτతʹݕূ͢ΔΑΓ΋ɺूத͢Δ΂͖՝୊Λઃఆͯ͠ਂ΅ͬ ͨ΄͏͕ɺద੾ͳΞϓϩʔνʹͨͲΓண͖΍͍͢ ෼ੳޙʹઃఆͨ͠՝୊ QɿલޙͷγʔϯͷมԽ͸௕ظͷ܏޲Λ֫ಘ͢Δͷʹ༗ޮ͔ʁ Qɿӡస࣌ͷঢ়گ͸༧ଌʹͲͷΑ͏ʹӨڹ͢Δͷ͔ʁʢྫ͑͹ߴ଎ಓ࿏ͩͱʁʣ
41. © 2024 Wantedly, Inc. ੜ੒AIػೳͰ࣮ݧεϐʔυΛૣ͘͢Δ ࣮ݧαΠΫϧ͕ैདྷͷ1/2~1/3ͷ࣌ؒͰճͤΔΑ͏ʹ • ࣮ݧͷઃܭ͔ΒݕূʢσόοάʣʹࢸΔ·Ͱͷ࣌ؒͷେ෯ͳ୹ॖ • ΊΜͲ͘͞… ͱ͍͏৺ཧతϋʔυϧΛେ෯ʹԼ͛Δʢਖ਼௚͜Ε͕େ͖͍ʣ • ྫ͑͹ɺID୯Ґ→Scene୯Ґ΁ͷมߋɺมߋՕॴ͕ଟͯ͘ਏ͍ • Ͳ͏ઃܭ͢Δ΂͖͔ɺͲ͏͍͏มߋՕॴ͕͋Δ͔ɺͲ͏࣮૷͢Δ͔Λ౴͑ ͯ΋Β͏ɻͦͯ͠ίέͨΒσόοάͷࡐྉΛ΋Β͏ ΞΠσΞͷ ݕ౼ ઃܭ ࣮૷ ݕূ