Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Turing × atmaCup #18 - 1st Place Solution
Search
Shuhei Goda
December 13, 2024
Technology
0
680
Turing × atmaCup #18 - 1st Place Solution
Turing × atmaCup #18 の表彰式での登壇資料です
https://turing.connpass.com/event/338583/
Shuhei Goda
December 13, 2024
Tweet
Share
More Decks by Shuhei Goda
See All by Shuhei Goda
ジョブマッチングサービスにおける相互推薦システムの応用事例と課題
hakubishin3
3
800
とある事業会社にとっての Kaggler の魅力
hakubishin3
8
2.1k
課題の解像度が荒かったことで意図した改善ができなかった話
hakubishin3
3
950
Wantedly におけるマッチング体験を最大化させるための推薦システム
hakubishin3
4
1.1k
Recommendation Industry Talks #1 Opening
hakubishin3
1
340
会社訪問アプリ「Wantedly Visit」での シゴトに関する興味選択機能と推薦改善
hakubishin3
0
580
論文紹介: Improving Implicit Feedback-Based Recommendation through Multi-Behavior Alignment(Xin Xin et al., 2023)
hakubishin3
0
560
Feedback Prize - English Language Learning における擬似ラベルの品質向上の取り組み
hakubishin3
0
920
ウォンテッドリーにおける推薦システムのオフライン評価の仕組み
hakubishin3
7
6.6k
Other Decks in Technology
See All in Technology
20240522 - 躍遷創作理念 @ PicCollage Workshop
dpys
0
300
OPENLOGI Company Profile for engineer
hr01
1
17k
Storage Browser for Amazon S3を触ってみた + α
miura55
0
100
The key to VCP-VCF
mirie_sd
0
150
[JAWS-UG新潟#20] re:Invent2024 -CloudOperationsアップデートについて-
shintaro_fukatsu
0
150
Agentic RAG with LangGraph
atsushii
0
120
深層学習と3Dキャプチャ・3Dモデル生成(土木学会応用力学委員会 応用数理・AIセミナー)
pfn
PRO
0
390
ソフトウェア開発における「パーフェクトな意思決定」/Perfect Decision-Making in Software Development
yayoi_dd
2
2.6k
ヤプリQA課題の見える化
gu3
0
140
The future we create with our own MVV
matsukurou
0
1.1k
I could be Wrong!! - Learning from Agile Experts
kawaguti
PRO
8
2k
大規模言語モデルとそのソフトウェア開発に向けた応用 (2024年版)
kazato
2
430
Featured
See All Featured
Six Lessons from altMBA
skipperchong
27
3.5k
The Language of Interfaces
destraynor
155
24k
Improving Core Web Vitals using Speculation Rules API
sergeychernyshev
2
150
It's Worth the Effort
3n
183
28k
Principles of Awesome APIs and How to Build Them.
keavy
126
17k
Building Adaptive Systems
keathley
38
2.3k
Fantastic passwords and where to find them - at NoRuKo
philnash
50
2.9k
Sharpening the Axe: The Primacy of Toolmaking
bcantrill
38
1.9k
The Success of Rails: Ensuring Growth for the Next 100 Years
eileencodes
44
6.9k
Dealing with People You Can't Stand - Big Design 2015
cassininazir
365
25k
How to Think Like a Performance Engineer
csswizardry
22
1.3k
The Art of Programming - Codeland 2020
erikaheidi
53
13k
Transcript
© 2024 Wantedly, Inc. 1st Place Solution Dec. 14 2024
- Shuhei Goda Turing × atmaCup #18 දজࣜ&ৼΓฦΓձ
© 2024 Wantedly, Inc. ໊લɿ ߹ా पฏ Shuhei Goda
ॴଐͱׂɿ ΥϯςουϦʔגࣜձࣾ ɾData Team Manager ɾMachine Learning Tech Lead ɾProduct Manager Kaggle Tierɿ Kaggle Competitions Grandmaster @jy_msc ࣗݾհ https://www.kaggle.com/shuheigoda
© 2024 Wantedly, Inc. Turing × atmaCup #18 ʹ͍ͭͯ •
։࠵ظؒɿ2024/11/15 17:30 ʙ 2024/11/24 18:00 • ࣗಈंͷߦγʔϯͷΧϝϥը૾ं྆ͷঢ়ଶσʔλͳͲ͔Βɺ0.5 ~ 3s ޙͷ ࣗंͷҐஔΛਪఆ͢ΔλεΫʢي༧ଌʣ
© 2024 Wantedly, Inc. ࠓճͷ Public LeaderBoard : 1st Place
🎉 Private LeaderBoard : 1st Place 🎉
© 2024 Wantedly, Inc. λΠϜϥΠϯʢ11/19͔ΒࢀՃʣ
© 2024 Wantedly, Inc. ϓϩηεʢͬ͘͟Γͱʣ 1.ੳͱํܾΊ
© 2024 Wantedly, Inc. ϓϩηεʢͬ͘͟Γͱʣ 2. 1st-stageͷվળ
© 2024 Wantedly, Inc. ϓϩηεʢͬ͘͟Γͱʣ 3. 2nd-stageͷվળ
© 2024 Wantedly, Inc. ϓϩηεʢͬ͘͟Γͱʣ 4. Ξϯαϯϒϧͷ४උͱ࣮ࢪ
© 2024 Wantedly, Inc. 1. ੳͱํܾΊ
© 2024 Wantedly, Inc. 1. ੳͱํܾΊ EDAϕʔεϥΠϯΞϓϩʔν͔ΒɺΛѲͯ͠ํΛߟ͑Δ • σʔλଟ͘ͳ͘ɺγϯϓϧͳ͕Βςʔϒϧಛྔ͕ڧͦ͏ •
ͱ͍ͬͯɺ༧ଌʹ͓͍ͯը૾ͷใ༗ޮͦ͏ʹݟ͑Δʢৄࡉޙड़ʣ → ଞࢀՃऀͱͷେ͖ͳࠩҟͱͳΓಘΔͷʮը૾ใͷѻ͍ํʯͩͱߟ͑ͨ
© 2024 Wantedly, Inc. 1. ੳͱํܾΊ ਐΊํͱ࣌ؒͷ͍ํΛܾΊΔ • 1st-stage Ϟσϧʢը૾Λѻ͏ϞσϧʣͷվળΛॏతʹΔ
• 2nd-stage ϞσϧʢςʔϒϧಛྔϝΠϯͷϞσϧʣͷվળΛগ͠Δ • ͋ͱϞσϧΛՔ͙࡞ۀΛߦ͍Ξϯαϯϒϧ͢Δ
© 2024 Wantedly, Inc. 1. ੳͱํܾΊ ࣮ݧʹ༻ͨ͠ϕʔεϥΠϯ • tk@tnkcoder ͞Μ͕ެ։ͨ͠ϕʔεϥΠϯϞσϧΛར༻
• [CV 0.2008/LB 0.2017] LightGBM + CNN stacking baseline (LightGBM + CNN) 1st-stage: CNN 2nd-stage: GBDT Image Tabular 1st-stage Predictions Submission
© 2024 Wantedly, Inc. 2. 1st-stageͷվળ
© 2024 Wantedly, Inc. 2-1. Target ͷվળ ֤ Target ͷ࠷େͰׂͬͨ
Target Λֶशɾ༧ଌ͢Δ • ݁ՌɿX ͱ Y ͷ༧ଌੑೳ͕վળɻt ͕খ͍͞΄ͲޮՌ͕େ͖͍ • ղऍɿ༧ଌ࣌Ͱλʔήοτͷεέʔϧ͕େ͖͘ҟͳΔɻεέʔϧΛ߹ΘͤࠐΉ͜ͱͰɺ ֤༧ଌ࣌ͷใΛ·ͱΊͯޮՌతʹֶशͰ͖ΔͷͰͳ͍͔
© 2024 Wantedly, Inc. 2-1. Target ͷվળ ֤༧ଌ࣌ͷՃΛ Auxiliary Target
ͱֶͯ͠शɾ༧ଌ͢Δ • ྫ͑ Target ͷ x_0, x_1 ͔Β vx_1 Λࢉग़͢Δ͜ͱ͕Ͱ͖Δ • ݁Ռͱͯ͠ɺ1st-stage CV: 0.2312 → 0.2288 (-0.0024) ʹվળ • ·ͨɺAuxiliary Target ʹର͢Δ༧ଌΛޙஈͷಛྔͱͯ͠Ճ͢Δ͜ͱͰ ɺ2nd-stage ͷείΞ͕վળʢ2nd-stage CV: 0.1963 → 0.1933ʣ
© 2024 Wantedly, Inc. 2-2. HorizontalFlip ࢥͬͨ͜ͱɿࣗಈं͔ΒࡱӨ͞Εͨը૾ɺਫฏసͤͯ͞ҧײ͕গͳ͍ Ͳ͕ͬͪΦϦδφϧʁ
© 2024 Wantedly, Inc. 2-2. HorizontalFlip ֶश࣌ɾਪ࣌ʹ HorizontalFlip ΛՃ͑Δ •
ֶश࣌ɿp=0.5 Ͱ HorizontalFlip • ਪ࣌ɿΦϦδφϧը૾ͷਪ݁Ռͱਫฏసͨ͠ਪ݁ՌΛฏۉ͢Δ • ͜ΕΒʹΑͬͯείΞ৳ͼΔ͕ɺ1st-stage Ϟσϧʹೖྗ͢Δςʔϒϧಛྔ ͷసΛΕͯ͠·͏ͱείΞ͕ٯʹԼ͕ͬͯ͠·͏ͷͰҙ • 1st-stageͰѻ͏ςʔϒϧಛྔۃྗγϯϓϧʹ͑Δඞཁ͕͋ΔɻΘ Γʹ 2nd-stage ʹෳࡶͳFEΛدͤΔ͜ͱ͕Ͱ͖Δ {“steeringAngleDeg”: 15, “leftBlinker”: True, “rightBlinker”: False} → {“steeringAngleDeg”: -15, “leftBlinker”: False, “rightBlinker”: True}
© 2024 Wantedly, Inc. 2-3. Scene୯Ґ ࢥͬͨ͜ͱɿಉҰγʔϯͷલޙͷࢹ֮తใΛ͏͜ͱͰɺ୯ҰID(t-1.0 ~ t)͚ͩͩ ͱࠔͳ༧ଌͰ͖ΔΑ͏ʹͳΔͷͰʁΑΓظతͳӡసঢ়گͷѲ͕ॏཁ
ྫ1ɿ sec=2.0, t-0.5 sec=2.0, t-1.0 sec=2.0 sec=12.0 12secޙͷใ͔Βɺͦͷ··ਐ͢Ε ྑ͔ͬͨ͜ͱ͕Θ͔Δ
© 2024 Wantedly, Inc. 2-3. Scene୯Ґ ࢥͬͨ͜ͱɿಉҰγʔϯͷલޙͷࢹ֮తใΛ͏͜ͱͰɺ୯ҰID(t-1.0 ~ t)͚ͩͩ ͱࠔͳ༧ଌͰ͖ΔΑ͏ʹͳΔͷͰʁΑΓظతͳӡసঢ়گͷѲ͕ॏཁ
ྫ2ɿ sec=2.0 sec=12.0 12secޙͷใ͔ΒɺࣼΊʹਐΊ ྑ͔ͬͨ͜ͱ͕Θ͔Δ
© 2024 Wantedly, Inc. 2-3. Scene୯Ґ ظͷมԽʢ1ඵ୯Ґʣ ɾٸͳૢ࡞มԽ ɾՃݮ ɾंઢมߋ
ɾӈࠨં ظͷมԽʢ୯Ґʣ ɾߦత ɾӡసελΠϧ ɾతͷܦ࿏
© 2024 Wantedly, Inc. 2-3. Scene୯Ґ ϘτϧωοΫͱͳ͍ͬͯΔʮظมԽʯΛޮՌతʹϞσϧԽ͢Δʹʁ • ϕʔεϥΠϯͰ2nd-stageʹ͓͍ͯલޙͷظใΛߟྀͨ͠༧ଌ͕Մೳͩ ͕ɺ1st-stageʹ͓͍֤ͯID͕ಠཱͳͷͱͯ͠ѻ͏&ܦ࿏༧ଌͷ݁Ռͱͯ͠ͷ
ใΛൖͤ͞ΔܗʹͳΔͷͰඇޮʹݟ͑Δ • 1st-stage ͷNNͷஈ֊ͰɺظͷมԽʹجͮ͘ΛֶशͰ͖ΔΑ͏ʹ͢Δ 1st-stage: CNN 2nd-stage: GBDT sceneA,ID1 1st-stage Predictions 1st-stage: CNN 1st-stage Predictions FE(e.g. shift features) sceneA,ID2 Shared
© 2024 Wantedly, Inc. 2-3. Scene୯Ґ ۩ମతͳΞϓϩʔνɿScene୯ҐͰ 2.5D-CNN + LSTM
CNN 1st-stage Predictions (B×S×N) BiLSTM … Tabular sec=20 MLP sec=120 Scene
© 2024 Wantedly, Inc. 2-3. Scene୯Ґ sec=20 sec=120 Pad Pad
Pad Pad sec=220 sec=320 sec=520 Pad Pad Pad sec=20 sec=120 sec=220 sec=320 sec=420 sec=520 scene=A scene=B scene=C όονͷ࡞Γํ • Scene͝ͱʹ͕͞ҟͳΔͷͰɺ٧ΊͯPadding • αϯϓϧؒͰ࣌ܥྻతͳҐஔ͕ؔҟͳΔͷͰɺscene_sec scene_num ʢsceneͷத ͰԿ൪ʹొͨ͠ID͔ʣΛಛྔͱͯ͠ೖྗ
© 2024 Wantedly, Inc. 2-3. Scene୯Ґ ۩ମతͳΞϓϩʔνɿ2.5D-CNN + LSTM Λ࠾༻͢Δ
• ͜ͷΞʔΩςΫνϟʹมߋ͢Δ͜ͱͰɺCNN୯ମͰ Private 4Ґ૬ͷείΞʹ ྫ1ɿ ྫ2ɿ
© 2024 Wantedly, Inc. 3. 2nd-stageͷվળ
© 2024 Wantedly, Inc. ͍͔ͭ͘ͷಛྔͷՃ ͍ͣΕͦͦ͜͜ͷվળʹد༩ͨ͠ • 1st stage ͷ
target (x_0 ~ z_5) ͷ༧ଌʹՃ͑ͯɺaux target ͷ༧ଌಛྔ ͱͯ͠ར༻͢Δ • 2छྨͷं྆ϞσϧʢϢχαΠΫϧϞσϧͱಈྗֶతόΠγΫϧϞσϧʣͷ༧ଌ ݁ՌΛಛྔͱͯ͠ར༻͢Δ
© 2024 Wantedly, Inc. 4. Ξϯαϯϒϧ
© 2024 Wantedly, Inc. Ξϯαϯϒϧ ༷ʑͳόοΫϘʔϯͰϞσϧΛ࡞ͬͯ Weighted Average • جຊతʹϞσϧΛ૿͢΄ͲείΞ͕େ͖͘৳ͼΔɻ࠷ऴతʹ11ݸࠞͥͨɻ
• ͬͨόοΫϘʔϯɿresnext, efficientnet, resnet, swin-transformer ͳͲ
© 2024 Wantedly, Inc. ࠷ऴ݁Ռ
© 2024 Wantedly, Inc. ֤ϞσϧͷύϑΥʔϚϯε model cv public private private
ॱҐ ɹsingle 1st stage 0.1906 0.1958 0.1808 4Ґ ɹsingle 2nd stage 0.1883 0.1928 0.1785 1Ґ ɹensemble 0.1792 0.1885 0.1754 1Ґ
© 2024 Wantedly, Inc. ϕʔεϥΠϯʹൺͯ͏·͍͘͘Α͏ʹͳͬͨྫ - ظతͳঢ়گѲ͕ޮ͍͍ͯΔ าಓʹಥͬࠐ ·ͳ͘ͳͬͨ
นʹಥͬࠐ ·ͳ͘ͳͬͨ ରंઢʹ ৵ೖ͠ͳ͘ͳͬͨ ΨʔυϨʔϧ ʹಥͬࠐ·ͳ ͘ͳͬͨ
© 2024 Wantedly, Inc. ૬มΘΒͣ͏·͍͔͘ͳ͍ྫ - ͦͷʹ͓͚Δঢ়گѲ͕ͳ͔ͳ͔͍͠ ࠨ͔Β ं͕ग़͖ͯͨ
τϥοΫͰ ৴߸͕ݟ͑ͳ͍ ETCϨʔϯ ঃߦ͠ͳ͍ͱ ͍͚ͳ͍ ԣஅาಓۙ͘ʹ ௨ߦਓ͍ͳ͍
© 2024 Wantedly, Inc. ຊίϯϖʹର͢ΔऔΓΈํʹ͍ͭͯ
© 2024 Wantedly, Inc. എܠ ࢠͲ͕ੜ·Ε͔ͯΒɺॳΊͯͷσʔλੳίϯϖͷࢀՃ ύύKagglerʹͳΓ·ͨ͠
© 2024 Wantedly, Inc. എܠ ͔ͤͬ͘ࢀՃ͢ΔͳΒPrizeݍʹೖΓ͍ͨ… Ͱ • ͕ͬͭΓίʔυΛॻ͚Δͷɺൺֱత͘৸ͯ͘ΕΔਂͷΈ •
։࠵ظؒͷલͱޙՈఉͷ༻ࣄͰ1த͕࣌ؒऔΕͳ͍ ͋Μ·Γ࣌ؒऔΕͳ͍ɺͲ͏͠Α͏
© 2024 Wantedly, Inc. Ͳ͏औΓΉ͖͔ Do everything
© 2024 Wantedly, Inc. Ͳ͏औΓΉ͖͔ Do everything Δ͜ͱɾΒͳ͍ ͜ͱΛܾΊΔ
© 2024 Wantedly, Inc. ελϯε Δ͜ͱ • ڝ૪༏ҐͱͳΔٕज़՝ʢղܾ͖͍͢ʣΛਪఆ͠ɺͦΕʹṌ͚ͯऔΓΉ • ֎ΕͨΒૉʹఘΊΔɺΘΜͪΌΜϗʔϜϥϯͶΒ͍
Βͳ͍͜ͱ • ࡉ͔͍վળɺϋΠύϥνϡʔχϯάͳͲ • ܭࢉϦιʔε͕ۭ͍͍ͯͯɺͳΜͱͳ͘Ͱ࣮ݧΛճ͞ͳ͍Α͏ʹ͢Δ
© 2024 Wantedly, Inc. Ͳ͏͍͏՝Λղ͖͔͘Λઃఆ͢Δ Ͳ͏͍͏͍ʢnot Ξϓϩʔνʣ͕ࠩผԽϙΠϯτʹͳΔͷ͔ߟ͑Δ • ΞΠσΞΛεϙοτతʹݕূ͢ΔΑΓɺूத͢Δ͖՝Λઃఆͯ͠ਂ΅ͬ ͨ΄͏͕ɺదͳΞϓϩʔνʹͨͲΓண͖͍͢
ੳޙʹઃఆͨ͠՝ QɿલޙͷγʔϯͷมԽظͷΛ֫ಘ͢Δͷʹ༗ޮ͔ʁ Qɿӡస࣌ͷঢ়گ༧ଌʹͲͷΑ͏ʹӨڹ͢Δͷ͔ʁʢྫ͑ߴಓ࿏ͩͱʁʣ
© 2024 Wantedly, Inc. ੜAIػೳͰ࣮ݧεϐʔυΛૣ͘͢Δ ࣮ݧαΠΫϧ͕ैདྷͷ1/2~1/3ͷ࣌ؒͰճͤΔΑ͏ʹ • ࣮ݧͷઃܭ͔ΒݕূʢσόοάʣʹࢸΔ·Ͱͷ࣌ؒͷେ෯ͳॖ • ΊΜͲ͘͞…
ͱ͍͏৺ཧతϋʔυϧΛେ෯ʹԼ͛Δʢਖ਼͜Ε͕େ͖͍ʣ • ྫ͑ɺID୯Ґ→Scene୯ҐͷมߋɺมߋՕॴ͕ଟͯ͘ਏ͍ • Ͳ͏ઃܭ͢Δ͖͔ɺͲ͏͍͏มߋՕॴ͕͋Δ͔ɺͲ͏࣮͢Δ͔Λ͑ ͯΒ͏ɻͦͯ͠ίέͨΒσόοάͷࡐྉΛΒ͏ ΞΠσΞͷ ݕ౼ ઃܭ ࣮ ݕূ