Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
画像ディープラーニングコンペの基本
Search
Jun Koda
August 07, 2025
1
1k
画像ディープラーニングコンペの基本
上位Kagglerに学ぶ~画像コンペの戦い方~
2025-08-07
Jun Koda
August 07, 2025
Tweet
Share
More Decks by Jun Koda
See All by Jun Koda
第3回関東kaggler会 🤔 妙だな... (Jun Koda)
junkoda
10
3k
Featured
See All Featured
Distributed Sagas: A Protocol for Coordinating Microservices
caitiem20
332
22k
Producing Creativity
orderedlist
PRO
346
40k
GitHub's CSS Performance
jonrohan
1031
460k
Writing Fast Ruby
sferik
628
62k
For a Future-Friendly Web
brad_frost
179
9.9k
The Psychology of Web Performance [Beyond Tellerrand 2023]
tammyeverts
48
2.9k
The Straight Up "How To Draw Better" Workshop
denniskardys
235
140k
Thoughts on Productivity
jonyablonski
69
4.8k
The Myth of the Modular Monolith - Day 2 Keynote - Rails World 2024
eileencodes
26
3k
I Don’t Have Time: Getting Over the Fear to Launch Your Podcast
jcasabona
33
2.4k
Site-Speed That Sticks
csswizardry
10
750
Fight the Zombie Pattern Library - RWD Summit 2016
marcelosomers
234
17k
Transcript
ը૾σΟʔϓϥʔχϯάίϯϖͷجຊ ্Ґ KagglerʹֶͿɹʙը૾ίϯϖͷઓ͍ํʙ 2025-08-07 Jun Koda ᅳాɹ३ʢίμɹδϡϯʣ https://hakuhodo-technologies.connpass.com/event/361499/
ཧ GMɻը૾ GM Ͱͳ͍ ⇧ ᠳΜͰ࡛ۄͷʮ࡛ۄʹॅΜͰΔͷʹ౦ژͬͯॻౕ͘ʯͰக໋ইΛෛͬͨ AI ʹΑͬͯ Notebook ͕ෆཁʹͳΔલʹ
Notebook GM ʹͳΖ͏͔ͳ ϙευΫͱͯ͠ӉཧΛ͖ͬͯͨ ࠓۀҕୗͰػցֶशݚڀ։ൃ
ඈߦػӢ segmentation. U-Net 70×1000 ͱ͍͏αΠζ͚ͩͲ U-Net ͓ೃછΈͷҩྍը૾εϥΠε ৴߸ॲཧཁૉ͋Γͷը૾ྨ ͡Ίͯͷը૾ίϯϖ ଟνϟϯωϧ͚ͩͲ2Dը૾ྨ
RTX2080Ti Λങͬͯઓ 3D ݂ segmentation ը૾ίϯϖ
G2Net (2021) ͷࢥ͍ग़ ϒϥοΫϗʔϧ߹ମ৴߸͕͋Δ͔Ͳ͏͔Λఆ͢Δ ࣌ܥྻ৴߸Λ spectrogram ͬΆ͘ը૾ʹ͢Δ “࠷େͷϒϨʔΫεϧʔ͕ learning rate
Λ্͛Δͩͬͨ” • Public notebook Λࣸܦ • ৴߸ॲཧΛ͢Δϝμϧݍ͔Β΄ Ͳԕ͍ • ৽͍͠ public notebook ΛΈΔ • lr 1e-5 → 1e-4 ͰείΞര্͕Γ
https://medium.com/@junkoda/kaggle-ॏྗͰۜϝμϧ-1c7135e69817 ࣌ͷϒϩάΑΓ খ͍͞ lr local minimum ʹϋϚΔ G2Net (2021)
ͷࢥ͍ग़
ɾɾɾ ɾɾɾ 0.005 େࠩ ↑ prize ↓ ݍ֎
Hyperparameter tuning Ͱ Kaggle Λউͭ͜ͱͰ͖ͳ͍ ͕ͩෛ͚Δ͜ͱͰ͖Δ
جຊతͳσΟʔϓϥʔχϯά܇࿅ͷ ίίϩॻ͍ͯ͋Δ learning rate warm up, batch size, ͳͲͳͲ খ͍͞
batch size over fi t Λ͙͚ͲɺͦͷͨΊʹখ͘͢͞Δͷअಓͳؾ͕͢Δ
https://www.kaggle.com/competitions/hms-harmful-brain-activity-classi fi cation/discussion/488083 େGrandmater ҙ֎ͱࡉ͔͘ௐͯ͠Δ
Grid search ʹ͢Δʁ Optunaʁ ͦΕͱ G • S • Dʁ
ͱ͜ΖͰ Hyperparameter optimization Ͳ͏ͯ͠·͔͢ʁ
Kaggle Ͱ graduate student ࢲͨͪࣗɻࢲखͰͪ·ͪ·ௐͯ͠Δ
Augmentation େࣄ
Albumentatations RandomRain: https://explore.albumentations.ai/transform/RandomRain ˚ ܇࿅σʔλʹͳ͍ʹରԠͰ͖ΔΑ͏ʹͳΔʢe.g. Ӎͷࣸਅʣ ˕ σʔλΛ૿ͯ͘͠܇࿅Ͱ͖ΔΑ͏ʹ͢Δ Augmentation ͷޮՌ
Augmentation ͍͢͝ͷਤ ඈߦػӢίϯϖ ࠷ॳͷ 10 epoch ͚ͩͩͱҧ͍গͳ͍ ͘܇࿅Ͱ͖Δͷ͕ϙΠϯτ
b d ͜Ε b HFlip ͜Ε d Augmentation Λ͍͚ͯ͠ͳ͍߹ ҰจࣈΞϧϑΝϕοτྨͰ
Horizontal fl ip Ϟσϧࠔ
ΠϯυਓΛӈʹ ϋϯυϧΛࠨʹ ߹ੑͷऔΕΔΑ͏ʹస - atmacup ंͷيಓ༧ଌ hakubisin ͞Μ 1st place
solution https://speakerdeck.com/hakubishin3/turing-x-atmacup-number-18-1st-place-solution ը૾ɾϋϯυϧ֯ɾΟϯΧʔͳͲΛҰ؏ͯ͠స͢Δ ※ ͜ΕͰं͕ӈଆΛΓͩͨ͠Βେมͳ߹μϝ
ରশੑΛճ෮ͤ͞Δ - ճసɾస͕͏·͍͔͘ͳͯ͘मਖ਼Ͱ͖Δ͜ͱ https://www.kaggle.com/competitions/waveform-inversion/writeups/ruby-14th-place-solution ίϯϖɾసͤ͞Δͱݯ͕̍ͭζϨΔ ͳΒɺసͤͯ͞1 pixel ͣΒ͍͍ͤ͡Όͳ͍ tascj ͞ΜͷඈߦػӢ
solution 0.5 pixel ͣΒ͢ ڥք͕ؾʹͳΔ͚ͲେৎΒ͍͠ ← సͯͣ͠Β͢ ճసɾస͕Ͱ͖ͳͯ͘ͻͱͰͰ͖Δ߹
σʔλ͕গͳ͍ͱ͖ Augmentation Λڧ͘ɺϞσϧখ͞Ί Theo Viel https://www.kaggle.com/competitions/rsna-2023-abdominal-trauma-detection/writeups/on-strike-2nd-place-solution Cut Mix Yun et
al (2019) https://arxiv.org/pdf/1905.04899 ϥϯμϜʹը૾ΛࠞͥΔ ϥϕϧ໘ੵൺ soft label ͦΜͳཚͳ! RSNA 2023 ೣ͕4ʹݘ͕6ʂ
Augmentation ڧ͍ Ճσʔλͬͱڧ͍ G2Net (2021) ѹత1Ґ https://www.kaggle.com/competitions/g2net-gravitational-wave-detection/writeups/kdl-top-1-solution-deep-learning-part (Geophysical Waveform
Inversion) https://www.kaggle.com/competitions/waveform-inversion/leaderboard 13TB Ҏ্ͷσʔλΛੜ On-the- fl y σʔλੜͷͨΊʹ CUDA Λॻ͍ͨ Augmentation ٖࣅతͳՃσʔλͳͷ͔ͩΒͦΕͦ͏ ࢲ: 660 GB ͷσʔλ͕େ͖ͯ͘ਏ͍ → ශऑ!
ը૾ίϯϖಓͳվળ͕ඞཁ • େ͖ͳΞΠσΟΞͰυϯͱείΞ͕Α͘ͳΔͷͰͳ͍ • ಉ͡Α͏ʹݟ͑ͯখ͞ͳվળͷੵΈॏͶ͕େ͖ͳࠩʹ զʑ RSNA ίϯϖϨδΣϯυͷΑ͏ʹ͍͔ͳ͍ ಓʹίπίπࢼߦࡨޡɾܦݧΛੵΉʁ ࡉ͔͍ςΫχοΫΛ
prize solution ίʔυΛಡΜͰձಘ͠Α͏ʢࢲͰ͖ͯͳ͍ʣ
ʮʓʓ͚ͨ͠Ͳޮ͔ͳ͔ͬͨʯҙຯ͕ͳ͍ • ͕݅ͦΖͬͯॳΊͯޮՌΛൃش͢Δ͜ͱ͕Α͋͘Δ • ྫ͑ը૾Λେ͖ͨ͘͠ͱ͖ɺϞσϧΛେ͖ͨ͘͠ͱ͖ɺ͘܇࿅ͨ͠ͱ͖ʹॳΊͯ ޮՌΛൃش͢Δ • ؆୯ͳྫͩͱ augmentation ͨ͠Β
epoch Λ૿͢ • ͦͷ··ͩͱɺ୯ʹσʔλ͕ྼԽ͚ͨͩ͠ tattaka ͞ΜʮҰࣺͯͨΞΠσΟΞʹҙࣝతʹཱͪฦΖ͏ͱߟ͑ͯΔʯ ؔ౦ Kaggler ձ 2025 य़ private communication ޙ͔Βߟ͑Δͱਖ਼ղͷۙ͘ΛԿ௨Γա͗ͯΔɺͱ͍͏͜ͱԿ͋Δ
Kenshin ͞Μ https://tech.preferred.jp/ja/blog/kaggle-contrails-3rd-place/ https://blog.knshnb.com/posts/journey-to-grandmaster/ ඈߦػӢίϯϖ ࢲΛؚΊͨଟ͘ͷਓʮ2.5D Ϟσϧ͏·͍͔͘ͳ͍ʯ ৴೦ͱࢼߦࡨޡʂ
G2Net Ͱҹʹͬͨ͜ͱ έϩοϐઌੜʮը૾Ͱ 0.88 ͑ΒΕͳ͍ɺ৴߸Λݟͳ͍ͱʯ ࣮ࡍɺ্Ґ solution ͦͷ௨Γͩͬͨ GMʹͳΔͱඍௐͰ͑ΒΕͳ͍ݶք͕ѲͰ͖ΔΒ͍͠ ͳͷͰ
Hyperparameter tuning ࡉ͔͍ௐ͖ͬͱྑ͍ܦݧ ͍Ζ͍Ζࢼߦࡨޡͯ͠ɺͲͷ͘Β͍·ͰվળͰ͖Δ͔Θ͔ΕҰਓલ ʢࢲΘ͔Βͳ͍ʣ
• ແ৬Λଓ͚Δ͓͕ۚ͏ͳ͍ • Grandmaster ʹͳΓ͍ͨ • ͓͕ۚͳ͍ͷͰϥετνϟϯε • Cloud GPU
ΛͬͯͰۚϝμϧ • ࠷ޙͷ1िؒʹ A100 Λ 4~5 • େ͖ͳϞσϧ → ࣦഊ • ը૾Λେ͖͘ • 256 pixel ͷೖྗը૾Λ 1024 ʹ Ryushi ͞Μ͔Βͷ͓ ্Ґೖ࣌ʹߟ͑ͨ͜ͱ ্Ґ..ೖ...? ͏͍͜ͱۚϝμϧऔͬͯͳ͍ͷͰΕ·ͨ͠ ը૾ΛͰ͔ͬ͘ 1024×1024 ඈߦػӢίϯϖ Google Research - Identify Contrails to Reduce Global Warming ৴߸͕ࡉ͍͔Βޮ͍ͨͷ͔ɻG2Net Ͱޮ͍ͨ
ίϯϐϡʔλࣄ ϩʔΧϧϚγϯ Ubuntu RTX 4090 (RAM 24 GB) Intel Core
i5 13500 (TDP 65W) ਫྫྷΛආ͚ۭͯྫྷʹ͢ΔͨΊ߇͑Ίʹ͚ͨ͠ͲɺίΞ͕ͨ͘͞Μ͋Δͱ͍͑ҙ֎ͱCPU͏ M.2 SSD WD Black 2TB (Gen 4 ~7000MB/s) σʔλಡΈࠐΈҙ֎ͱେࣄ ۚϝμϧ͕औΕͦ͏ͳΒ࠷ޙͷ1िؒʹ Cloud GPU Fractal Design "North"
औΕ·ͤΜͰͨ͠!!! Yale/UNC-CH - Geophysical Waveform Inversion
ۚϝμϧ͕औΕͦ͏ͳΒ࠷ޙͷ1िؒʹ Cloud GPU ʮۚϝμϧΛऔΔʯͳͲͱࢤ͕͍͔ΒۜϝμϧʹͳΔͷͰ͢ Cloud GPU ʹ͓ۚΛ͗ࠐΉͳΒ1ҐΛऔΓͳ͍͞
1Ґ͕औΕͳ͍ͳΒ Cloud GPU ʹ͓ۚΛ͗ࠐΉͷΊͳ͍͞ ରۮ ͦΜͳ͜ͱΑΓ prize solution Λࣸܦ͠Α͏ ϓϥΠεϨε
👍
ѱຐͷᅤ͖ ۚͰޙ͔Βิరͱ͔͍͏ ѱຐͷ༠ʹෛ͚ͳ͍ ๏Χ...ʢͦΕҎ্͍͚ͳ͍ʂ
30ສԁΛਓੜʹࢿʁ RTX 5070 (12G), 5070Ti (16G) 10ສԁલޙͷ GPU ͔Βελʔτ͢ΔͷΞϦͳؾ͕͢Δ
ࢲͷ͓͢͢ΊϩʔυόΠΫ ݈߁େࣄ ӡಈʹޮ͘ 10͑Δ 30ສԁΛਓੜʹࢿʁ RTX 5070 (12G), 5070Ti (16G)
10ສԁલޙͷ GPU ͔Βελʔτ͢ΔͷΞϦͳؾ͕͢Δ Shimano 105 ΛͬͯΔͭ