Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
NIPS2017reading_3Dreconstruction
Search
望月紅葉さんと幸せな家庭を築きたい
January 27, 2018
Research
0
1.5k
NIPS2017reading_3Dreconstruction
望月紅葉さんと幸せな家庭を築きたい
January 27, 2018
Tweet
Share
More Decks by 望月紅葉さんと幸せな家庭を築きたい
See All by 望月紅葉さんと幸せな家庭を築きたい
shadow-detection-with-conditional-generative-adversarial-networks
momijifullmoon
0
150
unsupervised-learning-of-depth-and-ego-motion-from-monocular-video-using-3d-geometric-constraints
momijifullmoon
0
450
ABEJA Innovation Meetup NIPS PointNet++
momijifullmoon
1
490
Other Decks in Research
See All in Research
心理言語学の視点から再考する言語モデルの学習過程
chemical_tree
2
640
アニメにおける宇宙猫ミームとその表現
yttrium173340
0
100
財務諸表監査のための逐次検定
masakat0
0
150
問いを起点に、社会と共鳴する知を育む場へ
matsumoto_r
PRO
0
670
20250624_熊本経済同友会6月例会講演
trafficbrain
1
710
AIスパコン「さくらONE」のLLM学習ベンチマークによる性能評価 / SAKURAONE LLM Training Benchmarking
yuukit
2
750
J-RAGBench: 日本語RAGにおける Generator評価ベンチマークの構築
koki_itai
0
840
Integrating Static Optimization and Dynamic Nature in JavaScript (GPCE 2025)
tadd
0
110
機械学習と数理最適化の融合 (MOAI) による革新
mickey_kubo
1
400
国際論文を出そう!ICRA / IROS / RA-L への論文投稿の心構えとノウハウ / RSJ2025 Luncheon Seminar
koide3
10
5.7k
不確実性下における目的と手段の統合的探索に向けた連続腕バンディットの応用 / iot70_gp_rff_mab
monochromegane
2
200
EOGS: Gaussian Splatting for Efficient Satellite Image Photogrammetry
satai
4
710
Featured
See All Featured
Dealing with People You Can't Stand - Big Design 2015
cassininazir
367
27k
The Pragmatic Product Professional
lauravandoore
36
7k
Product Roadmaps are Hard
iamctodd
PRO
55
11k
Design and Strategy: How to Deal with People Who Don’t "Get" Design
morganepeng
132
19k
Easily Structure & Communicate Ideas using Wireframe
afnizarnur
194
16k
Understanding Cognitive Biases in Performance Measurement
bluesmoon
31
2.7k
Designing Dashboards & Data Visualisations in Web Apps
destraynor
231
53k
Why You Should Never Use an ORM
jnunemaker
PRO
59
9.6k
10 Git Anti Patterns You Should be Aware of
lemiorhan
PRO
658
61k
Java REST API Framework Comparison - PWX 2021
mraible
34
8.9k
Keith and Marios Guide to Fast Websites
keithpitt
411
23k
Building Better People: How to give real-time feedback that sticks.
wjessup
369
20k
Transcript
̏࣍ݩ෮ݩʹؔͯ͠ Learning a Multi-View Stereo Machine NIPS2017จಡΈձˏΫοΫύου 1 ಛʹදه͕ͳ͍ݶΓɺҎԼͷࢿྉ͔ΒҾ༻ https://arxiv.org/pdf/1708.05375.pdf
Learning a Multi-View Stereo Machine ▸ චऀ • Abhishek Kar,
Christian Häne, Jitendra Malik ʢUC Berkeley) ▸ ֓ཁ • Multi View StereoʢMVSʣʹΑΔີͳ3࣍ݩ෮ݩΛDeep LearningͰEnd2Endʹֶश • MVSΛ”ֶशͰ͖Δ”ͷͰແ͍͔ͱ͍͏ٙʹ͑Δ 2
എܠ ▸ Multi View Stereoͱ 1. ಛநग़ 2. Ϛονϯά 3.
̏࣍ݩ෮ݩ 4. Τϥʔͷআڈ 3
എܠ ▸ Multi View Stereoͱ 1. ಛநग़ 2. Ϛονϯά 3.
̏࣍ݩ෮ݩ 4. Τϥʔͷআڈ ==> DeepԿͰશͯղܾͰ͖ͦ͏ 4
എܠ ▸ Multi View Stereoͱ 1. ಛநग़ɹ← CNNͰ͍͚Δ 2. Ϛονϯά
3. ̏࣍ݩ෮ݩ 4. Τϥʔͷআڈ 5
എܠ ▸ Multi View Stereoͱ 1. ಛநग़ 2. Ϛονϯάɹ← CNNͱRNNͰ͍͚Δ
3. ̏࣍ݩ෮ݩ 4. Τϥʔͷআڈ 6
എܠ ▸ Multi View Stereoͱ 1. ಛநग़ 2. Ϛονϯά 3.
̏࣍ݩ෮ݩɹ← DeconvͰ͍͚Δ 4. Τϥʔͷআڈ 7
എܠ ▸ Multi View Stereoͱ 1. ಛநग़ 2. Ϛονϯά 3.
̏࣍ݩ෮ݩ 4. Τϥʔͷআڈɹ← Encoder-DecoderͰ͍͚Δ 8
DeepԿͰࡾ࣍ݩ෮ݩ ▸ 3DR2N2(ECCV2016) • ෳը૾ΛΤϯίʔυ͠ɺLSTMͰϚονϯά 9 http://3d-r2n2.stanford.edu
DeepԿͰࡾ࣍ݩ෮ݩ ▸ 3D Shape Reconstruction by Modeling 2.5D Sketch (NIPS2017)
• ϦΞϧͷը૾͔Β2.5DͷεέονΛى͜͠ɺ2.5DεέονΛͱʹ 3DshapeਪఆΛEnd2EndֶशͰ͢Δ 10 https://arxiv.org/pdf/1711.03129.pdf
͢༰ ▸ શମ૾ ▸ ख๏ ▸ ࣮ݧ ▸ ·ͱΊ 11
શମ૾ 12 http://bair.berkeley.edu/blog/2017/09/05/unified-3d/
શମ૾ 13 Learnt Stereo Machines
ख๏ ▸ Image Encoder • Encoder-DecoderܕʢU-netʣͷઃܭ • Ϛονϯάʹ༻͍Δ̎DͷಛϚοϓ࡞ • ࣍ݩ2DnಛϚο
14
ख๏ ▸ Unplojection ▸ 2࣍ݩͷಛϚοϓ3࣍ݩͷຊདྷ͋Δ͖ಛϚοϓ͔ΒࣹӨ ▸ 3࣍ݩάϦουʹٯࣹӨ 15 http://bair.berkeley.edu/blog/2017/09/05/unified-3d/
ख๏ ▸ Unplojection ▸ 2࣍ݩͷಛϚοϓ3࣍ݩͷຊདྷ͋Δ͖ಛϚοϓ͔ΒࣹӨ ▸ 3࣍ݩάϦουʹٯࣹӨ 16 http://bair.berkeley.edu/blog/2017/09/05/unified-3d/
ख๏ ▸ Unplohection ▸ 2࣍ݩͷಛϚοϓ3࣍ݩͷຊདྷ͋Δ͖ಛϚοϓ͔ΒࣹӨ ▸ 3࣍ݩάϦουʹٯࣹӨ 17 http://bair.berkeley.edu/blog/2017/09/05/unified-3d/
ख๏ ▸ Unplohection ▸ 2࣍ݩͷಛϚοϓ3࣍ݩͷຊདྷ͋Δ͖ಛϚοϓ͔ΒࣹӨ ▸ 3࣍ݩάϦουʹٯࣹӨ 18 http://bair.berkeley.edu/blog/2017/09/05/unified-3d/
ख๏ ▸ Recurrent Grid Fusion • 3࣍ݩͷಛϚοϓͷϚονϯάΛGated Recurrent Unit(GRU)Ͱ •
GRUʹ͍࣋ͬͯͨ͘Ίɺ3D convolutionΛ༻ • ͜ͷաఔ͕MVSͷܭࢉϚονϯάΛ୲ • ֶशͷࡍը૾ͷೖྗॱΛϥϯμϜʹೖΕସ͑Δ 19
ख๏ ▸ 3D Grid Reasoning • GRUͰ̏࣍ݩάϦουʹͨ͠ΒϊΠζ͕ଟ͔ͬͨɻ • 3U-netͰEncode Decode͢ΔͱFilteringͰ͖Δ
20
ख๏ ▸ Differentiable Projection • Depthͷ෮ݩʹL1 loss(high frequency informationͷͨΊ) •
Voxelͷ෮ݩʹvoxel͝ͱͷcross entropy loss 21
࣮ݧ ▸ σʔληοτ • ShapeNetσʔλΛར༻ • ̏࣍ݩCADϞσϧͷެ։σʔληοτ 22 https://shapenet.cs.stanford.edu/shrec17/
࣮ݧ • ೖྗը૾ ▸ ShapeNetͷ3DϞσϧΛϨϯμϦϯάͯ͠224x224x3 ▸ ̍ࢹ͋ͨΓ̐ຕ ▸ Χϝϥϙʔζ •
Ξτϓοτ ▸ Depth: 224x224x3 ▸ Voxel: 32x32x32 23
࣮ݧ ▸ ݁Ռ 24 3DR2N2ͱൺɺࡉ͔͍෮ݩ͕Մೳ
࣮ݧ ▸ ݁Ռ 25 3DR2N2ͱൺɺগͳ͍ຕͰ෮ݩ͕Մೳ ຕ૿͑Δͱੑೳ্͕͕Δ
࣮ݧ ▸ ݁Ռ 26 stereo matchingͰ෮ݩ͠ͳ͍ ૭෮ݩՄೳ
࣮ݧ ▸ ݁Ռ 27 stereo matchingʹൺ গͳ͍ຕͰ෮ݩ͕Մೳ චऀᐌ͘ CNNͷίϯςΫετΛݟΔྗ ैདྷͷstereo
matchingΛ͙྇ DepthMapͷਪఆ݁ՌΛෳΈ߹Θͤͯ̏࣍ݩ෮ݩͨ͠
·ͱΊ ▸ Learnt Stereo MachinesΛఏҊ ▸ ෳࢹ͔Βͷೖྗը૾Λݩʹɺ DepthMapͱVoxelͷਪఆ͕Մೳͱͳͬͨ ▸ ՝
• ग़ྗVoxel͕32x32x32ͱখ͍͞ 28