Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
NIPS2017reading_3Dreconstruction
Search
望月紅葉さんと幸せな家庭を築きたい
January 27, 2018
Research
0
1.5k
NIPS2017reading_3Dreconstruction
望月紅葉さんと幸せな家庭を築きたい
January 27, 2018
Tweet
Share
More Decks by 望月紅葉さんと幸せな家庭を築きたい
See All by 望月紅葉さんと幸せな家庭を築きたい
shadow-detection-with-conditional-generative-adversarial-networks
momijifullmoon
0
140
unsupervised-learning-of-depth-and-ego-motion-from-monocular-video-using-3d-geometric-constraints
momijifullmoon
0
380
ABEJA Innovation Meetup NIPS PointNet++
momijifullmoon
1
480
Other Decks in Research
See All in Research
The Fellowship of Trust in AI
tomzimmermann
0
130
文化が形作る音楽推薦の消費と、その逆
kuri8ive
0
160
20241115都市交通決起集会 趣旨説明・熊本事例紹介
trafficbrain
0
230
Matching 2D Images in 3D: Metric Relative Pose from Metric Correspondences
sgk
1
320
marukotenant01/tenant-20240826
marketing2024
0
510
いしかわ暮らしセミナー~移住にまつわるお金の話~
matyuda
0
150
Weekly AI Agents News! 7月号 論文のアーカイブ
masatoto
1
230
工学としてのSRE再訪 / Revisiting SRE as Engineering
yuukit
19
11k
ニュースメディアにおける事前学習済みモデルの可能性と課題 / IBIS2024
upura
3
510
20240820: Minimum Bayes Risk Decoding for High-Quality Text Generation Beyond High-Probability Text
de9uch1
0
120
論文読み会 SNLP2024 Instruction-tuned Language Models are Better Knowledge Learners. In: ACL 2024
s_mizuki_nlp
1
350
「並列化時代の乱数生成」
abap34
3
820
Featured
See All Featured
Designing the Hi-DPI Web
ddemaree
280
34k
The Success of Rails: Ensuring Growth for the Next 100 Years
eileencodes
44
6.8k
Visualization
eitanlees
145
15k
Raft: Consensus for Rubyists
vanstee
136
6.6k
5 minutes of I Can Smell Your CMS
philhawksworth
202
19k
Statistics for Hackers
jakevdp
796
220k
10 Git Anti Patterns You Should be Aware of
lemiorhan
654
59k
実際に使うSQLの書き方 徹底解説 / pgcon21j-tutorial
soudai
169
50k
The Web Performance Landscape in 2024 [PerfNow 2024]
tammyeverts
0
95
XXLCSS - How to scale CSS and keep your sanity
sugarenia
246
1.3M
How to train your dragon (web standard)
notwaldorf
88
5.7k
4 Signs Your Business is Dying
shpigford
180
21k
Transcript
̏࣍ݩ෮ݩʹؔͯ͠ Learning a Multi-View Stereo Machine NIPS2017จಡΈձˏΫοΫύου 1 ಛʹදه͕ͳ͍ݶΓɺҎԼͷࢿྉ͔ΒҾ༻ https://arxiv.org/pdf/1708.05375.pdf
Learning a Multi-View Stereo Machine ▸ චऀ • Abhishek Kar,
Christian Häne, Jitendra Malik ʢUC Berkeley) ▸ ֓ཁ • Multi View StereoʢMVSʣʹΑΔີͳ3࣍ݩ෮ݩΛDeep LearningͰEnd2Endʹֶश • MVSΛ”ֶशͰ͖Δ”ͷͰແ͍͔ͱ͍͏ٙʹ͑Δ 2
എܠ ▸ Multi View Stereoͱ 1. ಛநग़ 2. Ϛονϯά 3.
̏࣍ݩ෮ݩ 4. Τϥʔͷআڈ 3
എܠ ▸ Multi View Stereoͱ 1. ಛநग़ 2. Ϛονϯά 3.
̏࣍ݩ෮ݩ 4. Τϥʔͷআڈ ==> DeepԿͰશͯղܾͰ͖ͦ͏ 4
എܠ ▸ Multi View Stereoͱ 1. ಛநग़ɹ← CNNͰ͍͚Δ 2. Ϛονϯά
3. ̏࣍ݩ෮ݩ 4. Τϥʔͷআڈ 5
എܠ ▸ Multi View Stereoͱ 1. ಛநग़ 2. Ϛονϯάɹ← CNNͱRNNͰ͍͚Δ
3. ̏࣍ݩ෮ݩ 4. Τϥʔͷআڈ 6
എܠ ▸ Multi View Stereoͱ 1. ಛநग़ 2. Ϛονϯά 3.
̏࣍ݩ෮ݩɹ← DeconvͰ͍͚Δ 4. Τϥʔͷআڈ 7
എܠ ▸ Multi View Stereoͱ 1. ಛநग़ 2. Ϛονϯά 3.
̏࣍ݩ෮ݩ 4. Τϥʔͷআڈɹ← Encoder-DecoderͰ͍͚Δ 8
DeepԿͰࡾ࣍ݩ෮ݩ ▸ 3DR2N2(ECCV2016) • ෳը૾ΛΤϯίʔυ͠ɺLSTMͰϚονϯά 9 http://3d-r2n2.stanford.edu
DeepԿͰࡾ࣍ݩ෮ݩ ▸ 3D Shape Reconstruction by Modeling 2.5D Sketch (NIPS2017)
• ϦΞϧͷը૾͔Β2.5DͷεέονΛى͜͠ɺ2.5DεέονΛͱʹ 3DshapeਪఆΛEnd2EndֶशͰ͢Δ 10 https://arxiv.org/pdf/1711.03129.pdf
͢༰ ▸ શମ૾ ▸ ख๏ ▸ ࣮ݧ ▸ ·ͱΊ 11
શମ૾ 12 http://bair.berkeley.edu/blog/2017/09/05/unified-3d/
શମ૾ 13 Learnt Stereo Machines
ख๏ ▸ Image Encoder • Encoder-DecoderܕʢU-netʣͷઃܭ • Ϛονϯάʹ༻͍Δ̎DͷಛϚοϓ࡞ • ࣍ݩ2DnಛϚο
14
ख๏ ▸ Unplojection ▸ 2࣍ݩͷಛϚοϓ3࣍ݩͷຊདྷ͋Δ͖ಛϚοϓ͔ΒࣹӨ ▸ 3࣍ݩάϦουʹٯࣹӨ 15 http://bair.berkeley.edu/blog/2017/09/05/unified-3d/
ख๏ ▸ Unplojection ▸ 2࣍ݩͷಛϚοϓ3࣍ݩͷຊདྷ͋Δ͖ಛϚοϓ͔ΒࣹӨ ▸ 3࣍ݩάϦουʹٯࣹӨ 16 http://bair.berkeley.edu/blog/2017/09/05/unified-3d/
ख๏ ▸ Unplohection ▸ 2࣍ݩͷಛϚοϓ3࣍ݩͷຊདྷ͋Δ͖ಛϚοϓ͔ΒࣹӨ ▸ 3࣍ݩάϦουʹٯࣹӨ 17 http://bair.berkeley.edu/blog/2017/09/05/unified-3d/
ख๏ ▸ Unplohection ▸ 2࣍ݩͷಛϚοϓ3࣍ݩͷຊདྷ͋Δ͖ಛϚοϓ͔ΒࣹӨ ▸ 3࣍ݩάϦουʹٯࣹӨ 18 http://bair.berkeley.edu/blog/2017/09/05/unified-3d/
ख๏ ▸ Recurrent Grid Fusion • 3࣍ݩͷಛϚοϓͷϚονϯάΛGated Recurrent Unit(GRU)Ͱ •
GRUʹ͍࣋ͬͯͨ͘Ίɺ3D convolutionΛ༻ • ͜ͷաఔ͕MVSͷܭࢉϚονϯάΛ୲ • ֶशͷࡍը૾ͷೖྗॱΛϥϯμϜʹೖΕସ͑Δ 19
ख๏ ▸ 3D Grid Reasoning • GRUͰ̏࣍ݩάϦουʹͨ͠ΒϊΠζ͕ଟ͔ͬͨɻ • 3U-netͰEncode Decode͢ΔͱFilteringͰ͖Δ
20
ख๏ ▸ Differentiable Projection • Depthͷ෮ݩʹL1 loss(high frequency informationͷͨΊ) •
Voxelͷ෮ݩʹvoxel͝ͱͷcross entropy loss 21
࣮ݧ ▸ σʔληοτ • ShapeNetσʔλΛར༻ • ̏࣍ݩCADϞσϧͷެ։σʔληοτ 22 https://shapenet.cs.stanford.edu/shrec17/
࣮ݧ • ೖྗը૾ ▸ ShapeNetͷ3DϞσϧΛϨϯμϦϯάͯ͠224x224x3 ▸ ̍ࢹ͋ͨΓ̐ຕ ▸ Χϝϥϙʔζ •
Ξτϓοτ ▸ Depth: 224x224x3 ▸ Voxel: 32x32x32 23
࣮ݧ ▸ ݁Ռ 24 3DR2N2ͱൺɺࡉ͔͍෮ݩ͕Մೳ
࣮ݧ ▸ ݁Ռ 25 3DR2N2ͱൺɺগͳ͍ຕͰ෮ݩ͕Մೳ ຕ૿͑Δͱੑೳ্͕͕Δ
࣮ݧ ▸ ݁Ռ 26 stereo matchingͰ෮ݩ͠ͳ͍ ૭෮ݩՄೳ
࣮ݧ ▸ ݁Ռ 27 stereo matchingʹൺ গͳ͍ຕͰ෮ݩ͕Մೳ චऀᐌ͘ CNNͷίϯςΫετΛݟΔྗ ैདྷͷstereo
matchingΛ͙྇ DepthMapͷਪఆ݁ՌΛෳΈ߹Θͤͯ̏࣍ݩ෮ݩͨ͠
·ͱΊ ▸ Learnt Stereo MachinesΛఏҊ ▸ ෳࢹ͔Βͷೖྗը૾Λݩʹɺ DepthMapͱVoxelͷਪఆ͕Մೳͱͳͬͨ ▸ ՝
• ग़ྗVoxel͕32x32x32ͱখ͍͞ 28