Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
NIPS2017reading_3Dreconstruction
Search
望月紅葉さんと幸せな家庭を築きたい
January 27, 2018
Research
0
1.5k
NIPS2017reading_3Dreconstruction
望月紅葉さんと幸せな家庭を築きたい
January 27, 2018
Tweet
Share
More Decks by 望月紅葉さんと幸せな家庭を築きたい
See All by 望月紅葉さんと幸せな家庭を築きたい
shadow-detection-with-conditional-generative-adversarial-networks
momijifullmoon
0
140
unsupervised-learning-of-depth-and-ego-motion-from-monocular-video-using-3d-geometric-constraints
momijifullmoon
0
400
ABEJA Innovation Meetup NIPS PointNet++
momijifullmoon
1
490
Other Decks in Research
See All in Research
ナレッジプロデューサーとしてのミドルマネージャー支援 - MIMIGURI「知識創造室」の事例の考察 -
chiemitaki
0
250
Weekly AI Agents News! 12月号 プロダクト/ニュースのアーカイブ
masatoto
0
360
AIトップカンファレンスからみるData-Centric AIの研究動向 / Research Trends in Data-Centric AI: Insights from Top AI Conferences
tsurubee
3
2.1k
Gemini と Looker で営業DX をドライブする / Driving Sales DX with Gemini and Looker
sansan_randd
0
190
研究を支える拡張性の高い ワークフローツールの提案 / Proposal of highly expandable workflow tools to support research
linyows
0
360
Weekly AI Agents News! 12月号 論文のアーカイブ
masatoto
0
250
請求書仕分け自動化での物体検知モデル活用 / Utilization of Object Detection Models in Automated Invoice Sorting
sansan_randd
0
140
LLM 시대의 Compliance: Safety & Security
huffon
0
630
Pix2Poly: A Sequence Prediction Method for End-to-end Polygonal Building Footprint Extraction from Remote Sensing Imagery
satai
3
120
Bluesky Game Dev
trezy
0
180
Vision Language Modelと完全自動運転AIの最新動向
tsubasashi
1
310
プロシェアリング白書2025_PROSHARING_REPORT_2025
circulation
1
250
Featured
See All Featured
Save Time (by Creating Custom Rails Generators)
garrettdimon
PRO
30
1.1k
I Don’t Have Time: Getting Over the Fear to Launch Your Podcast
jcasabona
32
2.2k
Principles of Awesome APIs and How to Build Them.
keavy
126
17k
Stop Working from a Prison Cell
hatefulcrawdad
268
20k
"I'm Feeling Lucky" - Building Great Search Experiences for Today's Users (#IAC19)
danielanewman
227
22k
Distributed Sagas: A Protocol for Coordinating Microservices
caitiem20
331
21k
How STYLIGHT went responsive
nonsquared
99
5.4k
Making Projects Easy
brettharned
116
6.1k
Build The Right Thing And Hit Your Dates
maggiecrowley
34
2.6k
Statistics for Hackers
jakevdp
798
220k
GraphQLの誤解/rethinking-graphql
sonatard
70
10k
Building an army of robots
kneath
304
45k
Transcript
̏࣍ݩ෮ݩʹؔͯ͠ Learning a Multi-View Stereo Machine NIPS2017จಡΈձˏΫοΫύου 1 ಛʹදه͕ͳ͍ݶΓɺҎԼͷࢿྉ͔ΒҾ༻ https://arxiv.org/pdf/1708.05375.pdf
Learning a Multi-View Stereo Machine ▸ චऀ • Abhishek Kar,
Christian Häne, Jitendra Malik ʢUC Berkeley) ▸ ֓ཁ • Multi View StereoʢMVSʣʹΑΔີͳ3࣍ݩ෮ݩΛDeep LearningͰEnd2Endʹֶश • MVSΛ”ֶशͰ͖Δ”ͷͰແ͍͔ͱ͍͏ٙʹ͑Δ 2
എܠ ▸ Multi View Stereoͱ 1. ಛநग़ 2. Ϛονϯά 3.
̏࣍ݩ෮ݩ 4. Τϥʔͷআڈ 3
എܠ ▸ Multi View Stereoͱ 1. ಛநग़ 2. Ϛονϯά 3.
̏࣍ݩ෮ݩ 4. Τϥʔͷআڈ ==> DeepԿͰશͯղܾͰ͖ͦ͏ 4
എܠ ▸ Multi View Stereoͱ 1. ಛநग़ɹ← CNNͰ͍͚Δ 2. Ϛονϯά
3. ̏࣍ݩ෮ݩ 4. Τϥʔͷআڈ 5
എܠ ▸ Multi View Stereoͱ 1. ಛநग़ 2. Ϛονϯάɹ← CNNͱRNNͰ͍͚Δ
3. ̏࣍ݩ෮ݩ 4. Τϥʔͷআڈ 6
എܠ ▸ Multi View Stereoͱ 1. ಛநग़ 2. Ϛονϯά 3.
̏࣍ݩ෮ݩɹ← DeconvͰ͍͚Δ 4. Τϥʔͷআڈ 7
എܠ ▸ Multi View Stereoͱ 1. ಛநग़ 2. Ϛονϯά 3.
̏࣍ݩ෮ݩ 4. Τϥʔͷআڈɹ← Encoder-DecoderͰ͍͚Δ 8
DeepԿͰࡾ࣍ݩ෮ݩ ▸ 3DR2N2(ECCV2016) • ෳը૾ΛΤϯίʔυ͠ɺLSTMͰϚονϯά 9 http://3d-r2n2.stanford.edu
DeepԿͰࡾ࣍ݩ෮ݩ ▸ 3D Shape Reconstruction by Modeling 2.5D Sketch (NIPS2017)
• ϦΞϧͷը૾͔Β2.5DͷεέονΛى͜͠ɺ2.5DεέονΛͱʹ 3DshapeਪఆΛEnd2EndֶशͰ͢Δ 10 https://arxiv.org/pdf/1711.03129.pdf
͢༰ ▸ શମ૾ ▸ ख๏ ▸ ࣮ݧ ▸ ·ͱΊ 11
શମ૾ 12 http://bair.berkeley.edu/blog/2017/09/05/unified-3d/
શମ૾ 13 Learnt Stereo Machines
ख๏ ▸ Image Encoder • Encoder-DecoderܕʢU-netʣͷઃܭ • Ϛονϯάʹ༻͍Δ̎DͷಛϚοϓ࡞ • ࣍ݩ2DnಛϚο
14
ख๏ ▸ Unplojection ▸ 2࣍ݩͷಛϚοϓ3࣍ݩͷຊདྷ͋Δ͖ಛϚοϓ͔ΒࣹӨ ▸ 3࣍ݩάϦουʹٯࣹӨ 15 http://bair.berkeley.edu/blog/2017/09/05/unified-3d/
ख๏ ▸ Unplojection ▸ 2࣍ݩͷಛϚοϓ3࣍ݩͷຊདྷ͋Δ͖ಛϚοϓ͔ΒࣹӨ ▸ 3࣍ݩάϦουʹٯࣹӨ 16 http://bair.berkeley.edu/blog/2017/09/05/unified-3d/
ख๏ ▸ Unplohection ▸ 2࣍ݩͷಛϚοϓ3࣍ݩͷຊདྷ͋Δ͖ಛϚοϓ͔ΒࣹӨ ▸ 3࣍ݩάϦουʹٯࣹӨ 17 http://bair.berkeley.edu/blog/2017/09/05/unified-3d/
ख๏ ▸ Unplohection ▸ 2࣍ݩͷಛϚοϓ3࣍ݩͷຊདྷ͋Δ͖ಛϚοϓ͔ΒࣹӨ ▸ 3࣍ݩάϦουʹٯࣹӨ 18 http://bair.berkeley.edu/blog/2017/09/05/unified-3d/
ख๏ ▸ Recurrent Grid Fusion • 3࣍ݩͷಛϚοϓͷϚονϯάΛGated Recurrent Unit(GRU)Ͱ •
GRUʹ͍࣋ͬͯͨ͘Ίɺ3D convolutionΛ༻ • ͜ͷաఔ͕MVSͷܭࢉϚονϯάΛ୲ • ֶशͷࡍը૾ͷೖྗॱΛϥϯμϜʹೖΕସ͑Δ 19
ख๏ ▸ 3D Grid Reasoning • GRUͰ̏࣍ݩάϦουʹͨ͠ΒϊΠζ͕ଟ͔ͬͨɻ • 3U-netͰEncode Decode͢ΔͱFilteringͰ͖Δ
20
ख๏ ▸ Differentiable Projection • Depthͷ෮ݩʹL1 loss(high frequency informationͷͨΊ) •
Voxelͷ෮ݩʹvoxel͝ͱͷcross entropy loss 21
࣮ݧ ▸ σʔληοτ • ShapeNetσʔλΛར༻ • ̏࣍ݩCADϞσϧͷެ։σʔληοτ 22 https://shapenet.cs.stanford.edu/shrec17/
࣮ݧ • ೖྗը૾ ▸ ShapeNetͷ3DϞσϧΛϨϯμϦϯάͯ͠224x224x3 ▸ ̍ࢹ͋ͨΓ̐ຕ ▸ Χϝϥϙʔζ •
Ξτϓοτ ▸ Depth: 224x224x3 ▸ Voxel: 32x32x32 23
࣮ݧ ▸ ݁Ռ 24 3DR2N2ͱൺɺࡉ͔͍෮ݩ͕Մೳ
࣮ݧ ▸ ݁Ռ 25 3DR2N2ͱൺɺগͳ͍ຕͰ෮ݩ͕Մೳ ຕ૿͑Δͱੑೳ্͕͕Δ
࣮ݧ ▸ ݁Ռ 26 stereo matchingͰ෮ݩ͠ͳ͍ ૭෮ݩՄೳ
࣮ݧ ▸ ݁Ռ 27 stereo matchingʹൺ গͳ͍ຕͰ෮ݩ͕Մೳ චऀᐌ͘ CNNͷίϯςΫετΛݟΔྗ ैདྷͷstereo
matchingΛ͙྇ DepthMapͷਪఆ݁ՌΛෳΈ߹Θͤͯ̏࣍ݩ෮ݩͨ͠
·ͱΊ ▸ Learnt Stereo MachinesΛఏҊ ▸ ෳࢹ͔Βͷೖྗը૾Λݩʹɺ DepthMapͱVoxelͷਪఆ͕Մೳͱͳͬͨ ▸ ՝
• ग़ྗVoxel͕32x32x32ͱখ͍͞ 28