Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
NIPS2017reading_3Dreconstruction
Search
望月紅葉さんと幸せな家庭を築きたい
January 27, 2018
Research
0
1.5k
NIPS2017reading_3Dreconstruction
望月紅葉さんと幸せな家庭を築きたい
January 27, 2018
Tweet
Share
More Decks by 望月紅葉さんと幸せな家庭を築きたい
See All by 望月紅葉さんと幸せな家庭を築きたい
shadow-detection-with-conditional-generative-adversarial-networks
momijifullmoon
0
140
unsupervised-learning-of-depth-and-ego-motion-from-monocular-video-using-3d-geometric-constraints
momijifullmoon
0
380
ABEJA Innovation Meetup NIPS PointNet++
momijifullmoon
1
480
Other Decks in Research
See All in Research
Inside Phishing Groups: Trust No One
0x1shu
0
110
Weekly AI Agents News! 9月号 プロダクト/ニュースのアーカイブ
masatoto
1
110
Matching 2D Images in 3D: Metric Relative Pose from Metric Correspondences
sgk
1
300
KDD論文読み会2024: False Positive in A/B Tests
ryotoitoi
0
170
LiDARとカメラのセンサーフュージョンによる点群からのノイズ除去
kentaitakura
0
110
大規模言語モデルを用いた日本語視覚言語モデルの評価方法とベースラインモデルの提案 【MIRU 2024】
kentosasaki
2
500
第79回 産総研人工知能セミナー 発表資料
agiats
1
120
MIRU2024チュートリアル「様々なセンサやモダリティを用いたシーン状態推定」
miso2024
3
2.1k
20240820: Minimum Bayes Risk Decoding for High-Quality Text Generation Beyond High-Probability Text
de9uch1
0
110
言語処理学会30周年記念事業留学支援交流会@YANS2024:「学生のための短期留学」
a1da4
1
230
[依頼講演] 適応的実験計画法に基づく効率的無線システム設計
k_sato
0
100
医療分野におけるLLMの現状と応用可能性について
kento1109
11
3.5k
Featured
See All Featured
[Rails World 2023 - Day 1 Closing Keynote] - The Magic of Rails
eileencodes
32
1.8k
Large-scale JavaScript Application Architecture
addyosmani
510
110k
Done Done
chrislema
181
16k
Thoughts on Productivity
jonyablonski
67
4.3k
Producing Creativity
orderedlist
PRO
341
39k
Designing on Purpose - Digital PM Summit 2013
jponch
115
6.9k
Art, The Web, and Tiny UX
lynnandtonic
296
20k
Designing the Hi-DPI Web
ddemaree
280
34k
Stop Working from a Prison Cell
hatefulcrawdad
267
20k
Fontdeck: Realign not Redesign
paulrobertlloyd
81
5.2k
Sharpening the Axe: The Primacy of Toolmaking
bcantrill
37
1.8k
Scaling GitHub
holman
458
140k
Transcript
̏࣍ݩ෮ݩʹؔͯ͠ Learning a Multi-View Stereo Machine NIPS2017จಡΈձˏΫοΫύου 1 ಛʹදه͕ͳ͍ݶΓɺҎԼͷࢿྉ͔ΒҾ༻ https://arxiv.org/pdf/1708.05375.pdf
Learning a Multi-View Stereo Machine ▸ චऀ • Abhishek Kar,
Christian Häne, Jitendra Malik ʢUC Berkeley) ▸ ֓ཁ • Multi View StereoʢMVSʣʹΑΔີͳ3࣍ݩ෮ݩΛDeep LearningͰEnd2Endʹֶश • MVSΛ”ֶशͰ͖Δ”ͷͰແ͍͔ͱ͍͏ٙʹ͑Δ 2
എܠ ▸ Multi View Stereoͱ 1. ಛநग़ 2. Ϛονϯά 3.
̏࣍ݩ෮ݩ 4. Τϥʔͷআڈ 3
എܠ ▸ Multi View Stereoͱ 1. ಛநग़ 2. Ϛονϯά 3.
̏࣍ݩ෮ݩ 4. Τϥʔͷআڈ ==> DeepԿͰશͯղܾͰ͖ͦ͏ 4
എܠ ▸ Multi View Stereoͱ 1. ಛநग़ɹ← CNNͰ͍͚Δ 2. Ϛονϯά
3. ̏࣍ݩ෮ݩ 4. Τϥʔͷআڈ 5
എܠ ▸ Multi View Stereoͱ 1. ಛநग़ 2. Ϛονϯάɹ← CNNͱRNNͰ͍͚Δ
3. ̏࣍ݩ෮ݩ 4. Τϥʔͷআڈ 6
എܠ ▸ Multi View Stereoͱ 1. ಛநग़ 2. Ϛονϯά 3.
̏࣍ݩ෮ݩɹ← DeconvͰ͍͚Δ 4. Τϥʔͷআڈ 7
എܠ ▸ Multi View Stereoͱ 1. ಛநग़ 2. Ϛονϯά 3.
̏࣍ݩ෮ݩ 4. Τϥʔͷআڈɹ← Encoder-DecoderͰ͍͚Δ 8
DeepԿͰࡾ࣍ݩ෮ݩ ▸ 3DR2N2(ECCV2016) • ෳը૾ΛΤϯίʔυ͠ɺLSTMͰϚονϯά 9 http://3d-r2n2.stanford.edu
DeepԿͰࡾ࣍ݩ෮ݩ ▸ 3D Shape Reconstruction by Modeling 2.5D Sketch (NIPS2017)
• ϦΞϧͷը૾͔Β2.5DͷεέονΛى͜͠ɺ2.5DεέονΛͱʹ 3DshapeਪఆΛEnd2EndֶशͰ͢Δ 10 https://arxiv.org/pdf/1711.03129.pdf
͢༰ ▸ શମ૾ ▸ ख๏ ▸ ࣮ݧ ▸ ·ͱΊ 11
શମ૾ 12 http://bair.berkeley.edu/blog/2017/09/05/unified-3d/
શମ૾ 13 Learnt Stereo Machines
ख๏ ▸ Image Encoder • Encoder-DecoderܕʢU-netʣͷઃܭ • Ϛονϯάʹ༻͍Δ̎DͷಛϚοϓ࡞ • ࣍ݩ2DnಛϚο
14
ख๏ ▸ Unplojection ▸ 2࣍ݩͷಛϚοϓ3࣍ݩͷຊདྷ͋Δ͖ಛϚοϓ͔ΒࣹӨ ▸ 3࣍ݩάϦουʹٯࣹӨ 15 http://bair.berkeley.edu/blog/2017/09/05/unified-3d/
ख๏ ▸ Unplojection ▸ 2࣍ݩͷಛϚοϓ3࣍ݩͷຊདྷ͋Δ͖ಛϚοϓ͔ΒࣹӨ ▸ 3࣍ݩάϦουʹٯࣹӨ 16 http://bair.berkeley.edu/blog/2017/09/05/unified-3d/
ख๏ ▸ Unplohection ▸ 2࣍ݩͷಛϚοϓ3࣍ݩͷຊདྷ͋Δ͖ಛϚοϓ͔ΒࣹӨ ▸ 3࣍ݩάϦουʹٯࣹӨ 17 http://bair.berkeley.edu/blog/2017/09/05/unified-3d/
ख๏ ▸ Unplohection ▸ 2࣍ݩͷಛϚοϓ3࣍ݩͷຊདྷ͋Δ͖ಛϚοϓ͔ΒࣹӨ ▸ 3࣍ݩάϦουʹٯࣹӨ 18 http://bair.berkeley.edu/blog/2017/09/05/unified-3d/
ख๏ ▸ Recurrent Grid Fusion • 3࣍ݩͷಛϚοϓͷϚονϯάΛGated Recurrent Unit(GRU)Ͱ •
GRUʹ͍࣋ͬͯͨ͘Ίɺ3D convolutionΛ༻ • ͜ͷաఔ͕MVSͷܭࢉϚονϯάΛ୲ • ֶशͷࡍը૾ͷೖྗॱΛϥϯμϜʹೖΕସ͑Δ 19
ख๏ ▸ 3D Grid Reasoning • GRUͰ̏࣍ݩάϦουʹͨ͠ΒϊΠζ͕ଟ͔ͬͨɻ • 3U-netͰEncode Decode͢ΔͱFilteringͰ͖Δ
20
ख๏ ▸ Differentiable Projection • Depthͷ෮ݩʹL1 loss(high frequency informationͷͨΊ) •
Voxelͷ෮ݩʹvoxel͝ͱͷcross entropy loss 21
࣮ݧ ▸ σʔληοτ • ShapeNetσʔλΛར༻ • ̏࣍ݩCADϞσϧͷެ։σʔληοτ 22 https://shapenet.cs.stanford.edu/shrec17/
࣮ݧ • ೖྗը૾ ▸ ShapeNetͷ3DϞσϧΛϨϯμϦϯάͯ͠224x224x3 ▸ ̍ࢹ͋ͨΓ̐ຕ ▸ Χϝϥϙʔζ •
Ξτϓοτ ▸ Depth: 224x224x3 ▸ Voxel: 32x32x32 23
࣮ݧ ▸ ݁Ռ 24 3DR2N2ͱൺɺࡉ͔͍෮ݩ͕Մೳ
࣮ݧ ▸ ݁Ռ 25 3DR2N2ͱൺɺগͳ͍ຕͰ෮ݩ͕Մೳ ຕ૿͑Δͱੑೳ্͕͕Δ
࣮ݧ ▸ ݁Ռ 26 stereo matchingͰ෮ݩ͠ͳ͍ ૭෮ݩՄೳ
࣮ݧ ▸ ݁Ռ 27 stereo matchingʹൺ গͳ͍ຕͰ෮ݩ͕Մೳ චऀᐌ͘ CNNͷίϯςΫετΛݟΔྗ ैདྷͷstereo
matchingΛ͙྇ DepthMapͷਪఆ݁ՌΛෳΈ߹Θͤͯ̏࣍ݩ෮ݩͨ͠
·ͱΊ ▸ Learnt Stereo MachinesΛఏҊ ▸ ෳࢹ͔Βͷೖྗը૾Λݩʹɺ DepthMapͱVoxelͷਪఆ͕Մೳͱͳͬͨ ▸ ՝
• ग़ྗVoxel͕32x32x32ͱখ͍͞ 28