Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Fuzzy Metaballs: Approximate Differentiable Ren...

Avatar for tattaka tattaka
October 22, 2022

Fuzzy Metaballs: Approximate Differentiable Rendering with Algebraic Surfaces

2022/10/22に開催されたECCV2022読み会@PRMU研究会において発表した資料です。
本研究のProject pageはこちら(https://leonidk.github.io/fuzzy-metaballs/)

Avatar for tattaka

tattaka

October 22, 2022
Tweet

More Decks by tattaka

Other Decks in Research

Transcript

  1. ࠓճ঺հ͢Δݚڀ “Fuzzy Metaballs” • metaballͱ͍͏ղऍ͕͠΍͍͢3DදݱΛ 
 ඍ෼ՄೳͳܗͰϨϯμϦϯά͢Δख๏ͷఏҊ • ଞͷඍ෼ՄೳϨϯμϥΛ༻͍ͨख๏ΑΓϩόετ͔ͭ 


    ߴ଎ͳ͜ͱΛෳ਺ͷλεΫͰ֬ೝ • ݸਓతͳਪ͠ϙΠϯτɿݹయతͳख๏Λݱ୅෩ʹ࠶ղऍɾ 
 CPUͰ΋ߴ଎ͳ఺΍ϩόετੑ͕ߴ͍ͳͲଞख๏ͱ͸ҧͬͨ࢖͍ํ͕Ͱ͖ΔͷͰ͸ • Jax࣮૷΋͋Γ·͢(https://github.com/leonidk/fuzzy-metaballs)
  2. ৭ʑͳඍ෼ՄೳϨϯμϦϯά • MeshදݱΛϨϯμϦϯά͢Δ ◦ ϨΠτϨʔγϯά(ྫ: Mitsuba3) ◦ ϥελϥΠζ(ྫ: Neural Mesh

    Renderer, Soft Rasterizer) • Neural Rendering ◦ ྫ: Neural Voxel Renderer, NeRF, GPNR • ӄؔ਺දݱΛϨϯμϦϯά͢Δ ◦ ྫ: SDFDiff, 
 Differentiable Signed Distance Function Rendering
  3. ৭ʑͳඍ෼ՄೳϨϯμϦϯά • MeshදݱΛϨϯμϦϯά͢Δ ◦ ϨΠτϨʔγϯά(ྫ: Mitsuba3) ◦ ϥελϥΠζ(ྫ: Neural Mesh

    Renderer, Soft Rasterizer) • Neural Rendering ◦ ྫ: Neural Voxel Renderer, NeRF, GPNR • ӄؔ਺දݱΛϨϯμϦϯά͢Δ ← ࠓճ͸ίϨʂ ◦ ྫ: SDFDiff, 
 Differentiable Signed Distance Function Rendering
  4. Fuzzy Metaballsͷ࣮૷ɾଞͷ3Dදݱͱͷม׵ • ཁ͢Δʹࠞ߹Ψ΢ε෼෍ͳͷͰ 
 sklearn.mixture.GaussianMixtureΛ࢖͏ • ϝογϡ΍఺܈→Fuzzy Metaballs: 


    ද໘΋͘͠͸಺෦͔Β఺ΛαϯϓϦϯά͠ɺGMMͰFit͢Δ • Fuzzy Metaballs→ ◦ ఺܈: GMM͔ΒαϯϓϦϯάΛߦ͏ ◦ ϝογϡ: Λ࠷దԽ্ͨ͠ͰMarching Cube๏Ͱද໘ΛுΔ
  5. ༻͍Δσʔλ • Stanford Model Repository͔Β 
 arma, buddha, dragon, lucy,

    bunny • Thingi10K͔Β gear, eiffel, rebel • ShapeNet͔Β Plane • Sketchfab dataset͔Β Yoga (ࠨͷը૾͸Ұ෦) http://graphics.stanford.edu/data/3Dscanrep/ https://ten-thousand-models.appspot.com/ https://arxiv.org/abs/1901.05567
  6. ͬ͘͟Γ༻͍Δඍ෼ՄೳϨϯμϥʹ͍ͭͯ • Soft Rasterizer ◦ ϨϯμϦϯά݁ՌΛ΅΍͔͢͜ͱͰޯ഑Λ࡞Δ • Direct Surface Splatting

    ◦ ఺܈Λପԁମͷू߹ͱଊ͑ͯϨϯμϦϯά͢Δ • Pulser ◦ ্ͱಉ͡Α͏ʹ఺Λପԁମͱͯ͠ଊ͑Δ͕ϨϯμϦϯάର৅ͷ ٿͷબ୒Λ޻෉͢Δ͜ͱͰߴ଎Խ
  7. ࣮ݧ: Pose Estimation • +-45°ͷճసͱ3DϞσϧͷ50%·Ͱͷ 
 ϥϯμϜͳฏߦҠಈͨ͠΋ͷͷσϓεϚοϓΛ༩͑Δ • લड़ʹՃ͑ͯICP๏΋࣮ݧ •

    ճసޡࠩͱฒਐޡࠩͷزԿฏۉ • ༩͑ΔσϓεϚοϓʹ 
 ϊΠζΛՃ͑ͨ΋ͷͱ 
 ͦ͏Ͱͳ͍΋ͷΛ༻͍Δ(ϊΠζͷ 
 ৄࡉ͸ৄ͘͠͸ॻ͍͍ͯͳ͍)
  8. ٞ࿦ɾLimitations • ຊख๏Ͱ͸ܗঢ়ΛϨϯμϦϯά͢Δ͜ͱʹয఺Λ౰ͯͨ ◦ ࣮ݧ͸͍ͯ͠ͳ͍͕Fuzzy Metaballs͝ͱʹ৭ΛׂΓ౰ͯΔ͜ͱ͸Մೳ • GPUʹಛԽͨ͠࠷దԽ͸͍ͯ͠ͳ͍ͷͰߴ଎Խͷ༨஍͋Γ • Ϟσϧͷࣗ༝౓ͷ௿͔͞Β҉໧తͳਖ਼ଇԽ͕ಇ͖ϊΠζʹڧ͍ͱߟ͑ΒΕΔ

    ◦ ηάϝϯςʔγϣϯϞσϧग़ྗͷΤϥʔमਖ਼ͳͲʹར༻Մೳ • ࠷దԽ͞ΕͨFuzzy Metaballsͷύϥϝʔλ͸زԿతʹղऍ͕Ͱ͖Δ 
 (GMMͳͷͰ) • ࣮ࡍͷϢʔεέʔεͱͯ͠ɺߴ඼࣭ͳը૾औಘ͕೉͍͠ͱ͖ʹ࢖͑ͦ͏
  9. ࢀߟจݙ • هࡌͷͳ͍ݶΓ͸Fuzzy Metaballsͷ࿦จ͔ΒҾ༻ • ͦͷଞ ◦ ϨΠτϨʔγϯάʹΑΔίϯϐϡʔλάϥϑΟΫεೖ໳ - ϝλ

    Ϙʔϧ - ◦ ඍ෼ՄೳϨϯμϦϯά (CVIMݚڀձ νϡʔτϦΞϧ) ◦ Differentiable Rendering: A Survey