EcoSplat: Efficiency-controllable Feed-forward 3D Gaussian Splatting from Multi-view Images

既存のFeed-forward3DGS手法が１ピクセルに１つのガウシアンを推論する冗長性を解決する手法。どのガウシアンが重要かをランク付けしながら学習を行い、推論時には必要なリソースに合わせてその場で最適な3DGSを推論する。5%という大幅なガウシアン数の削減に対しても品質の劣化を抑えることができる。

Transcript

1. EcoSplat: Efficiency-controllable Feed-forward 3D Gaussian Splatting from Multi-view Images CVPR 2026 Highlight Jongmin Park1, Minh-Quan Viet Bui1, Juan Luis Gonzalez Bello2, Jaeho Moon1, Jihyong Oh3, Munchurl Kim1 1KAIST, 2Flawless AI, 3Department of Imaging Science, GSAIM, Chung-Ang University Spatial AI Network ษڧձ 2026/06/23 ໼઒ྞᠳʢܚጯେֶʣ
2. ࿦จ৘ใ • ࿦จɿhttps://arxiv.org/abs/2512.18692 • Project pageɿhttps://kaist-viclab.github.io/ecosplat-site/ • Githubɿhttps://github.com/KAIST-VICLab/EcoSplat 2
3. എܠ
   طଘ'FFEGPSXBSE
   %(4ͷ৑௕ੑ • ଟࢹ఺ը૾͔ΒFeed-forwardʹ3D Gaussian SplattingΛਪ࿦͢Δݚڀ͸γʔϯ ͝ͱͷ࠷దԽ͕ෆཁͰ͋Δར఺͕͋Δ • ҰํͰ1ϐΫηϧʹ1ͭͷΨ΢γΞϯΛਪ࿦͢ΔͨΊɺೖྗࢹ఺͕ີͳը૾ͷࡍ ࢹ఺਺(N)xग़ྗղ૾౓(HW)ͷ෼͚ͩΨ΢γΞϯΛग़ྗ͢Δ৑௕ੑ͕ଘࡏ͢Δ • 1ϐΫηϧ1Ψ΢γΞϯͷઃఆΛղܾͯ͠ɺਪ࿦࣌ʹࣗ༝ʹΨ΢γΞϯͷ਺Λ มߋͯ͠΋ϨϯμϦϯάͨ͠ը૾ͷ඼࣭Λ཈͑Δ͜ͱ͕໨త 3
4. ؔ࿈ݚڀ
   'FFEGPSXBSE
   %(4 • ଟࢹ఺ը૾Λೖྗͯ͠Feed-forwardʹ3DGSΛਪ࿦͢Δطଘݚڀ͸ɺͲΕ΋ ϐΫηϧͷ਺͚ͩΨ΢γΞϯΛ഑ஔ͢Δߏ଄Λऔ͍ͬͯΔ ྫʣpixelSplat, SplatterImage, MVSplat, DepthSplat, Noposesplat, … L ղ૾౓΍ࢹ఺਺ʹΑͬͯա৒ͳ਺ͷΨ΢γΞϯΛग़ྗͯ͠͠·͏ 5 1ピクセル1ガウシアンの構造 pixelSplat Charatan et al., pixelSplat: 3D Gaussian Splats from Image Pairs for Scalable Generalizable 3D Reconstruction
5. ؔ࿈ݚڀ
   &GGJDJFOU
   'FFEGPSXBSE
   %(4
   • ਪ࿦͞Εͨ3DGSΛࣄલʹܾΊͨVoxelͷ਺ʹѹॖ͢Δख๏ (AnySplat, WorldMirror) • Ψ΢γΞϯ͝ͱͷࢹ఺ؒͷؔ࿈ੑ͔Β৑௕ͳͷΛ࡟Δख๏ (GGN) L ᮢ஋ϕʔεͰ࡟ݮ͢Δख๏ͷͨΊ໌ࣔతʹ࢒ΔΨ΢γΞϯͷ਺Λௐ੔Ͱ͖ͳ͍ 6 Voxelベースにガウシアン数を減らす手法の概念図 GGNの概念図 Zhang et al., Gaussian graph network: Learning efficient and generalizable gaussian representations from multi-view images, Neurips, 2024
6. ఏҊख๏ J ඞཁͳϦιʔεʹ߹Θͤͯɺͦͷ৔Ͱ࠷దͳ3DGSΛਪ࿦͢Δ J Ψ΢γΞϯͷ਺Λ10%, 5%ʹݮΒͯ͠΋PSNRͷྼԽ͕େ෯ʹ཈͑ΒΕΔ 7 Park et al., EcoSplat: Efficiency-controllable Feed-forward 3D Gaussian Splatting from Multi-view Images, CVPR, 2026
7. &DP4QMBU
   ஈ֊ͷֶशϓϩηε • Pixel-aligned Gaussian Training (PGT) શϐΫηϧʹରͯ͠Ψ΢γΞϯΛਪఆ͢ΔجૅϞσϧΛֶश (NoPoSplatΛϕʔε) • Importance-aware Gaussian Finetuning (IGF) ͲͷΨ΢γΞϯ͕ॏཁ͔ΛϥϯΫ෇͚͢ΔػೳΛֶशɻࢦఆ͞Εͨ਺(K)ʹԠͯ͡ ύϥϝʔλΛಈతʹௐ੔ 8 Park et al., EcoSplat: Efficiency-controllable Feed-forward 3D Gaussian Splatting from Multi-view Images, CVPR, 2026
8. ॏཁ౓ϚεΫʹΑΔΨ΢γΞϯͷબผ • Variation Map: ը૾ͷޯ഑ʢ৭ʣͱ๏ઢͷมԽʢܗ ঢ়ʣ͔Βෳࡶ͞Λܭࢉ • దԠతબ୒: • มԽͷܹ͍͠ྖҬ->ͦͷ··อ࣋ • ฏୱͳྖҬ->K-meansͰ౷߹ • Opacity Loss: ੜ੒ͨ͠ϚεΫΩΛٙࣅGTͱ͠ɺ༧ଌͨ͠ෆಁ໌౓ͱ ͷؒʹBCELossΛ͔͚Δɻ͜ΕʹΑΓෆཁͳΨ΢γΞ ϯͷෆಁ໌౓Λ໌ࣔతʹԼ͛Δ 10 Park et al., EcoSplat: Efficiency-controllable Feed-forward 3D Gaussian Splatting from Multi-view Images, CVPR, 2026
9. 1SPHSFTTJWF
   -FBSOJOH
   PO
   (BVTTJBO
   $PNQBDUJPO
   1-($ • ଟ༷ͳλʔήοτ਺(K)ʹϩόετʹରԠ͢ΔͨΊͷֶशઓུ • ৗʹݻఆͷ࡟ݮ཰Ͱֶश͢Δͱෆ҆ఆʹͳΔ • ΠςϨʔγϣϯ͕ਐΉʹͭΕɺαϯϓϦϯά͢ΔΨ΢γΞϯ਺(K)ͷԼݶ(𝐾!"# )Λ ঃʑʹԼ͍͛ͯ͘ • ֤εςοϓͰ𝐾~𝑈[𝐾!"# , 𝐾!$% ]͔ΒϥϯμϜʹબͿ͜ͱͰɺ؇΍͔ͳ࡟ݮ͔Β աࠅͳ࡟ݮ·ͰదԠͤ͞Δ 𝐾!$% = 0.95 - 𝑁𝐻𝑊 (্ݶ͸ݻఆ) 𝐾!"# = max(0.85 − 𝜆&'($) * + , 0.05) - 𝑁𝐻𝑊 (ԼݶΛঃʑʹԼ͛Δ) 11
10. ਪ࿦࣌ʹࢹ఺͝ͱͷϦιʔε഑෼ • શࢹ఺Ұ཯ʹݮΒ͢ͷͰ͸ͳ͘ɺࢹ఺ͷෳࡶ౓ʹԠͯ͡഑෼Λม͑Δ • ςΫενϟ͕ࡉ͔͍ࢹ఺ʹ͸ΑΓଟ͘ͷΨ΢γΞϯΛׂΓ౰ͯɺฏୱͳྖҬ͕ଟ ͍ࢹ఺Ͱ͸࡟ݮ཰ΛߴΊͯશମͷ߹ܭ਺(K)Λ཈͑Δ • ֤ࢹ఺ͷप೾਺੒෼Λߴ଎ʹܭࢉ͠ɺਪ࿦࣌ʹॠ࣌ʹ഑෼Λܾఆ 12 Park et al., EcoSplat: Efficiency-controllable Feed-forward 3D Gaussian Splatting from Multi-view Images, CVPR, 2026 𝜌! : 大 𝜌" : 小
11. ࣮ݧ • Realestate10kσʔληοτͰֶश͠ಉσʔληοτͱACIDσʔληοτͰධՁ • ධՁࢦඪ͸PSNR, SSIM, LPIPSͱγʔϯΛߏ੒͢ΔΨ΢γΞϯͷ਺ • ݕূͷϙΠϯτ • ۃ୺ͳ࡟ݮ࣌ͷϩόετੑ: Ψ΢γΞϯ਺Λ5%, 10%ʹݮΒͨ͠ࡍɺطଘͷޮ཰Խख๏ͱ ൺֱͯ͠඼࣭Λҡ࣋Ͱ͖Δ͔ • ࠷ॳ͔Β਺Λࢦఆͯ͠ਪ࿦͢Δ(EcoSplat) vs ग़ྗޙʹࣄޙѹॖ͢Δख๏ͷ༏Ґੑ • ൚Խੑೳ: ࢹ఺਺͕มԽͨ͠ࡍʹ΋εέʔϧ͢Δ͔ 13
12. Ψ΢γΞϯͷ਺Λௐ੔ͯ͠ൺֱ • ൺֱख๏ͷ෼ྨ • طଘͷޮ཰ԽFeed-forwardख๏ͱͷൺֱ (GGN, AnySplat, WorldMirror) • Χεέʔυख๏ʢFeed-forward3DGS + ࣄޙѹॖʣͱͷൺֱ ([13]…LightGaussian, [17]…PUP 3D-GS) • 5%, 10%ͱ͍ͬͨۃ୺ͳ࡟ݮઃఆʹ͓͍ͯɺଞख๏͸େ͖͘ྼԽ͢Δ͕ɺ EcoSplat͸ߴ͍ϨϯμϦϯά඼࣭Λҡ࣋ 14 Park et al., EcoSplat: Efficiency-controllable Feed-forward 3D Gaussian Splatting from Multi-view Images, CVPR, 2026
13. ఆੑධՁ ఆੑతʹݟͯ΋5%, 10%Ͱଞख๏͸େ͖͘ྼԽ͢Δ͕ɺEcoSplatͰ͸มΘΒͳ͍ ϨϯμϦϯά඼࣭Λ࣮ݱ 15 Park et al., EcoSplat: Efficiency-controllable Feed-forward 3D Gaussian Splatting from Multi-view Images, CVPR, 2026
14. ఆੑධՁ ఆੑతʹݟͯ΋5%, 10%Ͱଞख๏͸େ͖͘ྼԽ͢Δ͕ɺEcoSplatͰ͸มΘΒͳ͍ ϨϯμϦϯά඼࣭Λ࣮ݱ 16 Park et al., EcoSplat: Efficiency-controllable Feed-forward 3D Gaussian Splatting from Multi-view Images, CVPR, 2026
15. 'FFEGPSXBSE%(4ख๏ͱͷൺֱ • SPFSplatΑΓ΋൒෼Ҏ্গͳ͍Ψ΢γΞϯ਺Ͱಉఔ౓ͷϨϯμϦϯά඼࣭Λ࣮ݱ • 5%ͷΨ΢γΞϯ਺Ͱ6/7ͷख๏ΑΓ΋ߴ͍ϨϯμϦϯά඼࣭Λ࣮ݱ 17 Park et al., EcoSplat: Efficiency-controllable Feed-forward 3D Gaussian Splatting from Multi-view Images, CVPR, 2026
16. "CMBUJPO • PGTͳ͠: جૅతͳΨ΢γΞϯਪఆ͕Ͱ͖͍ͯͳ͍ͨΊɺશମతʹϊΠζ͕৐Δ • IGFͳ͠: EcoSplatͷ؊ɻ͜Ε͕ͳ͍ͱͦ΋ͦ΋೚ҙͷK΁ͷѹॖʹରԠͰ͖ͣɺ #G 5%࣌ʹPSNR͕க໋తʹ௿Լ͢Δ • 𝐿", (Opacity Loss) ͳ͠: ॏཁ౓ʹԠͨ͡Ψ΢γΞϯͷׂΓ౰͕ࣦͯഊ͠ɺ࢒͢΂͖ ߏ଄ʹ͕ۭ݀͘ • PLGC ͳ͠: ͍͖ͳΓۃ୺ͳ਺ʹݮΒֶ͢श͕ೖΓ࠷దԽ͕ෆ҆ఆԽɻ΅΍͚ͨ ը૾ʹͳΔ 18 Park et al., EcoSplat: Efficiency-controllable Feed-forward 3D Gaussian Splatting from Multi-view Images, CVPR, 2026
17. ݶքͱߟ࡯ • ݶք: ݱࡏ͸੩తγʔϯͷΈɻಈతγʔϯΛѻ͏ʹ͸Cross-attention૚ʹ࣌ؒతͳ Ұ؏ੑΛֶशͤ͞Δ֦ு͕ඞཁ • ߟ࡯1: طଘͷFF-3DGSख๏͸2ࢹ఺Ͱ΋ਪ࿦Մೳ͕ͩɺຊ࿦จ͸16ʙ24ࢹ఺ͱ͍ͬͨ Dense viewͰͷ݁Ռ͔͍ࣔͯ͠͠ͳ͍ɻ 2ࢹ఺ͷΑ͏ͳ΋ͱ΋ͱ৑௕ੑ͕গͳ͍Sparse viewઃఆʹ͓͍ͯɺ͜ͷ਺ࢦఆ(K) ͷΞϓϩʔν͕Ͳ͜·Ͱ༗ޮʹػೳ͢Δͷ͔͸ෆಁ໌ • ߟ࡯2: CVPR 2026Ͱ͸ɺଞʹ΋OffthegridɺsparsesplatɺTokenGSɺTokensplat, anchorsplatͱ͍ͬͨEfficient feed-forward 3DGSͷݚڀ͕ෳ਺௨͍ͬͯΔɻ ͜ΕΒͷ࠷৽ख๏ͱಉ͡৚݅Լʢࢹ఺਺΍࡟ݮλʔήοτʣͰଗ͑ͯൺֱ͢Δ͜ ͱͰɺ֤ख๏ͷڧΈ͕ΫϦΞʹͳΔ͔΋ 19