2018: Software engineer/manager at an insurtech startup Since 2024: Freelance; building ML models and infrastructure at a machine learning startup Website: https://sawara.dev/ I share the latest updates on social media. Follow me! X (Twitter): @xhiroga GitHub: @xhiroga LinkedIn: @hiroga VRChat: @hiroga YouTube: @hiroga
The first photogrammetry research was reported in Germany in 1867, and photo-based surveying began to be used for mapping 2000s SfM / MVS Structure-from-Motion and Multi-View Stereo methods emerge to estimate camera poses and generate point clouds from digital photos 2020 NeRF Mildenhall et al. publish NeRF, drawing attention to high-quality novel view synthesis with neural radiance fields 2023 3D Gaussian Splatting Kerbl et al. present 3D Gaussian Splatting, enabling real-time rendering
photographs—especially recovering 3D data from images. Photogrammetry photo (photograph) gram (drawing, record) metry (measurement) Generating mesh data from photos is often referred to as photogrammetry.
SfM (Structure from Motion) and MVS (Multi-View Stereo) SfM estimates camera poses and sparse point clouds, while MVS produces dense point clouds followed by optional meshing The pipeline looks like the diagram below 1. Hokkaido University Digital Geometry Processing Laboratory, "Enhancing the Quality of SfM-MVS." ↩︎ [1]
space with neural networks to achieve high-quality novel view synthesis Output is neural network weights (e.g., .safetensors ) rather than meshes or point clouds
Gaussian ( ガウシアン) Gaussian refers to the normal distribution. Instead of dense point clouds or meshes, each sparse point stores scale, color, and opacity as a 3D Gaussian distribution Splatting ( スプラッティング) One approach for rendering 3D graphics data into 2D. Ray tracing: Shoot rays from the camera into the scene and render the first intersected surface Splatting: Project each point onto the screen Triangle rasterization: Fill the nearest triangles; suitable for meshes The reconstructed data is commonly called a “splat.”
and scale to let each point cover a wide area. Open the mkkellogg/GaussianSplats3D demo and press P to toggle between 3D Gaussian splatting and point-cloud rendering modes.
Photogrammetry Faithfully reproduces matte materials High geometric accuracy Struggles with transparent and specular surfaces Manual optimization is labor-intensive NeRF Excels at novel view interpolation and high-frequency textures Difficult to run in real time or handle transparency Post-processing is complex 3D Gaussian Splatting Handles transparency and reflections well Great at representing fine details like foliage Strong for real-time rendering Pipeline can be highly automated Weaker for matte materials and unseen viewpoints Lower geometric fidelity
can be published on SuperSplat. Until you check "List," only people with the link can view it. You can review your own splats from Manage. 1. SuperSplat (demo by the speaker) ↩︎ [1]
variety of image-to-3D reconstruction methods including NeRF and 3D Gaussian Splatting gsplat: Core 3DGS implementation for nerfstudio; available on PyPI ( pip install gsplat ) Inria 3DGS: Official implementation of the original “3D Gaussian Splatting for Real-Time Radiance Field Rendering” paper Commercial Software postshot: Windows application with 3DGS support. Free trial available; exporting PLY and other formats requires a paid plan. Scaniverse: 3D scanning app by Niantic of Pokémon GO fame. Supports both photogrammetry and 3D Gaussian Splatting. LumaAI: Captures on iOS/Android and processes in the cloud to generate 3DGS in about 20–30 minutes. Exports PLY, OBJ/USDZ, and other splat- friendly formats.
is no industry-standard format dedicated to Gaussian Splatting. Key Characteristics Size / Quality Primary Use Cases & Tools PLY Originally for point clouds Uncompressed Common default in tools like nerfstudio SPLAT Originated from the official Inria implementation Smaller than PLY but implementation-dependent GraphDeco Viewer, etc. NPZ NumPy-compressed format output by Compact 3DGS Higher compression than PLY with near-lossless quality Compact-3DGS, etc. SOG WebP textures plus meta.json 15–20× smaller than PLY with lossy compression PlayCanvas, etc. SPZ Gzip-based binary proposed by Niantic Around 10× smaller than PLY with production-ready quality Scaniverse, etc.
Control exposure and lighting Keep the background simple Move the camera instead of using a turntable (unless you intend to remove the entire background) Example footage (feel free to use it!)
the xhiroga fork) 2. Install COLMAP 3. Run a command such as scripts/splatfacto.sh $VIDEO_FILE Processing time and file-size reference Example: 12-second HD video → 34 minutes total (10 minutes for COLMAP, 24 minutes for splat generation) → 35 MB PLY file
yet support Gaussian Splatting, but exporting in COLMAP format prepares the data for splat creation. That way you can start from stable LiDAR captures. 1. RealityScan & postshot (demo by the speaker) ↩︎ [1]
generated with nerfstudio or postshot directly in a browser. Steps 1. Open the PlayCanvas Model Viewer 2. Drag and drop your .ply or .sog files [1]: PlayCanvas Model Viewer (demo by the speaker)
3D and 4D Gaussian Splatting. Typical applications include: Bringing expressions that once required CG into live-action footage Intentionally embracing the distinctive look of Gaussian Splatting rather than aiming for pristine realism
Gaussian Splatting. The team fixed the camera angle while introducing glitch-like disruptions to emphasize the volumetric nature of the footage. [1]: Framestore, "https://www.instagram.com/reel/DO6DbZkjBfu/"
you can share splats you create. These are just examples. VRChat: Build your own worlds and embed splats. There is also a community library. Arrival.Space: Emerging metaverse accessible in the browser. Gracia: App that displays 3D/4D Gaussian Splatting on SteamVR and Meta Quest. Scaniverse: Supports capturing, publishing, and searching splats directly within the app.