Introduction to Point Tiler

Transcript

1. OPENING 01 / 20 Introduction to Point Tiler A tool and techniques for converting large point clouds into 3D Tiles Point Cloud 3D Tiles 1.1 Rust CLI Satoru Nishio / MIERUNE Inc. / @nokonoko1203
2. ABOUT ME 02 / 20 Satoru Nishio (@nokonoko̲1203) Engineering Manager at MIERUNE Inc. GIS & web development. 3D WebGIS Point Cloud AWS Community Builder (AI Engineering) PLATEAU ADVOCATE / Cesium Certified Developer X: twitter.com/nokonoko̲1203 LinkedIn: linkedin.com/in/satoru-nishio
3. REALITY 03 / 20 In theory 3 steps ̶ in reality, several problems ① Pick a tool › ② Point it at your files › ③ Run it ̶ in theory 01 Points land in the wrong place 02 Distributed data is too big 03 Memory dies mid-conversion 04 Output is heavy too JP plane rectangular CRS has X and Y swapped Ships as plain LAS. e.g. Expo site ≈ 940GB City-scale data does not fit in RAM 1000s of files, tens of GB. Each failure = half a day If conversion is fast and stable, trial and error works
4. POINT TILER 04 / 20 LAS/LAZ/CSV → 3D Tiles 1.1(GLB) Rust CLI, MIT license One CLI command → automate it with cron / CI curl -sSf .../install.sh | bash Linux / macOS via script · Windows: .exe from GitHub Releases
5. BENCHMARK Benchmark Input Osaka Expo site point cloud ̶ ≈ 940GB of LAS Time ≈ 3 hours Compare In the past, converting a 260GB point cloud took about 7 hours ※ This is the out-of-memory external sort path (mechanism in ③ Memory) 05 / 20
6. COMMAND The command = today's agenda 4 option groups ̶ one answer per trap from the previous slide. ptiler --input /path/to/data/*.laz ¥ --output /path/to/output ¥ --input-epsg 6677 --output-epsg 4979 ¥ # ① CRS & input --min 15 --max 18 ¥ # ② tile design --max-memory-mb 8192 --threads 8 ¥ # ③ memory & threads --quantize --meshopt --gzip-compress # ④ compression 06 / 20
7. ① INPUT ̶ LAS AND LAZ 07 / 20 LAS or LAZ ̶ Point Tiler reads both as they are LAS̶ big, common LAZ̶ small, handy › Point Tiler ̶ Point cloud data is often shared as LAS. LAS files are big. ̶ LAZ is the packed form of LAS. Same points, much smaller. Handy for storing and sharing. ̶ Point Tiler reads both. No need to unpack LAZ first.
8. ① LAZ INTERNALS 08 / 20 Inside LAZ: chunks of 50,000 points Header metadata › Chunk #1 › Chunk #2 50k points › Chunk #N … ̶ Point Tiler unpacks one chunk at a time ̶ Only one chunk is in memory ̶ nothing is written back as LAS › Chunk table offsets
9. ① CRS TRAP → PROJ 09 / 20 The CRS trap: explicit EPSG codes + PROJ JP plane rectangular CRS (EPSG:6669‒6687) X = north, Y = east ̶ the opposite of math Mixing them up throws NO error ̶ points appear rotated 90° or in the wrong place Fix 1 You give input and output EPSG codes ̶ no guessing Fix 2 The math is done by PROJ ̶ the standard GIS library
10. ② LOD BASICS 10 / 20 LOD = geometricError; "zoom level" is just a ruler LOD switches by geometricError (roughness in meters), not by zoom level min / max use 2D-style zoom as a simple ruler z15 geometricError ≈ 64m z18 geometricError ≈ 8m
11. ② QUADTREE 11 / 20 Generate from detailed → coarse All points → max zoom › merge 4 children, decimate ̶ Source points are read only once ̶ Upper levels get lighter › … › min zoom
12. ② DECIMATION 12 / 20 Decimation: voxels proportional to geometricError Voxel size follows geometricError One rule for all zooms ̶ coarser zoom, bigger voxel ̶ Keep 1 real point per voxel ̶ the one closest to the center Start from the defaults (z15‒18) → close views too thin? max +1 Output is for display ̶ for survey-grade accuracy, use LAS/LAZ or COPC
13. ③ MEMORY BUDGET 13 / 20 Just declare: "this machine may use 8GB" --max-memory-mb 8192 estimated size × 5 ≤ limit › All in memory estimated size × 5 > limit › External sort estimated size = point count × record length (from file headers) Why ×5: real memory use is several times the raw points
14. ③ EXTERNAL SORT 14 / 20 External sort: the basic idea An old, well-known way to sort data that does not fit in memory 1 2 Sort small pieces, write each piece to disk Merge all sorted files → whole dataset sorted Tiling = points of the same tile come together = sorting
15. ③ READ → SORTED FILES 15 / 20 Read with a queue → files sorted by tile ID Reader threads 1 point = › Tile ID Bounded queue XYZ coords › Sort by tile ID RGB color Tile IDs follow a Hilbert curve ̶ close tiles → close IDs › Write to disk = 38 bytes fixed
16. ③ SHARD + MERGE 16 / 20 Shard = one min-zoom cell; merge inside it shard Merge sorted files › Tile done → thin, write › GLB out Shard done → temp files deleted right away Temp disk ≈ 1.2‒1.5× the LAS size
17. ③ I/O + PARALLELISM 17 / 20 Disk speed matters ̶ each stage runs in its own way Files are written and read back many times → use a fast local drive Read + transform Shard merge Aggregation + GLB Threads + a queue One shard at a time In parallel with Rayon
18. ④ QUANTIZE 18 / 20 Compression stage 1 ̶ quantize: 16 → 12 bytes per point 16B/pt Normal f32 ×3 RGB · 12B/pt (−25%) --quantize u16 ×3 · RGB · Tile-local coords have a small range → 16-bit is fine ⚠ Required glTF extension ̶ old viewers cannot open it. CesiumJS OK since 1.97 (2022)
19. ④ MESHOPT + GZIP 19 / 20 Stages 2 & 3 ̶ meshopt and gzip ① Quantize ② meshopt ③ gzip Data representation Binary compression Transport compression ⚠ .glb files hold gzip bytes → server must send Content-Encoding: gzip The 3 stages are independent ̶ when in doubt, turn them all on
20. WRAP UP / FIN Takeaways ̶ and where to find Point Tiler 01 Read the compressed input directly 02 Ask for the CRS ̶ never guess 03 Let the user set a memory limit; switch paths automatically github.com/MIERUNE/point-tiler(MIT) For CityGML buildings → 3D Tiles: PLATEAU GIS Converter Issues / PRs welcome ̶ thank you! 20 / 20