Rust for Geospatial Data Processing: A Case Study with CityGML Converter for PLATEAU, Japan's Open Digital Twin Models / FOSS4G ASIA 2024

https://foss4g.asia/2024/
https://talks.geoinfo-lab.org/foss4g-asia-2024/talk/LXQ87Z/

code: https://github.com/MIERUNE/plateau-gis-converter

Transcript

1. Rust for Geospatial Data Processing A Case Study with CityGML Converter for PLATEAU, Japan's Open Digital Twin Models 2024-12-16 Sorami Hisamoto FOSS4G ASIA 2024
2. Rust PLATEAU nusamai Fast, reliable, & ef fi cient programming language Open-source CityGML data conversion tool Digital Twin project led by the Japanese ministry
3. https: // github.com/MIERUNE/plateau-gis-converter
4. https: // github.com/MIERUNE/plateau-gis-converter PLATEAU GIS Converter Convert 3D models of Japanese cities into various geospatial formats. codename “nusamai” Written in Rust, developed by MIERUNE Inc.
5. https: // github.com/MIERUNE/plateau-gis-converter github.com/MIERUNE/plateau-gis-converter OPEN SOURCE (MIT License)
6. @sorami HISAMOTO Sorami ‣ MIERUNE Inc., a geospatial company in Japan ‣ Exploring “Tools for Thought” with computers ‣ Love open source, love community! Hello, FOSS4G community!
7. We are MIERUNE ! ‣ A geospatial company in Japan ‣ Since 2016 - currently 30+ members ‣ A company born from the FOSS4G community Founded by the organizers of FOSS4G Hokkaido
8. Contributions to the community QGIS 5PUBM
   € 
   d
   
   € 
   WJB
   UIF
   DFSUJ fi DBUJPO
   QSPHSBN MapLibre 5PUBM
    
   d
   0UIFS
   TQPOTPST
   JODM
   "84
   .FUB
   
   .JDSPTPGU
9. Contributions to the community ‣ QGIS Country User Group, Voting Member ‣ MapLibre, Voting Member ‣ OSGeo, Charter Member ‣ OSGeo.JP, Institutional Member ‣ etc., ...
11. FOSS4G ASIA 2024 ‣ We are a GOLD SPONSOR ! ‣ plus 7 presentations !
12. Visit our SPONSOR BOOTH !
13. Rust PLATEAU nusamai Fast, reliable, & ef fi cient programming language Open-source CityGML data conversion tool Digital Twin project led by the Japanese ministry
14. Rust PLATEAU nusamai Fast, reliable, & ef fi
15. https: // www.mlit.go.jp/plateau/about/
16. https: // www.mlit.go.jp/plateau/about/ ‣A project to create and utilize 3D city models of Japanese cities ‣Led by the government (MLIT: Ministry of Land, Infrastructure, Transport & Tourism) ‣Started in year 2020
18. https: // plateauview.mlit.go.jp/
19. https: // plateauview.mlit.go.jp/ Not just simple geometry, but data re fl ecting the real world; year, altitude, land use, etc.
20. https: // www.mlit.go.jp/plateau/use-case/uc22-021/ Use Case: Potential Estimation of Solar Power Generation
21. https: // www.mlit.go.jp/plateau/use-case/uc23-12/ Use Case: Urban planning through gami fi cation; via simulation game “Cities: Skylines”
22. An example in the entertainment domain https: // www.youtube.com/watch?v=Jb6Zlg30rgk
23. https: // www.mlit.go.jp/plateau/open-data/ Open Data ! Currently 218 cities, 20 million features
24. Huge, complex speci fi cation https: // www.mlit.go.jp/plateaudocument/
25. Original data: CityGML format ‣ XML-based format for 3D models of cities and landscapes ‣ Classes & relationships for representing urban features Buildings, roads, terrain, vegetation, etc. ‣ Multi-Level of Detail (LoD) Simple footprints (LoD0) to highly detailed interior structures (LoD4) OGC Standard since 2008
26. Image source: Project PLATEAU, MLIT Japan. https: // www.mlit.go.jp/plateau/learning/tpc01-2/ Box (Building + Height) + Roof Shape + Exteriror struct. + Interior struct.
27. Example: Osaka city data ‣ 24.36GB (ZIP archive: 1.42GB) ‣ LOD1: 225.3kᶷ ‣ LOD2: 5.238kᶷ ‣ LOD3: 0.009kᶷ
28. ‣ Overly complex and huge XML format data ‣ Intermediate data: Requires conversion for use ‣ Need costly commercial software for conversion → Handly, free & open tool ? PLATEAU data: The Problems
29. Rust PLATEAU nusamai Fast, reliable, & ef fi cient programming language Open-source CityGML data conversion tool Digital Twin project led by the Japanese ministry
30. Rust PLATEAU nusamai Fast, reliable, & ef fi cient programming language Open-source CityGML data conversion tool Digital Twin project led by the Japanese ministry
31. https: // www.rust-lang.org/
32. https: // www.rust-lang.org/ ‣Modern systems programming langauge First stable release in 2015 ‣Performance, reliability, and productivity Highly performant, comparable to C and C++ Reliable via its elegant language design Modern developer experience ‣Community-led, open source
33. Ferris the crab, unof fi cial mascot for Rust
35. https: // georust.org/
36. https: // github.com/georust/geo/pull/797 3D support is not (yet) available
37. https: // github.com/georust/geo/pull/797 Cross-platform apps with Rust & Web technologies (all major desktop & mobile platforms) (Think “Electron for Rust”)
38. Rust PLATEAU nusamai Fast, reliable, & ef fi cient programming language Open-source CityGML data conversion tool Digital Twin project led by the Japanese ministry
39. Rust PLATEAU nusamai Fast, reliable, & ef fi
40. https: // github.com/MIERUNE/plateau-gis-converter
41. https: // github.com/MIERUNE/plateau-gis-converter PLATEAU GIS Converter Convert 3D models of Japanese cities into various geospatial formats. codename “nusamai” Written in Rust
42. Cross-platform App, with Tauri https: // github.com/MIERUNE/plateau-gis-converter/releases/tag/v0.0.0-alpha.37
43. CityGML → Various data formats ‣ GeoPackage ‣ GeoJSON ‣ 3D Tiles ‣ MVT Mapbox Vector Tiles ‣ glTF ‣CZML ‣KML ‣Minecraft ‣Wavefront OBJ ‣Shapefile 🪦
44. Why Rust ? ‣Performance To process large data in realistic time ‣Developer Experience Nice type system & modern tooling helps a lot ‣Cross-platform, GUI Windows & macOS, GUI instead of CUI → Tauri!
45. Why Rust ? ‣Performance To process large data in realistic time ‣Developer Experience Nice type system & modern tooling helps a lot ‣Cross-platform, GUI Windows & macOS, GUI instead of CUI → Tauri! .. and for FUN ! The team wanted to try Rust
46. Python or JavaScript? ‣ Great GIS ecosystem ‣ But … ‣ Lack of nice GUI tooling ‣ Performance issue ‣ GUI with Electron ‣ But … ‣ Lack of libraries for the domain ‣ Performance issue (?)
47. Rust is great, but we had to create a lot of things from scratch MIERUNE/ fl atgeom-rs Ef fi cient Flat data structures instead of jagged arrays https: // github.com/MIERUNE/flatgeom-rs MIERUNE/earcut-rs A Rust port of the Earcut polygon triangulation library https: // github.com/MIERUNE/earcut-rs plus, Projection transformation, XML (CityGML) parsing, 3D Tiles generation, MVT generation, etc., …
48. Yes! In the end, are we happy with Rust? I don’t think this could have been achieved with other languages or setups.
49. In the end, are we happy with Rust? Rust was perfect for this speci fi c problem as we were building a fundamental tool. However, in many cases, you may be better off quickly using existing great libraries, such as those available in Python. The right tool for the right job; there is no language or setup that fi ts all problems.
50. In the end, are we happy with Rust? Learning and writing Rust was FUN! You truly feel like you are learning the real fundamentals of computing. As a software engineer, that is a great experience and will make you a better developer, even if you don’t use Rust on a day-to-day basis.
51. More detail in this talk: Rust.Tokyo 2024, Satoru Nishio (MIERUNE) (Sorry, in Japanese language; feel free to open an issue for questions!) https: // speakerdeck.com/nokonoko1203/ri-ben-quan-guo-du-shi-3dhua-hurosiekuto-plateau-todetabian-huan-oss-plateau-gis-converter-nogong-kai
52. https: // github.com/MIERUNE/plateau-gis-converter github.com/MIERUNE/plateau-gis-converter OPEN SOURCE (MIT License) Have a look !
53. Rust PLATEAU nusamai Fast, reliable, & ef fi cient programming language Digital Twin project led by the Japanese ministry Open-source CityGML data conversion tool