Building and Consuming Urban Digital Twins with Open-Source Tools

This engaging workshop will explore the innovative use of digital twins in urban planning through the utilization of open-source geospatial technologies. Participants will learn how to transform vector data and point clouds into dynamic 3D models using the MapStore WebGIS framework, supported by Cesium JS.

This hands-on session is designed to equip attendees with the skills needed to create, visualize, and analyze digital replicas of urban environments, enhancing urban management and planning. Attendees will also work with open data from the city of Washington D.C., understanding the crucial role of interoperability in geospatial data management, setting the stage for smarter, sustainable urban futures.

1. Building and Consuming Urban Digital Twins with Open-Source Tools Stefano Bovio Tobia Di Pisa Georgios Petrakis Digital Twin Toolbox 13th of June 2024
2. Introduction Digital Twin Toolbox 13th of June 2024
3. Introduction Consuming 3D data in WebGIS applications has increasingly become a requirement over the last years. 3D Tiles became one of the most common OGC standards for streaming and rendering 3D geospatial contents on the web such as: • Photogrammetry like LiDAR-derived meshes • 3D Buildings (.obj, .gltf, .glb ...) • Point Clouds • and more ... Digital Twin Toolbox 13th of June 2024
4. Introduction In response to ever-growing and more specific needs in this context, it is usually necessary to: • Identify the best tools for viewing 3D data in 3D Tiles format • Identify tools for converting datasets into 3D Tiles format in a correct and performing way but... what about doing that using Open Source tools? Digital Twin Toolbox 13th of June 2024 Use MapStore to visualize them first!
5. Introduction The Digital Twin Toolbox borns with the aim to support with the conversion processes to 3D Tiles by using Open Source tools: Digital Twin Toolbox 13th of June 2024 • Defining pipelines for common data sources such as SHP and LAS files • Providing necessary tools for visually inspecting and assessing datasets • Allowing ways to manage classification, colorization and resampling • Providing reliable tools to manage tiling options, CRS and georeferencing adjustments • and many more…
6. Main objectives are: • Collect the best OS tools and libraries to process common data sources in the urban environment (SHP and LAS files for now) • Provide workflows to orchestrate a well-driven set of processing chains and methodologies to ◦ Inspect and evaluate data ◦ Prepare/process data ◦ Convert input data in 3D Tiles ◦ Preview data step by step • Provide an user friendly UI to facilitate the work All in a Dockerized environment! Introduction Digital Twin Toolbox 13th of June 2024
7. Use Case Digital Twin Toolbox 13th of June 2024
8. Use Case Objective: Reconstruction of a digital twin model in 3D Tiles format using sample data of the Washington D.C. city Digital Twin Toolbox 13th of June 2024
9. Use Case Steps overview: • Download of data samples from the open data portal of Washington D.C. • Data processing and 3D Tiles conversion using the open source digital- twin-toolbox solution • Visualization of generated 3D Tiles inside a MapStore map viewer Data: • Building volumes in Shapefile format (polygon geometries) • Trees in Shapefile format (point geometries) • Urban environment in LAS format in combination with orthophotos for colorization (point cloud structure) Link to sample data Digital Twin Toolbox 13th of June 2024
10. Use Case Digital Twin Toolbox is a collections of different tools/libraries and workflows inside a docker environment to generate 3D Tiles from common data sources such as Shapefiles and LAS files. This project is still a work in progress and this application is not production ready Digital Twin Toolbox 13th of June 2024
11. Use Case 3D Tiles format is designed for streaming and rendering massive 3D geospatial content such as Photogrammetry, 3D Buildings, BIM/CAD, Instanced Features, and Point Clouds Digital Twin Toolbox 13th of June 2024
12. Use Case MapStore is an highly modular Open Source WebGIS framework to create, manage and share maps, dashboard and geostories. The mapping engines currently supported are OpenLayers, LeafletJS and Cesium.js. Digital Twin Toolbox 13th of June 2024
13. Digital Twin Toolbox Setup Digital Twin Toolbox 13th of June 2024
14. Digital Twin Toolbox Setup Prerequisites: • Docker - open platform for developing, shipping and running application • Git - free and open source distributed version control system Setup steps: • Clone the repository in your workspace directory git clone https://github.com/geosolutions-it/digital-twin-toolbox • Navigate in the cloned digital-twin-toolbox repository cd ./digital-twin-toolbox • Create an .env file by copying the provided .env.sample and renaming it cp .env.sample .env • Start and build the application docker compose up Digital Twin Toolbox 13th of June 2024
15. Digital Twin Toolbox Setup Application by default runs at http://localhost:3000 Digital Twin Toolbox documentation: https://github.com/geosolutions-it/digital-twin-toolbox/wiki Digital Twin Toolbox 13th of June 2024
16. User Interface Digital Twin Toolbox 13th of June 2024
17. User Interface Controls: • located on the top right corner of the screen • change list of properties and action buttons based on the selected input file Digital Twin Toolbox 13th of June 2024
18. User Interface Data preview: • covers all the screen background • preview sample data in 3D • live update of geometries for Shapefiles Digital Twin Toolbox 13th of June 2024
19. User Interface Process feedback: • located on the top left corner of the screen • shows logs for all the actions and processes initialized with the controls panel Digital Twin Toolbox 13th of June 2024
20. Workflows Digital Twin Toolbox 13th of June 2024
21. Workflows Digital Twin Toolbox provides following workflows: • Conversion of shapefile data (polygons, lines, points) into 3D Tiles • Processing point cloud data to fix/manage CRS, resample and colorization • Conversion of point cloud data to 3D Tiles • [Experimental] Conversion of lidar data to a 3D Mesh file and 3D Tiles Digital Twin Toolbox 13th of June 2024
22. Workflows The static/data/ folder contains all the input data (Shapefiles, LAS, …) The sample data needs to be unzipped and the content copied inside this folder: • Shapefile as .zip folder with all sidecar files included • Point cloud file as .las file • Orthophoto for point cloud colorization as .tif files Digital Twin Toolbox 13th of June 2024
23. Workflows Conversion of shapefile polygons data into 3D Tiles Input data: Building_Footprints_join_Building_3D_EPSG_26985.zip Main tools dependencies: PostGIS, pg2b3dm Output: 3D Tiles Digital Twin Toolbox 13th of June 2024
24. Workflows Select Building_Footprints_join_Building_3D_EPSG_26985 (.zip) from input data section Data preview renders the building footprint surfaces Digital Twin Toolbox 13th of June 2024
25. Workflows Update the geometry by creating a volume based on properties: • change the Upper limit height to create a volume using following expression: $minZ + (MAX_Z - MIN_Z) // $minZ is the average minimum z of the feature geometry (computed) // MAX_Z and MIN_Z are properties of the features Digital Twin Toolbox 13th of June 2024
26. Workflows Generate 3D Tiles using the default properties by clicking on the Create tileset button Tip: minimum and maximum geometric error could be adjusted to define the visibility range of the tileset in the map Digital Twin Toolbox 13th of June 2024
27. Workflows Preview of generated 3D Tiles in Cesium.js by clicking on the Tileset preview button in the feedback box Digital Twin Toolbox 13th of June 2024
28. Workflows Conversion of shapefile points data into 3D Tiles Input data: Urban_Forestry_Street_Tree_Benefits_EPSG_26985.zip Main tools dependencies: PostGIS, i3dm.export Output: 3D Tiles Digital Twin Toolbox 13th of June 2024
29. Workflows Select Urban_Forestry_Street_Tree_Benefits_EPSG_26985 (.zip) from input data section Data preview renders the tree points as cubes by default Digital Twin Toolbox 13th of June 2024
30. Workflows Update geometry replacing the default cube with a tree model: • change the Model (.glb) field with the following string: “tree.glb” Notes: • Additional 3D model in .glb format can be added in the static/glb folder • Rotation and scale options supports expression or static values Digital Twin Toolbox 13th of June 2024
31. Workflows Generate 3D Tiles using the default properties by clicking on the Create tileset button Digital Twin Toolbox 13th of June 2024
32. Workflows Preview of generated 3D Tiles in Cesium.js by clicking on the Tileset preview button in the feedback box Digital Twin Toolbox 13th of June 2024
33. Workflows Processing point cloud data to fix/manage CRS, resample and colorization Input data: 1916.las, 1916.tif Main tools dependencies: PDAL Output: LAS (1916_processed.las) Digital Twin Toolbox 13th of June 2024
34. Workflows Select 1916 (.las) from input data section Data preview is empty by default Digital Twin Toolbox 13th of June 2024
35. Workflows Click on the Create a data sample for preview to generate the point cloud preview Note: this sample is limited to a maximum of 500000 points Digital Twin Toolbox 13th of June 2024
36. Workflows Retrieve metadata information by clicking on the Get projection button The projection authority code will be used in the point cloud processing Digital Twin Toolbox 13th of June 2024
37. Workflows Generate a new 1916_processed.las file with following changes: • vertical datum shift with the EGM2008 geoid model From Crs = +init=EPSG:26985 +geoidgrids=./vdatum/egm08_25.gtx To Crs = EPSG:26985+4269 • colorize the point cloud using the 1916.tif image Raster image = 1916.tif Digital Twin Toolbox 13th of June 2024
38. Workflows New file 1916_processed.las available in the data folder Digital Twin Toolbox 13th of June 2024
39. Workflows Conversion of point cloud data to 3D Tiles Input data: 1916_processed.las Main tools dependencies: py3dtiles Output: 3D Tiles Digital Twin Toolbox 13th of June 2024
40. Workflows Select 1916_processed (.las) from input data section then click the Create a data sample for preview button Now the preview shows the color of the raster image applied to the processed point cloud Digital Twin Toolbox 13th of June 2024
41. Workflows Retrieve metadata information by clicking on the Get projection button The projection authority code will be used in the tiling process Digital Twin Toolbox 13th of June 2024
42. Workflows Generate 3D Tiles by clicking on the Create tileset button using following properties: • Add the projection authority number 26985 in the CRS number field • Add a geometric error scale factor of 0.15 Digital Twin Toolbox 13th of June 2024
43. Workflows Preview of generated 3D Tiles in Cesium.js by clicking on the Tileset preview button in the feedback box Digital Twin Toolbox 13th of June 2024
44. Workflows [Experimental] Conversion of point cloud data to mesh (.ply format) Input data: 1916_processed.las Main tools dependencies: PDAL, Open3D Output: Mesh (1916_processed_mesh.ply) Digital Twin Toolbox 13th of June 2024
45. Workflows [Experimental] Select 1916_processed (.las) from input data section Nothing is rendered on the data preview for this type of process Digital Twin Toolbox 13th of June 2024
46. Workflows [Experimental] Click the Create mesh button with the default properties to generate a new .ply file named 1916_processed_mesh in the static/data/ folder Digital Twin Toolbox 13th of June 2024
47. Workflows [Experimental] Conversion of .ply mesh to 3D Tiles Input data: 1916_processed_mesh.ply, 1916.tif Main tools dependencies: Blender Python Output: 3D Tiles Digital Twin Toolbox 13th of June 2024
48. Workflows [Experimental] Select 1916_processed_mesh (.ply) from input data section Digital Twin Toolbox 13th of June 2024
49. Workflows [Experimental] Click on the Create a data sample for preview to generate the point cloud preview Note: this sample is a simplified version of the original mesh Digital Twin Toolbox 13th of June 2024
50. Workflows [Experimental] Generate 3D Tiles by clicking on the Create tileset button using: • CRS property equal to EPSG:26985 • Orthophoto image equal to 1916.tif Note: this process can take up to ~ 30 minutes Digital Twin Toolbox 13th of June 2024
51. Workflows [Experimental] Preview of generated 3D Tiles in Cesium.js by clicking on the Tileset preview button in the feedback box Digital Twin Toolbox 13th of June 2024
52. Tilesets Preview Digital Twin Toolbox 13th of June 2024
53. Tilesets Preview The static/tilesets/ folder contains all generated 3D Tiles outputs and they can be previewed in a Cesium.js client using the url: http://localhost:3000/preview/?{tileset-name} Digital Twin Toolbox 13th of June 2024
54. Tilesets Preview Alternatively tilesets can be previewed in an embedded MapStore map viewer application by selecting tilesets on the controls panel and then clicking on “Show selected tilesets in MapStore” button Tip: refresh the page to visualize the latest generated tilesets Digital Twin Toolbox 13th of June 2024
55. Tilesets Preview Here the final map with all the generated 3D Tiles layer for the Washington D.C. urban sample rendered in MapStore with the help of the Cesium.js map engine MapStore offers different tools to interact with 3D Tiles Digital Twin Toolbox 13th of June 2024
56. Tilesets Preview [Example] Point cloud shading properties to improve points visibility Digital Twin Toolbox 13th of June 2024
57. Tilesets Preview [Example] Style editor with possibility to filter the 3D Tiles feature by property Identify panel with information about selected feature Digital Twin Toolbox 13th of June 2024
58. Tilesets Preview [Example] Style editor with possibility to filter the 3D Tiles feature by property Identify panel with information about selected feature Digital Twin Toolbox 13th of June 2024
59. Tilesets Preview [Example] Different type of measurements for the 3D environment: polyline distance, area, point coordinates, height from terrain, angle and slope Digital Twin Toolbox 13th of June 2024
60. Tilesets Preview Here we presented few examples of the MapStore capabilities, Below links to learn more about the project: • Documentation with user and developer guides https://docs.mapstore.geosolutionsgroup.com/en/v2024.01. 01/ • Repository https://github.com/geosolutions-it/MapStore2 • Latest release https://github.com/geosolutions- it/MapStore2/releases/tag/v2024.01.01 Digital Twin Toolbox 13th of June 2024
61. Ongoing Work: Point cloud Classification Digital Twin Toolbox 13th of June 2024
62. What is point cloud classification ? Digital Twin Toolbox 13th of June 2024 Point clouds images source: Atik et al. 2024 Classified point clouds
63. Why point cloud classification is important? Digital Twin Toolbox 13th of June 2024 Point cloud classification is a crucial part of 3D urban reconstruction since it provides: • Improved data interpretation • Acceleration of the data analysis • Information about the land use / land cover of the scene • Potential tool for further point cloud processing • Useful for many other applications (e.g Self-driving cars, inspection, autonomous navigation etc)
64. Point cloud classification using AI Digital Twin Toolbox 13th of June 2024 • Machine learning algorithms: SVM, Random Forest, Gradient Boosted Trees etc • Deep learning architectures: PointNet, PointNet++, PointCNN, etc Random Forest architecture Image source: https://www.javatpoint.com/machin e-learning-random-forest-algorithm
65. Experimentation & Results Digital Twin Toolbox 13th of June 2024 Experimentation pipeline, using AI-based algorithms for point cloud classification Dataset preparation heterogeneity Classes selection Parameterization with unknown datasets Training Model extraction Model testing Comparison with other models
66. Results of a pre-trained model Digital Twin Toolbox 13th of June 2024 Results using a pre-trained model, trained with the classes: ground, buildings, vegetation and vehicles Point cloud by ODMSemantic3D Prediction of the pre-trained model with ground in gray, buildings in orange, vegetation in green and vehicles in yellow
67. Results of a pre-trained model Digital Twin Toolbox 13th of June 2024 Prediction of the pre-trained model using the Washington DC open LiDAR
68. Training & fine-tuning Digital Twin Toolbox 13th of June 2024 • Goal: Focused on urban areas • Classes: Vegetation, buildings, ground, water • Parameters tuning Training dataset by Washington DC open LiDAR
69. Training & fine-tuning Digital Twin Toolbox 13th of June 2024 Results using samples from densely urban scenes of Washington DC open LiDAR
70. Training & fine-tuning Digital Twin Toolbox 13th of June 2024 Results using samples from sparsely urban scenes of Washington DC open LiDAR
71. Training & fine-tuning Digital Twin Toolbox 13th of June 2024 Results using samples by GeoPortal of Genova
72. Classification results on 3D Tiles Digital Twin Toolbox 13th of June 2024 Results by Washington DC open LiDAR dataset, displayed on the 3D Tiles of the Digital-Twin-Toolbox
73. Future work Digital Twin Toolbox 13th of June 2024
74. 1. Further improvement of classification capabilities of point cloud data (including UI support) 1. More advanced and complete support for photogrammetry processes 1. Automation of the processing chains 1. Support to include LODs and further improve the Tiling System We have in plan to work on a bunch of significant functionalities to enrich the toolbox capabilities: That’s all for a first release of the Digital Twin Toolbox this year! Future work Digital Twin Toolbox 13th of June 2024
75. Check it out on Github: https://github.com/geosolutions- it/digital-twin-toolbox Pre-Release at: https://github.com/geosolutions-it/digital-twin-toolbox/releases/tag/v1.0.0-rc Online Documentation: https://github.com/geosolutions- it/digital-twin-toolbox/wiki Tutorials are also available in the WIKI: https://github.com/geosolutions-it/digital-twin-toolbox/wiki/Tutorials Online resources Digital Twin Toolbox 13th of June 2024
76. That’s all folks! Questions? info@geosolutionsgroup.com Digital Twin Toolbox 13th of June 2024