GenerationAI Paris 2025 | City of Light (COL)

City of Light (COL): A City-Scale, Geo-Anchored Urban Simulator with Multi-Sensor Streams designed for AI applications
Speaker: Ilias Sarbout, Research Engineer at Rothschild Foundation Hospital

City of Light (COL) is a city-scale, geo-anchored urban simulation platform presented by Ilias Sarbout, Research Engineer at Rothschild Foundation Hospital, designed to support high-throughput AI research and data generation. The talk introduced COL’s inner-Paris digital twin, which combines public GIS data with a Unity-based environment to generate synchronized multi-sensor streams—including RGB, depth, surface normals, and semantic labels—at extremely high frame rates on standard hardware.

A key highlight was TURBO, a lightweight Unity–Python shared-memory bridge that eliminates costly data serialization, enabling real-time, closed-loop AI training at scale. Together, COL and TURBO enable faster dataset generation, scalable reinforcement learning, and more accurate sim-to-real transfer—addressing critical challenges in modern computer vision, robotics, and embodied AI research.

------------------------------------

Conference Details:
Conference: GenerationAI Paris 2025, part of FOST (Future of Software Technologies)
Theme: Enterprise GenAI-readiness with the API mindset
Date: 1 - 3 December 2026 • CNIT Forest – Paris
--------------------------

Resources from apidays:
Join our upcoming conferences: https://www.apidays.global/
Read the latest API news: https://www.apiscene.io
Explore the API Landscape: https://apilandscape.apiscene.io/

Transcript

1. City of Light (COL): A City-Scale, Geo-Anchored Urban Simulator with Multi-Sensor Streams designed for AI applications
2. City of Light (COL): A City-Scale, Geo-Anchored Urban Simulator with Multi-Sensor Streams designed for AI applications Ilias Sarbout Rothschild BRAIN Lab, Hôpital Fondation A. de Rothschild , Sorbonne Université, Équipe Inria “Aramis” PhD Supervision : Daniel Racoceanu, PhD, Dan Milea, MD, PhD
3. Outline • From visual impairment to virtual environments • City-Of-Light • Questions (5mn)
4. 200 000 ~ 600 000 blind people in France 5% < use white cane 5% < use guide dog What can we do for the remaining 90% ?
5. Chen et al. 2022 Optogenetics Cortical prostheses Niketeghad et al. 2018 Qiu et al. 2020 Sonogenetics 1. Brain focused approaches
6. Visual prostheses : concept
7. 2. Retinal prostheses Alpha-AMS : CE Argus II : CE, FDA PRIMA : ongoing FDA
8. 2. Retinal prostheses Alpha-AMS : CE dissolution in 2019 Argus II : CE, FDA bankruptcy in 2020 PRIMA : ongoing FDA judicial liquidation in 2024
9. 2. Retinal prostheses
10. 2. Retinal prostheses I. Sarbout et al. Eye and Brain, 2025
11. 3. Sensory Substitution Devices Bach-Y-Rita, Brain Mechanisms in Sensory Substitution, 1972
12. 3. Sensory Substitution Devices Bach-Y-Rita, Brain Mechanisms in Sensory Substitution, 1972
13. 3. Sensory Substitution Devices
14. Tongue Display Unit (Kaczmarek, 2011) 3. Sensory Substitution Devices
15. 3. Sensory Substitution Devices Activates the visual cortex Amit et al. (2012)
17. AI AI integration in Sensory Substitution Devices
18. I. Sarbout et al. (under review manuscript) Current use of AI for Sensory substitution Devices
19. I. Sarbout et al. (under review manuscript) Current use of AI for Sensory substitution Devices
20. Current use of AI for Sensory substitution Devices Trained on specific data (3) I. Sarbout et al. (under review manuscript)
21. Hypothesis : We can specifically train an AI system adapted to blind patients.
22. Hypothesis : We can specifically train an AI system adapted to blind patients. AI Training requires Data Main challenge : Data in blind patients is missing
23. Hypothesis : We can specifically train an AI system adapted to blind patients. AI Training requires Data Main challenge : Data in blind patients is missing Let’s use synthetic data from a blind digital twin
24. Forehead RGB view Forehead Semantic View 1st proof of concept: NavIndoor
25. Safety level Advice NavIndoor : model training
27. RESULTS : our AI output w.r.t indoor real world photographs
29. 29 Real world navigation Adafruit ESP32 Feather S3 + Lipo 3V Jetson Orin Nano + Wide Angle Camera 3D printed handheld device* *3D design and printing by Dune Amr–Tardif
30. 30 Encouraging results but : • Indoor • Static & deterministic • Env. Lacks realism
33. City Of Light : Toward (more) realistic high performance urban simulation.
34. Features
35. Features - Full 1:1-scale reconstruction of the city of Paris
36. Toward A Digital Twin of Paris Division des plans de voirie de Paris
37. Toward A Digital Twin of Paris Division des plans de voirie de Paris Data processing & fusion
38. 38 Examples of processing step…
39. 39 Examples of processing step…
40. 40 Examples of processing step…
41. Toward A Digital Twin of Paris Division des plans de voirie de Paris Data processing & fusion
42. Toward A Digital Twin of Paris Division des plans de voirie de Paris Data processing & fusion 2D to 3D
43. Features - Full 1:1-scale reconstruction of the city of Paris
44. Features - Full 1:1-scale reconstruction of the city of Paris - Embodied agent equipped with multi-sensor support
45. Toward A Digital Twin of Paris Division des plans de voirie de Paris Data processing & fusion 2D to 3D
46. Toward A Digital Twin of Paris Division des plans de voirie de Paris Data processing & fusion 2D to 3D
47. Features - Full 1:1-scale reconstruction of the city of Paris - Embodied agent equipped with multi-sensor support
48. Features - Full 1:1-scale reconstruction of the city of Paris - Embodied agent equipped with multi-sensor support - Stochastic behaviors (pedestrians, vehicles)
49. Features - Full 1:1-scale reconstruction of the city of Paris - Embodied agent equipped with multi-sensor support - Stochastic behaviors (pedestrians, vehicles)
50. 50 Vehicles & Pedestrians
51. 51 Vehicles & Pedestrians
52. Features - Full 1:1-scale reconstruction of the city of Paris - Embodied agent equipped with multi-sensor support - Stochastic behaviors (pedestrians, vehicles)
53. Features - Full 1:1-scale reconstruction of the city of Paris - Embodied agent equipped with multi-sensor support - Stochastic behaviors (pedestrians, vehicles) - High Performance
54. 54 TURBO : Shared Memory Exchange Buffer Up to 1300 FPS (up to * 600 gain)
55. TURBO : Shared Memory Exchange Buffer
56. TURBO : Shared Memory Exchange Buffer
57. TURBO : Shared Memory Exchange Buffer
58. TURBO : Shared Memory Exchange Buffer
59. 59 TURBO : Shared Memory Exchange Buffer
60. 60 TURBO : Shared Memory Exchange Buffer
61. Features - Full 1:1-scale reconstruction of the city of Paris - Embodied agent equipped with multi-sensor support - Stochastic behaviors (pedestrians, vehicles) - High Performance
62. Features - Full 1:1-scale reconstruction of the city of Paris - Embodied agent equipped with multi-sensor support - Stochastic behaviors (pedestrians, vehicles) - High Performance
63. Features - Full 1:1-scale reconstruction of the city of Paris - Embodied agent equipped with multi-sensor support - Stochastic behaviors (pedestrians, vehicles) - High Performance - And more…
64. Features - Full 1:1-scale reconstruction of the city of Paris - Embodied agent equipped with multi-sensor support - Stochastic behaviors (pedestrians, vehicles) - High Performance - And more…
65. Features - Full 1:1-scale reconstruction of the city of Paris - Embodied agent equipped with multi-sensor support - Stochastic behaviors (pedestrians, vehicles) - High Performance - And more…
66. Features - Full 1:1-scale reconstruction of the city of Paris - Embodied agent equipped with multi-sensor support - Stochastic behaviors (pedestrians, vehicles) - High Performance - And more…
67. Mehdi Ounissi PhD Ilias Sarbout MSc, Eng Prof. Dan Milea Working Team Prof. Daniel Racoceanu Theo Cazenave-Coupet, Intern
68. Mehdi Ounissi PhD Ilias Sarbout MSc, Eng Prof. Dan Milea Working Team Prof. Daniel Racoceanu https://github.com/iliassarbout/CityOfLight Project Page : Theo Cazenave-Coupet, Intern
69. Mehdi Ounissi PhD Ilias Sarbout MSc, Eng Prof. Dan Milea Working Team Prof. Daniel Racoceanu https://github.com/iliassarbout/CityOfLight Project Page : Theo Cazenave-Coupet, Intern AAAI, January, 2026, Singapore (accepted paper) Contact: ilias.sarbout@gmail.com
70. Mehdi Ounissi PhD Ilias Sarbout MSc, Eng Prof. Dan Milea Working Team Prof. Daniel Racoceanu https://github.com/iliassarbout/CityOfLight Project Page : Theo Cazenave-Coupet, Intern Contact: ilias.sarbout@gmail.com Thank you for your attention !