Satellite Data and Super-Resolution to enhance a Slope Soaring Simulator by Schalk Heunis

Imagine soaring the slopes of el-Capitan in Yosemite, Rainbow mountain in Peru or even replicating the wingsuit glide path of Jeb Corliss! And doing it under lockdown

Slope soaring requires one to go outside and spend time with friends which was not possible during lockdown, so the next best thing is a simulator.

This talk will cover how a 10 year old, abandoned open source simulator (SSS - http://www.rowlhouse.co.uk/sss/index.html) was enhanced using Python with:

* Digital Elevation Models from Google Earth Engine
Enhancing Elevation Model Super Resolution using Deep Learning
* Satellite imagery as scenery from anywhere in the world courtesy of Bing
* Multi-player networking

The result is stunning scenery and the ability to fly any slope you can dream of.

The talk will cover:

* Introduction to the slope soaring simulator (SSS) and motivation for the wanting to use this under COVID'19 lockdown
* Multi-player networking using Python cstruct and socket libraries
* Extracting digital elevation models (DEM) and satelite data using Google Earth Engine Python API
* Downloading aerial photography from Bing using Python and mapping it to the DEM
* Downloading Hi-res LIDAR point cloud data and processing as target for Super Resolution using PDAL Python API
* Enhancing the elevation model using Deep Learning Super Resolution from 30m to 5m accuracy
* Adding skybox scenery to complete the picture

This talk is for anyone interested in working with satellite data, super-resolution and slope soaring simulations

Transcript

1. Satellite data & Super resolution to enhance a Slope Soaring Simulator Schalk Heunis
2. Previous Pycon Talks • 2019 -Edge Computing at the Edge of the World
5. Amphitheatre – Climb & Fly 12 April
6. Slope Soaring Simulator
7. Slope Soaring Simulator
9. Multi Two player
10. Two player à Multi player Python Server SSS SSS SSS SSS struct
11. cstruct • pip install cstruct
13. Import DEM • Add slide with orog documention
14. Digital Elevation Models
15. Digital Elevation Model
16. Digital Elevation Model
17. Digital Elevation Model - Python • pip install rasterio
18. Digital Elevation Model - Python
19. https://touchterrain.geol.iastate.edu
20. Satellite Data
21. Google Earth Engine • pip install earthengine-api
22. Google Earth Engine
25. Higher Resolution Data: LiDAR
26. LiDAR
27. https://usgs.entwine.io/
28. Super Resolution Learn transform Low Res (30m) High Res (5m)
29. Super Resolution • GAN • Unsupervised • Hard to train • Cool • Supervised • Simple • Fast to train
30. Fast.AI
31. Fast.ai • conda install -c fastai -c pytorch -c anaconda fastai gh anaconda • Lesson 7: Resnets from scratch; U-net; Generative (adversarial) networks • https://course19.fast.ai/ • https://github.com/fastai/course- v3/blob/master/nbs/dl1/lesson7-superres.ipynb
32. U-Net: Convolutional Networks for Biomedical Image Segmentation 41m parameters Fast AI: Pretrained Resnet34
33. Super Resolution – fast.ai – UNet
34. LiDAR - Python • pip install PDAL
35. PDAL : resource • https://usgs-lidar-public.s3-us-west-2.amazonaws.com/{resource}/ept.json • resource:
36. Training data • 16 446 tiles • Each tile 128x128 @ 5m • 0.4 km2 • 6 736 km2
37. Train & Validate Low Res (30m) Predicted Hi Res (5m)
38. Amphitheatre Low Res (30m) Predicted (5m)
39. Amphitheatre Low Res (30m) Predicted (5m)
40. Add ground texture • Bing Maps • https://github.com/llgeek/Satellite-Aerial-Image-Retrieval
41. Ampitheatre
45. El-Capitan Yosemite
46. Rainbow Mountain Peru
47. Glider & cockpit view
48. Jeb Corliss – “Grinding The Crack”