Do Deep Neural Networks Learn Full Waveform LiDAR Data?
I have talked about raw full waveform lidar data analysis using deep learning method called pointnet at Joint Student Seminar on Remote Sensing and Geoinformatics.
“waveform” discretely. • Providing not only 3D point clouds, but also additional information about the target properties. nData Analysis • Costly and time consuming at manual processing. Automatic analysis method for "raw” waveform data is needed. 3 Cited from “Urban land cover classification using airborne LiDAR data: A review”
• 2 stages supervised classification ‣ 1st stage: Waveform analysis by 1D CNN ⁃ Highly miss classification ‣ 2nd stage: Spatial analysis by 2D CNN ⁃ Raw classification results are converted to grid data ⁃ Prediction the class each grid. • Problems ‣ Spatial raw waveforms are not used in 1st stage. Þ Spatial learning method for raw waveform data are needed to improve performance. 4 Waveform Spatial
deep learning method for spatially irregular data. • Input order invariant network. nAuto Encoder as representation learning • Representation learning is data driven feature extraction method. • Auto Encoder is one of the method. • Auto Encoder can extract low dimensional latent vector from high dimensional data such as Image or some spatial data. Objective in this study: Using a deep learning method, a new representation learning method for spatially distributed raw full-waveform data. 5
Encoder: PointNet based • Decoder: simple Multi Layer Perceptron(MLP) 7 62 2,048 … 1D Conv features features features features features features … Input full waveform LiDAR Data Output full-waveform LiDAR Data Max Pool MLP 62 2,048 Latent vector PointNet based Encoder Decoder T-nets
CNN: Extract local features • MaxPool: Extract grobal features • T-nets: Extract rotation invariant features 8 62 2,048 … 1D Conv features features features features features features … Input full waveform LiDAR Data Output full-waveform LiDAR Data Max Pool MLP 62 2,048 Latent vector PointNet based Encoder Decoder T-nets
layers to produce reconstructed data ! of 2,048 × 62 dimensions, i.e., the same as those of input data 9 62 2,048 … 1D Conv features features features features features features … Input full waveform LiDAR Data Output full-waveform LiDAR Data Max Pool MLP 62 2,048 Latent vector PointNet based Encoder Decoder T-nets
novel representation learning method for spatially distributed full-waveform data observed from an ALS using an AE-based architecture called FWNetAE. • The results demonstrate a generalization error for invisible test data. • Moreover, the FWNetAE encoded a meaningful latent vector and the decoders reconstructed the spatial geometry and its waveform value from the encoded latent vector. • However, the PointNet-based encoders could not extract features at various resolutions. nFuture Study • Modern Hieratical learning: PointNet++, Dynamic Graph CNN • Application for Supervised Learning 17