Scene Completion for Satellite Image-Based Reconstructions

9c134fbb f5cc 40de a1ab 213791c286cc.png 500 500

Starting Date: earliest start: (2018-05-01) latest end: (2018-12-31)

Organization: Photogrammetry and Remote Sensing (Prof. Schindler)

Involved Host(s): Berger Monique

Abstract: Despite the impressive level of detail, reconstructions based on satellite images are typically incomplete due to restrictions on the viewing directions in image collections. The goal of this project is to develop advanced methods for completing such partial reconstructions.

Description: The advent of very high resolution (VHR) satellite imagery has opened up the exciting possibility of detailed 3D reconstruction only from satellite images [3]. However, due to restrictions on the viewing directions in satellite image collections, the reconstructed surfaces are typically incomplete. The goal of the project is to develop advanced methods for completing such partial reconstructions. Satellite surveys often do not contain views from all sides, such that the reconstructions remain incomplete in occluded areas, respectively noisy and contaminated by errors in weakly observed areas. The project aims to mitigate these problems and deliver complete surface models, using recent deep machine learning approaches for scene completion [0-2]. Context information, like symmetry in appearance and geometry, as well as a semantic interpretation of the observed objects may be important cues for completion and filtering of digital surface models. Satellite and airborne data, as well as experience with deep learning frameworks (Keras, TensorFlow) is available. [0]: Häne C, Tulsiani S, Malik J. "Hierarchical surface prediction for 3d object reconstruction." arXiv preprint 1704.00710 (2017). [1]: Dai A et al. "ScanComplete: Large-Scale Scene Completion and Semantic Segmentation for 3D Scans." arXiv preprint 1712.10215 (2017). [2]: Song S et al. "Semantic scene completion from a single depth image." Computer Vision and Pattern Recognition (CVPR),. [3]: Bosch M et al. "Metric Evaluation Pipeline for 3d Modeling of Urban Scenes." ISPRS Archives 42 (2017)

Goal: The goal is to design, implement and evaluate a deep neural network for the completion and post-processing of incomplete digital surface models. The idea is to train the network with complete, high-quality reconstructions derived from airborne photogrammetric survey, such that it can fill in and/or correct missing parts in models reconstructed from satellite data.

Contact Details: Mathias Rothermel,; Martin Oswald,

Satellite imagery 3D reconstruction Neural network Deep learning Scene completion

Labels: Bachelor Thesis Master Thesis CLS Student Project (MPG ETH CLS)
Topics: Information, Computing and Communication Sciences Engineering and Technology