Robotic Scene Understanding with Deep Semi-Supervised Learning

4112524f b58e 4ab2 837c 7aabc2af7781.jpeg 500 500

Organization: Autonomous Systems Lab

Involved Host(s): Cadena Cesar

Abstract: The goal of this project is implementing, developing and evaluating a machine learning system for robotic scene understanding over time from multiple sensor modalities.

Description: Scene understanding is a crucial capability for a robotic system for reliable and robust operations in different applications. While this topic has been extensively study from single and static camera images, it is not that well understood in the robotics context where several challenges have to be solved. - A robot provides a sequential stream of sensor information from potentially many modalities (RGB, IMU, LiDAR, thermal, ...). - There is an absence of synchronized datasets for different configurations in the sensor-suite for every robot. - The system should be capable of real-time performance under hardware constrained resources. - The system should opportunistically use the sensor information available while being robust to missing data due to sensor failures. **What we offer** - Contributing to ongoing research in an exciting emerging field. - Working with one of the largest robotics research teams in the world. - Strong ties with major industrial partners and numerous spin-off companies.

Work Packages: - Implement, train, and evaluate a state of the art multi-modal autoencoder for scene understanding from single frames. - Extend the (or propose a novel) model for learning from different dataset with different sensor information. - Extend the (or propose a novel) model for learning from temporal sequences. - Extend the (or propose a novel) model for opportunistic (or on-demand) use of a sensor information. - Evaluate the performance in terms of accuracy, computational cost, and robustness to missing data. The WPs will be adapted depending on the timeline available for the project.

Requirements: - Highly motivated and independent student. - Strong C++ and/or Python coding skills. - A good understanding of probability, and linear algebra. - Excellent command of machine learning algorithms, particularly Neural Networks, and experience in using typical machine learning frameworks such as TensorFlow or Caffe. - Students from outside of D-MAVT (particularly, from D-INFK, D-ITET, D-PHYS, and D-MATH) are also highly encouraged to apply.

Contact Details: If you are interested, please send your grade transcripts, CV, and a few sentences about your coding and machine learning background to Cesar Cadena (

Machine Learning TensorFlow Caffe Sensor fusion Autoencoder

Labels: Semester project Master Thesis CLS Student Project (MPG ETH CLS)
Topics: Information, Computing and Communication Sciences Engineering and Technology

© 2017, Copyright Max-Planck-Gesellschaft - Imprint