Scene understanding is a crucial capability for a robotic system for reliable and robust operations in different applications. While this topic has been extensively study from single and static camera images, it is not that well understood in the robotics context where several challenges have to be solved.
- A robot provides a sequential stream of sensor information from potentially many modalities (RGB, IMU, LiDAR, thermal, ...).
- There is an absence of synchronized datasets for different configurations in the sensor-suite for every robot.
- The system should be capable of real-time performance under hardware constrained resources.
- The system should opportunistically use the sensor information available while being robust to missing data due to sensor failures.
**What we offer**
- Contributing to ongoing research in an exciting emerging field.
- Working with one of the largest robotics research teams in the world.
- Strong ties with major industrial partners and numerous spin-off companies.
- Implement, train, and evaluate a state of the art multi-modal autoencoder for scene understanding from single frames.
- Extend the (or propose a novel) model for learning from different dataset with different sensor information.
- Extend the (or propose a novel) model for learning from temporal sequences.
- Extend the (or propose a novel) model for opportunistic (or on-demand) use of a sensor information.
- Evaluate the performance in terms of accuracy, computational cost, and robustness to missing data.
The WPs will be adapted depending on the timeline available for the project.
- Highly motivated and independent student.
- Strong C++ and/or Python coding skills.
- A good understanding of probability, and linear algebra.
- Excellent command of machine learning algorithms, particularly Neural Networks, and experience in using typical machine learning frameworks such as TensorFlow or Caffe.
- Students from outside of D-MAVT (particularly, from D-INFK, D-ITET, D-PHYS, and D-MATH) are also highly encouraged to apply.
If you are interested, please send your grade transcripts, CV, and a few sentences about your coding and machine learning background to Cesar Cadena (email@example.com).
CLS Student Project (MPG ETH CLS)
Information, Computing and Communication Sciences
Engineering and Technology