3 dagen geleden - Radboud Universiteit (RU) - Nijmegen
PhD Position Multi-modal self-supervised deep learning for self-driving vehicles
Challenge: Learn 3D environment representations from all vehicle sensor modalities (vision, lidar but also audio or radar) jointly with limited data annotation.
Change: Develop joint self-supervised learning for multi-modal sensor data, a novel shared environment representation, and exploit geometric and temporal constraints.
Impact: Simplify and improve learning-based perception for new vehicle sensors and sensor configurations for safer highly automated driving.
- Mekelweg, Delft, Zuid-Holland
- Tijdelijk contract / Tijdelijke opdracht
- Uren per week:
- 38 - 40 uur
- € 2395 - € 3061 per maand
In this project, you will develop novel multi-modal deep learning techniques to detect the dynamic and static obstacles in the urban environment in self-driving applications. Existing approaches often focus on training models for a single sensor modality (vision, lidar, radar, or audio) or a specific pair of sensors. The overarching goal here is to develop a generic multi-modal self-supervised learning approaches to exploit the complementary nature of all available sensors during training, and thus compensate for the lack of annotated data for some sensors modalities (e.g. acoustics, radar). A possible direction is to simultaneously learn a shared 3D environment representation, and the sensor models to map their measurements to this shared space while preserving measurement uncertainty. To regularize the task, we will seek to exploit geometric constraints in the representation and self-supervised learning strategies. Additionally, past and future observations could be incorporated into the learning strategy to exploit the temporal consistency of the dynamic scene, reducing uncertainty in the constructed representation and possibly capturing cues to anticipate future traffic events. If successful, the developed methods enable self-driving vehicles to achieve state-of-the-art online perception performance for both well-known (vision, lidar) and novel sensing modalities (audio) without the need for extensive data annotation.
The host research group – the Intelligent Vehicles group - focuses on environment perception, dynamics & control, and interaction with humans for intelligent vehicles and automated driving in complex unstructured urban environments. The group is part of the Cognitive Robotics department at the 3ME Faculty, which aims to develop intelligent robots and vehicles that will advance mobility, productivity and quality of life. The department combines fundamental research with work on physical demonstrators in areas such as self-driving vehicles, collaborative industrial robots, mobile manipulators and haptic interfaces.
The new AI lab on 3D Understanding (3DUU) initiated by the 3D Geoinformation Research Group and the Intelligent Vehicles Group is one of the Delft Artificial Intelligence Labs (DAI-Labs). It is a cross-disciplinary research lab seeking to develop state-of-the-art AI techniques for interpreting 3D data and reconstructing 3D objects for large-scale urban applications.
3DUU is a Delft Artificial Intelligence Lab (DAI-Lab). Artificial intelligence, data, and digitalization are becoming increasingly important when looking for answers to major scientific and societal challenges. In a DAI-lab, experts in ‘the fundamentals of AI technology’ along with experts in ‘AI challenges’ run a shared lab. As a PhD, you will work with at least two academic members of staff and three other PhD candidates. In total, TU Delft will establish 24 DAI-Labs where 48 Tenure Trackers and 96 PhD candidates will have the opportunity to push the boundaries of science by using AI. Each team is driven by research questions that arise from scientific and societal challenges and contribute to the development and execution of domain-specific education. Instead of the usual 4-year contract, you will receive a 5-year contract. Approximately a fifth of your time will be allocated to developing ground breaking learning materials and educating students in these new subjects. The experience you will gain by teaching will be invaluable for future career prospects. All team members have many opportunities for self-development. You will be a member of the thriving DAI-Lab community that fosters cross-fertilization between talents with different expertise and disciplines.
- Completed a MSc degree in computer science, artificial intelligence, applied mathematics, or robotics;
- Strong interests and expertise in machine learning (in particular deep learning), numerical linear algebra, statistics, robotics and/or geoinformatics, preferably with a strong publication record;
- An affinity with teaching and guiding students;
- Excellent Python and/or C++ programming skills, and experience with Git;
- Experience with deep-learning frameworks (PyTorch/TensorFlow) at the MSc level, preferably applied to point cloud data and 3D sensors (LiDAR, stereo vision), 2D computer vision, radar, and/or acoustic event detection and localization;
- Ability to act independently as well as to collaborate effectively with members of a larger interdisciplinary team, take initiative, be result oriented, organized and creative;
- Good command of verbal and written English.
TU Delft offers DAI-Lab PhD-candidates a 5-year contract (as opposed to the normal 4-years), with an official go/no go progress assessment after one year. Salary and benefits are in accordance with the Collective Labour Agreement for Dutch Universities, increasing from € 2395 per month in the first year to € 3061 in the fourth year. As a PhD candidate you will be enrolled in the TU Delft Graduate School. The TU Delft Graduate School provides an inspiring research environment with an excellent team of supervisors, academic staff and a mentor. The Doctoral Education Programme is aimed at developing your transferable, discipline-related and research skills.
The TU Delft offers a customisable compensation package, discounts on health insurance and sport memberships, and a monthly work costs contribution. Flexible work schedules can be arranged. For international applicants we offer the Coming to Delft Service and Partner Career Advice to assist you with your relocation.
- Paid vacation 232 hours (when working 38 hours a week).
- Partially paid parental leave.
- On TU Delft campus there is an international day care centre and an international primary school.
- The international secondary school is located close to campus.
For information about this vacancy or the selection procedure, you can contact Julian Kooij, Assistant Professor, email: J.F.P.Kooij@tudelft.nl.
For information about the application procedure, please contact Hilma Bleeker, HR Advisor, email: email@example.com