PhD position on the intersection of deep learning, causality, and information theory
Six cross-theme PhD positions in Computer Science The Informatics Institute is renowned for its research in complex information systems at large with a focus …
- Science Park, Amsterdam, Noord-Holland
- Vast contract
- Uren per week:
- 38 - 38 uur
- € 2325 - € 2972 per maand
Six cross-theme PhD positions in Computer Science
The Informatics Institute is renowned for its research in complex information systems at large with a focus on three themes: artificial intelligence, computational science, and systems and networking. Within these themes, specific topics are selected and pursued through theory and application development, using a broad range of methods from computer science and engineering. We also highly value topics that are crossing the themes. This brings together the different fields of expertise and creates the synergy and creativity needed for innovative approaches. To strengthen collaboration we have opened six cross-theme PhD positions on various scientific challenges.
We seek a multi-disciplinary researcher who can bring advanced information-theoretic concepts, in particular synergistic information, from the field of complex adaptive systems into the field of deep learning and causal inference. The focus of this PhD project is on adapting and further developing the theory of synergy with the goal of making it viable for optimization. The end goal is to train deep representations that are robust and meaningful for un-/semi-supervised learning, transfer learning, and causal inference. For these purposes we anticipate that a ‘synergistic bottleneck principle’ needs to be formulated, in analogy to the ‘information bottleneck principle’. It should be worked out in terms of variational methods and applied to benchmark data sets. Furthermore, the question will be explored of how the optimization guided by synergistic information may serendipitously lead to causal representations.
This PhD project is a close collaboration between the Computational Science Lab (CSL, promotor prof. Peter Sloot) and the Amsterdam Machine Learning Lab (AMLAB, promotor prof. Max Welling).
With the increasing amount of data as well as the size and depth of learning representations, finding useful lower dimensional representations (for ‘downstream tasks’) in an unsupervised manner is of utmost importance and one of the current big research topics. State-of-the-art representations include versions of variational auto-encoders (VAE) or decoder-free representations.
Furthermore, an information-theory based constraint was proposed to guide the learning of efficient representations: the information bottleneck principle. Since its introduction in 2015 its use has become widespread. The basic idea is simple: constraining the amount of information that a learning algorithm is allowed to propagate will lead to compressed representations. It turns out that it is a very useful paradigm for unsupervised learning in various settings.
However it is based on the Shannon mutual information function, which is the most basic and pairwise function in Shannon information theory. In recent research in the complex systems domain it turns out that multivariate information measures (particularly ‘synergy’) are actually more effective at predicting complex emergent systemic behaviors. They also appear to be predictive of stability (defined as resilience to local noise) of a dynamical system.
Therefore our main idea is to extend the idea of information bottleneck to multivariate information measures, in particular the ‘information synergy’ measure, which is currently an active research area in itself within complex systems science. This will combine the state-of-the-art research activities of both PIs (Dr Patrick Forré and Dr Rick Quax).
What are you going to do?
You are expected to:
- have a strong analytical skills as well as technical skills;
- have an interdisciplinary mindset and an open and proactive personality in interacting with researchers from different disciplines;
- take part in ongoing educational activities, such as assisting in a course and guiding student thesis projects, at the BSc or MSc level;
- publish high quality research articles in respected, peer-reviewed scientific journals;
- have a proactive mindset.
- An MSc degree in Computer Science, Computational Science, AI, (Applied) Mathematics, Statistics, Statistical Physics, or a closely related field;
- a strong scientific interest in understanding complex theoretical concepts; the concepts of (deep) learning and information theory in particular;
- familiarity with Shannon information theory (pre);
- familiarity with unsupervised/semi-supervised learning, deep learning, optimization (pre);
- experience with mathematical/computational modelling;
- proficiency with probability theory and statistics;
- fluent proficiency in English, both written and spoken.
A temporary contract for 38 hours per week for the duration of 4 years, preferably starting as soon as possible but no later dan 1 September. The initial appointment will be for a period of 18 months and after satisfactory evaluation it will be extended for a total duration of 4 years and should lead to a dissertation (PhD thesis). We will draft an educational plan that includes attendance of courses and (international) meetings. We also expect you to assist in teaching undergraduates and master students.
The salary will be €2,325 to €2,972 (scale P) gross per month, based on full-time employment (38 hours a week). These amounts are exclusive 8% holiday allowance and 8,3% end-of-year bonus. A favorable tax agreement, the ‘30% ruling’, may apply to non-Dutch applicants. The Collective Labour Agreement of Dutch Universities is applicable.
Are you curious about our extensive package of secondary employment benefits like our excellent opportunities for study and development? Then find out more about working at the Faculty of Science.
Do you have questions about this vacancy? Or do you want to know more about our organisation? Please contact: