1. Vacatures
  2. Universiteit van Amsterdam (UvA)
  3. PhD candidate in Reinforcement Learning

Helaas, deze vacature staat inmiddels niet meer online

Kijk gerust verder naar andere vacatures.

PhD candidate in Reinforcement Learning

The Informatics Institute invites applications for a PhD candidate to work on the development and application of a novel framework for personalised health …

9 maanden geleden


Spui, Amsterdam, Noord-Holland
Tijdelijk contract / Tijdelijke opdracht
Uren per week:
38 uur
€ 2222 - € 2840 per maand


The Informatics Institute invites applications for a PhD candidate to work on the development and application of a novel framework for personalised health interventions that combines elements of contextual bandits, causal prediction, and online learning.

Contextual bandits are often used to personalise user experience for optimising web search results, content displayed to a user and ranking advertisements. In the health care domain, there are many scenarios where we can use a similar approach to personalise the patient’s treatment experience in order to develop the most effective health intervention strategies. However, due to the additional complexity of treatments affecting various outcomes which may underlie a causal relationship and variations in pre- and post-intervention clinical features, it may be more appropriate to extend the contextual bandit framework to incorporate a causal modelling framework. Such an extension can have high impact on the personalisation and effectiveness of health intervention strategies.

The goal of this project is to develop a framework for personalised health interventions by combining elements of contextual bandits, causal prediction, and online learning in a novel way. We motivate the setting using scenarios from personalized medicine, but it applies more generally to other domains where interventions directly target features that are causally related to the features we want to control.

The developed framework will be refined and applied within the context of an online therapeutic health intervention to support and promote positive behaviour change and mental wellbeing. The student will use information available from an app developed for mental well-being to identify whether we can optimise treatment strategies to improve mental health outcomes for heterogeneous groups of patients with mental health conditions using the framework described in the causal contextual bandit strategy. During the application phase of the PhD, the student will engage with the clinical team in order to understand the problem domain and how to refine the causal contextual bandit framework to the mental health domain.

You will be employed full-time at the University of Amsterdam (UvA) working with dr Joris Mooij (promotor) and prof. Max Welling (co-promoter) in AMLAB. You will conduct research in close collaboration with dr Danielle Belgrave (co-supervisor) of Microsoft Research Cambridge. You would visit a Microsoft Research Cambridge during your research. You are expected to develop this research line, and to assist in teaching and in supervising Master’s students.


We are looking for highly motivated and creative individuals who enjoy working in a multidisciplinary research environment.

If you are the successful applicant we are looking for, you need:

  • a Master's degree in artificial intelligence, computer science, mathematics, statistics, or closely related area;
  • a solid understanding and working knowledge of reinforcement learning;
  • excellent mathematical skills (especially in probability theory and statistics, calculus, and linear algebra);
  • excellent programming skills (preferably in at least one of the following languages: C++, Python, R);
  • strong communication, presentation and writing skills and excellent command of English;
  • commitment and a cooperative attitude;

Hands-on experience with causal discovery methods or contextual bandits will be a plus.


Preferred starting date: between September 2018 and January 2019.

The appointment will be full-time (38 hours a week) on a temporary basis for a period of 4 years (initial employment is 18 months). Periodic evaluations will be held after 9 and 14 months, and upon positive evaluation, the appointment will be extended to a total of 48 months. The appointment must lead to a dissertation (PhD thesis). An educational plan that includes attendance of courses, summer and/or winter schools, and national and international meetings will be drafted for the PhD candidate. We also expect the PhD candidate to assist in teaching of undergraduate students.

The salary is in accordance with the university regulations for academic personnel and will range from €2.222 (first year) up to a maximum of €2.840 (last year) before tax per month (scale P) based on a full-time appointment. There are also secondary benefits, such as 8% holiday allowance per year and the end of year allowance of 8.3%. The Collective Labour Agreement for Dutch Universities (CAO) is applicable.

Among other things, we offer:

  • competitive pay and good benefits;
  • top-50 university worldwide;
  • very friendly, interactive and international working environment;
  • excellent computing facilities;
  • new building located near the city center (10 minutes by bicycle) of one of Europe's most beautiful and lively cities.

English is the working language within the Informatics Institute. Since Amsterdam is a very international city where almost everybody speaks and understands English, you need not be afraid of the language barrier.

Additional information

You may send informal inquiries by email to:

You may submit your application to:

To process your application immediately, please quote vacancy number 18-197 and the position you are applying for in the subject-line.

Your application should include:

  • a curriculum vitae;
  • a motivation letter that explains why you have chosen to apply for this specific position;
  • a copy of your Master’s thesis;
  • a complete record of Bachelor and Master courses (including grade transcripts and the explanation of the grading system), and
  • the names and contact information of two academic references (please do not include any recommendation letters).

Please group all these in a single PDF attachment.

Please do not send or copy your application to dr Joris Mooij or prof. Max Welling – they will get lost.

We will accept applications until 1 June 2018. The committee does not guarantee that late or incomplete applications will be considered.

We will not process applications not mentioning the vacancy number and the title of the position you are applying for in the subject-line.