Everything Flo's on Floris den Hengst's Homepage

Best Paper: AI in Medicine 2026

Mon, 13 Jul 2026 14:52:31 +0200

Our paper on ``Addressing Incomplete Data in Survival and Quality of Life Prediction: a Pancreatic Cancer Case Study" won the best paper award at AIME 2026 (AR ~20%) !

In this work, we address the joint prediction of survival and quality-of-life for patients suffering from pancreatic cancer, in the face of various forms of missing data distributed heterogeneously over time. Focusing on the technical contributions, there are two:

we extend random survival forest with a missing-aware component implemented in the split criterium. It penalizes features with many ‘missings’ so that during tree formation these features are preferred less when forming a new data split node. This yields comparable predictive performance without having to do imputation (introduces bias) or missingness indicator features (increases #features and harms interpretability)
we combine two types of conditional generative models based on diffusion for synthesizing a pre-treatment quality-of-life baselines, and longitudinal sequences of quality of life conditioned on the treatment at arbitrary time windows. The predicted quality of life trajectories can be used during treatment planning.

We hope this can help make patients more informed decisions during a critical point in their care journey when they have to balance to “how long?” with “how well?” in choosing their personal treatment plan.

Symbolic Quantile Regression

Wed, 20 May 2026 12:59:21 +0200

Throwback to this post about ICT.OPEN!

This research with Cas Hoekstra has by now been published by Transactions on Machine Learning Research.

We introduce Symbolic Quantile Regression (SQR), a framework that bridges the gap between high-performance numerical prediction and human-level interpretability:

SQR generates concise, closed-form mathematical expressions that explain why a specific quantile (like the median or the 90th percentile) is being predicted. Transparency is built-in.
In an extensive evaluation of 122 datasets, SQR outperformed existing transparent models and performed comparably to strong black-box baselines.
SQR’s preference for simpler models helps it to generalize better to out-of-distribution data compared to more complex models.

We also apply the framework on an airline fuel usage prediction task. Here, the goal was to understand why some flights use much more fuel than others.

HIAL Guest Lecture

Thu, 19 Mar 2026 17:02:26 +0100

Today I gave a guest lecture for the course Human-interactive Agent Learning. I presented SHARPIE, our tool for controlled human-RL experiments.

During the lecture, I showed how SHARPIE can be used to implement a simple evaluative feedback experiment using TAMEr in the MountainCar environment. Nothing beats putting the pressure on as a bit of live coding does, but then it remains so much fun when you can actually show how simple it is to implement this experiment with the right tools at hand.

University Teaching Qualification

Thu, 13 Nov 2025 15:52:31 +0100

Small post to celebrate that I obtained my University Teaching Qualification.

This has been useful in designing teaching material, presenting it in a way to reach an audience with a diverse background, supervising students, creating tests and evaluation material, and to understand how to continuously improve on all these aspects.

Tutorial in PhD course

Tue, 21 Oct 2025 16:38:32 +0200

We contributed a tutorial on doing real-world experiments involving humans and sequential decision making agents to a national PhD course on Reinforcement Learning for Adaptive Hybrid Intelligence. This two-day course is about making AI agents adapt to humans using reinforcement learning.

We set two goals for our tutorial:

Hands-on experience with hybrid RL example
Familiarize participants with SHARPIE and a hybrid RL algorithm

SHARPIE

SHARPIE is a framework for creating live demo’s and conducting real-world experiments involving humans and RL agents. It’s core idea is to allow researchers to incorporate a human at any point in the RL agent-environment interface:

ICT.OPEN 2025

Wed, 16 Apr 2025 11:29:45 +0200

I spoke at ICT.OPEN 2025 about a joint work with Cas Oude Hoekstra on predicting conditional percentiles with fully transparent models using the pinball loss and symbolic regression.

More to follow soon hopefully!

How to easily make a GenAI SVG logo

Tue, 17 Dec 2024 12:28:05 +0100

Recently, I needed a logo for a project I am working. I love SVG images for their versatility, but have the ability nor interest in creating on myself from scratch. So why not use an LLM for this?

I tried to create an SVG directly with all the current (2024-12-17) easily available offerings¹. They all failed miserably.

Figure 1: SVG "icon depicting two fingers (one human, one robotic) holding a Sharpie-style pen", according to OpenAI GPT-4o.

This was not working out, so instead I opted to first generate the image “in vector art style” and then convert it to svg using this useful tool which traces the image and converts it to svg flawlessly.

Bluesky

Tue, 19 Nov 2024 09:43:55 +0100

Today I joined Bluesky, a social media platform that aims to put users front-and-centre again.

See here for my profile and use this list of starter packs:

BNAIC/BeNeLearn 2024

Sat, 19 Oct 2024 09:32:44 +0100

I will attend the 2024 BNAIC/BeNeLearn to present our work on guideline-reinforcement informed reinforcement learning in critical care.

See the poster here (PDF) and slides here (PDF).

Research Visit DFKI Saarbrücken

Thu, 29 Aug 2024 14:56:45 +0200

I visited Verena Wolf’s Neuro-Mechanistic Modeling in Saarbrücken and met with her many interesting group members there to discuss RL, safety, formal methods and RL, and causality.

The visit was a great opportunity to discuss recent trends and share ideas.

Workshop and Demo Contributions to HHAI 2024

Wed, 12 Jun 2024 13:00:44 +0200

We have contributed papers to the health care workshop and demo track of the Hybrid Human AI Conference. Our contributions target the development of a lifestyle support chatbot for diabetes patients based on knowledge, semantic user modeling and active information gathering.

English Proficiency Assessment

Tue, 30 Apr 2024 11:12:31 +0200

Teaching staff who are active in one of our English programmes must prove that their English level is at EFSET C1.

I am happy to share that, based on an oral assessment with two examiners from the VU language centre, my English level has been assessed as C2.

I am still allowed to teach Msc. courses!

CICC: Conformal Intent Classification and Clarification

Fri, 29 Mar 2024 21:30:50 +0100

Our work on hashtag#task-oriented dialogue systems has been accepted at ACL’s 2024 NAACL Findings.

Task-oriented dialogue systems help users solve tasks in their daily lives. In order to do so, these systems have to identify the users intent.

Dialogue systems may be uncertain about the user intent. In these cases, the system can ask the user to clarify, i.e. select from the intents that the system considers likely given the user input.

AI & Health Winter School 2024

Mon, 11 Mar 2024 08:58:11 +0200

I contributed a talk on guideline-informed reinforcement learning with Annette ten Teije to the AI & Health Winter School organized by the VU Campus Center Artificial Intelligence & Health.

The winter school was attended by over 100 in-person and virtual participants interested in the intersection of AI & Health.

You may find the slides which focus on how the knowledge-driven AI (information in the guidelines) was combined with reinforcement learning (learning optimal decisions from data) here.

PhD. Defense Talk

Tue, 14 Nov 2023 13:19:56 +0100

Today I succesfully defended my PhD. thesis Learning to Behave - Reinforcement Learning in Human Contexts. I am now a doctor of philosophy!

In the Netherlands it is customary to give a 10 minute talk prior to the actual defence. This talk is aimed at a general audience because the defense is a public affair and because it is nice to tell friends and family what the thesis is about!

Please find the transcript below and the accompanying slide deck (in Dutch) here.

Student Supervision

Fri, 20 Jan 2023 15:48:55 +0100

For the past couple of years, I have always loved working with Msc. and Bsc. students. Usually, I have been involved in the role of supervisor for the thesis. In doing so, I have learnt a lot about what makes a good AI research project and what I can do to improve these projects.

The following is a distillation of some of that knowledge. Some of it applies only to theses, but most of it applies more broadly.

IJCAI 2022

Mon, 01 Aug 2022 16:43:48 +0200

Last week I attended the International Joint Conference on Artificial Intelligence (IJCAI) in Vienna. Being `co-hosted’ with the European Conference on AI, the official name of this years’ edition was IJCAI-ECAI 2022. It was a great week with too many impressions and learnings to list. Here are some personal highlights:

presented our paper on Option Machines as a long talk in the `Deep Reinforcement Learning’ session
presented our paper on Safe & Efficient RL with Planning for Potential at the Safe RL workshop
met many great people with great humor, ideas and experience
attended keynotes by two Giants in AI: Stuart Russel and Judea Pearl
had a nice visit of the Vienna. Managed to see some of the sights and experience the city. Went to see the 1922 movie Nosferatu with live piano music in a 1870s theatre-turned-movie hall.
AI computer chess championship being held during the conference
traveled there by a sleeper coach on the Night Jet, which was a great experience as well

The conference had a focus that seems pretty typical of AI research in Europe at the moment:

Reinforcement Learning Summer School 2022

Sat, 16 Jul 2022 10:27:22 +0200

Past week, I attended and assisted at the Reinforcement Learning Summer School organized by Vincent François-Lavet. We had some great lectures on a wide variety of topics, including pure exploration in bandits, the exploration-exploitation tradeoff in RL, MCTS, symmetries and state similarities, world models and hierarchical and distributational RL.

The summer school was hosted at one the theatre halls of our university’s new building, which serves as a movie theatre at night: great seats and air quality do help with focusing for 8+ hours! A fun detail was the dramatic Indiana Jones score randomly starting at the end of the final talk by Marc G. Bellemare.

A Brief intro to Gaussian Processes

Tue, 21 Jun 2022 14:12:24 +0200

Gaussian Processes are a fascinating tool for usage in RL due to modelling uncertainty and data efficiency
I briefly introduced GP’s and shown how/why they are used in RL

Gaussian processes (GPs) are a fascinating tool in the machine learning toolbelt. They stand out for a couple of reasons: some people will like them for their data efficiency, others love them for their ability to incorporate domain knowledge and yet others will love them for their visual or mathematical beauty. I was first introduced with GPs when studying ways to optimize chatbots or dialogue agents using data. In this blog post, I will introduce some of the concepts of GPs, discuss their strengths and weaknesses and discuss some applications in Reinforcement Learning (RL).

Paper on strategic workforce planning with DRL at LOD

Tue, 14 Jun 2022 14:49:38 +0200

A paper on Deep Reinforcement Learning (DRL) for strategic workforce planning co-authored with Yannick Smit, Sandjai Bhulai and Ehsan Mehdad is accepted as a long paper at the LOD conference.

In this paper, we model strategic workforce planning as a stochastic nonlinear optimization problem, learn a generative model from data and use it as a simulator in a simulation-optimization approach.

We show that the DRL approach enables optimizing an organizations’ strategic workforce goals directly. It significantly outperforms the strong baseline (linear programming) on a strategic objective while closely approaching it on an operational objective.

Option Machines paper accepted at IJCAI

Sat, 21 May 2022 14:44:12 +0200

The paper Reinforcement Learning with Option Machines co-authored by me, Vincent François-Lavet, Mark Hoogendoorn and Frank van Harmelen is accepted as a long presentation (~3% acceptance rate) at IJCAI.

Stay tuned for details!

Update 2022-08-01: I presented my paper at IJCAI, it was great fun!
Update 2022-09-23: The IJCAI proceedings are now available.

Safe and Efficient Reinforcement Learning with Planning for Potential

Thu, 24 Mar 2022 10:38:24 +0100

Reinforcement Learning has proven to be capable of outperforming humans on various tasks by interacting with and experimenting some environment. This makes it one of the most interesting and promising AI solutions to problems that require complex behaviors which we are unable to fully define upfront but can assign an objective score to.

In many settings of interest such as in healthcare and finance, we’ll want to enforce safety constraints on any system of interest. Arguably, these settings are the most interesting since, when the stakes are the highest we’ll care most about the potentially adversary outcomes. Although RL has a lot to offer, it cannot be readily applied in such safety-critical settings due to its lacking guarantees on safety.

Reinforcement Learning for Real Life Virtual Conference

Thu, 25 Jun 2020 21:14:18 +0200

Upcoming weekend will be a virtual conference on reinforcement learning (RL) in real-life.

It is of great interest to me primarily due to its program but also due to its organisation.

The virtual conference consists of two panel sessions and a virtual `poster’ session with pre-recorded videos. There is a slack workspace for discussion, and questions to the panelists can be submitted up-front. Poster presenters are to host their own video channel. The two panel sessions start at times convenient to the US Atlantic and Pacific coast.

Reinforcement Learning for Personalization: A Survey

Thu, 09 Apr 2020 09:22:30 +0100

Update 2020-06-27: this paper was presented as a `poster’(video) at the RL for Real Life virtual Conference. Read more here.

Reinforcement learning (RL) is becoming an increasingly popular tool to tackle hairy problems using data. A nice example of such a hairy problem is personalization. Personalization refers to a task central to many applications of data science and machine learning: to change a system so that its personal relevance to an individual or category of individuals is increased.

Collecting User Satisfaction Ratings for Dialogue Systems

Fri, 20 Mar 2020 11:35:13 +0100

Mickey, a master student I was co-supervising, recently published a paper based on his thesis work in ACM’s Conference for Human Information Interaction and Retrieval. His work focused on interfaces for collecting high-quality user satisfaction ratings for dialogue systems.

User satisfaction is an important indicator in the design, evaluation and adaptation of dialogue systems. Establishing user satisfaction ratings for conversation-based systems, however, remains challenging. User questionnaires may yield biased results and typically have low response rates (~1%). Third-party raters may be costly and gathering consistent and high-quality ratings from them is challenging. Mickey identified the issues that arise here, developed and open-sourced a tool with solutions to these issues and evaluated it in a case study on a public dataset.

Featured in Villamedia Blog

Wed, 18 Dec 2019 14:39:21 +0100

Today, my work is – very briefly – mentioned in a post by Nick Kivits.

The post is part of a blog series on innovation in the media and it is hosted on Villamedia, a Dutch website on journalism.

Personalized Dialogue Management

Mon, 21 Oct 2019 10:31:06 +0200

In a previous post I explained how reinforcement learning (RL) can be used to make chatbots better from its experience with users. RL allows chatbots to learn what to say by interacting with users. This allows for chatbots to tailor their behavior to preferences of groups of individuals, e.g. to personalize the interaction. I recently presented our paper on this topic, specifically on personalized dialog management at the Web Intelligence conference in Thessaloniki. In this conference, only 19% of all submitted papers were accepted as full paper. I’ll briefly describe the main contributions of the paper here.

How to Create a Music Bingo in Minutes

Thu, 05 Sep 2019 15:39:37 +0200

I was recently asked to help organise a ‘music bingo’ for students of horseriding association BLOK. To be completely honest with you, I feel that bingo is probably one of the most boring party games out there as the only ‘skills’ involved are paying attention and bookkeeping.

Music bingo, however, puts a nice twist to the original game that makes it more fun, exciting and skill-based. Music bingo is like regular bingo with some minor differences that make it just so much more fun. The first difference is in the bingo cards: these consist of a grid of cells like in regular bingo but each cell contains a song title instead of a number. Instead of drawing random numbers from a tumbler, songs are played from a (randomized) playlist. If a song is played, it can be marked on the card and once a player has five marked cells in a row, they shout ‘BINGO!’ and collect their prize.

ACAI Summer School 2019

Sat, 06 Jul 2019 16:41:29 +0200

Just returned from a lovely stay in Chania, Crete for the Advanced Course on Artificial Intelligence, a yearly summer school by EurAI. Besides the beautiful scenery and lovely people, there were lots of interesting talks to enjoy. Most talks stuck to the theme ‘AI for multi-agent worlds’ quite well.

Although I learned a lot in general, there were some key take-aways for me:

virtually any setting has some multi-agent aspect to it, but it might not be worthwile to actually take it into account
when going from a single- to a multi-agent setting, typically requires engineering and analysis of the resulting system from a game-theoretic point of view. Lots of the presented work had some deep connection with game theory, which is quite exciting in my point of view.
going from single- to a multi-agent setting – unfortunately – typically also results in problem spaces that are much harder: they are typically intractable, multimodal and sometimes even unstable. Typical solutions to get out involve leveraging some feature of the problem domain or resorting to approximation.
evaluation in the multi-agent domain may be quite challenging. Take the adversarial setting of learning to play a game. Here, a version \(n\) of an agent may beat version \(n-1\) but be beaten by version \(n-2\).

Overall, the summer school was well organized and was happy to meet many awesome people working on interesting problems in the AI space.

Bitter Lesson Response

Wed, 20 Mar 2019 23:27:28 +0100

A couple of days ago, RL founding father Rich Sutton posted a blog post. My reading is as follows:

Moore’s Law has consistently made general, compute-based methods outperform task-specific, domain knowledge-based methods for tasks in the AI `sphere of interest'
This has served as a bitter lesson for many (all?) researchers that have focused on developing such domain-specific methods
General methods will always prevail, especially those that scale with compute

Although I tend to agree with the overall idea that general methods are the way forward for AI, I feel that some nuance is in order. Specifically, I feel that task-specific methods fulfill a crucial role by showing what can be accomplished on a narrow task and thus serves like a telescope into the future. Furthermore, I believe that the role of data is not appreciated by Prof. Sutton. The availability data useful to some task may have natural limits.

How Reinforcement Learning is Applied to Dialogue Control

Fri, 15 Feb 2019 12:41:16 +0100

The value offering of most contemporary chatbot platforms consists of packaging state-of-art Automated Speech Recognition (ASR, or ‘speech-to-text’), Natural Language Understanding (NLU) and Voice Synthesis into a comprehensive API. The API typically also includes some programming model for dialog control such as DialogFlows’ Contexts and follow-up Intents and Alexa’s Dialog model. Implementing the right dialog controller is up to the developer. Figure 1 summarizes this in a diagram, with the handcrafted modules in green and with a keyboard in the top right. These handcrafted modules Developers can use their experience and common sense or even turn to focus groups and A/B tests in optimizing these handcrafted parts of the bot. Although making all optimization decisions upfront works well in small systems, it fails to scale to applications where many decisions are involved such as when a personalized chatbot is desired. Reinforcement Learning (RL) can help in such cases. In this blog post, I’ll describe the formalism underlying most dialog management models, explain how this formalism can be generalized to support more flexible dialog management and finally, how RL can be used to optimize dialog management using data.

Research Methods in Information and Knowledge Systems Course

Sun, 30 Dec 2018 13:18:10 +0100

I completed the SIKS course ‘Research Methods and Methodology in IKS’, spanning a wide range of topics such as research methods, Design Research, many issues involved with evaluation in ML and how the field of IR has a strong history of rigourous evaluation (much to learn there for ML/RL researchers!).

To top it all off, the course was held in the lovely location (see picture) and with lots of great minds from all over The Netherlands. Great course for first-year PhD in information systems research.

Scientific Writing Course

Tue, 18 Dec 2018 13:10:34 +0100

I just completed the course ‘Writing a Scientific Article’ at the VU Language Centre. Although I joined the course thinking I knew a fair share of writing in English, I learned a great deal about scientific writing and picked up some neat tricks to improve writing. As a bonus, I got some great feedback on an survey paper on Reinforcement Learning for personalization I’m working on with some colleagues in our group

Completed HPC Course

Thu, 29 Nov 2018 09:18:55 +0100

Today I completed the HPC course at VU University. The course was well organized and had some interesting courses. Some of the courses contained very little new information for me, though. I would recommend researchers/PhD. Students with some knowledge of programming/bash, Linux and clusters but with no experience working with SurfSARAs offerings to follow the following courses:

Intro to distributed systems & BigData: optional refresher
Intro to Linux and Clustercomputing: take, become familiar with Lisa
Intro to MPI parallel programming concepts: take, get familiar with Cartesius
Datamanagement: take to become familiar with various data storage/archiving solutions and make a data management plan for your research
HPC Cloud: optional if interested in virtualization
GPU Computing: as a refresher, somewhat theoretical
Singularity with containerized applications: optional if interested in reproducibility using containers (e.g. Docker)
Scientific Visualization: optional, if you like making nice graphs
Data intensive computing (spark/hadoop): take, pretty hands-on and very useful in industry collaborations

The Book that Predicted AlphaGo

Wed, 01 Aug 2018 21:54:28 +0100

Little under three years ago, in March 2016, a Reinforcement Learning (RL) algorithm beat Lee Sedol, a pro player in a match of Go. This game had been considered too hard for algorithms due to the astronomical number of board game configurations and the AlphaGo team therefore baffled experts in AI with their accomplishments. Nobody seemed to be aware that techniques were available to make such a win possible at that time and most experts considered computers beating humanity at Go to be a couple of decades away. Which is why I was surprised to find a mention of Go in a ‘Looking into the Future’ chapter of a book on RL from 2012, which was written at least four years prior to AlphaGo’s triumph:

Reinforcement Learning for Personalization Tool

Mon, 01 Jan 0001 00:00:00 +0000

You will be automatically redirected to the tool at http://rl4personalization.fht800.ops.few.vu.nl/…