https://florisdenhengst.github.io/ Recent content in Everything Flo's on Floris den Hengst's Homepage Hugo -- gohugo.io en-us Thu, 19 Mar 2026 17:02:26 +0100 https://florisdenhengst.github.io/posts/hial-guest-lecture/ Thu, 19 Mar 2026 17:02:26 +0100 https://florisdenhengst.github.io/posts/hial-guest-lecture/ Today I gave a guest lecture for the course Human-interactive Agent Learning. I presented SHARPIE, our tool for controlled human-RL experiments. During the lecture, I showed how SHARPIE can be used to implement a simple evaluative feedback experiment using TAMEr in the MountainCar environment. Nothing beats putting the pressure on as a bit of live coding does, but then it remains so much fun when you can actually show how simple it is to implement this experiment with the right tools at hand. https://florisdenhengst.github.io/posts/utq/ Thu, 13 Nov 2025 15:52:31 +0100 https://florisdenhengst.github.io/posts/utq/ Small post to celebrate that I obtained my University Teaching Qualification. This has been useful in designing teaching material, presenting it in a way to reach an audience with a diverse background, supervising students, creating tests and evaluation material, and to understand how to continuously improve on all these aspects. https://florisdenhengst.github.io/posts/siks-course-sharpie/ Tue, 21 Oct 2025 16:38:32 +0200 https://florisdenhengst.github.io/posts/siks-course-sharpie/ We contributed a tutorial on doing real-world experiments involving humans and sequential decision making agents to a national PhD course on Reinforcement Learning for Adaptive Hybrid Intelligence. This two-day course is about making AI agents adapt to humans using reinforcement learning. We set two goals for our tutorial: Hands-on experience with hybrid RL example Familiarize participants with SHARPIE and a hybrid RL algorithm SHARPIE SHARPIE is a framework for creating live demo’s and conducting real-world experiments involving humans and RL agents. https://florisdenhengst.github.io/posts/ictopen25/ Wed, 16 Apr 2025 11:29:45 +0200 https://florisdenhengst.github.io/posts/ictopen25/ I spoke at ICT.OPEN 2025 about a joint work with Cas Oude Hoekstra on predicting conditional percentiles with fully transparent models using the pinball loss and symbolic regression. More to follow soon hopefully! https://florisdenhengst.github.io/posts/genai-svg-logo/ Tue, 17 Dec 2024 12:28:05 +0100 https://florisdenhengst.github.io/posts/genai-svg-logo/ Recently, I needed a logo for a project I am working. I love SVG images for their versatility, but have the ability nor interest in creating on myself from scratch. So why not use an LLM for this? I tried to create an SVG directly with all the current (2024-12-17) easily available offerings1. They all failed miserably. Figure 1: SVG "icon depicting two fingers (one human, one robotic) holding a Sharpie-style pen" https://florisdenhengst.github.io/posts/bluesky/ Tue, 19 Nov 2024 09:43:55 +0100 https://florisdenhengst.github.io/posts/bluesky/ Today I joined Bluesky, a social media platform that aims to put users front-and-centre again. See here for my profile and use this list of starter packs: Neurosymbolic AI starter pack RL AI in healthcare AI https://florisdenhengst.github.io/posts/bnaic2024/ Sat, 19 Oct 2024 09:32:44 +0100 https://florisdenhengst.github.io/posts/bnaic2024/ I will attend the 2024 BNAIC/BeNeLearn to present our work on guideline-reinforcement informed reinforcement learning in critical care. See the poster here (PDF) and slides here (PDF). https://florisdenhengst.github.io/posts/dfki-saarbrucken/ Thu, 29 Aug 2024 14:56:45 +0200 https://florisdenhengst.github.io/posts/dfki-saarbrucken/ I visited Verena Wolf’s Neuro-Mechanistic Modeling in Saarbrücken and met with her many interesting group members there to discuss RL, safety, formal methods and RL, and causality. The visit was a great opportunity to discuss recent trends and share ideas. https://florisdenhengst.github.io/posts/hhai2024/ Wed, 12 Jun 2024 13:00:44 +0200 https://florisdenhengst.github.io/posts/hhai2024/ We have contributed papers to the health care workshop and demo track of the Hybrid Human AI Conference. Our contributions target the development of a lifestyle support chatbot for diabetes patients based on knowledge, semantic user modeling and active information gathering. https://florisdenhengst.github.io/posts/english_proficiency/ Tue, 30 Apr 2024 11:12:31 +0200 https://florisdenhengst.github.io/posts/english_proficiency/ Teaching staff who are active in one of our English programmes must prove that their English level is at EFSET C1. I am happy to share that, based on an oral assessment with two examiners from the VU language centre, my English level has been assessed as C2. I am still allowed to teach Msc. courses! https://florisdenhengst.github.io/posts/cicc/ Fri, 29 Mar 2024 21:30:50 +0100 https://florisdenhengst.github.io/posts/cicc/ Our work on hashtag#task-oriented dialogue systems has been accepted at ACL’s 2024 NAACL Findings. Task-oriented dialogue systems help users solve tasks in their daily lives. In order to do so, these systems have to identify the users intent. Dialogue systems may be uncertain about the user intent. In these cases, the system can ask the user to clarify, i.e. select from the intents that the system considers likely given the user input. https://florisdenhengst.github.io/posts/winter-school-ai-health/ Mon, 11 Mar 2024 08:58:11 +0200 https://florisdenhengst.github.io/posts/winter-school-ai-health/ I contributed a talk on guideline-informed reinforcement learning with Annette ten Teije to the AI & Health Winter School organized by the VU Campus Center Artificial Intelligence & Health. The winter school was attended by over 100 in-person and virtual participants interested in the intersection of AI & Health. You may find the slides which focus on how the knowledge-driven AI (information in the guidelines) was combined with reinforcement learning (learning optimal decisions from data) here. https://florisdenhengst.github.io/posts/phdthesis/ Tue, 14 Nov 2023 13:19:56 +0100 https://florisdenhengst.github.io/posts/phdthesis/ Today I succesfully defended my PhD. thesis Learning to Behave - Reinforcement Learning in Human Contexts. I am now a doctor of philosophy! In the Netherlands it is customary to give a 10 minute talk prior to the actual defence. This talk is aimed at a general audience because the defense is a public affair and because it is nice to tell friends and family what the thesis is about! https://florisdenhengst.github.io/posts/student_supervision/ Fri, 20 Jan 2023 15:48:55 +0100 https://florisdenhengst.github.io/posts/student_supervision/ For the past couple of years, I have always loved working with Msc. and Bsc. students. Usually, I have been involved in the role of supervisor for the thesis. In doing so, I have learnt a lot about what makes a good AI research project and what I can do to improve these projects. The following is a distillation of some of that knowledge. Some of it applies only to theses, but most of it applies more broadly. https://florisdenhengst.github.io/posts/ijcai2022/ Mon, 01 Aug 2022 16:43:48 +0200 https://florisdenhengst.github.io/posts/ijcai2022/ Last week I attended the International Joint Conference on Artificial Intelligence (IJCAI) in Vienna. Being `co-hosted’ with the European Conference on AI, the official name of this years’ edition was IJCAI-ECAI 2022. It was a great week with too many impressions and learnings to list. Here are some personal highlights: presented our paper on Option Machines as a long talk in the `Deep Reinforcement Learning’ session presented our paper on Safe & Efficient RL with Planning for Potential at the Safe RL workshop met many great people with great humor, ideas and experience attended keynotes by two Giants in AI: Stuart Russel and Judea Pearl had a nice visit of the Vienna. https://florisdenhengst.github.io/posts/reinforcement-learning-summer-school/ Sat, 16 Jul 2022 10:27:22 +0200 https://florisdenhengst.github.io/posts/reinforcement-learning-summer-school/ Past week, I attended and assisted at the Reinforcement Learning Summer School organized by Vincent François-Lavet. We had some great lectures on a wide variety of topics, including pure exploration in bandits, the exploration-exploitation tradeoff in RL, MCTS, symmetries and state similarities, world models and hierarchical and distributational RL. The summer school was hosted at one the theatre halls of our university’s new building, which serves as a movie theatre at night: great seats and air quality do help with focusing for 8+ hours! https://florisdenhengst.github.io/posts/intro-to-gps/ Tue, 21 Jun 2022 14:12:24 +0200 https://florisdenhengst.github.io/posts/intro-to-gps/ Gaussian Processes are a fascinating tool for usage in RL due to modelling uncertainty and data efficiency I briefly introduced GP’s and shown how/why they are used in RL Gaussian processes (GPs) are a fascinating tool in the machine learning toolbelt. They stand out for a couple of reasons: some people will like them for their data efficiency, others love them for their ability to incorporate domain knowledge and yet others will love them for their visual or mathematical beauty. https://florisdenhengst.github.io/posts/rl-for-swp-lod/ Tue, 14 Jun 2022 14:49:38 +0200 https://florisdenhengst.github.io/posts/rl-for-swp-lod/ A paper on Deep Reinforcement Learning (DRL) for strategic workforce planning co-authored with Yannick Smit, Sandjai Bhulai and Ehsan Mehdad is accepted as a long paper at the LOD conference. In this paper, we model strategic workforce planning as a stochastic nonlinear optimization problem, learn a generative model from data and use it as a simulator in a simulation-optimization approach. We show that the DRL approach enables optimizing an organizations’ strategic workforce goals directly. https://florisdenhengst.github.io/posts/option-machines-ijcai/ Sat, 21 May 2022 14:44:12 +0200 https://florisdenhengst.github.io/posts/option-machines-ijcai/ The paper Reinforcement Learning with Option Machines co-authored by me, Vincent François-Lavet, Mark Hoogendoorn and Frank van Harmelen is accepted as a long presentation (~3% acceptance rate) at IJCAI. Stay tuned for details! Update 2022-08-01: I presented my paper at IJCAI, it was great fun! Update 2022-09-23: The IJCAI proceedings are now available. https://florisdenhengst.github.io/posts/planning-for-potential/ Thu, 24 Mar 2022 10:38:24 +0100 https://florisdenhengst.github.io/posts/planning-for-potential/ Reinforcement Learning has proven to be capable of outperforming humans on various tasks by interacting with and experimenting some environment. This makes it one of the most interesting and promising AI solutions to problems that require complex behaviors which we are unable to fully define upfront but can assign an objective score to. In many settings of interest such as in healthcare and finance, we’ll want to enforce safety constraints on any system of interest. https://florisdenhengst.github.io/posts/rl-at-workshop/ Thu, 25 Jun 2020 21:14:18 +0200 https://florisdenhengst.github.io/posts/rl-at-workshop/ Upcoming weekend will be a virtual conference on reinforcement learning (RL) in real-life. It is of great interest to me primarily due to its program but also due to its organisation. The virtual conference consists of two panel sessions and a virtual `poster’ session with pre-recorded videos. There is a slack workspace for discussion, and questions to the panelists can be submitted up-front. Poster presenters are to host their own video channel. https://florisdenhengst.github.io/posts/rl-for-pers-survey/ Thu, 09 Apr 2020 09:22:30 +0100 https://florisdenhengst.github.io/posts/rl-for-pers-survey/ Update 2020-06-27: this paper was presented as a `poster’(video) at the RL for Real Life virtual Conference. Read more here. Reinforcement learning (RL) is becoming an increasingly popular tool to tackle hairy problems using data. A nice example of such a hairy problem is personalization. Personalization refers to a task central to many applications of data science and machine learning: to change a system so that its personal relevance to an individual or category of individuals is increased. https://florisdenhengst.github.io/posts/high-quality-ratings/ Fri, 20 Mar 2020 11:35:13 +0100 https://florisdenhengst.github.io/posts/high-quality-ratings/ Mickey, a master student I was co-supervising, recently published a paper based on his thesis work in ACM’s Conference for Human Information Interaction and Retrieval. His work focused on interfaces for collecting high-quality user satisfaction ratings for dialogue systems. User satisfaction is an important indicator in the design, evaluation and adaptation of dialogue systems. Establishing user satisfaction ratings for conversation-based systems, however, remains challenging. User questionnaires may yield biased results and typically have low response rates (~1%). https://florisdenhengst.github.io/posts/villamedia-blog/ Wed, 18 Dec 2019 14:39:21 +0100 https://florisdenhengst.github.io/posts/villamedia-blog/ Today, my work is – very briefly – mentioned in a post by Nick Kivits. The post is part of a blog series on innovation in the media and it is hosted on Villamedia, a Dutch website on journalism. https://florisdenhengst.github.io/posts/personalized-dm/ Mon, 21 Oct 2019 10:31:06 +0200 https://florisdenhengst.github.io/posts/personalized-dm/ In a previous post I explained how reinforcement learning (RL) can be used to make chatbots better from its experience with users. RL allows chatbots to learn what to say by interacting with users. This allows for chatbots to tailor their behavior to preferences of groups of individuals, e.g. to personalize the interaction. I recently presented our paper on this topic, specifically on personalized dialog management at the Web Intelligence conference in Thessaloniki. https://florisdenhengst.github.io/posts/music-bingo/ Thu, 05 Sep 2019 15:39:37 +0200 https://florisdenhengst.github.io/posts/music-bingo/ I was recently asked to help organise a ‘music bingo’ for students of horseriding association BLOK. To be completely honest with you, I feel that bingo is probably one of the most boring party games out there as the only ‘skills’ involved are paying attention and bookkeeping. Music bingo, however, puts a nice twist to the original game that makes it more fun, exciting and skill-based. Music bingo is like regular bingo with some minor differences that make it just so much more fun. https://florisdenhengst.github.io/posts/eurai-acai-2019/ Sat, 06 Jul 2019 16:41:29 +0200 https://florisdenhengst.github.io/posts/eurai-acai-2019/ Just returned from a lovely stay in Chania, Crete for the Advanced Course on Artificial Intelligence, a yearly summer school by EurAI. Besides the beautiful scenery and lovely people, there were lots of interesting talks to enjoy. Most talks stuck to the theme ‘AI for multi-agent worlds’ quite well. Although I learned a lot in general, there were some key take-aways for me: virtually any setting has some multi-agent aspect to it, but it might not be worthwile to actually take it into account when going from a single- to a multi-agent setting, typically requires engineering and analysis of the resulting system from a game-theoretic point of view. https://florisdenhengst.github.io/posts/bitter-lesson-response/ Wed, 20 Mar 2019 23:27:28 +0100 https://florisdenhengst.github.io/posts/bitter-lesson-response/ A couple of days ago, RL founding father Rich Sutton posted a blog post. My reading is as follows: Moore’s Law has consistently made general, compute-based methods outperform task-specific, domain knowledge-based methods for tasks in the AI `sphere of interest' This has served as a bitter lesson for many (all?) researchers that have focused on developing such domain-specific methods General methods will always prevail, especially those that scale with compute Although I tend to agree with the overall idea that general methods are the way forward for AI, I feel that some nuance is in order. https://florisdenhengst.github.io/posts/rl-for-dialog-management/ Fri, 15 Feb 2019 12:41:16 +0100 https://florisdenhengst.github.io/posts/rl-for-dialog-management/ The value offering of most contemporary chatbot platforms consists of packaging state-of-art Automated Speech Recognition (ASR, or ‘speech-to-text’), Natural Language Understanding (NLU) and Voice Synthesis into a comprehensive API. The API typically also includes some programming model for dialog control such as DialogFlows’ Contexts and follow-up Intents and Alexa’s Dialog model. Implementing the right dialog controller is up to the developer. Figure 1 summarizes this in a diagram, with the handcrafted modules in green and with a keyboard in the top right. https://florisdenhengst.github.io/posts/completed-research-methods-in-iks/ Sun, 30 Dec 2018 13:18:10 +0100 https://florisdenhengst.github.io/posts/completed-research-methods-in-iks/ I completed the SIKS course ‘Research Methods and Methodology in IKS’, spanning a wide range of topics such as research methods, Design Research, many issues involved with evaluation in ML and how the field of IR has a strong history of rigourous evaluation (much to learn there for ML/RL researchers!). To top it all off, the course was held in the lovely location (see picture) and with lots of great minds from all over The Netherlands. https://florisdenhengst.github.io/posts/completed-scientific-writing-course/ Tue, 18 Dec 2018 13:10:34 +0100 https://florisdenhengst.github.io/posts/completed-scientific-writing-course/ I just completed the course ‘Writing a Scientific Article’ at the VU Language Centre. Although I joined the course thinking I knew a fair share of writing in English, I learned a great deal about scientific writing and picked up some neat tricks to improve writing. As a bonus, I got some great feedback on an survey paper on Reinforcement Learning for personalization I’m working on with some colleagues in our group https://florisdenhengst.github.io/posts/completed-hpc-course/ Thu, 29 Nov 2018 09:18:55 +0100 https://florisdenhengst.github.io/posts/completed-hpc-course/ Today I completed the HPC course at VU University. The course was well organized and had some interesting courses. Some of the courses contained very little new information for me, though. I would recommend researchers/PhD. Students with some knowledge of programming/bash, Linux and clusters but with no experience working with SurfSARAs offerings to follow the following courses: Intro to distributed systems & BigData: optional refresher Intro to Linux and Clustercomputing: take, become familiar with Lisa Intro to MPI parallel programming concepts: take, get familiar with Cartesius Datamanagement: take to become familiar with various data storage/archiving solutions and make a data management plan for your research HPC Cloud: optional if interested in virtualization GPU Computing: as a refresher, somewhat theoretical Singularity with containerized applications: optional if interested in reproducibility using containers (e. https://florisdenhengst.github.io/posts/prediction-go-mcmc/ Wed, 01 Aug 2018 21:54:28 +0100 https://florisdenhengst.github.io/posts/prediction-go-mcmc/ Little under three years ago, in March 2016, a Reinforcement Learning (RL) algorithm beat Lee Sedol, a pro player in a match of Go. This game had been considered too hard for algorithms due to the astronomical number of board game configurations and the AlphaGo team therefore baffled experts in AI with their accomplishments. Nobody seemed to be aware that techniques were available to make such a win possible at that time and most experts considered computers beating humanity at Go to be a couple of decades away. https://florisdenhengst.github.io/rl4personalization/ Mon, 01 Jan 0001 00:00:00 +0000 https://florisdenhengst.github.io/rl4personalization/ You will be automatically redirected to the tool at http://rl4personalization.fht800.ops.few.vu.nl/…