Human-AI Interaction

Using eye-tracking data collected while students interact with an Intelligent Tutoring System, we train machine learning models to predict when students are experiencing boredom and curiosity. Which analyze which features are most relevant to detecting when students are engaged and curious vs. disengaged and bored.

My Master’s Thesis investigated the usefulness of different data sources for automatically predicting when students using an Intelligent Tutoring System were engaged and curious, or disengaged and bored. Detailed comparisons of machine learning algorithms trained with eye-tracking data, Electrodermal Activity (EDA) and distance from the screen revealed that distance (which can be obtained with cheap infra-red sensors) provided one of the simplest and most reliable signals of student engagement.

We conduct a study in which participants form American Sign Language hand signs while being recorded with a Microsoft Kinect. The resulting infra-red distance data are used to train both neural networks with dropout (dropout NN) and Random Forests; dropout NN perform significantly better.

An emerging field in user-adaptive systems is affect adaptivity: modeling and responding to an estimation of the user’s emotional state. Prior work used Dynamic Bayesian Networks to obtain adaptivity, but in this paper we represent the problem as a Partially Observable Markov Decision Process (POMDP) and find solutions that compute a plan of interventions for an Intelligent Tutoring System to take given an estimation of the user’s mood and goals.

This study uses eye tracking to assess how students interact with automatic, adaptive hints in an Intelligent Tutoring System. Specifically, we study Prime Climb, an educational game which provides individualized support for learning number factorization skills in the form of hints generated from a model of student learning.