Human-AI Interaction

Using inverse reinforcement learning to infer human preferences is challenging, because it is an underspecified problem. We use multi-task RL pre-training and successor features to learn a strong prior over the space of reasonable goals in an environment—which we call a basis—that enables rapidly inferring an expert’s reward function in only 100 samples.

We study the automatic generation of natural language navigation instructions in visually realistic indoor environments. Existing generators suffer from poor visual grounding, skip steps, and hallucinate objects. We address this using a large language model which incorporates visual landmark detection.. The model dramatically increases the quality of generated instructions, such that humans can follow them with a 71% success rate (SR); just shy of the 75% SR of real human instructions.

We train dialog models with interactive data from conversations with real humans, using a novel Offline RL technique based on KL-control. Rather than rely on manual ratings, we learn from implicit signals like sentiment, and show that this results in better performance.

Existing metrics for automatically evaluating dialog models correlate poorly with human judgements, and are evaluated on static conversation snippets. Instead, we deploy bots to interact live with humans, then approximate human ratings with state-of-the-art accuracy using conversations generated with self-play.

My PhD Thesis spans both Social Reinforcement Learning and Affective Computing, investigating how affective and social intelligence can enhance machine learning algorithms, and how machine learning can enhance our ability to predict and understand human affective and social phenomena.

We show the outputs of a generative model of sketches to human observers and record their facial expressions. Using only a small number of facial expression samples, we are able to tune the model to produce drawings that are significantly better rated by humans.

This demo deployed RL Tuner and other Magenta music generation models into an interactive interface in which users can collaborate creatively with a machine learning model. The interface supports call and response interaction, automatically generating an accompaniment to the user’s melody, or melody morphing: responding both with variations on the user’s melody and a bass accompaniment.

We collect observational data from real human conversations, and develop a measure of how much participants experienced bonding or chemistry. We analyze the effects of personality and attitudes on bonding, and find that attentiveness and excitement are more effective at promoting bonding than traits like attractiveness and humour.

Given only one-minute slices of facial expressions and body language, we use machine learning to accurately predict whether two humans having a conversation will bond with each other. We analyze factors which lead to bonding and discover that synchrony in body language and appropriate, empathetic facial expressions lead to higher bonding.

SmileTracker is an app that uses facial expression recognition to take a screenshot of the user’s screen whenever they smile. The screenshot and image of the user’s face are saved, to help them remember positive content they encountered during the day. Users can opt to share their images to a public gallery.