Natasha Jaques

Natasha Jaques

Assistant Professor, University of Washington

Senior Research Scientist, Google DeepMind

Social learning helps humans and animals rapidly adapt to new circumstances, and drives the emergence of complex learned behaviors. My research is focused on Social Reinforcement Learning—developing algorithms that combine insights from social learning and multi-agent training to improve AI agents’ learning, generalization, coordination, and human-AI interaction.

I am an Assistant Professor at the University of Washington Paul G. Allen School of Computer Science & Engineering, where I lead the Social RL Lab. I am also a Senior Research Scientist at Google DeepMind. If you are interested in joining my lab as a PhD student, check out our contact page for more information

During my PhD at MIT, I developed techniques for fine-tuning language models with RL and learning from human feedback which were later built on by OpenAI’s series of work on Reinforcement Learning from Human Feedback (RLHF) [1,2,3]. In the multi-agent space, I developed techniques for improving coordination through the optimization of social influence. I interned at DeepMind, Google Brain, and was an OpenAI Scholars Mentor. I was subsequently a Visiting Postdoctoral Scholar at UC Berkeley, in Sergey Levine’s group, and a Senior Research Scientist at Google Brain, where I built novel methods for adversarial environment generation to improve the robustness of RL agents. My work has received various awards, including Best Demo at NeurIPS, an honourable mention for Best Paper at ICML, and the Outstanding PhD Dissertation Award from the Association for the Advancement of Affective Computing. My work has been featured in Science Magazine, MIT Technology Review, Quartz, IEEE Spectrum, Boston Magazine, and on CBC radio, among others. I earned a Master's degree from the University of British Columbia, and undergraduate degrees in Computer Science and Psychology from the University of Regina.

Download my CV.

Interests
Education

  • PhD in the Media Lab, 2019

    Massachusetts Institute of Technology

  • MSc in Computer Science, 2014

    University of British Columbia

  • BSc in Computer Science, 2012

    University of Regina

  • BA in Psychology, 2012

    University of Regina

Selected Awards

Selected Press

  • Degrees Magazine. Cataldo, S. (2021, November 19). The sky’s the limit.
  • Science. Hutson, M. (2021, January 19). Who needs a teacher? Artificial intelligence designs lesson plans for itself.
  • IEEE Spectrum. Hutson, M. (2019, June 17). DeepMind Teaches AI Teamwork.
  • MIT Technology Review. Hao, K. (2019, June 20). Here are 10 ways AI could help fight climate change.
  • National Geographic. Snow, J. (2019, July 18). How artificial intelligence can tackle climate change.
  • Quartz. Gershgorn, D. (2018, February 16). Google is building AI to make humans smile.
  • MIT Technology Review. Knight, W. (2016, November 30). AI songsmith cranks out surprisingly catchy tunes.
  • Boston Magazine. Annear, S. (2015, January 5). Website tracks your happiness to remind you life’s not so bad.
  • CBC radio. Brace, S. (2015, January 5). Regina woman develops smile app at MIT.

Featured Publications

Michael Dennis*, Natasha Jaques*, Eugene Vinitsky, Alexandre Bayen, Stuart Russell, Andrew Critch, Sergey Levine
2020 In Neural Information Processing Systems (NeurIPS) Oral (top 1% of submissions)
Emergent Complexity and Zero-shot Transfer via Unsupervised Environment Design
Emergent Complexity and Zero-shot Transfer via Unsupervised Environment Design

PAIRED trains an agent to generate environments that maximize regret between a pair of learning agents. This creates feasible yet challenging environments, which exploit weaknesses in the agents to make them more robust. PAIRED significantly improves generalization to novel tasks.
PDF Cite Code Poster Slides Videos NeurIPS Oral Science article Google AI Blog
Natasha Jaques*, J. H. Shen*, A. Ghandeharioun, C. Ferguson, A. Lapedriza, N. Jones, S. Gu, R. Picard
2020 In Empirical Methods in Natural Language Processing (EMNLP)
Human-Centric Dialog Training via Offline Reinforcement Learning
Human-Centric Dialog Training via Offline Reinforcement Learning

We train dialog models with interactive data from conversations with real humans, using a novel Offline RL technique based on KL-control. Rather than rely on manual ratings, we learn from implicit signals like sentiment, and show that this results in better performance.
PDF Cite Code Dataset Slides EMNLP Talk
Natasha Jaques, A. Lazaridou, E. Hughes, C. Gulcehre, P. A. Ortega, D. J. Strouse, J.Z. Leibo, N. de Freitas
2019 In International Conference on Machine Learning (ICML) Best Paper Honourable Mention (top 0.26% of submissions)
Social Influence as Intrinsic Motivation for Multi-Agent Deep Reinforcement Learning
Social Influence as Intrinsic Motivation for Multi-Agent Deep Reinforcement Learning

Social influence is a unified mechanism for achieving coordination and communication in Multi-Agent Reinforcement Learning, through rewarding agents for having causal influence over other agents’ actions, thus increasing mutual information between agents’ actions. Optimizing for influence leads to agents learning emergent communication protocols. Unlike prior work, influence can be computed in a fully decentralized manner.
PDF Cite Poster Slides Videos ICML talk IEEE Spectrum article ICML 2019 Best Papers
Natasha Jaques*, S. Taylor*, E. Nosakhare, A. Sano, R. Picard
2017 In IEEE Transactions on Affective Computing (TAFFC) Best Paper; NeurIPS Machine Learning for Healthcare (ML4HC) Workshop Best Paper
Personalized Multitask Learning for Predicting Tomorrow's Mood, Stress, and Health
Personalized Multitask Learning for Predicting Tomorrow's Mood, Stress, and Health

Traditional, one-size-fits-all machine learning models fail to account for individual differences in predicting wellbeing outcomes like stress, mood, and health. Instead, we personalize models to the individual using multi-task learning (MTL), employing hierarchical Bayes, kernel-based and deep neural network MTL models to improve prediction accuracy by 13-23%.
Cite Code Video ML4HC Best Paper TAFFC Journal Best Paper

Publications

To find relevant content, try searching publications, filtering using the buttons below, or exploring popular topics. A * denotes equal contribution.

*
All Affective Computing Social Learning Multi-Agent Emergent Complexity Cooperation Communication and Language Human-AI Interaction Generalization
Concept-based Understanding of Emergent Multi-Agent Behavior
Interpreting whether multi-agent reinforcement learning (MARL) agents have successfully learned to coordinate with each other, versus finding some other way to exploit the reward function, is a longstanding problem. We develop a novel interpretability method for MARL based on concept bottlenecks, which enables detecting which agents are truly coordinating, which environments require coordination, and identifying lazy agents.
N. Grupen, Natasha Jaques, B. Kim, S. Omidshafiei
2022 In Preprint
Cite
Concept-based Understanding of Emergent Multi-Agent Behavior
Moral Foundations of Large Language Models
Moral Foundations theory decomposes human moral reasoning into five factors, which vary reliably across different human populations and political affiliations. We use moral foundations to analyze large language models like GPT-3 to determine what, if any, consistent moral values it brings to conversations, whether these can be deliberately manipulated, and whether holding a particular moral stance affects downstream tasks.
M. Abdulhai, C. Crepy, D. Valter, J. Canny, S. Levine, Natasha Jaques
2022 In Preprint
Cite
Moral Foundations of Large Language Models
In the ZONE: Measuring difficulty and progression in curriculum generation
Past work on curriculum generation in RL has focused on training a teacher agent to generate tasks for a student agent that accelerate student learning and improve generalization. In this work, we create a mathematical framework that formalizes these concepts and subsumes prior work, taking inspiration from the psychological concept of the Zone of Proximal Development. We propose two new techniques based on rejection sampling and maximizing the student’s gradient norm that improve curriculum learning.
R. E. Wang, J. Mu, D. Arumugam, Natasha Jaques, N. Goodman
2022 In Preprint
Cite
In the ZONE: Measuring difficulty and progression in curriculum generation
Multi-Agent Reinforcement Learning for Hardware Architecture Search: A Case Study on Domain-Specific DRAM Memory Controller Design
Reinforement Learning can potentially be a powerful tool for solving complex combinatorial optimization problems, such as microprocessor desgin. Here, we show that a multi-agent RL approach outperforms past work using single agent RL, since the problem can easily be decomposed into designing independent sub-systems.
S. Krishnan, Natasha Jaques, S. Omidshafiei, D. Zhang, I. Gur, V. J. Reddi, S. Faust
2022 In Preprint
Cite
Multi-Agent Reinforcement Learning for Hardware Architecture Search: A Case Study on Domain-Specific DRAM Memory Controller Design
Basis for Intentions: Efficient Inverse Reinforcement Learning using Past Experience
Using inverse reinforcement learning to infer human preferences is challenging, because it is an underspecified problem. We use multi-task RL pre-training and successor features to learn a strong prior over the space of reasonable goals in an environment—which we call a basis—that enables rapidly inferring an expert’s reward function in only 100 samples.
M. Abdulhai, Natasha Jaques, S. Levine
2022 In Preprint
PDF Cite Code Project
Basis for Intentions: Efficient Inverse Reinforcement Learning using Past Experience
Less is More: Generating Grounded Navigation Instructions from Landmarks
We study the automatic generation of natural language navigation instructions in visually realistic indoor environments. Existing generators suffer from poor visual grounding, skip steps, and hallucinate objects. We address this using a large language model which incorporates visual landmark detection.. The model dramatically increases the quality of generated instructions, such that humans can follow them with a 71% success rate (SR); just shy of the 75% SR of real human instructions.
S. Wang, C. Montgomery, J. Orbay, V. Birodkar, A. Faust, I. Gur, Natasha Jaques, A. Waters, J. Baldridge, P. Anderson
2022 In Computer Vision and Pattern Recognition (CVPR)
PDF Cite Dataset
Less is More: Generating Grounded Navigation Instructions from Landmarks
Wearables: an R package with accompanying Shiny application for signal analysis of a wearable device targeted at clinicians and researchers
Physiological signals like heart rate and skin conductance collected from wearable devices open up a range of interesting research for clinicians and psychologists, including studying physiological reactivity to daily events and stressors. We introduce a new R package and application for analyzing wearable physiological data which enables large scale processing, and ease of use in gaining insight into this data.
P. de Looff, R. Duursma, Noordzij. Noordzi, S. Taylor, Natasha Jaques, F. Scheepers, K. De Schepper, S. Koldijk
2022 In Frontiers in behavioral neuroscience
PDF Cite Code R Package
Wearables: an R package with accompanying Shiny application for signal analysis of a wearable device targeted at clinicians and researchers
Environment Generation for Zero-Shot Compositional Reinforcement Learning
We analyze and improve upon PAIRED in the case of learning to generate challenging compositional tasks. We apply our improved algorithm to the complex task of training RL agents to navigate websites, and find that it is able to generating a challenging curriculum of novel sites. We achieve a 4x improvement over the strongest web navigation baselines, and deploy our model to navigate real-world websites..
I. Gur, Natasha Jaques, K. Malta, M. Tiwari, H. Lee, A. Faust
2021 In Neural Information Processing Systems (NeurIPS)
PDF Cite Code
Environment Generation for Zero-Shot Compositional Reinforcement Learning
PsiPhi-Learning: Reinforcement Learning with Demonstrations using Successor Features and Inverse Temporal Difference Learning
PsiPhi-Learning learns successor representations for the policies of other agents and the ego agent, using a shared underlying state representation. Learning from other agents helps the agent take better actions at inference time, and learning from RL experience improves modeling of other agents.
A. Filos, C. Lyle, Y. Gal, S. Levine, Natasha Jaques*, G. Farquhar*
2021 In International Conference on Machine Learning (ICML) Oral (top 3% of submissions)
PDF Cite Project ICML talk
PsiPhi-Learning: Reinforcement Learning with Demonstrations using Successor Features and Inverse Temporal Difference Learning
Emergent Social Learning via Multi-agent Reinforcement Learning
Model-free RL agents fail to learn from experts present in multi-agent environments. By adding a model-based auxiliary loss, we induce social learning, which allows agents to learn how to learn from experts. When deployed to novel environments with new experts, they use social learning to determine how to solve the task, and generalize better than agents trained alone with RL or imitation learning.
Kamal Ndousse, Douglas Eck, Sergey Levine, Natasha Jaques
2021 In International Conference on Machine Learning (ICML); NeurIPS Cooperative AI Workshop Best Paper
PDF Cite Code Poster Slides Cooperative AI talk ICML talk
Emergent Social Learning via Multi-agent Reinforcement Learning
Explore and Control with Adversarial Surprise
Adversarial Surprise creates a competitive game between an Expore policy and a Control policy, which fight to maximize and minimize the amount of entropy an RL agent experiences. We show both theoretically and empirically that this technique fully explores the state space of partially-observed, stochastic environments.
A. Fickinger*, Natasha Jaques*, S. Parajuli, M. Chang, N. Rhinehart, G. Berseth, S. Russell, S. Levine
2021 In ICML Unsupervised Reinforcement Learning workshop
PDF Cite Code Project
Explore and Control with Adversarial Surprise
Joint Attention for Multi-Agent Coordination and Social Learning
Joint attention is a critical component of human social cognition. In this paper, we ask whether a mechanism based on shared visual attention can be useful for improving multi-agent coordination and social learning.
D. Lee, Natasha Jaques, J. Kew, D. Eck, D. Schuurmans, A. Faust
2021 In ICRA Social Intelligence Workshop Spotlight talk
PDF Cite Code Poster Video
Joint Attention for Multi-Agent Coordination and Social Learning
Emergent Complexity and Zero-shot Transfer via Unsupervised Environment Design
PAIRED trains an agent to generate environments that maximize regret between a pair of learning agents. This creates feasible yet challenging environments, which exploit weaknesses in the agents to make them more robust. PAIRED significantly improves generalization to novel tasks.
Michael Dennis*, Natasha Jaques*, Eugene Vinitsky, Alexandre Bayen, Stuart Russell, Andrew Critch, Sergey Levine
2020 In Neural Information Processing Systems (NeurIPS) Oral (top 1% of submissions)
PDF Cite Code Poster Slides Videos NeurIPS Oral Science article Google AI Blog
Emergent Complexity and Zero-shot Transfer via Unsupervised Environment Design
Human-Centric Dialog Training via Offline Reinforcement Learning
We train dialog models with interactive data from conversations with real humans, using a novel Offline RL technique based on KL-control. Rather than rely on manual ratings, we learn from implicit signals like sentiment, and show that this results in better performance.
Natasha Jaques*, J. H. Shen*, A. Ghandeharioun, C. Ferguson, A. Lapedriza, N. Jones, S. Gu, R. Picard
2020 In Empirical Methods in Natural Language Processing (EMNLP)
PDF Cite Code Dataset Slides EMNLP Talk
Human-Centric Dialog Training via Offline Reinforcement Learning
Approximating Interactive Human Evaluation with Self-Play for Open-Domain Dialog Systems
Existing metrics for automatically evaluating dialog models correlate poorly with human judgements, and are evaluated on static conversation snippets. Instead, we deploy bots to interact live with humans, then approximate human ratings with state-of-the-art accuracy using conversations generated with self-play.
A. Ghandeharioun*, J. H. Shen*, Natasha Jaques*, C. Ferguson, N. Jones, A. Lapedriza, R. Picard
2019 In Neural Information Processing Systems (NeurIPS)
PDF Cite Code Dataset Poster
Approximating Interactive Human Evaluation with Self-Play for Open-Domain Dialog Systems
Automatic Triage and Analysis of Online Suicide Risk with Document Embeddings and Latent Dirichlet Allocation
To predict which users are at risk of suicide based on a small dataset of online posts, we leverage pre-trained sentence embeddings from large language models, and achieve high F1 scores (.83-.92). We further analyze users’ posts to determine which topics are most associated with suicidal users.
N. Jones, Natasha Jaques, P. Pataranutaporn, A. Ghandeharioun, R. Picard
2019 In Affective Computing and Intelligence Interaction (ACII) workshop on Machine Learning for Mental Health
PDF Cite
Automatic Triage and Analysis of Online Suicide Risk with Document Embeddings and Latent Dirichlet Allocation
Hierarchical Reinforcement Learning for Open-Domain Dialog
For the first time, we use hierarchical reinforcement learning to train open-domain dialog models, enabling the optimization of long-term, conversational, rewards, including reducing the toxicity of generated language. Our approach provides significant improvements over state-of-the-art dialog models.
A. Saleh*, Natasha Jaques*, A. Ghandeharioun, J. H. Shen, R. Picard
2019 In Association for the Advancement of Artificial Intelligence (AAAI) Oral (top 7.8% of submissions)
PDF Cite Code Dataset Talk
Hierarchical Reinforcement Learning for Open-Domain Dialog
Social and Affective Machine Learning
My PhD Thesis spans both Social Reinforcement Learning and Affective Computing, investigating how affective and social intelligence can enhance machine learning algorithms, and how machine learning can enhance our ability to predict and understand human affective and social phenomena.
Natasha Jaques
2019 In Massachusetts Institute of Technology
PDF Cite Thesis Defense CV News write-up
Social and Affective Machine Learning
Social Influence as Intrinsic Motivation for Multi-Agent Deep Reinforcement Learning
Social influence is a unified mechanism for achieving coordination and communication in Multi-Agent Reinforcement Learning, through rewarding agents for having causal influence over other agents’ actions, thus increasing mutual information between agents’ actions. Optimizing for influence leads to agents learning emergent communication protocols. Unlike prior work, influence can be computed in a fully decentralized manner.
Natasha Jaques, A. Lazaridou, E. Hughes, C. Gulcehre, P. A. Ortega, D. J. Strouse, J.Z. Leibo, N. de Freitas
2019 In International Conference on Machine Learning (ICML) Best Paper Honourable Mention (top 0.26% of submissions)
PDF Cite Poster Slides Videos ICML talk IEEE Spectrum article ICML 2019 Best Papers
Social Influence as Intrinsic Motivation for Multi-Agent Deep Reinforcement Learning
Tackling Climate Change with Machine Learning
This paper comprehensively surveys the ways in which machine learning could be usefully deployed in the fight against climate change. From smart grids to disaster management, we identify high impact problems and outline how machine learning can be employed to address them.
D. Rolnick, P. L. Donti, L. H. Kaack, K. Kochanski, A. Lacoste, K. Sankaran, A. S. Ross, N. Milojevic-Dupont, Natasha Jaques, A. Waldman-Brown, A. Luccioni, T. Maharaj, E. D. Sherwin, S. K. Mukkavilli, K. P. Kording, C. Gomes, A. Y. Ng, D. Hassabis, J. C. Platt, F. Creutzig, J. Chayes, Y. Bengio
2019 In ACM Computing Surveys
PDF Cite CCAI Organization MIT Tech Review article National Geographic article ACM Computing Surveys
Tackling Climate Change with Machine Learning
Learning via Social Awareness: Improving a Deep Generative Sketching Model with Facial Feedback
We show the outputs of a generative model of sketches to human observers and record their facial expressions. Using only a small number of facial expression samples, we are able to tune the model to produce drawings that are significantly better rated by humans.
Natasha Jaques, J. McCleary, J. Engel, D. Ha, F. Bertsch, D. Eck, R. Picard
2018 In International Conference on Learning Representations (ICLR) workshop
PDF Cite Slides Quartz article
Learning via Social Awareness: Improving a Deep Generative Sketching Model with Facial Feedback
Vomit Comet Physiology: Autonomic Changes in Novice Flyers
During a zero-gravity parabolic flight, we recorded participants’ heart rate, accelerometer, and skin conductance measurements as well as their self-report nausea, anxiety, and excitement. Statistical analysis revealed that skin conductance is predictive of nausea, while heart rate is predictive of anxiety and excitement.
K. Johnson, S. Taylor, S. Fedor, Natasha Jaques, W. Chen, R. Picard
2018 In IEEE Engineering in Medicine and Biology Society (EMBC)
PDF Cite
Vomit Comet Physiology: Autonomic Changes in Novice Flyers
Importance of Sleep Data in Predicting Next-Day Stress, Happiness, and Health in College Students
We train personalized hierarchical Bayes models to predict individual’s next-day stress, happiness, and health, and examine the effect of including features related to sleep in the model. Including sleep features significantly improves performance when predicting happiness.
S. Taylor, Natasha Jaques, Sano, A. E. Nosakhare, E. B. Klerman, R. Picard
2017 In Journal of Sleep and Sleep Disorders Research (suppl_1)
PDF Cite
Importance of Sleep Data in Predicting Next-Day Stress, Happiness, and Health in College Students
Multimodal Autoencoder: A Deep Learning Approach to Filling in Missing Sensor Data and Enabling Better Mood Prediction
Predicting signals like stress and health depends on collecting noisy data from a number of modalities, e.g. smartphone data, or physiological data from a wrist-worn sensor. Our method can continue making accurate predictions even when a modality goes missing; for example, if the person forgets to wear their sensor.
Natasha Jaques, S. Taylor, A. Sano, R. Picard
2017 In International Conference on Affective Computing and Intelligent Interaction (ACII)
PDF Cite Code Slides
Multimodal Autoencoder: A Deep Learning Approach to Filling in Missing Sensor Data and Enabling Better Mood Prediction
Personalized Multitask Learning for Predicting Tomorrow's Mood, Stress, and Health
Traditional, one-size-fits-all machine learning models fail to account for individual differences in predicting wellbeing outcomes like stress, mood, and health. Instead, we personalize models to the individual using multi-task learning (MTL), employing hierarchical Bayes, kernel-based and deep neural network MTL models to improve prediction accuracy by 13-23%.
Natasha Jaques*, S. Taylor*, E. Nosakhare, A. Sano, R. Picard
2017 In IEEE Transactions on Affective Computing (TAFFC) Best Paper; NeurIPS Machine Learning for Healthcare (ML4HC) Workshop Best Paper
Cite Code Video ML4HC Best Paper TAFFC Journal Best Paper
Personalized Multitask Learning for Predicting Tomorrow's Mood, Stress, and Health
Predicting Tomorrow’s Mood, Health, and Stress Level using Personalized Multitask Learning and Domain Adaptation
Modeling measures like mood, stress, and health using a monolithic machine learning model leads to low prediction accuracy. Instead, we develop personalized regression models using multi-task learning and Gaussian Processes, leading to dramatic improvements in next-day predictions.
Natasha Jaques, O. Rudovic, S. Taylor, A. Sano, R. Picard
2017 In Proceedings of Machine Learning Research
PDF Cite Slides
Predicting Tomorrow’s Mood, Health, and Stress Level using Personalized Multitask Learning and Domain Adaptation
Sequence Tutor: Conservative Fine-Tuning of Sequence Generation Models with KL-control
To combine supervised learning on data with reinforcement learning, we pre-train a supervised data prior, and penalize KL-divergence from this model using RL training. This enables effective learning of complex sequence-modeling problems for which we wish to match the data while optimizing external metrics like drug effectiveness. The approach produces compelling results in the disparate domains of music generation and drug discovery.
Natasha Jaques, S. Gu, D. Bahdanau, J. M. Hernandez-Lobato, R. E. Turner, D. Eck
2017 In International Conference on Machine Learning (ICML)
PDF Cite Code ICML talk Generated music Magenta blog MIT Tech Review article
Sequence Tutor: Conservative Fine-Tuning of Sequence Generation Models with KL-control
BITxBIT: Encouraging Behavior Change with N=2 Experiments
To help promote behavior change, we leverage the power of social obligation, and conduct an experiment in which participants are paired together and asked to design a Behavioral Intervention Technology (BIT) customized to suit their partner’s behavior change goal.
Natasha Jaques, T. Rich, K. Dinakar, N. Farve, W.V. Chen, P. Maes, R. Picard
2016 In Proceedings of the CHI Conference Extended Abstracts on Human Factors
PDF Cite Code
BITxBIT: Encouraging Behavior Change with N=2 Experiments
Interactive Musical Improvisation with Magenta
This demo deployed RL Tuner and other Magenta music generation models into an interactive interface in which users can collaborate creatively with a machine learning model. The interface supports call and response interaction, automatically generating an accompaniment to the user’s melody, or melody morphing: responding both with variations on the user’s melody and a bass accompaniment.
A. Roberts, J. Engel, C. Hawthorne, I. Simon, E. Waite, S. Oore, Natasha Jaques, C. Resnick, D. Eck
2016 In Neural Information Processing Systems (NeurIPS) Best Demo
Cite Code Video NeurIPS Demo Magenta Blog post
Interactive Musical Improvisation with Magenta
Machine Learning of Sleep and Wake Behaviors to Classify Self-Reported Evening Mood
Machine learning applied to nightly data from sensors and smartphones, shows value for predicting college student’s mood the following evening. Using multi-task learning to simultaneously predicted related wellbeing factors like health, energy, stress, and alertness improves performance.
S. Taylor, Natasha Jaques, A. Sano, A. Azaria, A. Ghandeharioun, R. Picard
2016 In Sleep
PDF Cite Code
Machine Learning of Sleep and Wake Behaviors to Classify Self-Reported Evening Mood
Personality, Attitudes, and Bonding in Conversations
We collect observational data from real human conversations, and develop a measure of how much participants experienced bonding or chemistry. We analyze the effects of personality and attitudes on bonding, and find that attentiveness and excitement are more effective at promoting bonding than traits like attractiveness and humour.
Natasha Jaques, Y. K. Kim, \& Picard R. Picard R
2016 In Intelligent Virtual Agents (IVA)
PDF Cite
Personality, Attitudes, and Bonding in Conversations
Tuning Recurrent Neural Networks with Reinforcement Learning
Generating music using traditional supervised sequence models suffers from known failure modes, including the inability to produce coherent global structure. Music is an interesting sequence generation problem, because musical compositions adhere to known rules. We impose these rules with a novel algorithm combining RL and supervised learning.
Natasha Jaques, S. Gu, R. E. Turner, D. Eck
2016 In International Conference on Learning Representations (ICLR) - workshop
PDF Cite Code Magenta blog MIT Tech Review article
Tuning Recurrent Neural Networks with Reinforcement Learning
Understanding and Predicting Bonding in Conversations Using Thin Slices of Facial Expressions and Body Language
Given only one-minute slices of facial expressions and body language, we use machine learning to accurately predict whether two humans having a conversation will bond with each other. We analyze factors which lead to bonding and discover that synchrony in body language and appropriate, empathetic facial expressions lead to higher bonding.
Natasha Jaques, D. McDuff, Y. K. Kim, \& Picard R. Picard R
2016 In Intelligent Virtual Agents (IVA)
PDF Cite
Understanding and Predicting Bonding in Conversations Using Thin Slices of Facial Expressions and Body Language
Active learning for Electrodermal Activity classification
We use labels provided by domain experts to classify whether artifacts are present in an Electrodermal Activity signal. Through the use of active learning, we improve sample efficiency and reduce the burden on human experts by as much as 84%, while offering the same or improved performance.
V. Xia, Natasha Jaques, S. Taylor, S. Fedor, R. Picard
2015 In IEEE Conference on Signal Processing in Medicine and Biology (SPMB)
PDF Cite
Active learning for Electrodermal Activity classification
Automatic identification of artifacts in Electrodermal Activity data
Ambulatory measurement of Electrodermal Activity (EDA) from the wrist has important clinical benefits, such as predicting mood, stress, health, or even seizures. However, ambulatory measurement is noisy, and artifacts can easily be mistaken for true Skin Conductance Responses (SCRs). In addition to our paper which describes a machine learning method for detecting artifacts with 95% test accuracy, we built EDA Explorer, an open-source tool that allows users to automatically detect artifacts and SCRs within their data.
S. Taylor*, Natasha Jaques*, W. Chen, S. Fedor, A. Sano, R. Picard
2015 In International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC)
PDF Cite Code EDA Explorer tool Artifact detection tutorial SCR detection tutorial Research which uses EDA Explorer
Automatic identification of artifacts in Electrodermal Activity data
Engaging the workplace with challenges
The Challenge is a tool aimed at promoting social connections and decreasing sedentary activity in a workplace environment. Participants are paired with a partner to complete short physical challenges, leveraging social obligation and social consensus to drive behavior change.
Natasha Jaques, N. Farve
2015 In International Conference on Persuasive Technologies
PDF Cite Video Extended paper
Engaging the workplace with challenges
Multi-task Multi-Kernel Learning for Estimating Individual Wellbeing
Wellbeing is a complex internal state consisting of several related dimensions, such as happiness, stress, energy, and health. We use Multi-task Multi-kernel learning to classify them simultaneously, leading to significant performance approvements.
Natasha Jaques*, S. Taylor*, A. Sano, R. Picard
2015 In Neural Information Processing Systems (NeurIPS) Workshop on Multimodal Machine Learning
PDF Cite Code
Multi-task Multi-Kernel Learning for Estimating Individual Wellbeing
Predicting students' happiness from physiology, phone, mobility, and behavioral data
We train machine learning models to predict students’ happiness from extensive data comprising physiological signals, location, smartphone logs, and behavioral questions. Analyzing which features provide the highest information gain reveals that skin conductance during sleep, social interaction, exercise, and fewer phone screen hours are all positively associated with happiness.
Natasha Jaques*, S. Taylor*, A. Azaria, A. Ghandeharioun, A. Sano, R. Picard
2015 In International Conference on Affective Computing and Intelligent Interaction (ACII)
PDF Cite NCBI link
Predicting students' happiness from physiology, phone, mobility, and behavioral data
Prediction of happy-sad mood from daily behaviors and previous sleep history
We trained machine learning models to classify happy vs. sad moods in college students using data from surveys and wearable sensors. Factors such as poor health-related behavior, more academic activity hours, and more neutral social interactions were highly predictive of mood.
A. Sano, A. Z. Yu, A. W. McHill, A.J. Phillips, S. Taylor, Natasha Jaques, C. A. Czeisler, E. B. Klerman, R. Picard
2015 In International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC)
PDF Cite
Prediction of happy-sad mood from daily behaviors and previous sleep history
Recognizing academic performance, sleep quality, stress level, and mental health using personality traits, wearable sensors and mobile phones
SNAPSHOT was a large-scale study of college undergraduates which tracked detailed longitudinal data from smartphones, wearable sensors, behavioral data, and mental health and sleep quality surveys. This initial study analyzed relationships between sleep quality, stress, and GPA, and used machine learning to predict these indices from objective phone and sensor data.
A. Sano, A. J. Phillips, A. Z. Yu, A. W. McHill, S. Taylor, Natasha Jaques, C. A. Czeisler, E. B. Klerman, R. Picard
2015 In Wearable and Implantable Body Sensor Networks (BSN)
PDF Cite
Recognizing academic performance, sleep quality, stress level, and mental health using personality traits, wearable sensors and mobile phones
SmileTracker: Automatically and Unobtrusively Recording Smiles and their Context.
SmileTracker is an app that uses facial expression recognition to take a screenshot of the user’s screen whenever they smile. The screenshot and image of the user’s face are saved, to help them remember positive content they encountered during the day. Users can opt to share their images to a public gallery.
Natasha Jaques, W. V. Chen, R. Picard
2015 In Proceedings of the CHI Conference Extended Abstracts
PDF Cite Slides Boston Magazine article CBC news
SmileTracker: Automatically and Unobtrusively Recording Smiles and their Context.
Wavelet-based motion artifact removal for Electrodermal Activity
We propose a method for removing motion artifacts from Electrodermal Activity using a stationary wavelet transform. We modeled the wavelet coefficients as a Gaussian mixture distribution corresponding to the underlying skin conductance level and skin conductance responses. Our method achieves a greater reduction of artifacts while retaining motion-artifact-free data.
W. Chen, Natasha Jaques, S. Taylor, A. Sano, S. Fedor, \& Picard R. Picard R
2015 In International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC)
PDF Cite
Wavelet-based motion artifact removal for Electrodermal Activity
Predicting Affect from Gaze Data During Interaction with an Intelligent Tutoring System
Using eye-tracking data collected while students interact with an Intelligent Tutoring System, we train machine learning models to predict when students are experiencing boredom and curiosity. Which analyze which features are most relevant to detecting when students are engaged and curious vs. disengaged and bored.
Natasha Jaques, C. Conati, J. M. Harley, R. Azevedo
2014 In Intelligent Tutoring Systems
PDF Cite Slides
Predicting Affect from Gaze Data During Interaction with an Intelligent Tutoring System
Predicting Affect in an Intelligent Tutoring System
My Master’s Thesis investigated the usefulness of different data sources for automatically predicting when students using an Intelligent Tutoring System were engaged and curious, or disengaged and bored. Detailed comparisons of machine learning algorithms trained with eye-tracking data, Electrodermal Activity (EDA) and distance from the screen revealed that distance (which can be obtained with cheap infra-red sensors) provided one of the simplest and most reliable signals of student engagement.
Natasha Jaques
2014 In University of British Columbia
PDF Cite Slides
Predicting Affect in an Intelligent Tutoring System
A Comparison of Random Forests and Dropout Nets for Sign Language Recognition with the Kinect
We conduct a study in which participants form American Sign Language hand signs while being recorded with a Microsoft Kinect. The resulting infra-red distance data are used to train both neural networks with dropout (dropout NN) and Random Forests; dropout NN perform significantly better.
Natasha Jaques, J. Nutini
2013 In Unpublished manuscript
PDF Cite
A Comparison of Random Forests and Dropout Nets for Sign Language Recognition with the Kinect
Emotionally Adaptive Intelligent Tutoring Systems using POMDPs
An emerging field in user-adaptive systems is affect adaptivity: modeling and responding to an estimation of the user’s emotional state. Prior work used Dynamic Bayesian Networks to obtain adaptivity, but in this paper we represent the problem as a Partially Observable Markov Decision Process (POMDP) and find solutions that compute a plan of interventions for an Intelligent Tutoring System to take given an estimation of the user’s mood and goals.
Natasha Jaques
2013 In Unpublished manuscript
PDF Cite
Emotionally Adaptive Intelligent Tutoring Systems using POMDPs
Fast Johnson–Lindenstrauss transform for classification of high dimensional data
This paper investigates the utility of using the Fast Johnson-Lindenstrauss Transform to produce a low-dimensional random projection of eye-tracking data features that can be used for classifying emotion in an Intelligent Tutoring System. Interestingly, the FJLT provides similar or superior performance to more computationally expensive techniques.
Natasha Jaques
2013 In Unpublished manuscript
PDF Cite
Fast Johnson–Lindenstrauss transform for classification of high dimensional data
Understanding attention to adaptive hints in educational games: an eye-tracking study
This study uses eye tracking to assess how students interact with automatic, adaptive hints in an Intelligent Tutoring System. Specifically, we study Prime Climb, an educational game which provides individualized support for learning number factorization skills in the form of hints generated from a model of student learning.
C. Conati, Natasha Jaques, M. Muir
2013 In International Journal of Artificial Intelligence in Education
PDF Cite
Understanding attention to adaptive hints in educational games: an eye-tracking study

Popular Topics

Affective Computing Communication and Language Cooperation Deep Learning Electrodermal Activity Emergent Complexity Generalization Generative Models Healthcare Human-AI Interaction Human-Computer Interaction Intelligent Tutoring Systems Machine Learning Multi-Agent Physiology Reinforcement Learning Sensors Sequence Modeling Social Learning Wellbeing

Featured Talks

Adaptive Learning Agents workshop at AAMAS
This talk gives an overview of Social RL and dives into three topics: multi-agent social influence, emergent curriculum generation with PAIRED, and emergent social learning and PsiPhi-Learning. It is similar to the faculty job talk I gave at MIT.
2022 Virtual
Institute for Pure and Applied Mathematics (IPAM) Mathematics of Intelligence workshop
This talk focuses on how social learning from other agents can lead to learning more complex behaviors and enhance generalization. I discuss PAIRED and recent work on emergent social learning and PsiPhi-Learning in detail.
2022 UCLA
NeurIPS Political Economy of Reinforcement Learning Panel
Panel discussion on the near-term policy concerns related to reinforcement learning (RL). Other panelists include Michael Dennis, Mireille Hildebrandt, and Salome Viljoen.
2021 Virtual
NeurIPS Political Economy of Reinforcement Learning Panel
Conference on Robot Learning (CoRL) Tutorial
An overview of Social Reinforcement Learning, including multi-agent coordination, and using multi-agent training as a tool to induce emergent complexity and improve generalization.
2021 London, UK
Jay Shah Podcast
A casual discussion on my favourite research projects, how I think about generating research ideas, and future directions.
2021 Virtual
Re-Work Women in AI Podcast
A fun discussion of my research, career trajectory, and take on possible beneficial future directions for reinforcement learning research.
2021 Re-Work
Institute of Cognitive Science Deep Reinforcement Learning Workshop
In addition to talking about Social Reinforcement Learning, I participated in a panel discussion with Deepak Pathak.
2020 University of Osnabrück
UCL Deciding, Acting, and Reasoning with Knowledge (DARK) Seminar
An overview of Social Reinforcement Learning including using multi-agent competition to drive emergent complexity via PAIRED, increasing multi-agent coordination with Social Influence, and learning from human feedback in dialog with Offline RL.
2020 University College London (UCL)
Samsung Forum
A talk entitled “Towards Social and Affective Machine Learning”, which covers most of the same content as my PhD defense, including my early PhD work on using multi-task learning for personalized wellbeing prediction.
2020 Samsung Strategy & Innovation Center
Thesis Defense
My thesis defense at the MIT Media Lab. I cover work on Affective Computing, learning from affective signals in human-AI interaction, and multi-agent coordination. Includes an in-depth question period with my PhD committee.
2019 Virtual
See all events

Research Communities