Affective Computing

Wearables: an R package with accompanying Shiny application for signal analysis of a wearable device targeted at clinicians and researchers

Physiological signals like heart rate and skin conductance collected from wearable devices open up a range of interesting research for clinicians and psychologists, including studying physiological reactivity to daily events and stressors. We introduce a new R package and application for analyzing wearable physiological data which enables large scale processing, and ease of use in gaining insight into this data.

P. de Looff, R. Duursma, Noordzij. Noordzi, S. Taylor, Natasha Jaques, F. Scheepers, K. De Schepper, S. Koldijk

2022 In Frontiers in behavioral neuroscience

Wearables: an R package with accompanying Shiny application for signal analysis of a wearable device targeted at clinicians and researchers

Human-Centric Dialog Training via Offline Reinforcement Learning

We train dialog models with interactive data from conversations with real humans, using a novel Offline RL technique based on KL-control. Rather than rely on manual ratings, we learn from implicit signals like sentiment, and show that this results in better performance.

Natasha Jaques*, J. H. Shen*, A. Ghandeharioun, C. Ferguson, A. Lapedriza, N. Jones, S. Gu, R. Picard

2020 In Empirical Methods in Natural Language Processing (EMNLP)

Human-Centric Dialog Training via Offline Reinforcement Learning

Approximating Interactive Human Evaluation with Self-Play for Open-Domain Dialog Systems

Existing metrics for automatically evaluating dialog models correlate poorly with human judgements, and are evaluated on static conversation snippets. Instead, we deploy bots to interact live with humans, then approximate human ratings with state-of-the-art accuracy using conversations generated with self-play.

A. Ghandeharioun*, J. H. Shen*, Natasha Jaques*, C. Ferguson, N. Jones, A. Lapedriza, R. Picard

2019 In Neural Information Processing Systems (NeurIPS)

Approximating Interactive Human Evaluation with Self-Play for Open-Domain Dialog Systems

Automatic Triage and Analysis of Online Suicide Risk with Document Embeddings and Latent Dirichlet Allocation

To predict which users are at risk of suicide based on a small dataset of online posts, we leverage pre-trained sentence embeddings from large language models, and achieve high F1 scores (.83-.92). We further analyze users’ posts to determine which topics are most associated with suicidal users.

N. Jones, Natasha Jaques, P. Pataranutaporn, A. Ghandeharioun, R. Picard

2019 In Affective Computing and Intelligence Interaction (ACII) workshop on Machine Learning for Mental Health

Automatic Triage and Analysis of Online Suicide Risk with Document Embeddings and Latent Dirichlet Allocation

Social and Affective Machine Learning

My PhD Thesis spans both Social Reinforcement Learning and Affective Computing, investigating how affective and social intelligence can enhance machine learning algorithms, and how machine learning can enhance our ability to predict and understand human affective and social phenomena.

Natasha Jaques

2019 In Massachusetts Institute of Technology

Social and Affective Machine Learning

Learning via Social Awareness: Improving a Deep Generative Sketching Model with Facial Feedback

We show the outputs of a generative model of sketches to human observers and record their facial expressions. Using only a small number of facial expression samples, we are able to tune the model to produce drawings that are significantly better rated by humans.

Natasha Jaques, J. McCleary, J. Engel, D. Ha, F. Bertsch, D. Eck, R. Picard

2018 In International Conference on Learning Representations (ICLR) workshop

Learning via Social Awareness: Improving a Deep Generative Sketching Model with Facial Feedback

Vomit Comet Physiology: Autonomic Changes in Novice Flyers

During a zero-gravity parabolic flight, we recorded participants’ heart rate, accelerometer, and skin conductance measurements as well as their self-report nausea, anxiety, and excitement. Statistical analysis revealed that skin conductance is predictive of nausea, while heart rate is predictive of anxiety and excitement.

K. Johnson, S. Taylor, S. Fedor, Natasha Jaques, W. Chen, R. Picard

2018 In IEEE Engineering in Medicine and Biology Society (EMBC)

Vomit Comet Physiology: Autonomic Changes in Novice Flyers

Importance of Sleep Data in Predicting Next-Day Stress, Happiness, and Health in College Students

We train personalized hierarchical Bayes models to predict individual’s next-day stress, happiness, and health, and examine the effect of including features related to sleep in the model. Including sleep features significantly improves performance when predicting happiness.

S. Taylor, Natasha Jaques, Sano, A. E. Nosakhare, E. B. Klerman, R. Picard

2017 In Journal of Sleep and Sleep Disorders Research (suppl_1)

Importance of Sleep Data in Predicting Next-Day Stress, Happiness, and Health in College Students

Multimodal Autoencoder: A Deep Learning Approach to Filling in Missing Sensor Data and Enabling Better Mood Prediction

Predicting signals like stress and health depends on collecting noisy data from a number of modalities, e.g. smartphone data, or physiological data from a wrist-worn sensor. Our method can continue making accurate predictions even when a modality goes missing; for example, if the person forgets to wear their sensor.

Natasha Jaques, S. Taylor, A. Sano, R. Picard

2017 In International Conference on Affective Computing and Intelligent Interaction (ACII)

Multimodal Autoencoder: A Deep Learning Approach to Filling in Missing Sensor Data and Enabling Better Mood Prediction

Personalized Multitask Learning for Predicting Tomorrow's Mood, Stress, and Health

Traditional, one-size-fits-all machine learning models fail to account for individual differences in predicting wellbeing outcomes like stress, mood, and health. Instead, we personalize models to the individual using multi-task learning (MTL), employing hierarchical Bayes, kernel-based and deep neural network MTL models to improve prediction accuracy by 13-23%.

Natasha Jaques*, S. Taylor*, E. Nosakhare, A. Sano, R. Picard

2017 In IEEE Transactions on Affective Computing (TAFFC) Best Paper; NeurIPS Machine Learning for Healthcare (ML4HC) Workshop Best Paper

Personalized Multitask Learning for Predicting Tomorrow's Mood, Stress, and Health