Building novel machine learning and pattern analysis tools to understand and improve mental wellbeing

Emotions are an intricate part of the human condition. People experience stress, joy, varying levels of interest, anxiety, depression, and different mood changes. However, some individuals face serious emotional challenges that greatly impair their lives. Dr. Rosalind Picard, Professor of the Massachusetts Institute of Technology (MIT) Media Lab at MIT, develops novel technology to better understand and measure emotion in daily life. Using machine learning and pattern analysis of personal data. Dr. Picard’s research explores human emotions relating to mental health, neurological conditions, autistic disorders, and mood regulation. Her innovative wearable+smartphone technology directly improves the lives of those facing these challenges. 

Dr. Picard and her team of graduate students, research assistants, and postdoctoral fellows are building new machine learning and sensing tools for mental health. They create new ways to forecast mood and health conditions based on data from wearable technology and the internet of things. Their innovative tools measure data and patterns to improve healthy emotional wellbeing, prevent depression and suicide, and enable autistic students to decrease their anxiety and improve their learning experiences. Dr. Picard and her team’s wearable sensors also measure important mood-influencing factors for many neurological conditions, such as Alzheimer's disease, epilepsy, multiple sclerosis, and Parkinson's disease. These conditions activate regions of the brain involved in the emotion system; they measure these changes to better understand how they impact a patient’s quality of life. Their expansive research is actively tested in users and also used to collect basic data for new theory and algorithm development. They’ve discovered how to detect seizures with an Autonomic Nervous System Activity in Epilepsy wristband, and have spun out several companies—including Empatica and Affectiva—enabling faster scaling of their technology to reach more people. 

Current research topics include:

Managing Stress to Improve Mental Health - Current weather forecast tools are able to warn people to evacuate an area before, for example, a hurricane strikes. This technology developed as ever-changing weather factors, such as pressure, humidity, and temperature were measured in different cities and the data compared and communicated. Dr. Picard and her team are expanding on and adapting such predictive mathematical techniques for mental health. From their ongoing research in their SNAPSHOT Expose and SNAPSHOT Study, they are developing tools for users to assess how their behavior—including social interactions, stress, sleep, and activities—impacts their mood. They can help people see, for example, if making their sleep more regular this week might provide a higher probability of weekend happiness.  They compare the types of visualization strategies that are most meaningful and useful for acting on each theory, ultimately mitigating the probability of depression and suicide for a user before it happens. They are building customized technology—the software, sensors, signal processing, and machine-learning data—and running studies in partnership with sleep and suicide experts from Harvard University and Massachusetts General Hospital to improve mental health. For this ongoing project, Dr. Picard and her team aim to collect more longterm data from users through their interaction with their smartphones and (optional) wearables, which also capture how they interact with other people in their lives. By tracking behavioral patterns and understanding what puts an individual at risk for depression and suicide, they can identify changes in mood and prevent further mental decline before it occurs. 

Improving Communication and Learning for Non-Speaking Children - Dr. Picard and her team are building innovative tools for students who have difficulty with communication and socialization. Often, students with autism experience extreme stress that results in emotional meltdowns. They want to help students understand and regulate their emotions, while harnessing their special interests to motivate successful learning. Current software for children with learning disabilities is very limited, as interests and motivations are unique to each individual. One of their ongoing projects, Storyscape, encourages its users to participate in learning through interactive storybooks, which the team aims to greatly expand and build upon. Thus far, teachers have described Storyscape as “life-changing” for their students, even helping some students make friends. Their new project, SPRING, helps kids with severe motor and learning disabilities who have trouble succeeding in school due to a lack of motivation strong enough to overcome their challenges. With a focus on science-based reward and motivation for successful learning, they’ve built a platform that provides powerful motivations to a child with developmental challenges, such as a smartphone coupled with a toy that excites them. Their technology is transforming the classroom; it enables students with learning disabilities to expand their social interactions, while improving their mood, attention, task performance, and motivation to learn.

Can We Forecast and Reduce Migraines? - Dr. Picard and her team are developing wearable sensors to measure behaviors that can predict headaches. In collaboration with the American Headache and Migraine Association (AHMA), they are currently in the basic exploratory research stage, collecting large amounts of pilot data and information. Using machine learning and data forecasting, they want to build a wearable smart phone system that tracks behavior and forecasts when a migraine is most likely to occur. Then, the device will help a user prevent it. For example, the sensor will send a notification that reminds the user to take their medicine, or—after it identified that red wine disrupts the user’s sleep—warn them to avoid it at night. Dr. Picard and her team’s tools combine stress and emotion-related data with other individualized factors in a user’s life to make personalized predictive analytics for improved wellbeing. 



Dr. Rosalind Picard obtained her B.A. in electrical engineering from the Georgia Institute of Technology, and earned her master's and doctorate degrees in electrical engineering and computer science from MIT. 

She was the first person to build machine learning and pattern recognition tools that measure emotion, and began the field of “affective computing.” However, she didn’t begin her career in emotional technology. Like many others in the field of machine learning, Dr. Rosalind Picard was first interested in building an AI. However, her perspective shifted after her colleague said they would one day build AI machines that are so great, we will be lucky if they keep us around as a household pet. At the end of her life, Dr. Picard didn’t want to say that she converted human beings into “household pets.” Instead, she wanted to use machine intelligence in a good and powerful way. 

As a signal-processing expert building sensors and technology to understand human behavior and data, Dr. Picard has pioneered this area of research and published about 200 papers on it. Her early work provided machines with emotional intelligence, enabling them to identify when a person was annoyed, frustrated, or stressed. This work was previously used for AI and learning, market research, and customer service. However, she soon realized that humans didn’t understand feelings well themselves, especially misunderstanding people on the autism spectrum. Rather than enabling machines to understand our feelings and potentially use it for manipulation, Dr. Picard wanted to exclusively use her expertise to improve the lives of people.

Since 2012, she has focused her technology development on preventive mental health, neurology, and autism research. Dr. Picard and her team started building wearable technology that autistic students could wear and effectively communicate with others. She saw that these students would get stressed out and have meltdowns before they were aware their emotions were changing. She knew, just like weather forecasting, that they needed to collect data to understand what's causing stress before it overwhelms them. They created a wrist sensor that showed a child's stress levels. In many cases, they could see that when a child started pacing or rocking, the stress level decreased. They were amazed. Letting a child rock or pace reduced stress, so why were teachers telling them not to do that? By collecting this data and sharing it, they realized they could make a huge difference in people's lives. Dr. Picard was now hooked, and she continues to pursue emotional technology to change lives.

When she's not in the lab, Dr. Picard enjoys spending time with her husband and sons. 


#9 on “30 Most Innovative Women Professors” list (Condoleezza Rice was #1) 2016

CNN’s 7 tech Superheroes to Watch in 2015

Sigma Xi Walston Chubb Award for Innovation 2014

IEEE Trans on IT Systems best paper of the decade 2013

Popular Science Top Ten Inventions of 2011

New York Time's Magazine's "Best Ideas of the Year" 2006

Groden Network Distinguished Honorees, Research Award 2008


Using Affect Within A Gaming Context

U.S. Patent No. 9,247,903: “Using Affect Within A Gaming Context.” Inventors: Bender, Kaliouby, Picard, Sadowsky, Turcot, Wilder-Smith).

Method and Apparatus for Relating and Combining Multiple Images of the Same Scene or Object(s)

U.S. Patent No. 5,706,416: “Method and Apparatus for Relating and Combining Multiple Images of the Same Scene or Object(s)." Inventors: Picard and Mann.

Sensing and Display of Skin Conductivity

U.S. Patent No. 6415176: “Sensing and Display of Skin Conductivity.” Inventors: Picard, Scheirer, Tilbury and Farringdon.

Methods and apparatus for Monitoring Patients and Delivering Therapeutic Stimuli

U.S. Patent No. 8,655,441: “Methods and apparatus for Monitoring Patients and Delivering Therapeutic Stimuli.” Inventors: Picard, Fletcher, Eydgani and Williams. Issued February 18, 2014.

Biosensor Module with Leadless Contacts

U.S. Patent No. 8,774,893B2: “Biosensor Module with Leadless Contacts." Inventors: Picard and Wilder-Smith. Issued: July 8, 2014.

Biosensors with electrodes and pressure compensation

U.S. Patent No. 8965479: “Biosensors with electrodes and pressure compensation." Inventors: Picard and Wilder-Smith.  

Method for Biosensor Through Pressure Compensation

U.S. Patent No. US8396530B1: “Method for Biosensor Through Pressure Compensation.” Inventors: Picard and Wilder-Smith).

Washable wearable biosensor

U.S. Patent No. 8140143: “Washable wearable biosensor.” Inventors: Picard, et al.