Cogito and MGH test voice app to monitor moods

Cogito's Companion app uses the company's voice analysis software to detect emotions in your speaking voice.
Cogito's Companion app uses the company's voice analysis software to detect emotions in your speaking voice.

A Boston company has developed a smartphone app that can decipher emotions in the human voice, an early-warning system that could bring timely help to those with mental illness.

Cogito has been developing its voice analysis software for more than a decade and since last fall has been working with the US Department of Veterans Affairs to monitor the moods of service members through the app.

Now, researchers at Massachusetts General Hospital will conduct a two-year study of 1,000 patients with bipolar disorder and depression to track their mental health using the same app.

With permission, the app tracks how social a person is by looking at how often they text or call, and it uses location tracking data to detect whether a person isn’t leaving the house for long periods of time. A “voice diary” feature lets a user leave a short spoken message, which
Cogito’s software then analyzes to read their mood.

The voice is a rich source of data, said James Harper, cofounder of Sonde Health, a startup that is building technology that will use the voice as an indicator of health.

Harper said the right kind of software can analyze the voice stream to spot trouble in the nervous system or the respiratory system or the mind. “In general, I can say that mental health is emerging as the sector most in need of those capabilities.”

The question the VA study will focus on is whether veterans will actually use the app. About 500 veterans are expected to enroll in the study.

Cogito founder and CEO Joshua Feast said the idea is for the silent phone companion to alert doctors or family to unusual mood swings, or even predict the onset of a bad spell. The hope is that it could become an important tool for fighting the high suicide rate among military service members, who are killing themselves at the rate of about one every day. According to a 2015 study that examined the records of 1.3 million veterans, deployed and nondeployed veterans had a 41 percent and 61 percent higher suicide risk compared to the general population.

In 2013, Cogito was midway through a 100-person test-run of an early version of the software when the Tsarnaev brothers set off bombs at the finish line of the Boston Marathon. The software picked up on a dip in the mood of test participants in the weeks that followed. It also noticed that the event more acutely affected people who had a history of mental health issues. “That was one of the things that told us, ‘wow this is really going to work,’ ” said Feast.

The concept has commercial potential: The company’s first product is software for call centers that can give employees a real-time read of the mood of the person at the other end of the line.

In a clinical setting, doctors typically ask patients with bipolar disorder or depression to keep “mood diaries” to track their emotions over days and weeks, said Dr. Thilo Deckersbach, leader of the new study and associate director of the Bipolar Clinic and Research Program at MGH. Instead of keeping a written log of their moods — which happened fairly irregularly, it turns out — study participants will download and activate the Cogito Companion app.

“The nature of these disorders is that they’re episodic — there are periods in which you are hypermanic or depressed,” Deckersbach said.

The app would be particularly effective if it gave the user a readout in real time, said Danna Mauch, president and CEO of the Massachusetts Association for Mental Health, a nonprofit advocacy group. Such feedback would also be key to making sure people with mental illness will not discard the app after a few trials.

“It could get them to recognize mood changes that they may not be immediately aware of or confronting. So it could be very valuable,” Mauch said.

Unlike hypertension or diabetes, which have measurable indicators such as blood glucose levels or blood pressure, mental health lacks a continuous, “objective” measure.

“There’s a lot of noise in the measures of behavioral health and mental health, and this may let us get a better signal,” said David Ahern, director of the Program in Behavioral Informatics and eHealth in the Department of Psychiatry at Brigham and Women’s Hospital. But the key to realizing the potential of such technologies would be proving their efficacy. Ahern is leading another test of Cogito’s app with 300 people at Brigham and Women’s Advanced Primary Care Associates clinic in Jamaica Plain.

The search for such a reliable metric, in part, is motivating the National Institute of Mental Health to fund some 150 projects that use technology to monitor mental health or deliver care. Of late, many of those are investigating app-based tools, according to Dr. Adam Haim, chief of the Clinical Trials Operations and Biostatistics branch at the NIMH.

Only about 15 percent of Americans with mental illness receive minimally acceptable care, according to the institute. If it works, a well-distributed tool such as a phone app could bring help to a much larger number of people, Haim said.

“Technology exists to get more frequent and more accurate assessments of one’s behaviors and one’s thoughts and emotions,” Haim said. “Linked with well-delivered care, there is opportunity for significant benefits.”

This article was updated on Feb. 25 at 11:05 a.m. to include comment from David Ahern and Danna Mauch. 

Nidhi Subbaraman writes about science and research. Email her at [email protected]
Follow Nidhi on Twitter - Facebook