Linguistic sentiment may predict future changes in depression symptoms, study finds

A person’s choice of words may be an indicator of whether symptoms of major depressive disorder are worsening, according to a new Yale study. Using both human raters and the large-scale language model ChatGPT, the researchers demonstrated that written responses to open-ended questions could be used to predict who would experience more severe depressive symptoms a few weeks later.

The results, published on September 16 in the Proceedings of the National Academy of Sciencessuggest that automated procedures to assess language use could complement and enhance psychological assessments.

A growing body of research has shown a link between depression and the language a person uses. For example, people with depression use more words with negative emotional connotations on social media and in text messages. Word choice is also associated with how people respond to treatment.

For this study, the Yale researchers wanted to know if language could also provide insight into a person’s future symptoms. To better understand this phenomenon, they asked 467 participants to answer nine short, neutral, open-ended questions as well as the Patient Health Questionnaire (PHQ-9), which assesses the severity of depression. Three weeks later, all participants completed the PHQ-9 again.

Using a tool called Linguistic Inquiry and Word Count (LIWC) — which calculates the number of words that fall into a particular category — the researchers identified the number of words in participants’ written responses to short-answer questions that had a positive or negative emotional tone. Although LIWC scores were associated with depression severity at the time participants answered the questions, they did not predict depression severity three weeks later, the researchers found.

Feeling scores given by human raters, on the other hand, predicted future depressive symptoms.

“This showed us that human raters were picking up on something that simply counting emotional words couldn’t detect,” said Robb Rutledge, assistant professor of psychology in the School of Arts and Sciences at Yale and lead author of the study.

LIWC treats each word individually, which may explain why it is insufficient in this particular application, the researchers said.

“We wanted to study word order and the multidimensional aspect of language, which is key to shaping emotional tone,” said lead author Jihyun Hur, a doctoral student in Rutledge’s lab and that of co-author Jutta Joormann, the Richard Ely Foundation Professor of Psychology. “That’s when we got interested in ChatGPT.”

ChatGPT is an artificial intelligence tool that aims to mimic human conversational language. Therefore, word order and meaning within and across sentences are taken into account in a way that standard language analysis tools, such as LIWC, do not.

When the researchers asked ChatGPT versions 3.5 and 4.0 to rate the positive and negative tone of participants’ responses, the scores predicted future changes in depression severity, just as well as human raters’ scores.

The researchers believe the discovery is a starting point that lays the foundation for further research. Rutledge and his team, for example, are interested in how this approach could be applied to other psychiatric disorders and over longer periods of time.

This line of work is part of the lab’s ongoing research into the relationship between emotion and decision-making, in which anyone can participate by playing games on the lab’s free smartphone app, Happiness Quest.

Rutledge said he could see this type of language assessment as a useful addition to the clinician’s toolbox in the future.

“Analyzing the language that patients use provides additional information that doctors don’t currently have, and our approach could be a way for doctors to evaluate their patients,” Rutledge said. “You need a combination of tools that work with a large number of people and together can give you insight into an individual. If some of these tools are automated like this, it frees up the doctor to spend more time trying to help the patient.”

And finally, a better understanding of the symptoms and how to predict them would be beneficial.

“Artificial intelligence tools like ChatGPT open a new avenue for using the vast amount of language data already available in the clinical setting to better understand mental health,” Hur said.