What makes people good at having conversations? In a recent paper, Cornell researchers explored conversations on a crisis text service in order to figure out how to answer that question.

“The problem we always came up with was that we never knew if the things we observed were correlations, or if they could actually provide useful information to inform how the platform assigns counselors,” said Justine Zhang, doctoral student in information science and first author of “Quantifying the Causal Effects of Conversational Tendencies.”

The paper was presented at the Association for Computing Machinery Conference on Computer-Supported Cooperative Work and Social Computing, held virtually Oct. 17-21.

For example, the researchers noticed that positive language tended to appear in better conversations, but did this mean that the positive words improved the conversations or was it just easier to use positive words with people who are less distraught?

Using anonymized data from Crisis Text Line, a crisis counseling hotline where people in mental health distress can exchange texts with counselors, the researchers found that the effect of using positive language disappeared when they considered other factors, such as the time of day the conversation took place.

Understanding which language characteristics are tied to better conversations could offer Crisis Text Line and similar services a data-driven way to better allocate counselors to callers. Understanding effective language use could also help in other realms, such as customer service, tutoring and interviewing.

“We want to figure out how to make these conversations better, and we’d like for these recommendations to be data-driven,” said Zhang, who is advised by Cristian Danescu-Niculescu-Mizil, associate professor of information science and the paper’s co-author. “It’s hard. We’ve identified precisely why it’s hard. And once we identified some of these specific challenges, we offer recommendations for what you can do to get around these challenges.”

For example, the time of day could influence the types of problems people are calling about, or the severity of their situations. Researchers could control for this by exploring the language used by counselors on the same shift, when counselors working at the same time are assigned calls randomly.

Another problem is disentangling how the interaction with the caller shapes the counselor’s language. For instance, a straightforward analysis might show that counselors who frequently say “you’re welcome” have better conversational outcomes. But telling counselors to say “you’re welcome” more often is probably not going to lead to better conversations, because they’re probably responding to callers saying “thank you” – an indication that the call is already succeeding.

“It’s basically a signal that the conversation has already gone pretty well, rather than something they can actually do,” Zhang said.

One solution to that problem is considering only the beginnings of conversations, before the callers’ behavior starts influencing the counselors’ language – but this would help understand only the very start of a conversation. Building off the challenges they identified, the researchers developed new solutions that could address these problems, at least in settings such as the counseling hotline.

Though researchers found no impact from positive language on the success of conversations, they found that it may be more promising to allocate texters to counselors who tended to write longer messages or better echo the texters’ language. Conversations were considered successful if participants rated them highly in a survey.

The paper is among the first in the fields of machine learning and computational social science to examine the difference between correlation and causation in the context of conversations, Zhang said.

“Hopefully, by laying out all the challenges you could face and the ways you might get around these challenges, can be a starting point for more people to ask these questions in future research,” she said.

The paper was co-authored with Sendhil Mullainathan of the Chicago Booth School of Business. The research was supported in part by a National Science Foundation CAREER Award and a Microsoft Research Ph.D. fellowship; the collaboration with Crisis Text Line was supported by the Robert Wood Johnson Foundation.