When U.S. Supreme Court justices write opinions, they mostly talk about the case in question. But occasionally, they will discuss themselves or the court, using what is called the “monologic voice.”

Researchers in the Cornell Ann S. Bowers College of Computing and Information Science trained a large language model (LLM) to identify the monologic voice – used in an attempt to affirm one’s legitimacy, monologue style – including its collective and individualistic tones, in eight decades’ worth of U.S. Supreme Court opinions.

Among the findings: The monologic voice is used at approximately the same rate throughout, but majority opinions tend to use a collective tone, while concurring and dissenting opinions use more individualistic tones. Additionally, all current justices except Chief Justice John Roberts write overwhelmingly in individualistic tones, contrasting sharply with the William Rehnquist court, which used collective tones at a historically high rate.

“There’s a lot of research into how Supreme Court justices go about legitimating the court, but this often focuses on public moments – nomination hearings, television interviews, speeches and oral arguments. There’s not as much research into how they do legitimation in the court opinion,” said Rosamond Thalken, a doctoral candidate in the field of information science and corresponding author of “Judicial Self Fashioning: Rhetorical Performance in Supreme Court Opinions,” which published Sept. 29 in the journal Discourse Studies.

Her co-authors are David Mimno, associate professor and chair of the Department of Information Science (Cornell Bowers CIS), and Matthew Wilkens, associate professor of information science (Cornell Bowers CIS).

For the study, the researchers trained a large language model (LLM) to identify the monologic voice, including its collective and individualistic tones, in a total of 15,291 Supreme Court decisions between 1946 and 2022. This allowed them to examine how this language has changed over time.

“The Supreme Court, though very powerful, doesn’t have a lot of actual power in its hands,” Wilkens said. “It doesn’t have an army, it doesn’t have executive powers. It depends, ultimately, on the political branches going along with what it says. Therefore, it’s important not just to persuade politicians, but to persuade the public that it works in a legitimate way, so that the decisions it makes are supported.”

The researchers first labeled sentences from 10 politically noteworthy cases, then added 36 randomly sampled non-political opinions, to train the LLM to detect both rhetorical and non-rhetorical sentences. They labeled 8,001 sentences, then did another round of LLM training to find the monologic voice.

If a sentence’s focal point is the justice or the court, and it discusses judicial duties, the legitimacy of a prior decision or the purpose of the Court, it is assigned the monologic label.

“The monologic voice would be moments where judges talk about themselves and the role of the court in the court opinion,” Thalken said. “And then if the opinion is seen as unifying the court, that would be a collective tone; if it’s performing more criticism of the members, it would be the individualistic tone.”

A total of 5,982 rhetorical sentences were coded either monologic or non-monologic. Personal pronouns were another strong predictor of the monologic voice, and of one tone or the other. The researchers found that by simply marking all sentences that contained a personal pronoun, they could match 95% of all monologic predictions made by their LLM. However, sentences without a personal pronoun –  but with the words “court” or “plurality” – were also often correctly predicted as monologic.

That’s evidence of the power of LLMs, Thalken said.

“These models are able to pick up on something that is slightly more complicated, more nuanced,” she said. “Prior work would have just used word counts or something similar to that. But here, we’re using large language models to identify rhetorical concepts.”

In addition to detecting different tones between the Rehnquist and Roberts courts, the researchers were surprised to find that seniority really didn’t factor into writing styles as much as whether the opinion was majority or concurring/dissenting.

“And it’s the same with ideology,” Thalken said. “This isn’t something where you have the conservative justices writing in a certain way and the liberal justices are writing in an entirely different way.”

In terms of tone, Thalken said, one might think that the high court would want to portray, at least outwardly, a sense of unity when its decisions are marked by disunity. But that’s not the case, particularly in the most recent years studied.

“In the 2020s, at a time when the court is issuing a historically high number of divided opinions, we might have expected a greater use of the collective tone to try to cover for that fact of division, “ she said. “But that’s not what we’re seeing.”

As a result, the researchers said, the chief justice has gone in the other direction – attempting to portray a sense of unity in a divided court.

“We don’t know Roberts’ mind,” Wilkens said, “but our results are compatible with the hypothesis that he might perceive his function to be speaking for the court as a collective entity.”

Thalken said related work with Jed Stiglitz, associate dean for academic affairs and professor of law at Cornell Law School, has involved training the LLMs to detect a justice’s method of reasoning in their decision – either formal logic, which relies on “mechanical” reasoning (case law, precedents) or “grand” logic, in which the law is seen as evolving over time and can be interpreted using economic, social or political factors.

Their research, “Modeling Legal Reasoning: LM Annotation at the Edge of Human Agreement,” was published in Empirical Methods in Natural Language Processing, in December 2023. Thalken said future work may involve looking at the relationship between a justice’s rhetoric and reasoning.

Support for this work came from the National Science Foundation.