Skip to main content

Computers not yet able to understand human speech

Perhaps Hal from "2001: A Space Odyssey" may not have been wrong when he said: "I'm sorry, Dave, I'm afraid I can't do that." Machines -- even Apple's Siri -- cannot yet completely understand our natural language, a Cornell researcher says.

For the second installment of the School of Continuing Education and Summer Sessions lecture series, Cornell's Lillian Lee, professor of computer science, drew 225 faculty, students and guests to Kennedy Hall's Call Auditorium July 18. Lee detailed the progress in natural language processing (NLP) and machine learning, and the challenges that lie ahead.

"Understanding language is really hard, not just because of understanding the structure of language part ... it also involves understanding things about what human beings want," Lee explained. Scientists are trying to integrate the insight from linguistics into statistical models, but "we are not all the way there yet," Lee said.

What would happen if, in March 2012, you queried, "Is Snooki on stork watch?" into Google, or asked the question to "Watson," the machine that has beaten human champions in Jeopardy. "Google didn't know the answer!" Lee said. "I've argued that we need a probabilistic approach; a data approach. ... How would Watson figure this out? We have a lot of data. We as human beings may notice what answers the first question. Watson doesn't understand 'Snooki and fiancé Jionni LaValle are expecting their first child together' when asked about 'stork watch.'"

More Cornell summer events

Free Cornell summer events sponsored by the School of Continuing Education and Summer Sessions include "Journey West" with Max Buckholtz, July 24; and classical and improvisation with Malcolm Bilson and Roger Moseley, July 31. Lectures, Wednesdays at 7:30 p.m. in Kennedy Hall's Call Auditorium: "Lessons for Living: Tried and True Advice From the Wisest (and Oldest) Americans," July 25, with Karl Pillemer, professor of human development; and "Doing Math in Public," Aug. 1, with Steven Strogatz, professor of engineering and mathematics. Free concerts on the Arts Quad, Fridays at 7 p.m.: Latin dance band El Rumbon, July 27; and roots music with The Horse Flies, Aug. 3.

NLP seeks to create systems that can use human language as input or output. This includes speech-based interfaces, information retrieval (such as Google), automatic summarization of news, emails and postings, and automatic translation (such as Google Translate). According to Lee, the thrill of NLP is that it is "interdisciplinary, including fields of computer science, linguistics, psychology, communication, probability and statistics, and information theory."

"Why is understanding language so hard?" Lee answers her own question by providing the example: "I saw her duck with a telescope." According to Lee: "[This sentence] could mean a lot of things. If you look at the word 'duck,' it could mean I'm 'ducking' because people are throwing potatoes at me. Or the word duck could be the animal. In both cases, you have to ask who's holding the telescope … seven simple little words, and this sentence could mean a bazillion things."

According to Lee, somewhere between science fiction and new technological advancement there is a dream and a promise of computers that can understand what people are saying. Human intelligence can be demonstrated by natural language conversation.

Even Siri has not been able to stand up to this test of intelligence. For example, Lee explains that telling her, "We can email you when you're back" generates "We can email you when you're fat."

The moral of Lee's story: "Today, we need to be careful before you hit, or now even say, the word 'send.'"

Rebecca Harrison '14 is a writer intern for the Cornell Chronicle.