A radical collaboration between a biologist and an engineer is supercharging efforts to protect grape crops. The technology they’ve developed, using robotics and AI to identify grape plants infected with a devastating fungus, will soon be available to researchers nationwide working on a wide array of plant and animal research.

The biologist, Lance Cadle-Davidson, Ph.D. ’03, an adjunct professor in the School of Integrative Plant Science (SIPS), is working to develop grape varieties that are more resistant to powdery mildew, but his lab’s research was bottlenecked by the need to manually assess thousands of grape leaf samples for evidence of infection.

Powdery mildew, a fungus that attacks many plants including wine and table grapes, leaves sickly white spores across leaves and fruit and costs grape growers worldwide billions of dollars annually in lost fruit and fungicide costs.

Cadle-Davidson is also a research plant pathologist with the U.S. Department of Agriculture’s Agricultural Research Service (USDA-ARS). He works in the Grape Genetics Research Unit in Geneva, New York, and his team developed prototypes of imaging robots that could scan grape leaf samples automatically – a process called high-throughput phenotyping – through the USDA-ARS funded VitisGen2 grape breeding project and in partnership with the Light and Health Research Center. This partnership led to the creation of a robotic camera they named “BlackBird.”

Credit: Allison Usavage/Provided

But extracting relevant biological information from these images was still a critical need.

Enter the engineer and computer scientist: Yu Jiang, an assistant research professor in SIPS’ Horticulture Section at Cornell AgriTech. Jiang’s research focuses on systems engineering, data analytics and artificial intelligence. The BlackBird robot can gather information at a scale of 1.2 micrometers per pixel – equivalent to a regular optical microscope. For each 1-centimeter leaf sample being examined, the robot provides 8,000 by 5,000 pixels of information.

Extracting useful information from such a large, high-resolution image was Jiang’s challenge, and his team used AI to solve it. Using breakthroughs in deep neural networks developed for computer vision tasks like face recognition, Jiang applied this knowledge to the analysis of microscopic images of grape leaves. In addition, Jiang and his team implemented the visualization of the network inferential processes, which help biologists better understand the analysis process and build confidence with AI models.

Working together, Cadle-Davidson’s team tests and validates what the robots see, enabling Jiang’s team to teach them how to identify biological traits more effectively. The results are astounding, Cadle-Davison said. Research experiments that used to take his entire lab team six months to complete now take the BlackBird robots just one day.

“It has revolutionized our science,” Cadle-Davidson said. “And we’re finding that Yu’s AI tools actually do a better job of explaining the genetics of these grapes than we can do sitting at a microscope for months at a time doing backbreaking work.”

In the month of July alone, the collaboration won an award and two new grants. On July 1, the team was awarded a $100,000 grant from the USDA-ARS to disseminate BlackBird to ARS field offices working on other crops that do the same kind of high-throughput phenotyping work.

“We hope to find collaborative labs who can join us in taking advantage of this tool,” Jiang said. “We see potential applications for this research in plant studies, animal fields or medical purposes.”

On July 12, the team’s article on their project won the Information Technology, Sensors, and Control Systems’ best paper award at the 2021 American Society of Agricultural and Biological Engineers annual international meeting. And on July 27, they were awarded a two-year, $150,000 grant from the Cornell Institute for Digital Agriculture Research Innovation Fund to begin upgrading the BlackBird robot to see beyond the red-green-blue color spectrum and into infrared.

Plant diseases like powdery mildew can show up in infrared before they are visible to the naked eye; if the researchers can develop tools to help farmers detect disease early, it would enable farmers to target fungicide sprays before infection spreads, meaning less fungicide and fewer lost crops. They’re also working to integrate AI more effectively with scientists in data analysis.

“This work is greatly accelerating the pace of breeding and genetics work in grape,” said Donnell Brown, president of National Grape Research Alliance. “Normally, when we in industry invest in research, we do it knowing that we may never see the outcome of our investments in our lifetimes – it’s really a faith-based investment in future generations of growers. But now, this technology is really shortening that timeline, for the benefit of growers and consumers.”

Krisy Gashler is a writer for the College of Agriculture and Life Sciences.