Fifteen years ago, labor accounted for about 45% of total costs at the Washington Fruit and Produce Co. Today, it’s over 60%. 

“Our costs have just been skyrocketing, largely around labor, but everything else as well – meanwhile, apples sell for the same price they did 20 years ago,” said Gilbert Plath, technology director for the Yakima, Washington-based orchard, which grows apples and cherries on 10,000 acres. “Who gets squeezed at that point? It’s not the warehouse, and it’s not the grocery store. It’s the farmers. We need to find ways to do more with less.”

Plath’s fourth-generation family of growers is one of nine organizations nationwide collaborating on a Cornell-led research project to develop robots that can perform labor-intensive orchard operations such as pollinating flowers, thinning fruits, harvesting apples and weeding between rows. The project is supported by a newly announced four-year, $7.5 million grant from the U.S. Department of Agriculture’s Specialty Crop Research Initiative. The grant will establish a Center of Excellence for Orchard Robotics in Cornell’s Department of Biological and Environmental Engineering. 

“Some of these farming operations are not just labor-intensive, they’re risky for human health and safety, whether that be through ladder falls, machinery-related accidents or repetitive motion injuries,” said Manoj Karkee, the Norman R. and Sharon R. Scott Professor of Agriculture and Life Sciences in the College of Agriculture and Life Sciences (CALS), who is leading the project alongside co-director Matthew Whiting, professor of tree fruit horticulture at Washington State University. 

“When you’re picking apples 12 hours a day for three months, a lot of people develop repetitive motion injuries,” Karkee said. “We’d like to automate these tasks as much as possible and create job opportunities for workers in manufacturing, maintaining and supervising these machines.”

With additional support from Federal Capacity Funds managed by the Cornell University Agricultural Experiment Station (Cornell AES), Karkee seeks to make the robots autonomous and self-driving. He’s collaborating in this work with dozens of academic and industry colleagues from Cornell, Washington State University, Oregon State University, Pennsylvania State University, Michigan State University, Carnegie Mellon University, Stanford University and the Israeli-based Fresh Fruit Robotics Company. 

Marrying old and new: Ag and AI

Apples are the most-consumed fruit in the U.S., contributing $21 billion annually to the national economy. New York ranks second for total apple production, after Washington, and ranks first in diversity of apple varieties grown. 

Any agricultural technology, no matter how revolutionary, will not be useful if farmers can’t afford it or don’t adopt it. To that end, Karkee’s collaborators include sociologists, economists, engineers and computer scientists, as well as horticulturalists. Cornell Cooperative Extension fruit extension specialist Mario Miranda Sazo will lead outreach to New York apple growers to gather their feedback and ensure the developed technologies meet their needs. In addition to engineering the actual robots, the project team will carry out tasks such as: developing digital twins of real orchards to aid horticultural analysis; training artificial intelligence to perceive fruit tree canopies so they can determine, for example, which fruitlets to thin early in the season; and analyzing the cultural and economic factors that affect technology adoption in farming. 

Most agricultural crops in the U.S. are harvested by machine, and apple growers have hoped for a robotic solution for many years, Karkee said. For more than two decades, he’s been working on creating soft, robotic hands that can handle fruits without bruising them, and training agricultural robots to move autonomously. The recent, rapid advances in AI and machine learning have supercharged his lab’s ability to train robots to recognize leaves, stems and fruits, and make independent decisions about which fruitlets to thin in the spring so remaining fruit can grow to marketable size, he said. 

“Recent models based on neural networks have revolutionized what we can do around data and image processing,” said Karkee, who runs research trials at the Cornell Orchards. “Where it is safe to go, where the fruits are located, how fruits are located in relation to a branch, a trellis, a wire, leaves, other fruits – all those things are being perceived and understood by robots using an AI model running behind the scenes.”

‘Turning innovation into reality’

Avi Kahani, founder and CEO of Fresh Fruit Robotics, began dreaming about building robots to automate orchard tasks decades ago when he worked for a time as the orchard manager at a kibbutz in northern Israel. The mechanical engineer and project collaborator has been working on developing robots to automate orchard tasks for more than 10 years. 

“Turning innovation into reality is still one of the major challenges for an industrial company like us,” Kahani said. “We see these incredible things that universities and other bodies develop, but then we have to see if it fits into reality – if it works and most importantly, if it’s cost effective.” 

For example, one of the main points of feedback he’s received from growers is that if a machine is very expensive and can only help with harvesting, that return-on-investment is not worth it for growers, he said. However, if the robot could help with multiple tasks throughout the year – such as pruning in the winter, pollinating and thinning in the spring, and harvesting in the fall – that would be more useful and cost-effective for growers.

“Nobody thinks about harvesting cotton or corn by hand anymore. The specialty fruit market indeed needs solutions for this niche,” Kahani said. “I believe that with the intelligent people involved in this – the growers, a company like us and the researchers – I’m hopeful that we will see robots out in commercial orchards in the not-too-distant future.”

Initial work for this project was supported by the CALS Research and Innovation Office, through grants focused on AI in agriculture, food security, sustainability, health and life sciences.

Krisy Gashler is a freelance writer for the Cornell University Agricultural Experiment Station.