A team of researchers from Cornell Tech has developed a new tool designed to revolutionize hardware troubleshooting, with the help of 3D phone scans. 

SplatOverflow – inspired by StackOverflow, a widely used platform for tackling software issues – brings a similar approach to hardware support, enabling users to diagnose and fix hardware issues asynchronously with the help of remote experts. 

A paper about the new tool will be presented April 30 at the Association of Computing Machinery’s CHI conference on Human Factors in Computing Systems, taking place April 26-May 1 in Yokohama, Japan.

SplatOverflow was developed in the Matter of Tech Lab at Cornell Tech, directed by Thijs Roumen, assistant professor at Cornell Tech and the Cornell Ann S. Bowers College of Computing and Information Science (Cornell Bowers CIS).

The tool, conceived by Amritansh Kwatra, a Ph.D. student in the field of information science at Cornell Tech, aims to address a major gap in hardware maintenance: the lack of scalable, structured support for troubleshooting issues.

“Currently, design information, documentation and end-user discussions have all existed in separate spaces,” Kwatra said. “They have also been disconnected from the actual hardware, so searching for this information is quite daunting for users whose expertise is not in hardware.”

SplatOverflow creates a 3D model of a malfunctioning device, paired with detailed information about the 3D model, which experts and users can examine together to determine a solution.

The model, known as a “SplatOverflow scene,” is made up of two parts: a 3D Gaussian Splat (a 3D rendering of the malfunctioning machine, based on phone scans); and a computer-aided design (CAD) model, a detailed blueprint for the machine.

By connecting a 3D Gaussian Splat with a CAD model, experts can virtually compare a malfunctioning machine with the blueprint of what it is supposed to look like. For users reporting a hardware issue remotely, they can simply point to where they think the issue is coming from directly within the virtual 3D model.

The Matter of Tech Lab focuses on digital fabrication technologies, like 3D printers and laser cutters. In their paper, researchers demonstrated SplatOverflow on a pick-and-place machine (a robotic device used in electronics assembly); a 3D printer; an open-source e-reader; and an Ikea bookshelf.

The researchers are exploring applications in other contexts.

“We are currently looking into using SplatOverflow for machine sanitation and maintenance in agriculture,” Roumen said. “We are also interested in seeing its deployment in education and other domains.”

Project adviser François Guimbretière, a professor of information science (Cornell Bowers CIS), highlighted how the tool could potentially assist farmers in the upkeep of expensive, complicated machinery in remote locations.

“This could have a great impact on food safety and ensuring crops are harvested at peak return value,” he said. “The paper is presenting our first prototype. Ultimately, we would like to develop the system further so that it could be deployed and evaluated by a large population over a long period of time.”

Looking ahead, the team hopes to expand SplatOverflow into an open-source ecosystem – much like StackOverflow.

“We invite others to explore the use of SplatOverflow in other contexts,” Roumen said. “We are thrilled to explore longer-term deployment, to see how we can gradually grow this into a real, open-source platform for hardware support.”

This research was supported by a Ph.D. fellowship to Kwatra from the Digital Life Initiative.

Grace Stanley is a staff writer-editor for Cornell Tech.