Male fruit flies don’t usually like each other. Socially, they reject their fellow males and zero in on the females they discern via chemical receptors – or so scientists thought.

New research from Cornell biologists suggests the fruit fly’s visual system, not just chemical receptors, are deeply involved with their social behaviors. The work sheds light on the possible origin of differences in human social behaviors, such as those seen in people with bipolar disorder and autism.

The paper, “Visual Feedback Neurons Fine-tune Drosophila Male Courtship via GABA-mediated Inhibition,” published Sept. 5 in Current Biology.

Many species of animals use vision to regulate their social behaviors, say the researchers, but the underlying mechanisms are largely unknown.

In fruit flies, vision is thought to be used explicitly for motion detection and following an object, not to regulate social behaviors. But the researchers found that may not be the case.

“In our study, we found that hyperactivating the visual system overran the inhibition generated by chemical signals emitted by the male fly to say to the other male, ‘I’m another male – don’t mess with me,’” said senior author Nilay Yapici, assistant professor of neurobiology and behavior and Nancy and Peter Meinig Family Investigator in the College of Arts and Sciences. “Surprisingly, increasing the visual gain in the brain somehow overrides the chemosensory inhibition, attracting male flies to other males.”

The researchers found that altering the GABARAP/GABAA receptor signaling in visual feedback neurons in the male brain affected the flies’ social inhibitions. When GABARAP is knocked down in the visual system, the males unexpectedly exhibit increased courtship toward other males.

The researchers have found that genes similar to those in the human brain control the fruit fly’s visual neurons. Decreasing GABA signaling in the human brain has been associated with social withdrawal characteristics in conditions such as autism and schizophrenia.

“Our results offer a promising avenue for investigating how these proteins regulate social behaviors in the mammalian brain and their potential contribution to human psychiatric conditions,” said lead author Yuta Mabuchi, Ph.D. ’23.

Because the fly brain is so much smaller than the human brain – only 150,000 neurons – it was possible for the researchers to identify the visual neuron changing the fly’s behavior and trace it to the central circuits that regulate fly social behaviors.

To find the gene, the researchers conducted a genome-wide RNA screen, knocking out each of the fly’s genes in the fly brain to see what effect it would have on its social behaviors. The GABARAP gene stood out because of its surprising impact on the male’s social interactions – though it took two years of painstaking work before Mabuchi could identify which neurons were required for GABARAP to regulate male behavior.

Mabuchi then used two-photon microscopy to trace the neurons affected by the gene and identify the entire circuit. Altogether, Yapici said, it took Mabuchi almost six years of “elegant experiments” to uncover the whole mechanism.

“Our lab studies how behaviors are generated and regulated in the brain on the level of genes and circuits,” Yapici said. “We mainly study innate behaviors in flies to investigate how an animal is born with the capacity to execute certain behaviors that aren’t learned, and we try to understand how neural circuits and genetic mechanisms regulate these behaviors.”

Other co-authors include doctoral candidate Xinyue Cui; Lily Xie ’21; Haein Kim, a postdoctoral researcher in neurobiology and behavior; and Tianxing Jiang ’20, M.Eng. ’21.

The research was supported by a Pew Biomedical Scholar Award, an Alfred P. Sloan Foundation Award and a National Institutes of Health grant.

Linda B. Glaser is news and media relations manager for the College of Arts and Sciences.