Cowley’s team recorded the fruit fly courtship process over a series of “dates” in a petri dish, tracking the behavior of the male (blue) in response to any sight of the female (red). Tiny microphones placed below this “arena” captured the songs the male produced by beating its wings. Credit: Cowley Lab/Cold Spring Harbor Laboratory
Research using AI reveals new insights into how vision influences fruit fly behavior, with implications for human visual research.
We’ve all heard the saying, “The eyes are the window to the soul.” But remember, windows offer a view in both directions. Our eyes are also our windows to the world. What we see and how we see it help determine how we move through the world. In other words, our vision helps guide our actions, including social behaviors.
Now, a young scientist at Cold Spring Harbor Laboratory (CSHL) has uncovered a major clue into how this works. He did it by building a special AI model of the common fruit fly brain.
Watch as the male fruit fly (blue) courts a female companion (red). The corresponding animation captures the male fly’s point of view. Credit: CSHL
AI and Fruit Fly Behavior
CSHL Assistant Professor Benjamin Cowley and his team honed their AI model through a technique they developed called “knockout training.” First, they recorded a male fruit fly’s courtship behavior—chasing and singing to a female. Next, they genetically silenced specific types of visual neurons in the male fly and trained their AI to detect any changes in behavior. By repeating this process with many different visual neuron types, they were able to get the AI to accurately predict how the real fruit fly would act in response to any sight of the female.
“We can actually predict neural activity computationally and ask how specific neurons contribute to behavior,” Cowley says. “This is something we couldn’t do before.”
Decoding Neural Pathways
With their new AI, Cowley’s team discovered that the fruit fly brain uses a “population code” to process visual data. Instead of one neuron type linking each visual feature to one action, as previously assumed, many combinations of neurons were needed to sculpt behavior. A chart of these neural pathways looks like an incredibly complex subway map and will take years to decipher. Still, it gets us where we need to go. It enables Cowley’s AI to predict how a real-life fruit fly will behave when presented with visual stimuli.
With this schematic in hand, Cowley’s team can now turn their attention to interrogating their AI model instead of running costly experiments on real fruit flies. Credit: Cowley lab/Cold Spring Harbor Laboratory
Implications for Human Brain Research
Does this mean AI could someday predict human behavior? Not so fast. Fruit fly brains contain about 100,000 neurons. The human brain has almost 100 billion. Referring to the subway map, Cowley says:
“This is what it’s like for the fruit fly. You can imagine what our visual system is like.”
Still, Cowley hopes his AI model will someday help us decode the computations underlying the human visual system. He says:
“This is going to be decades of work. But if we can figure this out, we’re ahead of the game. By learning [fly] computations, we can build a better artificial visual system. More importantly, we’re going to understand disorders of the visual system in much better detail.”
How much better? You’ll have to see it to believe it.
Reference: “Mapping model units to visual neurons reveals population code for social behaviour” by Benjamin R. Cowley, Adam J. Calhoun, Nivedita Rangarajan, Elise Ireland, Maxwell H. Turner, Jonathan W. Pillow and Mala Murthy, 22 May 2024, Nature.
DOI: 10.1038/s41586-024-07451-8
Starr Foundation, Simons Collaboration on the Global Brain, National Institutes of Health, NIH BRAIN Initiative, Howard Hughes Medical Institute, National Institute of Neurological Disorders and Stroke
