Drosophila’s Digital Twin
Professor Pavan Ramdya, School of Life Sciences at Ecole Polytechnique Federale de Lausanne (EPFL) says that NeuroMechFly was built using two types of data. To build a biomechanical model that is morphologically accurate, we first took a fly and did a CT scan. We also had data from the fly’s real limb movements. This was obtained with pose estimation software we developed over the past couple of years, which allows us to track the animal’s movements precisely.
Time is fleeting
Drosophilais one of the most common insects in life sciences. Ramdya has been focused on digitally tracking this animal and creating models for it for many years. His group published DeepFly3D in 2019, a deep-learning-based motion-capture program that uses multiple camera views and quantifies the movements of Drosophila within three-dimensional space.
In 2021 Ramdya and his team published LiftPose3D. This method reconstructs 3D animal poses using 2D images from one camera. These breakthroughs have provided the rapidly expanding fields of neuroscience and animal-inspired robots with tools that are of immense value.
NeuroMechFly is in many ways the culmination all these efforts. The model is based on morphological and Kinematic data from previous studies. It features separate computational parts that simulate different parts. The model includes a biomechanical exoskeleton that can articulate body parts such as the head, legs and wings. Proboscis segments, antennae, halteres (organs which help fly to determine its own orientation) and neural network controllers with motor output.
Why not create a digital copy of Drosophila instead?
Ramdya says, “How can we tell when we have understood a system?” Ramdya says that one way to know if a system is understood is to be able recreate it. Although we might attempt to create a robotic fly from scratch, it is much easier and faster to make a simulation of an animal. This work has a major goal: to build a model that incorporates all we know about the fly’s nervous systems and biomechanics, to see if this is sufficient to explain its behavior.
He adds that hypotheses are often what motivate us when we do experiments. “Until now we have relied on intuition and logic to form hypotheses or predictions. As neuroscience becomes more complex, we are increasingly reliant on models that combine many intertwined parts, play them out, then predict what might happen if there is a tweak.
The testbed
NeuroMechFly is a valuable tool for biomechanics and robotics research. However, it only accurately represents the animal in a digital environment. This was a major concern for the researchers. Ramdya says, “We did validation experiments that showed that we could closely reproduce the behavior of the real animal.”
Researchers first took 3D measurements of real grooming and walking flies. The researchers then recreated these behaviors in a physics-based simulation using NeuroMechFly’s biomechanical exoskeleton.