With advances in deep learning, machines are now able to “predict” a variety of aspects about life, including the way people interact on online platforms or the way they behave in physical environments. This is especially true in computer vision applications where there is a growing body of work on predicting the future behavior of moving objects such as vehicles and pedestrians.
“However, while machine-learning methods are now able to match — and sometimes even beat — human experts in mainstream vision applications, there are still some gaps in the ability of machine-learning methods to predict the motion of ‘shape-shifting’ objects that are constantly adapting their appearance in relation to their environment,” said Anuj Karpatne, assistant professor of computer science and faculty at the Sanghani Center for Artificial Intelligence and Data Analytics.
This is a problem encountered in many scientific fields, Karpatne said. For example, in mechanobiology, cells change their shape and trajectory as they move across fibrous environments in the human body, constantly tugging or pushing on the fibers and modifying the background environment, which in-turn influences the movement of cells in a perpetual loop.
“This is fundamentally different from mainstream applications in computer vision where changes in the background caused by pedestrians and vehicles are far less accelerated than those possible by the movement of living cells governed by the laws of mechanics and biology,” he said.
To address this challenge, the National Science Foundation has awarded a team of Virginia Tech scientists a $1 million grant to create a new avenue of research in physics-guided machine learning. The project will, for the first time, systematically integrate the mechanics of cell motion available as biological rules and physics-based model outputs to predict the movement of shape-shifting objects in dynamic physical environments.
As principal investigator, Karpatne will team with co-principal investigators Amrinder Nain, associate professor, and Sohan Kale, assistant professor in the Department of Mechanical Engineering, combining his expertise in machine learning with their specialties in cell mechanobiology and computational modeling, respectively.
“The work we are doing at the STEP Lab is a natural overlap,” said Nain, who founded the lab and pioneered research in designing nanofiber network platforms and experimental imaging to study cell motion.
“Cell shapes are highly dynamic and undergo limitless transformations as they sense and react to their environment. In addition, cell motion is constrained by the forces exerted by the cells on the background environment and the complex nature of cell-cell and cell-fiber interactions," Nain said. "While conventional methods for studying cell motion require manual tracking of images' features or running computationally expensive tools, our project will take advantage of our ability to create well-defined suspended nanofiber nanonets and advancements in machine learning to open to a new frontier to automatically describe new rules of cell behavior.”
Kale said his Mechanics of Living Materials Lab has already developed a computational method to estimate the forces exerted by cells from the deformed shapes of underlying fibers.
“This, combined with the deep learning framework from Anuj's group, provides a framework to measure forces directly from experimental images of cells moving on nanofiber networks. Our tool enables the study of cell mechanobiology in fibrous environments in a radically different way than existing approaches in the field,” said Kale.
“We are fully leveraging the principles of `convergence research’ in our project by integrating data, knowledge, and methodologies from our three different disciplines — machine learning, experimental cell imaging, and computational modeling,” said Karpatne. “The ultimate goal is to accurately predict and explain how cells move, interact with each other, and change their appearance in physiological environments inside our body.”
The project will contribute foundational innovations by going far and beyond current standards of black-box machine learning for motion prediction in scientific problems. “By anchoring our deep learning patterns with scientific theories, our work advances the frontiers of explainable machine learning by discovering new rules of cell behavior that are physically consistent and scientifically meaningful,” Karpatne said.
The research has potential impact on several scientific disciplines that routinely involve predicting the trajectories of shape-shifting objects in dynamic physical environments, for example, hurricane prediction, bird migration, and ocean eddy monitoring, he said.
The project will also lead to novel advances in mechanobiology.
“Studying cell migration is a major research frontier in the study of embryo development, wound closure, immune response, and cancer metastasis,” Nain said. “We expect that this research will also serve as a drug discovery, diagnostics, and testing platform in the context of cancer and wound healing biology where the spread of disease or repair of wound result from the constant change of cell and fibrous network shapes.”
The research team is committed to supporting Virginia Tech’s education and workforce development goals, especially toward training a diverse cadre of students who can address complex problems requiring interdisciplinary skills. These students include those majoring in computer science, mechanical engineering, physics, and biological sciences.
Three Ph.D. students will also be working on this project. They are Arka Daw in computer science, advised by Karpatne; Abinash Padhi in mechanical engineering, advised by Nain; and Maahi Tulukder in mechanical engineering, advised by Kale.
In conjunction with their research, Karpatne, Nain, and Kale will collaborate with the Center for Educational Networks and Impacts to create a hands-on exhibition on “Artificial Intelligence for Observing Cells” for the annual Virginia Tech Science Festival and Hokie for a Day field trip event.