Dynamic duo: a powerful pair of tools to learn about cells

With today’s advanced microscopes, scientists can capture videos of entire embryos developing in real time. But there’s a catch: turning those breathtaking images into clean, accurate trajectories of each cell's journey as it finds its proper place in a developing organism is incredibly hard.

The difficulty comes from cells moving, dividing, and sometimes vanishing altogether as they form the tissues and organs that will comprise a functioning adult animal. Using the cells’ nuclei as landmarks, scientists must find the boundaries of each one in a video frame — a process called segmentation — then track the cells from one video frame to the next. In addition to following developmental processes, accurate cell tracking is essential in understanding how diseases develop and how diseased cells respond to treatments.

Now, in Nature Methods, scientists at the Chan Zuckerberg Biohub San Francisco have unveiled Ultrack, a cell-tracking platform that can scale from following a few cells in a lab dish to tracking whole embryos in 3D videos. Ultrack has proven particularly strong at whole-embryo cell tracking when compared to other tools in the Cell Tracking Challenge, an international benchmarking initiative. The paper was published online on August 25.

“It’s easy to do tracking in 2D or on a few cells, but Ultrack pushes the limits on very hard scenarios, like 3D or full embryos,” says Loïc Royer, director of imaging AI at the San Francisco Biohub and senior author of the paper. “It’s very fast and scales well but also has a lot of practical features to make it easy to use.”

Working smarter, not harder

Cell Tracking algorithms traditionally perform their task in two steps — first, segmenting the cells in each frame of the video, and then linking the same cells across frames. The Achilles’ heel of this approach is the first step, defining all the cells in a large, blurry, 3D microscopy image and having to determine whether what appears to be one large cell in a single frame of a video is instead multiple cells passing by one another, or two cells that recently divided. Instead, Ultrack works smarter, not harder; it tackles the tracking problem by solving these two tasks, segmentation and linking, simultaneously.

Each time Ultrack inspects a candidate region in a frame, the algorithm constructs something called an ultrametric contour map — a hierarchy of boundaries, with the outlines of the possible cells represented as a composition of coarser to finer partitions. To decide which cell boundary is the correct one, Ultrack considers all frames, deciding on the cell boundaries (segments) which are the most consistent over time when connecting to the neighboring frames (the linking).

This way of thinking is similar to how you might figure out whether a cloud in the sky is one large structure or two smaller clouds passing each other. Your brain determines the boundaries of the clouds by comparing the larger structure to the two small clouds you saw before and after — moving together, then apart.

Ultrack simplifies this process even further by solely considering segmentation scenarios that follow the rules of cell biology — such as the fact that cells divide, but they generally don’t merge or make wild jumps from one place to another.

This efficient approach not only saves hours of computation, but also results in fewer tracking errors, meaning that researchers spend less time manually fixing mistakes. In images of dense tissue, where every correction requires considerable manual labor, Ultrack roughly halves the time scientists spend fixing segmentation and tracking mistakes.

“It does all this without the need for retraining deep‑learning models on each new dataset, which is a major hurdle for many labs,” says Biohub SF scientist Jordão Bragantini, first author of the paper. 

From zebrafish to sea squirts

To evaluate Ultrack’s performance in tracking organ development, the team chose the zebrafish neuromast — a mechanosensory organ that helps fish navigate — as a model system. Using guidelines from the Cell Tracking Challenge, Ultrack achieved near-perfect accuracy.

And to build Zebrahub, a zebrafish cell atlas recently published in Cell, the Royer team used Ultrack to reconstruct entire developmental trajectories of multiple embryos. And other Biohub scientists are using Ultrack to study things like the zebrafish immune system.

To make it easy to explore the massive cell-tracking datasets from Ultrack, Royer’s team, in collaboration with colleagues at the Chan Zuckerberg Initiative led by Chi-Li Chiu, developed an innovative tool they call inTRACKtive, also published in Nature Methods; at the Biohub this work was led by Teun Huijben. Using inTRACKtive’s intuitive browser-based interface, users can rotate embryos in 3D space, select groups of cells and follow their trajectories for deeper analysis, speed up or slow down developmental processes, and even see events run backward.

Royer’s team also uploaded datasets from five other model species (collected from the lightsheet community) — including mouse, C. elegans, and sea squirt — to create the Virtual Embryo Zoo, in which users can use inTRACKtive to explore the datasets. Since inTRACKtive runs in any browser, users can delve into datasets interactively from the laptop, desktop, and even their phone. 

“We encourage researchers to contribute to the Virtual Embryo Zoo by submitting their own whole-embryo datasets from other species. Contributions will help the resource grow, creating a comprehensive repository of embryonic development across different organisms,” says Huijben. “Next, we plan to expand inTRACKtive’s capabilities by integrating imaging data alongside cell tracking results. This would allow for even richer visualizations by layering cell behavior and tissue development with live microscopy.”

---

---