Cloudy with a chance of lifesaving and more cost-effective weather predictions

When Hurricane Katrina reached the Gulf Coast in 2005, emergency responders were blindsided by a storm surge that defied predictions. In Japan six years later, the destructive scale of a tsunami triggered by a massive earthquake outpaced early warnings. The 2020 wildfires that engulfed California overwhelmed air quality models.

In each of these disasters, comprehensive modeling—encompassing tropical cyclones, ocean waves, air quality, and broader climate variables—could have enhanced emergency responses, saved lives, and cut damage repair costs. However, processing such vast amounts of numerical data has traditionally been computationally intensive and expensive, often hindering timely decision-making.

Now, Paris Perdikaris of the University of Pennsylvania and his collaborators at Microsoft Research have developed a machine learning model capable of accurately forecasting a variety of Earth systems, including air quality, ocean waves, and tropical cyclone tracks. Their new model, Aurora, outperforms existing traditional systems at a fraction of the cost, and their findings could help emergency service providers better prepare for extreme weather events. Their findings are published in Nature.

“Earth’s climate is perhaps the most complex system we study—with interactions spanning from quantum scales to planetary dynamics,” says Perdikaris, an associate professor at the School of Engineering and Applied Science. “With Aurora, we addressed a fundamental challenge in Earth system prediction: how to create forecasting tools that are both more accurate and dramatically more computationally efficient.”

For example, the team’s model correctly predicted landfall of 2023’s Typhoon Doksuri—the costliest Pacific typhoon to date—in the northern Philippines four days ahead of the event, while official forecasts erroneously predicted landfall off the coast of northern Taiwan.

Perdikaris explains that the numerical models that have been the backbone of weather prediction for decades involve complex systems of differential equations derived from physics principles. He notes that instead of solving equations, Aurora identifies complex relationships in historical Earth system data and uses these to generate predictions.

“This makes Aurora dramatically faster—generating predictions in seconds rather than hours— while maintaining or even exceeding the accuracy of traditional models,” he says.

Better, faster, stronger

To achieve better results for less time and money, the team turned to a “foundation model,” an artificial intelligence (AI) model trained on a wide variety of data—much like OpenAI’s GPT. Aurora is trained on more than one million hours of diverse geophysical data, including temperature, wind speeds, humidity, ocean wave heights, and atmospheric chemical compositions, Perdikaris says. These come from weather analyses, reconstructions of historical weather, forecasts, and climate simulations.

The learning process involves two key phases, he explains. First, Aurora is fed this diverse data, learning to predict the evolution of Earth system variables with a six-hour lead time and providing the model with fundamental insights into planetary dynamics. Then, during fine-tuning, this pretrained model can be adapted to perform specific tasks, such as using chemical composition data to predict air quality or pressure patterns associated with storm systems to track tropical storms.

The model achieves faster predictions, the researchers explain, because it learns patterns directly from extensive observational and simulation datasets—bypassing the need for explicit mathematical equations typically required in traditional model—and employs AI techniques specifically designed to leverage parallel processing capabilities of graphics processing units.

The enhanced accuracy of their approach arises from several key factors. First, the model identifies and utilizes subtle patterns and correlations within data that conventional physics-based approaches might miss or not explicitly represent. Secondly, its neural network architecture is particularly well-suited for capturing complex physical processes occurring simultaneously at multiple scales.

Perdikaris says Aurora also employs transfer learning, which means that knowledge gained from one area, such as atmospheric dynamics used in weather forecasting, enhances its predictive performance in other domains, including air quality modeling or predicting tropical cyclone formation.

“This cross-domain learning is central to the foundation model philosophy that guides my broader research program,” says Perdikaris.

Beating the supercomputers

In testing Aurora’s predictive abilities, the team looked at a series of recent weather events as case studies and pitted their new AI against extant systems.

Perdikaris says that Aurora’s hurricane forecasting achievements are particularly remarkable. “When we compared Aurora to official forecasts from agencies like the National Hurricane Center, China Meteorological Administration, and others, Aurora outperformed all of them across different basins worldwide.”

To examine air quality, the team looked at a sandstorm that took place in Iraq in June 2022, one of a series that resulted in more than 5,000 hospitalizations. Their AI accurately predicted it one day in advance at a fraction of the cost it takes to run a forecast on the Copernicus Atmosphere Monitoring Services, the gold standard in Earth observation and atmospheric monitoring.

Perdikaris adds that what’s particularly impressive is the model’s ability to handle the challenges of air quality data—sparse observations, large dynamic ranges in pollutant concentrations, and complex chemical reactions through hundreds of equations—while accounting for human-generated emission pattern changes, like those seen during COVID-19.

Aurora “did not have any prior knowledge about atmospheric chemistry or how nitrogen dioxide, for instance, interacts with sunlight— that wasn't part of the original training,” says co-first author Megan Stanley of Microsoft Research. “And yet, in fine-tuning, Aurora was able to adapt to that because it had already learned enough about all of the other processes.” 

In testing Aurora’s predictive capabilities for the heights and directions of ocean waves, the team conducted a case study of Typhoon Nanmadol, which struck the southern coast of Japan in 2022 and was the most intense typhoon that year. Their model exceeded expectations by perceiving intricate wave patterns in greater detail, drawing from prevailing wind patterns, and accurately capturing the typhoon’s waves.  

The forecast for Aurora

“What makes these results particularly exciting is that they demonstrate how a single foundational approach can be applied across diverse domains,” says Perdikaris. “It’s something we’re now expanding to other scientific applications in my group.”

The researchers are interested in extending their model to generate predictions on Earth systems such as local and regional weather, seasonal weather, and extreme weather events like floods and wildfires. Perdikaris believes that this may represent a potential paradigm shift in how information on Earth systems is disseminated to key decision makers.

“The most transformative aspect is democratizing access to high-quality forecasts,” he says. “Traditional systems require supercomputers and specialized teams, putting them out of reach for many communities worldwide. Aurora can run on modest hardware while matching or exceeding traditional model performance.”

For cities and local governments, Perdikaris notes that this means having localized, high-resolution predictions for air quality, extreme rainfall, or heat waves without relying on downscaled global models. He says that the computational efficiency allows for more frequent updates and forecasts that better quantify uncertainty, which is critical for risk management.

“What excites me most about this technology is its broader applicability,” says Perdikaris. “At Penn, we’re exploring how similar foundation model approaches can address other prediction challenges beyond weather—from urban flooding to renewable energy forecasting to air quality management—making powerful predictive tools accessible to communities that need them most.”

Paris Perdikaris is an associate professor in the Department of Mechanical Engineering and Applied Mechanics in the School of Engineering and Applied Science at the University of Pennsylvania.

Megan Stanley is a senior researcher in the Machine Intelligence group at Microsoft Research.

Other authors include Johannes Brandstetter of Johannes Kepler University Linz and Microsoft Research, Chun-Chieh Wu of National Taiwan University; Ana Lucic and Max Welling of the University of Amsterdam and Microsoft Research; Anna Allen and Alexander T. Archibald of the University of Cambridge; Richard E. Turner of the University of Cambridge and Microsoft Research; Haiyu Dong, Kit Thambiratnam, and Jonathan A. Weyn of Microsoft Corporation; Wessel P. Bruinsma, Patrick Garvan, Elizabeth Heider, and Maik Riechert of Microsoft Research; and Cristian Bodnar and Jayesh K. Gupta of Microsoft Research and Silurian AI.

This research was supported by the Department of Energy’s Advanced Scientific Computing Research program (DE-SC0024563) and the Engineering & Physical Sciences Research Council Prosperity Partnership) between Microsoft Research and the University of Cambridge (EP/T005386/1).