Advances in artificial intelligence-assisted protein engineering are enabling breakthroughs in protein design, but they also introduce biosecurity challenges related to potential production of harmful proteins. Though screening software to detect harmful proteins exists, a new multi-month analysis of such software reports that this software has vulnerabilities; some proteins of concern could evade detection. Critically, the study also offers a way to improve detection rates of proteins of concern going forward. AI-assisted protein design (AIPD) enables powerful advances in medicine and biology, enabling researchers to modify existing proteins or design wholly new ones with novel structures and functions. However, this powerful technology could also be misused to design harmful proteins. A necessary step to make a protein in the lab is ordering the DNA that encodes it. The companies that provide these synthetic nucleic acids screen customer orders with biosecurity screening software (BSS) to identify and block genes that encode proteins of concern. However, protein sequence generative models can produce functional variants in which amino‑acid sequences differ enough from controlled examples to evade detection. Despite this, no systematic assessment of BSS vulnerabilities has been undertaken, and international governance regarding the potential biosecurity risks of generative protein design is lacking. These concerns were previously highlighted in Science in a Policy Forum by Bloomfield et al. [https://www.science.org/doi/full/10.1126/science.adq1977], as well as in an Editorial by David Baker and George Church [https://www.science.org/doi/10.1126/science.ado1671].

 

In this study, Bruce Wittmann and colleagues employ an “AI red teaming” approach to evaluate BSS models with the goal of improving them to enhance biosecurity. Using open-source AI protein design software, Whittmann et al. generated more than 75,000 variants of hazardous proteins and submitted them to four different BSS developers and found that, while all tools performed nearly flawlessly when screening the original wild-type proteins, their ability to detect reformulated variants was inconsistent. According to the authors, the results suggest that, although current BSS systems remain effective for unaltered sequences, they lack consistent sensitivity when faced with protein sequence homologs engineered using modern generative AI methods, despite being similar. Following the initial findings, and in collaboration with BSS providers, Wittmann et al. developed software patches, which were deployed by three of the four BSS providers’ systems. These updates resulted in improved detection rates for AI-generated variants without significantly increasing false positives. Nonetheless, the authors note that no tool achieved complete coverage: across providers, about 3% of the variants most likely to retain functionality still escaped detection. ““AI advances are fueling breakthroughs in biology and medicine, yet with new power comes the responsibility for vigilance and thoughtful risk management,” said Eric Horvitz, senior author of the study and Microsoft’s chief scientific officer. “Beyond identifying and working to mitigate this specific vulnerability, our aim was to develop and demonstrate an effective process: building a cross-sector team, applying rigorous scientific methods, and creating a framework for sharing sensitive data and insights in ways that advance the science while managing potential risks.””

 

The authors of this work have made Science aware that parts of the data and code should not be made available in a public repository due to potential for misuse. The authors have thus designed a tiered access scheme for data release in which interested parties can request access to the restricted material by contacting designated representatives from the International Biosecurity and Biosafety Initiative for Science (IBBIS), a non-profit organization (https://ibbis.bio). The requestor will need to provide their identity, affiliation, and a brief explanation for their use of the data and will be subject to a data usage agreement. A committee at IBBIS will evaluate applications and determine if the requestor information provided is accurate and the proposed usage is legitimate. The restricted data is organized in tiers according to information hazard level, with the highest tier including code used in the study, and it will be released according to suitability for the proposed usage. The Science editors have agreed the proposed scheme adequately balances the security concerns with the data and code availability requirements in Science Editorial Policies. The authors have also made provisions for future declassification and/or succession of data management. Any concerns about the availability of data through this scheme can be raised by contacting science_data@aaas.org.

 

***A related embargoed news briefing was held on Tuesday, 30 September, as a Zoom webinar. A recording, with transcript, can be found at https://aaas.zoom.us/rec/share/75vhLzuQGdN4nE16fEhGGqNNzHIXAEBIbvm1OwOsW-9Wjkr57kOhkmFLicUF-h4E.PelPeKq8ffXCVeg0 - the passcode is ^q707eRd. See Other Information for Journalists section for full briefing information.***

Catastrophic wildfires – those causing massive damage and soaring suppression costs – are increasing in frequency and intensity worldwide, a trend expected to worsen with climate change. In a Policy Forum, Robert Gray and colleagues use British Columbia (BC), Canada, as a case study of a government at a crossroads: continue reactive spending on suppression and recovery or invest in strategies to reduce future wildfire risk. “Although we focus on BC, this same tough question, along with lessons learned and our main recommendations, apply to regional and national governments in dozens of countries,” Gray et al. write. Over the past decade, BC has lost more than 7 million hectares to wildfire, with suppression and recovery costs topping $4.8 billion. Indirect costs, including health impacts and economic disruption, are far higher. Here, the authors argue that governments like BC are faced with an important policy choice; either continue to focus on response and recovery, or embrace proactive, long-term fire mitigation and prevention. Gray et al. highlight that scaling up wildfire mitigation requires public and political support for sustained, multidecade investments, along with cost-management strategies, yet governments continue to prioritize reactive spending despite evidence that prevention is far more cost-effective. They propose four recommendations: set clear resilience goals based on economic and risk analyses; foster public understanding and support for short-term sacrifices like prescribed burns; implement collaborative, hands-on landscape-scale interventions with scientists, Indigenous leaders, industry, and communities; and leverage public concern and scientific evidence to secure long-term political commitment and funding for preventive wildfire management.