The heads of several major artificial intelligence companies are calling on Congress to pass new laws that would make it harder for bad actors to create biological weapons with the help of AI.
Sam Altman of OpenAI, Dario Amodei of Anthropic, Demis Hassabis of Google DeepMind, and Mustafa Suleyman of Microsoft AI are among the signatories of a public letter. The letter asks lawmakers to require companies that sell synthetic DNA and RNA to screen both their customers and the orders they receive. The goal is to prevent the misuse of genetic material.
A Coordinated Call for Regulation
The letter was organized by the nonpartisan Institute for Progress and the right-leaning Foundation for American Innovation. It acknowledges that given the speed of AI development, "there is a real possibility that the knowledge barriers which have historically prevented bad actors from obtaining biological weapons will meaningfully erode."
The signers include other scientists, national security experts, and executives from gene synthesis companies like Twist Bioscience and Ansa Biotechnologies. These firms are members of the International Gene Synthesis Consortium, which was formed in 2009 to put voluntary screening practices in place.
The Growing Threat of AI-Designed Pathogens
Arthur Kornberg first successfully synthesized DNA in the 1950s. Today, the process is automated. Dozens of companies around the world use commercial synthesizers to print and sell custom genetic sequences for scientific research, drug development, and diagnostics. Many providers only sell to qualified researchers, biotech companies, and educational institutions. But not all of them vet their customers or the gene sequences they order.
In 2017, Canadian researchers raised alarm when they used $100,000 worth of mail-order DNA to reconstruct the extinct horsepox virus. Critics pointed out that the same method could be used to build smallpox, a closely related and deadly virus. Gene synthesis has only become cheaper since then.
Combined with advances in AI, it is now possible to design dangerous new toxins and pathogens using large language models. Some biology training would still likely be needed to make a functional virus from scratch. But bioterror attacks, though rare, have the potential to cause mass casualties, public panic, and economic loss. A major concern is that an AI-designed pathogen could accidentally or intentionally spark a global pandemic.
Stay updated
Get the day's AI and automation news in your inbox. No spam, unsubscribe anytime.
David Relman, a microbiologist and biosecurity expert at Stanford University who signed the letter, explained the danger. "AI tools enable a user to very quickly identify where to turn to order sequences that will not be subject to screening," he said. "If prompted appropriately, they can also tell you how to change the nature of your order, so that even those that are screening may be much less able to detect what it is you're trying to make."
Current Screening Practices and Their Limits
Many gene synthesis companies already use software to screen orders for "sequences of concern" that can contribute to an organism's toxicity or ability to cause disease. James Diggans, vice president of policy and biosecurity at Twist Bioscience, supports formal rules. "If you have technology that is capable of synthesizing DNA, then you should ensure that it's used responsibly," he said. "Part of that is making sure that you understand what you're making and who you're making it for." Twist has backed implementing formal rules for years.
Federal guidelines introduced during the Biden administration required scientists and companies that receive federal funding to order synthetic gene sequences from providers that screen purchases. A bipartisan bill introduced earlier this year in the Senate would require all gene synthesis providers operating in the US to screen orders and customers for bad actors or dangerous pathogens.
But screening tools are not perfect. Last year, Microsoft researchers published a study showing that AI protein design tools were able to generate potentially dangerous gene sequences that got past companies' screening software. The models suggested new protein sequences with structures similar to known dangerous ones.
Geoff Ralston, former president of Y Combinator and a partner at the Safe AI Fund, also signed the letter. He thinks AI labs with biology models should do their own screening of users. "It should be very difficult, if not impossible, to ask a model to help you do something imminently dangerous," he said.
Relman agreed that regulations around screening procedures are only part of the solution. "Given that the screening may fail in some cases, we must then have other points of control," he said. "That's where the AI companies are going to have to step up."