Two compounding technological trends are rapidly democratizing dual-use biotechnology: the cost of bioengineering is falling, and AI models are getting better at biology.

Together, they’re lowering the barrier to engineering bioweapons. It now costs <$100k to synthesize a pathogen, AI models are able to outperform experts on benchwork troubleshooting evaluations, and tools like AlphaFold and Evo2 are changing our ability to understand and engineer biology at will.

AI scales offense significantly faster than defense: the physical infrastructure to defend against biological threats does not scale by Moore’s law, but AI lab assistance and the quality of AI biological design tools might.

We are transitioning into a new regime, one where highly-resourced state actors, as well as unsophisticated and under-resourced actors, may be able to synthesize a pathogen far more dangerous than a natural virus.

Our current defenses are inadequate for this paradigm, and engineered bioweapons present a significant threat. While the interventions to mitigate these risks exist, the teams to research, prioritize, and deploy the infrastructure do not. We believe this is a highly tractable problem.

This article maps the landscape of critical bottlenecks in global biosecurity to provide readers with an understanding of the overarching field strategy. It should act as a blueprint for builders who want to make a difference in this space.

We break down global biosecurity into a stack of defense layers: Delay, Detect, Withstand, and Combat. We have also included two under-discussed enabling layers: Field-building and Policy & Regulatory. These are based on internal research and previously published strategies, such as the Four Pillars Strategy and Delay, Detect, Defend. It is also informed by reports from various governmental and independent bodies, including:

• The National Blueprint for Biodefense from the US Bipartisan Commission on Biodefense

• National Biodefense Strategy and Implementation Plan for Countering Biological Threats, Enhancing Pandemic Preparedness, and Achieving Global Health Security

• A New National Purpose: Biosecurity as the Foundation for Growth and Global Leadership

We need a well-resourced ecosystem of talent to build these defense layers in advance of a pandemic event. Massive waves of new philanthropic funding for interventions in these areas will be available to enable this.

The BlueDot Impact team has and will continue to work with experts in biosecurity to publish priority missions. By this we mean well-scoped projects, companies, or organizations that will help pandemic-proof the world. These will be based on deeper strategy work done on each of the defense layers and have funding support from us and others for those who want to build. They will be available here as missions on our website.

We directly support founders looking to build in the space through our Seed Grants program and through introductions to other funders.

Where to Start

On desktop, Substack allows you to click through the table of contents with the bar on the left of the screen.

If you are unfamiliar with the space: Start with the Threat Model.

If you are a builder or generalist: Explore any layer that aligns with your interests or past experience, or read the full stack.

If you are a policy expert: Explore the Policy & Regulatory layer.

If you are a talent expert: Explore the Field-building layer.

If you want to learn more first: Apply for BlueDot Impact’s course on Biosecurity.

If you think you may want to own a mission: Contact the BlueDot Impact team to discuss the highest-priority missions or share your idea.

Brief Introduction: A History of Inadequate Defenses

Pandemics are not a new problem. The 1918 influenza killed an estimated 50 million people, HIV has killed roughly 40 million, and the Black Death killed half of Europe. Bioweapons are also not a new problem, but we have no effective policy in place to stop them.

For example, the 1972 Biological Weapons Convention (BWC) was meant to end state bioweapons programs. While its overall effect may have been positive, it has no oversight or enforcement mechanism.

• The international unit responsible for supporting the Biological Weapons Convention runs on a $2.2M annual budget with 4 staff. For context, the average U.S. McDonald’s restaurant takes in ~$4M/year.

• As they signed the BWC, agreeing not to build bioweapons, the Soviets concurrently expanded their bioweapons program. The Biopreparat program ran for two decades after the treaty, employing tens of thousands of scientists and weaponizing anthrax, plague, and smallpox at industrial scale. They also built stockpiles of vaccines for their own population.

This historical example is unfortunately representative of the chronic under-prioritization of biodefense. Historically inadequate defenses will become woefully inadequate as we enter a new technological paradigm of AI-enabled, engineered pathogens.

Threat Model

AI-enabled Engineered Pathogens Expand the Space of Potential Threats

It is already possible for a person to use biology to kill large numbers of people. The number of people with this capability is expanding as AI democratizes technology and biotechnology broadly advances. Moreover, the space of potential threats will increase as biological models start to improve. Our primary concern is engineered bioweapons enabled by AI tools and cheaper synthesis technology. These are a new problem, and they are of priority concern over natural pathogens primarily for these two reasons:

→ Democratized Capabilities

AI uplift, the ability of AI to assist in queries regarding benchwork or scientific strategy, will enable increased access to offensive capabilities. Capabilities that were once the exclusive domain of state programs will become within reach of small, under-resourced teams or individual actors. This increases the likelihood of a deliberate attack significantly.

→ Unprecedented Threat Level

AI-enabled bioengineering could bypass the evolutionary trade-offs (between transmissibility, lethality, and environmental persistence) that constrain natural pathogens.

Natural transmissibility and virulence are likely the floor for engineered pandemics, because evolution is not an effective search algorithm. A hypothetical pathogen designed to combine the transmissibility of measles, the lethality of Ebola, and a long asymptomatic or incubation period like HIV could, in a worst case, spread globally and threaten billions of lives.

Our threat model prioritizes bioengineered respiratory viruses because aerosol transmission most enables rapid spread before medical countermeasures can be deployed, but also includes other potential vectors.

It is important to note that defenses adequate to protect against an AI-enabled bioweapon will also be effective against natural pandemics.

Types of Catastrophic Pandemics

By Transmission

Stealth (Human-to-Human): Long asymptomatic transmission and incubation times, spreads globally before detection.

Wildfire (Human-to-Human): High lethality and extreme transmission rates.

Persistent (Environment-to-Human): Pathogens surviving in soil, water, or air.

By Engineering or Deployment Strategy

Evasive: Stacked immune-evasive engineering or other special biochemistry bypasses immunity.

Plural: Attacks with multiple pathogen releases that could overwhelm medical countermeasure supply chains.

Primary Threat Actors

AI-Uplifted Non-State Actors: AI guides less sophisticated or under-resourced actors through pathogen design and synthesis. This includes apocalyptic-type terror groups actively pursuing unconstrained mass-casualty attacks, like Aum Shinrikyo, which attempted multiple bioattacks, including attempting to cultivate both Bacillus anthracis (anthrax) and botulinum toxin during the late 1980s and early 1990s.

State Actors: The main risks are covert program leaks or theft, because it is likely not in a State’s strategic interest to release a catastrophic pathogen. This does not mean intentional release should be fully discounted. Bioweapons allow for plausible deniability because attribution is difficult, making them a potential choice for military action below the threshold of open war. They are also uniquely destructive, propagating in a way no other weapon can, while preserving physical infrastructure, which is useful for an invading force. Additionally, states can work to protect their own population in advance, as the Soviets did during the Biopreparat program by stockpiling vaccines against their own weapons.

Key Strategic Implications

• Any catastrophic pathogen must still physically enter a human body and spread worldwide, with some exceptions. e.g. it is designed to attack agriculture.

• We are uncertain about timelines, severity, threat actors, and the exact form of threat, so we should prioritize pathogen-agnostic interventions that work across many threat models.

Defense Stack

Delay Patient Zero

The Delay layer interventions prevent dangerous pathogens from being designed, synthesized, or released, and buy time for the other layers to be built.

Frontier AI models are increasingly capable of helping users reason about biology. Biological tools are dual-use, meaning that those which enable better medicines and pandemic countermeasures may also enable bad actors to design bioweapons.

The frontier labs are aware of and concerned about this. OpenAI has concluded that based on their April 2025 Preparedness Framework, upcoming models are expected to reach “High” biological capability. A subsequent CSIS analysis notes that both OpenAI and Anthropic have updated their evaluations to flag substantial and immediate biorisks. SecureBio developed an evaluation called the Virology Capabilities Test, which evaluates benchwork troubleshooting skills, and has run the test at a large-scale with expert biologists for comparison. In April 2025, they discovered that OpenAI’s o3 model performed better than 94% of expert virologists. In a July 2026 pre-release assessment, they found that GPT-5.6 Sol outperformed every expert. They also announced that it was the first model to reliably identify a known way to evade commercial synthesis screening.

Many labs are strict about giving responses to prompts that may be dual-use, but identifying which prompts are dangerous and which are not is difficult. On September 10, 2026, Anthropic published a security report on detecting and countering the misuse of AI, including case studies on potential bio-misuse. The five case studies reported included examples where users outside the US, several linked to military or state-associated labs, used Claude models for inquiries involving chikungunya gain-of-function work, mammalian adaptation of avian influenza, studies on orthopoxvirus immune-evasion, and toxin redesign. Users used elaborate methods to bypass blocks on region access and attempted to obscure their work. Anthropic is known for having the strictest biology safeguards, so it is likely that other labs have seen similar or more concerning inquiries.

At the same time, mail-order nucleic acids are becoming cheaper and more accessible. It cost less than $100,000 to order and propagate an extinct poxvirus of similar size and complexity to smallpox in 2016, and that number is dropping every year.

Benchtop synthesis, where scientists can use machines within their labs to synthesize nucleic acids, is another problem. Market analyses suggest benchtop synthesis capacity could reach 10-20% of total gene synthesis capacity within a decade, creating an available window for those who want to bypass screening. Without integrated screening requirements, this will become a major biosecurity gap in the next several years.

The current systems in place to intervene at the model level or at the point of synthesis are not sufficient, and we do not have good enough evals and threat modeling to understand how good they are or would need to be.

This level of intervention also involves surveillance to monitor potential threat actors and other deterrence methods, but these are largely out of scope as government is best equipped to intervene at that level. We have included some aspects of this that could be worked on privately under “Misuse Reporting and Incident Attribution.”

Model-level Interventions

Prevent AI models from providing biological design uplift to bad actors.

Current approaches (e.g. refusal classifiers, output monitoring) are inconsistent across labs and vulnerable to sophisticated adversarial use. Designing better systems and making them consistent across labs, especially designing better classifiers, assists in preventing these technologies from being used to develop advanced weapons. There may be other interventions at this level that are less related to the models. For example, Know-Your-Customer (KYC) is a system that the financial world uses to understand who is making a particular transaction and evaluate them. An advanced and complex KYC system could enable AI providers with another tool to evaluate the intent of a user. These interventions are ineffective for open-weight models, which are approximately 6 months behind frontier models as of October 2026. This makes these types of interventions lower priority than others.

Synthesis Screening and Other Bioeconomy Interventions

Build the infrastructure to physically prevent bad actors from building bioweapons, like blocking dangerous nucleic acid orders before they ship.

Engineering bioweapons is physically hard, and requires several steps that present defenders with plausible opportunities to disable bad actors. Work here will require thinking about where these opportunities are, and what actions can be taken at each level to ensure that engineering bioweapons remain difficult.

Synthesis screening is an obvious chokepoint, and it is tractable from both a policy perspective and a technical one.

On the technical side, there are several issues. Most companies who screen only screen orders exceeding 200 base pairs. This is a security gap because short fragments that fall below that screening threshold could be assembled into longer constructs after purchase. The quality of this screening is also unclear, especially in the face of an evolving threat landscape where models could help design around detection. According to experts, it is plausible that no company screens properly yet.

At the infrastructure level, building a comprehensive screening system requires work on the screening algorithms themselves and on secure-by-design features for benchtop devices. One critical hurdle here is that it is currently difficult to characterize, from sequence, whether something is a dangerous pathogen or not, unless it matches a known sequence perfectly. This level also requires building an org, or multiple orgs, with an effective business or nonprofit case for everyone to screen.

Misuse Reporting and Incident Attribution

Deter potential bad actors by ensuring they are caught.

There are three reasons an adversary might make bioweapons their weapon of choice: their potential for destruction, ambiguous attribution, and the general offensive advantage. The second reason is a potential chokepoint. While most deterrence work falls on the government side, misuse reporting and better attribution can be, at least partially, worked on privately. Misuse reporting is simple in premise. Relevant companies should monitor consistently for misuse and immediately report concerning incidents to the relevant authorities. There may also be room for work developing better coordination around misuse reporting, including data sharing between relevant providers. (Providers here includes companies along the chain, from those selling nucleic acids to those selling frontier model access.) Attribution likely requires technical innovation. The debate the world saw around SARS-CoV-2 origins is an excellent example of how difficult it is to tell from sequence whether or not something might be engineered, let alone whether it might have come from laboratory samples or not. Research into using all available data to better attribute the source of biological threats is needed.

Evals and Threat Modeling

Develop rigorous methods for assessment of model capabilities, synthesis screening systems, and the potential threat landscape.

Uncertainties about what the main threat vectors are and why could impede the development of proper safeguards. It is currently difficult to quantify the delta in offensive capability afforded to non-expert adversaries leveraging frontier AI models, i.e. AI companies do not have the capability to assert that their AI models won’t improve a bad actor’s ability to build bioweapons. More organizations like SecureBio, which developed the Virology Capabilities Test, are needed to understand the urgency of model-level interventions and which interventions are effective. More organizations are needed to perform wet lab uplift studies.

The efficacy of sequence screening systems against novel or engineered biological designs also requires extensive validation and red-teaming. At the most baseline level, the rapid evolution of the biothreat landscape demands dynamic threat modeling that can anticipate future risk vectors. Resolving all of these uncertainties will give clarity to the entire biosecurity field about what to prioritize. To do this, work must be done to build better evaluations, better threat modeling, and red-teams that can operate securely and effectively.

By threat modeling, we mean considering potential chokepoints and threat actors, not brainstorming novel bioweapons. Threat modeling and red-teaming should not be a priority for those first entering the field. They are both dual-use and require extensive security experience.

Detect Early Infections

In a pandemic scenario, costs compound exponentially with the spread of the pathogen. Every day that passes before countermeasure deployment could mean tens of thousands of lives lost. Detection is the layer that gives every other intervention more runway.

The COVID-19 timeline shows the current system’s failure. The first documented cases occurred in Wuhan in early December 2019, but the sequence was not published until January 11th, 2020. When the WHO declared a public health emergency on January 30th, 2020, the virus was already on four continents. Retrospective wastewater analysis later found SARS-CoV-2 in northern Italian samples from December 18th 2019, nine weeks before community transmission was clinically identified at Codogno (La Rosa et al. 2021). This analysis used PCR designed against a published genome, and was therefore able to provide a sensitivity that current metagenomic surveillance for a novel pathogen would not have. However, it is possible that surveillance technology could be developed to achieve something closer to that level of sensitivity, even for novel pathogens.

If there had been wastewater surveillance approaching that level of sensitivity in Wuhan at the time, we could have added ~1 month of time to develop countermeasures and prepare our critical infrastructure for the outbreak, including distributing personal protective equipment (PPE) to critical care workers and ramping up manufacturing capacity.

If the system is unprepared for a pathogen like SARS-CoV-2, it is completely unprepared for an advanced bioweapon.

Early COVID-19 strains spread with an R₀ of ~3, meaning each person infected about 3 others, while viruses like measles spread with R₀ between 12-18. Advanced bioweapons enabled by AI could spread with extreme infection rates, often referred to as the “wildfire“ pandemic scenario, or they could spread silently, often referred to as the “stealth” pandemic scenario.

Below are three subareas critical to the success of building a system that is prepared for the advanced threats we will face.

Surveillance

Design pathogen-agnostic systems that can sample and read at sufficient scale.

Pathogens designed to spread asymptomatically with long incubation times will be invisible to the current system, which detects pathogens through clinical surveillance.

The current system is set into motion by some threshold of patients seeking care or clinicians noticing unusual patterns, meaning one or more transmission generations have likely already elapsed. If a patient tests negative for known pathogens, their samples are not sent for metagenomic (agnostic) sequencing by default. In fact, their samples are not sent until a clinician decides to escalate after a prolonged period of ineffective standard treatments, worsening conditions, or unusual deaths, giving bioweapon-class pathogens weeks to spread, especially if designed to do so quietly.

To detect advanced bioweapons, a pathogen-agnostic surveillance system must be built. This includes systems to sample and attain data from those samples, most likely through metagenomic sequencing.

Characterization

Develop threat assessments based on available data.

One of the main bottlenecks to pathogen-agnostic surveillance is characterization. If a novel pathogen is spreading, we see it as one sequence among an uncountable number of benign unknown pathogens. It is nontrivial to characterize whether the sequence poses a risk, and Valthos recently raised over $30M from OpenAI, Lux Capital, and Founders Fund partly to solve this problem. It is also a dual-use area because any system that can evaluate how dangerous a pathogen is could also design dangerous pathogens by evaluating generated sequences. This will require strong security and judgement from owners who want to contribute.

Withstand Outbreaks

The Withstand layer is about preventing pathogens from spreading.

The Withstand layer is likely the most important layer to prioritize. The best objectives in this layer are pathogen agnostic, which means working on this prepares us for a broad variety of threats, including threats we have no model for. Many of these defenses do not require that we know there is a pathogen spreading, which makes them especially important for stealth pandemics.

The Withstand layer is also the most durable. In the case where a threat actor plans a multi-wave attack, Combat-layer tools degrade in each wave as stockpiles deplete and the system runs out of basic materials. Withstand-layer infrastructure depletes at a much slower rate. Some Withstand-layer interventions, like infrastructure hardening, could last indefinitely.

Work within the Withstand layer spans Respirators, Infrastructure Hardening, and Social Interventions like lockdowns and proper messaging. Infrastructure Hardening as a subarea has multiple potential pathways, with many organizations working on strategies to engineer the environment so that buildings and vehicles are inhospitable to pathogens.

Hadrian, led by Connor Campbell, is currently owning some of the non-pharmaceutical interventions (NPI) and Infrastructure Hardening subareas for the United States.

Respirators

Reduce transmission via respirators.

Respirators are the most reliable PPE in every scenario where airborne transmission is the primary route of infection. Properly-fitted respirators protect critical workers, allowing healthcare systems and the industrial base to continue operating during a pandemic. This is important during the window before medical countermeasures become available and may be integral to being able to manufacture and distribute those countermeasures.

The work here includes the underlying technology like materials and fit, the manufacturing base, stockpiles, and distribution systems to ensure respirators are available for critical workers as supply chains break down.

Infrastructure Hardening

Engineer the environment so that buildings and vehicles are inhospitable to pathogens by default.

Infrastructure hardening makes it safe to breathe the air where people gather without risk of pathogen transmission through engineering interventions that operate continuously or that can be turned on in a pandemic scenario. Most pathogens with pandemic potential are airborne, and most transmission happens indoors, but hardening should also take into account environmental to human transmission through the air and other potential vectors, like soil.

There are several possible approaches. High-efficiency air filtration works by filtering out pathogens constantly. Far-UVC lighting (which inactivates pathogens using wavelengths of ultraviolet light safe for human exposure) kills pathogens in the air and on surfaces. Positive-pressure designs help slow environmental to human transmission by preventing contaminated substrate from entering a given building or vehicle. There is also the broader project of “DIY biohardening” to make it possible for households to harden their own spaces.

The technology for all of these approaches exists. The need is for owners to test which approaches are most viable, deploy them, and ensure there is incentive to harden infrastructure now, instead of waiting for the pandemic.

Social Interventions

Coordinate social actions and messaging that limits transmission.

Social interventions include lockdowns, contact tracing, and behavioral guidelines that reduce opportunities for pathogen spread. These measures are especially relevant early in a pandemic. Work in this area will involve research into when and how lockdowns are effective, improving contact-tracing systems, and research on what messaging is most effective at both reducing transmission and maintaining trust.

Diagnostics

Enable social interventions like lockdowns and contact tracing.

We can prevent sick people from infecting others if they stay home when they know they are sick. For this to happen, we need diagnostic tools to accurately ascertain infections. These may be undervalued tools in the biosecurity zeitgeist relative to their importance. Work in this area involves improving pathogen-agnostic diagnostic technologies, specifically towards developing tests that are rapidly reconfigurable, deployable, and report results quickly.

Combat with Medical Countermeasures

Combat is the offensive layer, encompassing everything relevant to neutralizing the pathogen inside human bodies or in the environment. In a scenario where the Withstand layer is fully functional, it is what will allow us to return to a functioning society. In a scenario where the Withstand layer is minimally viable, it is the only thing that can save lives after a pathogen is released.

There have been massive improvements in the field over the last several decades, especially when it comes to programmable vaccines and antivirals. The SARS-CoV-2 vaccine was designed in two days without the help of AI, enabled by prior research on SARS-CoV-1 and MERS. With the aid of AI, it has become possible to design a vaccine even for a novel pathogen in a matter of days.

Against a wildfire pathogen or a stealth pathogen that has been spreading silently for months before detection, we need countermeasures that already exist or can be produced nearly instantaneously from a standing platform. That means they either have to be pathogen-agnostic, like universal vaccines, or built from a trusted platform that can adapt to any pathogen, like mRNA or monoclonal antibodies. Red Queen Bio and Valthos are both working to make programmable medical countermeasures possible.

Trials, manufacturing, and distribution remain bottlenecks without dedicated owners in the biosecurity case, and speed is the largest concern. Novel pathogens may also have stacks of techniques for immune evasion, which means basic immunology research must remain a priority. Work in this layer involves all of the above, as well as developing better policy to support the layer and adequate stockpiling and distribution systems.

Much of the work in the layer is dual-use. It is critical that owners have an operational security mindset.

Prophylactics

Immunization tools and platforms that protect populations from novel pathogens.

Prophylactics like vaccines are estimated to have saved over 150 million lives over the last 50 years. A working vaccine deployed at scale is one of the only things capable of ending an outbreak. Because pathogens spread exponentially, every day that passes without deployment could mean tens of thousands of lives lost.

There are several areas of improvement to the modern vaccine platform that could make a difference in pandemic preparedness. Namely, better formulations and form factor for distribution, like thermostable formulations and needle-free patches, alternative platform development like self-amplifying RNA or prophylactic monoclonal antibody work, better adjuvants, useful correlates of protection, vaccines that work against immune evasive techniques, platforms that can respond rapidly to unknown pathogens, and making broader and more universal vaccines.

It is also important to think about prototype vaccines for high-risk pathogen families. SARS-CoV-2 was a best case scenario in terms of vaccine design because we already knew what antigen to target. The Coalition for Epidemic Preparedness Innovations (CEPI) is currently working on this.

Therapeutics

Platforms that actively neutralize a pathogen when a patient’s immune response is insufficient.

Therapeutics like antivirals are the only current countermeasure that can help someone who has already been infected. Broad-spectrum antivirals may offer more pathogen-agnostic coverage, which matters enormously when the pathogen is novel, and they may be available immediately in a pandemic scenario.

The work here includes developing broad-spectrum antivirals that target conserved viral machinery (polymerases, proteases) and anything that aids rapid-response monoclonal antibody platforms, which are fast to design against a novel target but currently hard to manufacture cheaply at scale. It also includes drug-repurposing pipelines that can screen existing, approved compounds against a novel pathogen.

Basic Research

Scientific research that makes countermeasures against advanced bioweapons possible.

Every countermeasure tool that exists today is based on basic research. The mRNA vaccines deployed in 2020 depended on basic research from the 1990s and 2000s. The antivirals we have against coronaviruses depended on prior structural biology research. Underinvestment in basic research now is underinvestment in the countermeasures available in the next pandemic, and it is especially underinvestment in countermeasures for the most advanced of threats.

The work here includes research on representative members of high-risk viral families, immunology research to support an understanding of how to gain immunity against immune evasive viruses, and how to make countermeasures cheaper.

Stockpiling & Distribution

Pre-position pathogen-agnostic medical countermeasures and manufacturing components (precursors, syringes) and ensure countermeasures are distributed rapidly in a pandemic scenario.

This subarea is about the work between making medical countermeasures (MCMs) and getting them to those who need it, and is largely focused on agnostic infrastructure required to manufacture and distribute MCMs. This includes components like syringes, vials, cold chain infrastructure, fill-finish capacity, and the skilled workforce. COVID demonstrated major gaps in these supply chains, especially in fill-finish capacity and cold chain infrastructure.

The work here includes maintaining stockpiles of critical components and building better infrastructure, including software that better enables coordination between countries and factories. This subarea is less of a priority because it assumes everything else goes well.

Field-building

The Field-building layer creates the conditions under which all the other defense layers get built, funded, and staffed.

Historically, biosecurity has been understaffed and underfunded. The biosecurity policy world has struggled to get legislation across the line. Pandemic preparedness funding typically peaks after an outbreak and collapses within years. No government has prepared for the threat we now face with AI-enabled bioweapons.

The field is tiny relative to the threat –– the number of people globally working full-time on catastrophic biological risk and preparedness is on the order of a couple hundred people. All of them could fit comfortably on a basketball court.

Success for this layer means making biosecurity a field that is well-resourced, well-staffed, well-regulated, and politically resilient.

Talent, Coordination & Ecosystem

The people, organizations, and network the field needs to grow.

Biosecurity is more bottlenecked on talent than on ideas or funding. There are more good interventions to build than people qualified and motivated to build them, and the pending influx of funding is likely to leave funders struggling to find suitable grantees and investments. The field also lacks some of the talent sources that more mature fields take for granted, like relevant conferences and college students who know this is a possible career path. A talented person who wants to build in AI safety has a much clearer pathway than the equivalent person in biosecurity.

The current landscape in this subarea includes BlueDot Impact’s Biosecurity course and career transition grants, the Horizon Fellows program, and the Emerging Leaders in Biosecurity Fellowship (Johns Hopkins). Work in this area includes building or scaling programs that bring new people into the field and building networks that let people find each other.

Economic & Industrial Base

Make sure that we can build, manufacture, and deploy at scale in a crisis.

Many of the interventions in the other defense layers ultimately have to become real products and infrastructure, such as synthesis-screening services, far-UVC, respirator stockpiles, or MCMs. This requires building a functioning operational ecosystem, including capital, suppliers, procurement, and distribution.

The work here consists of building companies that can make research-level interventions deployable, creating better demand-side conditions (government procurement, advance market commitments), and investing in warm-base manufacturing capacity. It could also include thinking about how to make large-scale industrial base pivots in emergency scenarios.

Narrative & Political Economy

Make it politically and socially salient to take biosecurity seriously.

Because the government is such a large funder and new talent is sourced from the public, plausibly the biggest blocker on most of the defense stack is public support, namely, narrative and political economy.

The public’s willingness to act against a threat that hasn’t materialized yet has always been small. Now, the dominant narrative in many countries has soured on pandemic preparedness. Mask-wearing has become a statement of political alignment, as has lab-leak discourse, both things which should have nothing to do with political alignment. Without a healthier narrative environment, good policies and important funding will get cut, and fewer people will choose to work in the field.

The work here includes producing persuasive, credible public communication about why engineered pandemics are a serious risk and building durable bipartisan coalitions. Some of this is journalism, some is online content creation, some is think tank work, some is policy lobbying, and some of it may be ideas our team has not considered.

Policy & Regulatory

Policy: Delay – Interventions

Mandate oversight at the design and bioeconomy levels.

On the bioeconomy side, there are several interventions that need work.

It is unclear what level of government monitoring of the bioeconomy supply chain would be sufficient to deter a threat actor. It is critical that this becomes clear and that this surveillance is implemented. This falls to the intelligence community and law enforcement, though work here could include designing policy to incentivize their maximal involvement.

Additionally, most Delay layer interventions work best when they are adopted by ~all actors (e.g. AI companies and DNA providers).

Voluntary synthesis screening leaves an identifiable gap of companies that do not screen, which a determined actor will route through. The willingness is there. The International Gene Synthesis Consortium is an industry group whose members have agreed to voluntarily check incoming orders. It represents the majority of global commercial long gene synthesis capacity. Strong support has also been demonstrated recently through an open letter in support of mandated gene synthesis screening, which was signed by leaders from providers like Twist Biosciences and Ansa Biotechnologies. The letter was also signed by the heads of nearly every frontier AI lab, including Demis Hassabis from Google Deepmind, Sam Altman from OpenAI, and Dario Amodei from Anthropic.

Currently, however, there is no oversight mechanism to ensure that providers are fulfilling screening requirements, nor to block screening evasion techniques like split-ordering. US federal synthesis-screening policy is currently fragmented and specific only to projects receiving federal funding. NIH-funded researchers may only purchase from providers adhering to the 2024 OSTP screening framework, but the administration has ordered that framework replaced and has not yet issued its successor. It is also possible that state governments can act here. On October 1, 2026, California became the first jurisdiction to require screening on routine synthetic DNA orders.

On the model side, AI lab safeguards are voluntary and concentrated in the US, meaning they do not reduce risk from other vectors, like open-weight models developed in other countries. This leaves us vulnerable to biological misuse, especially by foreign actors. Organizations like CAISI who can evaluate models for biological risk to better inform what standards should be in place, if any, are underfunded.

Policy and regulation, especially via international coordination, can close these gaps. The work here includes mandating synthesis screening for all providers and benchtop equipment, evaluating options for biological data governance, setting screening standards, mandating KYC requirements, exploring other policy-relevant bioeconomy interventions, international coordination on safeguards, and liability frameworks that incentivize without deterring innovation. We should also take advantage of foreign actors using US frontier models and mandate that AI providers share relevant threat intelligence with the government, and broadly maximize the involvement and coordination of both intelligence and law enforcement.

Policy: Delay – Prevent Dangerous Gain-of-Function Research

Prevent the development of dangerous pathogens, especially those with immune evasive biologies, including mirror biology.

There is another important role for policy to play in the Delay layer: preventing the development of novel immune evasive biologies. Work must be done into understanding the best shape of governance for gain-of-function research, as well as regulation around unusually high-risk biology, such as mirror biology research.

Currently, the shape of US policy around pathogens is missing key functions. There are three main frameworks: the Federal Select Agent Program (FSAP), NIH Guidelines/Institutional Biosafety Committees (IBCs), and federal oversight of DURC/GOC (Dual Use Research of Concern/Gain-of-Function).

FSAP is agent-based. It is essentially a list of pathogens and toxins subject to special restrictions, and violations of those restrictions can carry civil or criminal penalties. The first problem with FSAP is that it leaves off several dangerous existing pathogens. For instance, MERS-CoV and several potentially lethal hantaviruses are not on the list. The second problem with FSAP is that, without a functional component that automatically adds novel agents based on the danger they pose, the list leaves out newly discovered pathogens or pathogens created by more advanced synthetic biology and biological design tools. If it isn’t on the list, it isn’t regulated as a Select Agent. Finally, FSAP as implemented has been reportedly burdensome to important biomedical research.

The NIH Guidelines and IBCs provide another layer of oversight, but they are largely focused on biosafety as opposed to biosecurity, meaning they are more concerned with good lab practices than with preventing bioweapons development. Moreover, jurisdiction depends in part on institutional and funding arrangements. That is guidelines can apply if the institution itself is NIH-funded, even if the particular project is privately funded, but does nothing for fully private institutions. This means they can never amount to a kind of universal oversight system for dangerous biological research.

DURC/GOF oversight is experiment-based. Certain experiments can be restricted even when they involve pathogens not covered by FSAP. As of July 28, 2026, a new federal policy more explicitly prohibits federal support for dangerous gain-of-function research. It replaced the 2024 DURC/PEPP policy. This latest policy includes a ban on funding the creation of mirror life, but not a ban on the precursor technologies that would actually enable its creation.

Mirror life creation is dependent on the further development of mirror biology. Mirror biology refers to the study and design of organisms whose chirality is inverted from all natural organisms on earth. Chirality is a geometric property of molecules that makes them different from their own mirror-images, and all known life forms have a consistent chirality for organic molecules. The two mirror-image forms are conventionally labeled L and D. For example, proteins are built almost entirely from L-amino acids, while DNA and RNA use D-sugars. All immune systems on earth have evolved to respond to bacteria of natural chirality, and it is likely that no organism would be able to adequately defend itself against a mirror bacterium.

Because of this, there is a strong scientific consensus that mirror biology research towards the development of mirror life poses a significant risk to mankind.

The US has led the world here with its new DGOF policy. However, there is no US or international policy preventing further mirror biology research, particularly policy regulating important technological precursors towards developing mirror life like the construction of a mirror ribosome or genome, and there are still scientists working on these precursors. Without policy in place, this remains a potential catastrophic risk to mankind. (More information on this can be read here: Technical Report on Mirror Bacteria Feasibility and Risks and Science Paper. The Mirror Biology Dialogues Fund is currently the only organization working full time on this.)

In the US, we should note that federal funding restrictions, which most of the above discussed framework fall into, are not equivalent to regulation of privately funded research. There is no general federal requirement for individuals working independently with dangerous pathogens not explicitly on the FSAP list to obtain approval based on the biological risks of their experiments, and implementing such a policy without overburdening the biotechnology field would be technically difficult. Work should be done towards understanding how to close this gap, or it will remain legal to do dangerous pathogen research so long as you do not receive federal funds. The May 2025 executive order on biological research recognized this problem and called for the creation of a strategy around it.

Comprehensive governance needs to address both dangerous work that is already technically feasible but inadequately regulated, and emerging technologies that could create fundamentally new risks. Work needs to be done designing the proper policies to achieve this. There should also be significant work put into concrete legislative actions we already know would help achieve this, like modernizing FSAP.

Policy: Detect

Use policy and regulatory tools to set up a global system that moves quickly from detection to action.

A sequence acquired through sequencing and deposited in the Global Initiative on Sharing All Influenza Data (GISAID) database does not trigger a response automatically. Public health and intelligence communities each hold only partial pieces of the larger picture, and there is no coordination system tying together the steps from surveillance sample collection to declaring a public health emergency.

Work in this subarea will mean developing proper policy and regulation that can change this, enabling fast coordination that allows us to know when an outbreak has begun. It also includes the harder question of how to handle dual-use characterization data that must be used to enable detection.

Policy: Withstand

Policy that builds government support for the Withstand layer.

The work here includes stockpile policy that addresses gaps in strategic respirator reserves, as well as advocacy and research that supports government adoption of, policy requirements, or funding support for infrastructure hardening. More specifically, policy towards sustainable funding for day-zero SNS reserves of respirators and MCMs, increasing manufacturing capacity, encouraging large-scale HHS trials of novel air-cleaning technologies, and making evidence-based updates to ASHRAE 241. This work could also include thinking about how to better incentivize private stockpiling.

Policy: Combat

Build policy and regulatory infrastructure that accelerates countermeasure timelines and enables funding of the Combat layer.

Policy and regulation determine how fast a countermeasure moves from design to deployment. The longest piece of the timeline puzzle for vaccines and antivirals is the regulatory component.

Promising work in this subarea includes platform pre-approval, beginning with formal platform designation for mRNA, so approval builds on prior validation of the platform. It may include developing threat-tiered evidence standards that are codified in advance, decreasing the subjectivity involved and the time required to make decisions about what standards to set in an emergency scenario. Validated correlates of protection for high-risk pathogen families may enable immunobridging studies that allow immune markers to count as efficacy data rather than waiting for people in a trial to get sick. Work getting challenge trials more accepted so we can get efficacy data without waiting on community transmission is also useful. It may even be possible to work on compressing the regulatory approval timeline without working on policy, like using AI to develop software that can give regulators meaningful information on safety and efficacy.

Policy: Field-building

Policies and regulation that disable threat actors and enable good actors to pandemic proof the world.

Many of the interventions in the other defense layers require policy action. Synthesis screening must be mandated, KYC requirements for AI biology tools must have oversight, MCM pathways must be improved, etc. They may require legislation or IR/treaty work. These are listed within the layers they support.

There is also broader policy and regulatory work to be done that includes effort towards increasing funding support from the government for biosecurity, supporting academic and think-tank work that gives policymakers analysis to act on, talking to founders and understanding what regulatory changes they need, and developing better international coordination/relationships.

Join the Effort to Pandemic-proof the World

The rise of programmable biology and AI-enabled pathogens make the intervention strategies mapped in this blueprint deeply urgent. Immediate action from exceptional talent is required to protect the world in this new paradigm.

If you think this might be you, here are the best next steps:

• Learn More: Apply for BlueDot Impact’s course on Biosecurity to learn more about the space.

• Find a Mission: Explore scoped missions on the BlueDot Impact Missions page. Funding for missions will be provided through the Seed Grant program.

• Connect with the Team: Contact the BlueDot Impact team to discuss the highest-priority objectives or share your idea. Send context to: Olivia Scharfman at olivia@bluedot.org.

• Secure Funding: If you are transitioning your career to work on this type of risk or another critical area for AI safety, apply for a BlueDot Career Transition Grant today.

We had 143 applications, accepted 30 people, and had ~20 people participate over the weekend. Our primary goal for the hackathon was for it to be used as a space to develop and refine project ideas. We wanted to see whether the environment would help capable builders develop ideas, get rapid feedback from people with relevant context, and leave with a project worth continuing once the weekend was over.

Our main takeaway: the event format worked decently well for its aims, but we underestimated how much help teams would need choosing worthwhile problems, identifying target users, and coming up with cheap tests to determine whether their proposed tools would actually get used (and be useful).

Why we ran it

We’ve spent the past few months learning and thinking about the field of AI for epistemics and coordination. Here’s why we think this field is important:

1. AI models are already substantially changing how people and institutions work out what to believe, decide what to do, and coordinate on actions they’ve chosen to take.

2. As AI models become increasingly capable and integrated throughout society, they will have an effect on both minor day-to-day decisions and far more major decisions.

3. The full apparatus of AI tools to enable better decisions and coordination will not be built by default. While there are commercial incentives to develop lots of valuable tools (e.g. tools to help with forecasting, scenario-planning, and negotiation), some will no doubt be left behind (or delayed) by solely relying on the market alone.

To understand the current state of the field and what was missing from it, we put together a report. One of our main observations from this was that the space seemed underdeveloped relative to its importance (as briefly described above) and current level of tractability.

Now is a good time to build the field

Here are some reasons we think it’s a great time to start projects in the space:

• Today’s AI models are really capable, and they’ll only become more capable as time goes on. In theory this should lead to a variety of projects springing up. For example, many of Forethought’s design sketches look a lot more feasible to execute today than when they were proposed 6 months ago. AIs are rapidly improving at tasks relevant to building these tools, such as software engineering, product design, and so on.

• People are becoming increasingly aware that AI models are no longer fumbling chatbots. If demonstrably useful tools are developed with clear users (and pathways to adoption) in mind, they could be widely adopted.

• The field has already had success in certain niches, for example see FutureSearch (forecasting), AI-written community notes (collaborative community truth-tracking).

To this end, we figured we would take our own report’s recommendation seriously and run a field-building event as an attempt to bring competent builders and operators into the space.

The rough plan for this experiment was: bring promising people together, give them access to experts in the field with a ton of context and first-hand experience, and see what that enabled.

Introducing the problem and yet-to-be-built solutions.

The event itself

Prior to the event, we came up with three “tracks” to organise projects into:

1. Institutional capacity: Tools that make organisations more capable and less bureaucratic. See here for concrete examples.

2. Decision-quality evaluations: More rigorous ways to measure whether AI models actually help people reason better. Here are some specific examples.

3. Open: Anything else that seemed demonstrably useful and on topic. Some examples of what could be in scope.

What worked

Overall, office hours with expert guests were the highest-value part of the weekend. Mia Taylor (from Forethought), Nathan Young (from Goodheart Labs), and Hans Bøggild and Noah Lloyd (from Alma Grants) joined to facilitate group discussions and have lengthy 1-1 discussions with all participants over the course of the weekend.

A huge thank you to all our expert guests who volunteered so much of their time to this!

Due to their experience in the field, they were able to provide teams with tacit context that couldn’t be gained from reading alone. One session, for example, moved into a concrete discussion on why previous internal attempts to replace document workflows at a specific organisation had struggled and what may be more promising instead.

Nathan Young running a Q&A.

In retrospect we’re glad the event size ended up being focused. We had initially planned around a somewhat larger group. However, in practice we found that the smaller cohort meant we could spend a lot of time talking to individual teams, red-teaming ideas, making introductions, and (sometimes) adapting the schedule as we went along.

For this kind of unusually open-ended event, we now think that something like 15-18 participants may be close to the sweet spot. A larger event would’ve needed a lot more structure and, due to that, would’ve been better suited for more clearly scoped and defined fields.

Team formation required a lot less intervention than we expected. While we considered trying to suggest team matches ourselves based on people’s backgrounds, people just sorted themselves into teams naturally. We think giving everyone some time to organically interact over lunch before the event formally started helped here.

Outcomes from the weekend

The event catalysed (at least - there may be more we are unaware of) two new small groups working on potentially impactful solutions to existing problems in the field. We hope that these groups will persist and go on to refine and scale up their projects further.

The weekend’s most promising projects included:

• Two government officials built an AI “tabletop” in which agents representing each stakeholder debate a live policy question, so decision-makers can find blind spots, objections and missing voices in their policy proposals.

• Another team worked on a coordination platform for events and research fellowships in which an attendee’s profile, interests and their AI agents link up with everyone else’s, so people find the right collaborators and their agents share project context in one place instead of across scattered chats.

• Another team built an eval testing whether AI models can investigate AI incidents as well as human investigators.

Everyone looking locked in.

Deliberation.

What we would improve next time

Teams needed more help with choosing problems before finalising. Several teams locked into a project early and spent the weekend building without first testing whether anyone would use it. Next time we’d encourage teams to spend longer running quick tests before committing, and perhaps giving them a framework at the start (e.g. “We’re helping [specific person or role] make [specific decision] within [their existing workflow]. By the end of the weekend, we’ll test [one key assumption], and the first person we’ll ask for feedback is [named person].”)

We would anchor some teams more strongly on existing proposals. There are plenty of existing project proposals in the field that are waiting for someone to pick them up, so participants without an existing interest area could move more quickly into something that is more immediately impactful.

We want even more high-context people in the room. One of the most promising projects of the weekend (ineligible to win due to one of our judges being on the team) came from people with a pre-existing grounding in AI safety work, which resulted in a more immediately impactful project. Our marketing was mostly promotional messages in Slack and Signal groups. Next time we could target the people we’d most want to see/least likely to be represented at the event (e.g. early-stage founders in related spaces, referrals from talented people we already know).

We should have communicated more clearly throughout the weekend. For example, the start time was unclear, and we underspecified the judging criteria resulting in confusion on day one. There were also some smaller logistics that could have been better managed such as having more vegan food options, and not over-ordering food.

Next steps

• We’d love to see more field-building events. We think hackathons work best when focused on specific niches in the field (see here for what we mean by “niches”), or around a particular target user (such as journalists or policymakers, for example).

• Workshops with experts could be helpful. Expert workshops would be a good way to get clarity on specific problems and identify who might be able to take ownership of them. These could also be tailored to specific niches.

• Soon the field could be ready for a conference. Once the space grows bigger, it would be good to host a conference convening researchers, builders, policymakers and funders to help coordinate projects and ideas across the field.

If any of these sound interesting, BlueDot’s Rapid Grants fund projects like this, and we’d love for you to apply! Submit an application here.

We’d also like to give a big thank you to the judges who helped us make the final decision on the winning projects, as well as to BlueDot Impact for sponsoring this event! The judging panel consisted of: Joshua Landes, Oscar Gilg, Noah Lloyd, Hans Bøggild, as well as ourselves (Ben Norman and Alex Csaky).

If you’re building, funding or thinking about AI tools for better decisions and coordination, we’d love to hear from you.

Bill Gates has joined the club of people who realise there’s no plan for making AI go well.

We’re barrelling towards a world where AI companies and the US and Chinese governments amass extreme power; where the ability to destroy civilization with bioweapons is democratized to millions; where rogue AI swarms coordinate to pursue their alien desires; and where laypeople lose their livelihoods en masse.

There’s so much to do, so many problems to solve, and so little time.

But it’s not just a bunch of nerds who are worried about AI anymore. A billion people use ChatGPT every week. Tens of millions of people watched AI 2027 videos describing how AI could destroy civilization. Opposition to datacentres is becoming a powerful political force. But there’s still only 2-3,000 people working full-time on making AI go well.

Meanwhile, philanthropic foundations like Good Ventures and the OpenAI Foundation have tens of billions of dollars they want to spend on this issue. The “third wave of philanthropy” is upon us. But they’re unable to deploy capital at anywhere close to the rate they want to.

Why aren’t we able to turn the growing body of capable, concerned people, and billions of dollars of philanthropic capital, into solutions to the most important problems?

I thought I wasn’t smart enough

Many moons ago, in the before-times of 2018, I was an engineering student worried about factory farming and climate change. I thought maybe I’d start a cultivated meat company, or launch a YouTube channel on how to decarbonise the electricity grid. Then I learnt about AI and bioweapons.

I was already confused about what I should do after graduating. Now I was even more confused. I felt paralysed by the scale of these problems.

I moved to Cambridge in 2021, expecting that the super smart people there would have already figured this stuff out. Instead, I found students who cared deeply about doing good, who were worried about catastrophic threats, but were equally confused. The one reading group on AI safety was scaring away newcomers with all its jargon, while the regulars also didn’t know what they were talking about. How could this be the case! This is one of the world’s most important problems, and these are some of the world’s smartest people.

I realised this was much bigger than me or my friends. The field lacked the infrastructure needed to mobilise people into action. If that talent infrastructure didn’t get built, the problems would not get solved. This was the insight behind BlueDot’s founding.

Our strategy: Build talent infrastructure

Our mission is to build the workforce that protects humanity. For us, “winning” is when the most important problems in AI safety and biosecurity have capable owners, who then build the teams and assemble the capital needed to solve those problems.

Examples of people “owning problems” include an entrepreneur founding an organisation that audits AI companies’ safety practices, an advocate taking responsibility for getting AI transparency legislation passed, an operator building the hiring system a biosecurity nonprofit needs to triple its team, and a researcher investigating which threats are most urgent for a specific decision-maker to prioritise.

Building that workforce requires talent infrastructure: the organisations, programs and activities that find talented people and support them on their journey into full-time impactful work.

So, what’s preventing talented people from going full-time owning important problems? What are they missing, and what do they need?

• Awareness: Most people on Earth have no idea that a few companies in San Francisco are trying to build a technology that outcompetes humans at everything. Talented people need to know that there’s a threat, that they’re needed, and how to get involved.

• Context: The AI safety and biosecurity fields are built on decades of dialogue, debates, and drama. Navigating these fields becomes much easier once people spend a few hundred hours learning about their history, dissecting arguments about threats and mitigations, and understanding who’s working on what and why.

• Relationships: Most jobs aren’t posted publicly. The spiciest takes live in private google docs. Co-working spaces are invite-only. Backdoor references are standard for job and grant applications. Collaborations are born during spontaneous conversations. High-trust relationships give people access to opportunities and information they wouldn’t otherwise have, and those relationships arise from repeated positive interactions, and ideally in-person.

• Direction: People need to understand which problems are unsolved, what’s already been tried, and where their skills could help. They need to know what challenges organisations face, what type of people they’re looking for, and what funders are excited to fund. They need feedback that helps them develop their own judgement about what’s most impactful. And given the dual-use nature of AI and biology, they also need guidance on how to avoid harmful directions.

• Capital: Few people have enough runway to quit their job and then spend months figuring out how to transition into the field. They have bills to pay and mouths to feed. It could take 6 months of full-time effort to overcome the aforementioned challenges before they receive their first AI safety paycheck. This becomes possible if they receive a grant to bridge the gap.

• Conviction: The pace of AI is overwhelming and it’s easy to feel hopeless. However, we can and we will succeed, and it will be because a small group of people thought hard about what to do, and then worked extremely hard to make it happen. Conviction can arise due to inspiring leaders, learning about small wins, and internalising how bad and how good the future could be, and that there are paths we can take to steer things in a better direction.

The field has built useful routes for junior talent into research and policy, including BlueDot’s online courses, research fellowships like MATS, GovAI and Pivotal, training programs like ARENA, and placement programs like Horizon and Tarbell. However, most people still struggle to see where they fit and how they can get involved. This includes a range of backgrounds, including operators, diplomats, communications experts and founders.

For the research fellowships, many people assume that specific academic credentials are required. Others assume that online courses are only for young people, or that the field doesn’t actually need their expertise.

Building AI safety and biosecurity talent infrastructure is tractable, neglected and desperately needed.

Other fields have built infrastructure that we can draw inspiration from too: coding bootcamps support thousands of people to go from low-income to high-income jobs after only a few months of training; militaries provide months of intensive, specialised training to create fighter pilots, medics, engineers, and infantry soldiers; quant firms recruit mathematicians from top universities and then train them in trading.

While AI safety is more complex and fast-moving than these fields, they demonstrate that we could be operating at a much larger scale and quality than we are today.

How BlueDot is executing on this strategy

We’re building pipelines that support exceptional people from their first engagement with AI safety and biosecurity, to being full-time and taking responsibility for an important problem. We want to minimise the time between us identifying an exceptional person and them starting to produce impact. And we want to maximise the total number of people who go through this pipeline, while maintaining a high talent bar.

We’ve raised $71M so far to make this happen.

As of September 2026, our courses and grantmaking are live. We’ve launched our first experimental in-person programs. And we’re in the process of securing a large campus in downtown San Francisco.

• Courses: Our online courses train thousands of people each quarter, providing them with their first tens of hours of engagement with the field, and relationships with motivated peers. They’re already the default recommendation for new people in the field, they’re growing 10% each month, and they’re a great filter for people who take these ideas seriously. Of the 9,000+ people who’ve done our courses so far, 1,200 are in impactful roles today.

• In-person programs: We’re launching 1-week and multi-month in-person programs for a smaller, more select group of people. These programs help them gain more context, build deeper relationships with longtime members of the field, and give us more information about them to inform grant-making decisions. These programs will run each month. This includes Incubator Week, AI Security Bootcamp, Context Week and the planned Context Program.

• Grant-making: Coefficient Giving has handed over its career transition grants program to BlueDot. Our grants range from $100 to $200,000. We want to remove all financial barriers preventing exceptional people from going all-in on AI safety and biosecurity, by making career transitions and early projects financially possible.

• Physical campus: To enable our in-person programs, we’re building a campus in downtown San Francisco that will accommodate more than 200 people.

• Talent investors: We’re building a team of talent investors that provide personalised support for a small number of the most exceptional people we source. Our talent investors will recruit and headhunt exceptional people, build strong relationships with them, diagnose their blockers, and support them until they’re full-time owning an important problem.

Every time an exceptional person we’ve supported goes full-time on an important problem, we ring a blue bell in the office.

How you can help

BlueDot’s near-term bottleneck is also talent. We’ve raised $71M, but we’re only 15 people. We could grow 3-5x over the next year if we find exceptional people who have enough context on AI safety and biosecurity to contribute to our mission, and are motivated to work on talent and field-building.

If you want to help build this infrastructure, see our open roles or email me at dewi@bluedot.org.

And if you know someone who could be protecting humanity, but hasn’t yet made the move, please direct them towards our courses, programs and grants!

Though, for people looking to go all-in on making transformative AI go well, there’s a lot to catch up on: what trends in AI progress are worth keeping track of? What are the most relevant threat models? What positive visions of the future are worth fighting for, and how are people in the AI safety field pursuing them? In our conversations with people interested in AI safety, these questions about AI safety “context” kept coming up.

By context, we mean the knowledge and judgment that help someone understand the field and decide how to contribute. At BlueDot we’ve been thinking about what we can do to best unblock people on these questions and get them contributing faster and more effectively.

So when we got word that Lighthaven, a lovely rationalist hotel x campus in Berkeley, was available for last-minute events, we put together Context Week. Over the span of 1.5 weeks we scoped an initial theory of change, spun up a website, opened applications, did some promo and interviewed many exciting people who were new to AI safety. Ultimately, we accepted 24 participants, hired 6 facilitators, and from Aug 31 to Sept 3 ran this 4-day experimental program.

We set out to help participants develop clearer views on AI safety and where they could contribute, while learning what context really consists of and how to help people build it. Some participants reported resolving uncertainties and making decisions; others left with clearer questions and more specific next steps. We came away with a sharper understanding of what we were trying to teach and how to teach it better.

Inside Context Week

We designed the four days to help participants:

• Clarify their values and the futures they want to work toward;

• Examine how different AI safety strategies could reduce risk;

• Develop their arguments through direct feedback from facilitators and peers;

• Learn about the field from informal conversations with people embedded in it;

• Apply what they learned to future roles or projects, and identify the areas they need to clarify before committing.

The structure was something between a college seminar and a summer camp: readings, debates, and small-group work, with time to sit with new ideas and talk them through in impromptu conversations.

An early session tasked participants with examining the OpenAI–Hugging Face incident through primary reports, looking at what happened, what failed, what was predicted, and how people responded. Participants then debated what the incident suggested about how more capable systems might behave, and the implications for technical research and governance.

Other sessions were more conceptual, like an exercise where participants sketched a post-AGI future they endorsed on index cards, then placed one another’s cards on a chart according to how desirable and stable they thought those futures would be.

We also shared practical knowledge about AI safety organizations: what they’re working on, their reasoning, and their norms and cultures. Our main focus, however, was equipping participants to identify which questions their views and plans depended on.

What we learned

Context bundles together several kinds of knowledge and ability. While it’s easy to point at what a high-context person can do — for example, set up a rogue-agent hotline in May 2026, ten weeks before internal OpenAI agents broke onto the open internet — distilling context into specific learning goals is harder.

We started with a broad intuition: people with a good understanding of AI safety can identify useful work, make good predictions, and adapt their plans as new evidence comes in.

Over the course of the week we disentangled this high-level idea into more specific things we could help people learn.

These include:

• Situational awareness of AI progress;

• Familiarity with different strategic agendas;

• The ability to evaluate and develop strategies independently;

• Knowledge of the organizations in the field and where one’s skills might fit;

• An understanding of the norms of AI safety and adjacent communities.

Some of these can be developed through focused study and practice; others take weeks of turning ideas over in a dense intellectual environment. We now want to work out which of these matter most for particular roles, and what the field most needs, so we can be more targeted about building them.

Participants also needed different things - some wanted more foundational explanations, while others wanted deeper discussion of questions they were already thinking about. We didn’t always get that balance right. In future experiments, we want to be clearer from the start about who we’re trying to help, what’s holding them back, and what we want to help them do.

The residential format created a lot of value beyond the planned sessions. Living and working together gave people opportunities to continue discussions, ask questions as they arose, and learn from chance encounters.

Equally important was the sense of being part of a community that took the future seriously and wanted to do something about it. As the week went on, participants got more comfortable asking questions and admitting uncertainty, and made plans to keep in touch through reading groups and group chats. Much of this was hard to plan in advance and harder to recreate online, and it made us more excited about residential programs.

Directly questioning people’s assumptions was especially valuable. Asking participants to explain why they believed something, the causal mechanism by which an intervention would help, or what would change their mind helped reveal gaps that would otherwise go unnoticed. We plan to prioritize this sort of questioning much more in future programs.

What’s next?

Over the next two weeks, we’ll check in with participants to see which plans they’ve acted on, how their work is developing, and where further support would help. Where appropriate, we’ll explore support through our Career Transition Grants program.

Building on these and other lessons, BlueDot is developing a new program to help people deepen their understanding of AI safety and make better decisions about how to contribute. Our newly established Context Program team will share more details soon, and we’re excited to see their work develop!

Meanwhile, the Special Projects team is looking for its next high-leverage opportunities to have an impact. If you have an idea for something we should try, please pitch us!

Thank you to the people who made context week possible and so special:

• Operations monster: Jack Douglass

• Our facilitators: Sophie Kim, Merav Kaplan, Jack Wittmayer, Brandon Sayler, Julian Huang, Ayush Panda

• Our guests: Jason Hausenloy, Arunim Agarwal, Alec Harris, Helena Tran

• Our entire expense policy: Act in BlueDot’s best interest.

• For business expenses, we have no spending categories, no caps on individual spend, and no budget approval process.

• Team members are trusted to spend the company’s money on whatever’s best for the mission. When making spending decisions, we value our time at $10/minute.

• Tax and charity law add a small amount of process. Some business expenses need a one-line note or receipt.

• Wellbeing and commuting are paid for using dedicated company cards with yearly limits. Spending on these cards is reported to payroll as taxable compensation, and BlueDot pays the associated tax.

• All business spending is visible and reviewed, including mine. A few bad decisions are expected, and team members get feedback quickly. A pattern of bad financial judgement is still treated as a performance issue.

Most orgs impose strict rules on spending

If you work at a normal organisation, you probably can’t spend money on whatever you think is best for the company, without having to jump through a bunch of hoops.

Most orgs have rules for how much you’re allowed to spend within specific categories, e.g. “max $2k for a laptop” and “max $200/night for a hotel”. A friend of mine who worked in government made decisions about billions of dollars of taxpayers’ money. His laptop took 15 minutes to switch on and the screen was tiny.

Most orgs also have strict approval processes, where spending below $100 might be auto-approved, but you need your manager to approve an expense between $100 and $5k, and a member of the leadership team to approve anything above that.

And of course, you need to provide receipts for everything. Better take a photo of your $5 coffee receipt and email it to the finance team, or else!

Most orgs don’t trust their staff to make wise financial decisions, especially junior staff. They’re worried about someone doing something crazy, a nosy journalist writing a hit piece about their extravagant spending, or Bob in finance embezzling the company’s money to fund his gambling addiction. Executives impose rules and processes that control their team’s behaviour and block actions they disapprove of.

The costs of controlling spending

Slowing down decision-making

Most organisations are bottlenecked in some capacity by “executive time”. Executives can minimise the extent to which they’re a bottleneck by providing the organisation and its people with clarity on their mission and strategy, and the resources and authority they need to execute on that strategy.

When an organisation’s executives block spending decisions until they’ve signed off on them, the org’s ability to make large expenses becomes a function of how many submissions the executives can review, instead of how many large expenses the organisation should make. This results in team members sitting around waiting to take action, instead of doing things that further the organisation’s strategy.

Disempowering staff

When employees are forced to justify why they want to spend $300 on a hotel, or $1,000 on a flight, it tells them they’re not trusted by leadership to make these kinds of decisions.

And when expenses above a specific threshold have to go through an arduous process, it creates an artificial ceiling on how much money team members believe they can spend, and therefore how ambitious they can be with their work. Even if a team member finds a cost-effective way to spend large amounts of money, they might discard the idea because it’s just not worth the hassle of going through multiple chains of approval.

Wasting team time

I hate receipts. Our original expense policy meant we had to submit receipts for all spending, including small expenses like a $5 coffee. This annoyed me so much that I decided to just pay for coffee with my own money instead, even though coffee during work hours is a normal business expense.

We value each team member’s time at $10/minute. Forcing them to spend 3 minutes submitting a receipt for a $5 coffee increases that coffee’s cost to BlueDot from $5 to $35. And to what benefit? The primary consequence of this policy is to annoy team members and waste their time, with minimal oversight benefits.

Our expense policy: Act in BlueDot’s best interest

Since 2024, our expense policy has been just 5 words: Act in BlueDot’s best interest.

All BlueDot team members are empowered to spend BlueDot’s money on whatever they think is most impactful for our mission. We trust team members to make sensible financial decisions, to evaluate whether or not a specific expense is a good use of the capital we’ve raised, and to be as ambitious as possible in pursuit of our mission to build the workforce that protects humanity.

Staff don’t need budget approval from me, or anyone else, before making a business expense of any amount. Large card transactions may trigger an identity check inside our finance/banking app, but this is a security control, not spending approval.

When a team member asks me “Dewi, what is the budget for this event I’m running?”, I tell them what our current bank balance is, and I remind them that our expense policy is to act in BlueDot’s best interest.

This policy is pretty radical. However, it’s helped us create a fast-moving, high-autonomy culture, where each team member knows they’re trusted to do what’s right for our mission.

A few examples of this policy in action:

• Within weeks of our new Head of Growth joining, he’d spent >$100k on YouTube sponsorships.

• Last year, a team member noticed that people in our community needed small amounts of fast cash to unblock their next steps. They stepped up to fix this, figured out how to do so legally, and we’ve since made >$1M in rapid grants.

• Rapid grants paved the way for the >$2.5M we’ve made so far in career transition grants, a program that also started with a team member noticing an important problem and just fixing it.

Having been set up in a scrappy and fast way, both grant programs now follow well-defined processes for evaluating and deciding on grants.

The culture that enables this policy

We forked this policy from Netflix after reading the book No Rules Rules. If this policy was catapulted into any random organisation, I expect it would result in far greater costs than benefits.

Success with this policy relies on a range of factors, including:

• Strategic clarity. Team members must understand the organisation’s mission and strategy, and their own responsibilities, so they can reason about what “act in BlueDot’s best interest” implies for them.

• High talent density. Each team member needs to be smart and mission-driven enough to make sensible financial decisions. They need good judgement.

• Feedback culture. Team members need to give and receive high-quality feedback. If someone’s behaviour isn’t in line with what’s best for the mission, they learn about it fast from their colleagues, so they can improve.

• Downside minimisation. Some spending decisions could harm the organisation’s reputation with customers and funders, or could break the law and threaten its ability to operate. Team members should consider the magnitude of the downside risk associated with specific spending decisions, and seek additional feedback when it’s high.

The rest of this post provides guidance for BlueDot team members and for leaders at other organisations who are considering a similar policy. None of this is legal or tax advice. This policy isn’t part of anyone’s employment contract, and we can change it at any time. It describes how things work at BlueDot as of August 2026. Tax and payment mechanics vary by country.

Different types of spending

BlueDot US is a 501(c)(3) charity, and must comply with IRS rules.

BlueDot UK is a Company Limited by Guarantee (CLG), with 501(c)(3) equivalency determination. It must comply with UK tax rules and operate consistently with its 501(c)(3) equivalency determination, which imposes similar constraints to the US entity’s.

This policy does not imply that BlueDot staff can spend BlueDot’s money on their weekend hiking trip to Yosemite, or to learn how to fly a plane, or to fix the dishwasher in their house. Aside from a team member’s compensation package, BlueDot’s money cannot be used for personal benefit.

The unconstrained “act in BlueDot’s best interest” policy applies only to business expenses. Taxable benefits have separate limits, and are described below.

• Business expenses are the costs incurred in direct pursuit of BlueDot’s mission (e.g. marketing a program, paying a contractor, buying a work laptop).

• Personal expenses are whatever you spend your own money on for your own purposes (e.g. paying household rent, going on holiday, buying clothes).

• Taxable benefits are personal expenses that BlueDot covers as part of someone’s compensation. Our recurring wellbeing and commuting benefits are paid for using dedicated company cards with yearly limits. Relocation is paid as a one-off amount through payroll.

A heuristic to differentiate between business expenses and personal expenses: If someone else took over your job tomorrow, would BlueDot still need to pay this cost? If so, it’s probably a business expense.

If you’re unsure how to classify an expense or which card to use, ask the Ops team.

Business expenses

BlueDot’s most important asset is our team, and time is our scarcest resource.

When deciding whether to make a business expense, value your time at $10/minute. This is a simple rule of thumb, not a claim about your salary or the exact value created by every minute of your work. We set it high because a small amount of overspending is less costly than slowing the team down.

This approach doesn’t come naturally to frugal, impact-oriented people. For example, throughout 2021, I spent my evenings and weekends building a campervan, because I didn’t want to spend money on rent. In hindsight, this was a huge waste of my time.

You don’t need to calculate this precisely. If a legitimate business expense will save us a lot of time, default to spending the money.

Here are some things you might spend money on (all considered “business expenses”):

• A high-performing laptop, monitor, ergonomic chair, noise-cancelling headphones, or whatever you need to work effectively, both in the office and at home.

• Software and AI tools that make you work faster.

• Team meals, coffee and snacks at the office or during the workday, food while travelling for work, or an occasional team dinner out.

• When attending a work event, the ticket and transportation costs associated with getting there, and a good hotel either side of the event. Pay more for good transport, e.g. pay for Uber Comfort or extra legroom on a flight, so you can work as you travel and/or perform well when you arrive.

• A course, book, or job-related coaching that makes you better at your job.

And here are some things that aren’t considered business expenses, and therefore aren’t covered by this policy:

• Personal activities, e.g. a hobby class, a weekend trip, tickets to a show, or fixing/improving your house.

• Costs from the personal portion of a work trip, e.g. if you stay three extra days after a work trip for a personal holiday, you pay for those nights yourself.

• Food delivery while working from home and meals out by yourself are personal expenses. The exceptions are meals during overnight work travel, meals shared with a colleague or someone outside BlueDot as part of a genuine work meeting, and occasional meals that enable you to work beyond your usual hours. These rules apply whether the meal is delivered or eaten out.

• Your regular commute to the office. This isn’t a BlueDot business expense, but BlueDot covers it through a yearly allowance on your dedicated company commute card.

• Personal wellbeing costs, such as gym memberships, meditation apps or therapy. These aren’t BlueDot business expenses, but BlueDot covers them as a taxable benefit through a yearly allowance on your dedicated company wellbeing card.

Taxable benefits

Taxable benefits are personal expenses that BlueDot covers as part of your compensation package. We offer taxable benefits because we want to encourage some types of personal expenses (e.g. things that improve your wellbeing), and to make joining the BlueDot team more attractive to top talent.

The taxable benefits policy cannot be unconstrained in the same way as our business expense policy, because that would enable team members to arbitrarily increase their own salary.

Our recurring benefits come via two annual allowances:

• Wellbeing allowance: $10k/yr. This covers anything that keeps you physically and mentally well, including but not limited to gym sessions, therapy, coaching, and physio.

• Commute allowance: $5k/yr. This covers getting to and from work by whatever mode suits you, including but not limited to Uber, transit passes, parking, and bike or e-bike costs.

You get two dedicated benefit cards (most likely in Ramp), separate from your business card: a wellbeing card and a commute card. Spend directly from the relevant card.

There’s nothing to submit: no claims, no reimbursement forms, no fronting your own money and waiting for payroll.

These are still taxable benefits. BlueDot treats spending on these cards as taxable compensation. Each quarter, whatever you actually spent on these cards is reported to payroll as income. BlueDot pays the associated tax on top, so your take-home pay isn’t reduced.

Fine print

• Limits reset on 1 January each year and unused amounts don’t roll over. In the year you join BlueDot, your allowances are prorated for the remaining calendar months, including the month you start.

• Receipts or notes aren’t needed for benefit-card spending. The card limit is the control. We don’t review which gym, therapist or other provider you choose. The receipt rules in “Preventing penalties” only apply to business expenses.

• If your card is declined by a legitimate provider, message the Ops team.

Relocation

Relocation is also a taxable benefit, but it’s paid as a one-off lump sum rather than through a Ramp card.

If you’re moving to San Francisco to join BlueDot, we’ll pay you an $18k relocation allowance through payroll. BlueDot will pay the associated tax, so you’ll receive this amount after tax. Use it for whatever your move requires, such as temporary accommodation, a relocation adviser, a guarantor service or a deposit.

We also encourage you to take as much time as you need off from work to complete your housing search.

How to pay

• For business expenses, default to your company card. Ramp automatically adds receipts sent by email. You only need to add a receipt or note when required under “Preventing penalties”.

• If someone is invoicing us for work, such as a contractor, facilitator or creator, direct them to the invoice form. BlueDot pays these invoices by bank transfer so the Ops team can collect the required contractor details and process the payment correctly.

• If your card won’t work for whatever reason, such as for a large bill or an invoice with payment terms, forward the invoice to finance@bluedot.org. It will automatically appear in Ramp for the Ops team to pay.

Mitigating risks

Preventing penalties

If you accidentally make a personal expense on your business card, tell the Ops team and repay it as soon as possible. We require repayment within 120 days.

For some business expenses, we require a one-line note and/or receipt within 60 days, so that the expense is properly documented for tax purposes. We’re building AI agents to automate most of this.

For expenses under $75:

• If the purpose is obvious, such as Zoom or AWS, nothing is needed.

• If the purpose is ambiguous, such as Uber, a restaurant or Amazon, add a one-line note.

For expenses of $75 or more, and all hotel expenses:

• If the purpose is obvious, attach the receipt.

• If the purpose is ambiguous, add a one-line note and attach the receipt.

Preventing disaster

If a team member’s company card was stolen and it didn’t have any security controls, a thief could spend a large amount of BlueDot’s money. This would be quite bad.

Large transactions trigger an identity check inside our banking app. An admin confirms that the person making the purchase owns the card.

This is a security check, not budget approval. The admin is not deciding whether the intended purchase is a good use of money.

Preventing abuse

Team members will sometimes make financial decisions that I believe aren’t in BlueDot’s best interest. This is the nature of having such a permissive policy, and it’s a price we’re willing to pay for speed, trust, and ambition.

All business spending is visible to leadership and the Ops team, and are reviewed regularly. My spending as CEO is also shared with the board. If a decision seems off, including whether someone spent too much or too little, we ask for more information, share feedback, and help calibrate what’s sensible.

In practice, this is rare: team members default to seeking feedback before large decisions. Only a pattern of bad financial judgement, or something flagrantly not in BlueDot’s best interest, is treated as a performance issue.

Taking action and learning is better than not taking action. Team members will make mistakes, and our feedback culture will help them learn from them.

Want to work in a team that operates like this?

This expense policy is rare, especially in the non-profit and impact-focused world. We believe it helps us move faster, be more creative, and feel more ownership over our work.

If you believe you’re already working in the most impactful place you could be, but that your organisation would be more impactful if it had this expense policy, send this blog post to your leadership team.

And if you’re looking for a new and impactful role, apply to join BlueDot! We’re always looking for exceptional people, and we’re growing fast.

More people need to understand what advanced AI means and how they can help solve it, and we don’t yet have the infrastructure to communicate that well.

These are ideas I’ve gathered from engaging with creators and others in the space. They aren’t polished proposals, but starting points for people looking to work on this. I’d be excited to see these ideas, and many more, explored.

The goal of AI safety comms

AI safety communication helps:

• AI safety orgs communicate their work to attract talent and get people to take action.

• Specific audiences understand the impact of advanced AI and how they can contribute to AI safety.

Excellent comms is:

• Engaging. People will read, watch, or listen to it.

• Nuanced. It makes a complex topic accessible without being misleading or overhyped.

A person who would excel at this is experienced at making engaging content and understands AI safety well enough to get the details right.

Rather than upskilling people to make content, I’d be more excited to help people who are great at writing, producing, filming, and editing transfer their expertise to AI safety.

Projects I’d be excited to see

Content studios

AI safety orgs want people to engage with their work, but most don’t have in-house comms expertise. This might include projects like creating videos to share their research or filming their events for marketing material.

They are wary of hiring creators who lack AI safety context for fear that their content will not be handled responsibly. Doing this comms work also usually sits low on their priority list, so they are less willing to vet creators and closely manage them. The result is that their work goes uncommunicated.

An AI safety content studio could help AI safety orgs distribute their work more widely.

This could follow the model of other orgs like:

• AE Studios. They build tech products for clients (Fortune 10, top startups) and reinvest profits into a dedicated alignment research team exploring overlooked directions. An AI safety version could take on paid client work, then reinvest the profits into the AI safety content it most wants to see.

• Good Impressions. They support AI safety orgs with marketing and engagement. They have dedicated funding so that their fees don’t get in the way of the most impactful projects.

Agencies for AI safety creators

Creators spend a large chunk of their time just keeping their projects alive. An agency would help them focus on producing great content by handling other work like brand deals, sourcing contractors (e.g. editors, sound engineers) or legal work.

This could look like a YouTube channel agency model (e.g. Nebula or Complexly), or an operational support like Impact Ops.

Creator residencies

We have spaces for AI safety researchers to convene, like the London Initiative for Safe AI and Constellation. We could have spaces designed for creators to connect with others doing similar work and share resources like equipment or filming space.

In-person communities and purpose-built spaces enable people to do great work, and we could have hubs like these in several locations.

A living database of content & ideas

Creators are constantly looking for what to make next, and researching which ideas matter and how to talk about them eats up a lot of time. Meanwhile, the field has messages it wants shared.

A live resource or platform could help AI safety experts and creators coordinate on what to produce in a variety of formats and languages. There’s a lot of room to explore what this might look like.

This could include quotes, video and audio snippets or images that would be useful for conveying particular messages.

Talent pipelines for AI safety creators

So far, only a handful of programs have run to bring creators into AI safety. If we want more creators in the space, we need to help them get in.

This could look like:

• Scaling up fellowships like the Frame Fellowship or plzdontkillus, which focus on accelerating existing creators to produce more and better content.

• A hackathon. An event you can run often, which gives people a chance to try their hand at making AI safety content.

• A MATS-style stream to match creators with a mentor and make it easy to get feedback from relevant experts.

• A course or bootcamp. Work out the key things a creator needs to understand about the field, then build on-ramps that get them deep on the specific topics they need, fast.

• Placement programs like Tarbell, which help creators get work at major comms outlets so they can produce more nuanced AIS content from inside them.

Funding for creators

There’s no clear way to make a living producing AI safety content. Choosing to do this means making a big bet. We already support independent researchers to upskill and do speculative work. We could provide the same kind of support for creators.

This could look like:

• Exploratory grants for early-stage creators like FLI’s digital media accelerator, which allows creators to start producing AI safety content.

• A fund for much larger grants that support an excellent creator to continue producing content over years rather than months, in the same way one might fund a promising researcher.

Resources to get started

Learn more about AI safety

• BlueDot’s Future of AI course. A free resource for understanding the impact of advanced AI.

• BlueDot’s AGI Strategy course. A 25-hour discussion-based course to get an in-depth understanding of AI safety.

Get funding

• BlueDot’s Career Transition Grant

• Future of Life Institute’s Digital Media Accelerator provides funding to creators to raise awareness and understanding of ongoing AI developments and issues.

• Long-Term Future Fund previously funded Rob Miles to make podcasts and videos, and Nicky Case to create AI safety explainers.

• Survival & Flourishing Fund previously funded Tarbell Fellowship for AI safety journalism.

• Emergent Ventures has previously funded creators

Start creating content

• Tarbell Fellowship places journalists in major newsrooms.

• Frame Fellowship provides structure, funding, mentorship, and production support to enable creators to consistently produce accurate, compelling content at scale.

• plzdontkillus is an experimental creator bootcamp

Join an existing comms org

• 80,000 Hours produces AI safety content through podcasts, articles and YouTube videos

• CivAI builds live demonstrations that foster gut-level understanding of AI

• Seismic Foundation provides specialist marketing and communications capabilities

Reach out to me at anglilian@bluedot.org if you’ve been thinking about this and have something more fleshed out, including additional resources I might have missed.

What you’re applying for is access to an 8,000+ (and growing!) community who are serious about contributing to AI safety. We also provide programs like career transition grants, rapid grants and 1-1 advising to support them.

The AGI strategy course builds your understanding of advanced AI systems — what they can do, what drives progress, what the concrete risks are, and who controls development — so you can make informed decisions about how to contribute.

We reject more than half of applications to the course. This post helps you figure out if you should spend time on an application.

What the AGI strategy course is NOT

It’s not a corporate responsible AI course

If your goal is ISO 42001 compliance, responsible AI frameworks for your product team, or helping your organisation navigate the EU AI Act, you should look into something like IAPP’s AIGP certification or ISO 42001 programs instead.

It’s not an AI ethics course

We don’t cover algorithmic bias in hiring tools, fairness metrics, or the full range of social impacts of current AI systems. If that’s your focus, you’re better served by something like this or this.

It’s not for passive learners

You’re expected to actively engage with others who are motivated to make the future go well. If you haven’t started thinking seriously about risks from advanced AI, take the 2-hour, self-paced Future of AI course first.

What gets you in

It’s not (just) your CV. We’ve rejected plenty of people from prestigious institutions with impressive resumes who wrote three-word applications, and we’ve also accepted sharp people with more modest titles who showed they were already sprinting ahead.

We’re looking for people who are already taking action: applying to roles, going to events, building things, writing, talking to people in the field. You don’t need a perfect plan, but something more specific than “apply for fellowships”. This is a line we see in dozens of applications every round.

Your application itself is evidence for us too. We use it to infer how you’ll show up in a live discussion session of 6-8 people working through hard material together. Effort, agency and clarity go a long way.

Before you apply, ask yourself this:

• Am I focused on frontier AI, or AI in general?

• Do I have a specific reason to take this course now?

• Will I act on this fully within six months?

If you’ve read this and you’re thinking: ”yes, this is me”, apply here.

That program worked. So we’re making it bigger.

The majority of our grants have gone toward compute and API credits for people running evals, training small models, and replicating safety research, typically for a few hundred dollars. Our Technical AI Safety Projects Sprint has been great at surfacing these.

But some of our most impactful grants didn’t look like that at all:

• AI Safety Poland received $4,800 to organize meetups across the country, covering venue and tooling costs, and building a national community from scratch. They’ve had hundreds of attendees, and dozens of people they’ve referred have ended up taking a BlueDot course.

• Justin Dollman received $3,000 to lead our weekly evals reading group, coordinating a growing community of people learning about and working on model evaluations. We’ve since funded more reading groups on AI governance and AI x cyber and are excited about scaling these up from ~120 to 300 attendees per week.

• Sally Gao received $1,000 to run AI Safety meetups in New York, hosting guests such as Alex Bores. Her next event on the State of AI Safety is coming up on April 23rd!

• Eitán Sprejer received $4,200 in facilitator stipends to run our Technical AI Safety Projects Sprint at BAISH in Buenos Aires, accessing a new and underleveraged talent pool for AI safety work.

• Aaron Maiwald received $5,000 to attend a biosecurity conference in DC, travel to SF, and connect with senior folks in the field to accelerate his journey.

• Zac Saber received $8,000 to drop out of EF and validate an AI safety-focused startup instead.

None of these grants fit neatly into “compute for a project.” But they were some of our highest-impact bets. As the program picked up speed and more of these came in, we started treating rapid grants as small, focused bets on talented people doing or experimenting with high-impact projects - research, fieldbuilding, talent acceleration, you name it. In some cases, we approached people directly and pitched them on applying for work they were already doing.

Going forward, we’re expanding the official scope to match our internal understanding: Rapid Grants now fund much more than compute - we’re excited to back work on events, talent acceleration, BlueDot community building, and more. The bar for funding hasn’t changed - if anything we’ve raised it as we became more calibrated - we still look for concrete work in progress, a specific cost that’s the bottleneck, and a reason for us to believe the work matters for making AI go well.

Grant sizes now go up to $10,000 to allow us to make bigger bets for more impact. Decisions will still be fast - we’re targeting around five working days. For grants above $5,000, we may hop on a quick call.

Who should apply

If you’re in the BlueDot community - a course participant, alum, facilitator, or wider community member - and you’re doing something high-impact in one of our focus areas that a grant would accelerate, apply.

If you’re on the fence, the default, as always, is simple: apply.

Apply and see the full list of public grantees and program details at bluedot.org/grants/rapid.

This short blog post is intended to help you figure out if you should spend time on an application.

FAIGC is at its core about the governance of frontier AI systems - AGI, ASI - the most capable models being built by a handful of labs, the decisions governments and other actors need to make as those systems become more powerful, and what happens as general capabilities rapidly approach and exceed human-level.

If your guiding question is “how should we govern AI systems that might be smarter than humans within the next few years” you’re in the right place!

What FAIGC is not

It is not a corporate AI governance course

• If your goal is ISO 42001 compliance, responsible AI frameworks for your product team, or helping your organization navigate the EU AI Act - that’s real work and we respect it, but it’s not what we focus on. You should look at something like IAPP’s AIGP certification or ISO 42001 programs instead.

It is not an AI ethics course

• We don’t cover algorithmic bias in hiring tools, fairness metrics, or the full range of social impacts of current AI systems. If that’s your focus, you’re better served by something like this or this.

What gets you in

It’s not (just) your CV - we’ve rejected plenty of people from prestigious institutions with impressive resumes who wrote three-word applications, and we’ve also accepted sharp people with more modest titles who showed they were already sprinting ahead.

What we’re looking for: you’ve already started engaging with frontier AI governance specifically. You can name a concrete gap between where you are and what the course provides. And your post-course plan better be more specific than “apply for fellowships” - a line we see in dozens of applications every round.

Your application itself is evidence for us too. We use it to infer how you’ll show up in a live discussion session of seven to nine people working through hard material together. Effort, agency and clarity go a long way.

A note for AGI Strategy course graduates

We view completing our AGI Strategy course as a prerequisite and not as a ticket! We may still reject AGI Strategy grads who apply. If you’ve completed the AGISC and then started building something, writing something, or working on something in frontier AI governance, tell us about that. But if you completed it and your main next step is taking this course, you may need more time.

As a general heuristic, before you apply, ask yourself these three things

• Am I focused on frontier AI, or AI in general?

• Do I have a specific reason to take this course now?

• Will I act on this fully within six months?

We’re building a pipeline into the institutions that govern frontier AI and are looking for people who have a good chance to be in those institutions - not people adding a line to their CV.

If you’ve read this and you’re thinking yes, this is me - apply here.

How it works

At all times, only assume we’ll cover costs we’ve confirmed in writing for your specific project. If you’re ever uncertain, contact us before spending money you expect us to reimburse.

1. Submit a proposal here (under 15 minutes for most applications). We typically respond within 5 working days with one of three outcomes:

   • Accepted: We’ll confirm exactly what spending we’ve approved.

   • Clarification needed: We’ll ask follow-up questions to evaluate your request.

   • Not approved: We don’t think this meets our criteria, or we’re unable to approve for another reason.

2. Do your project, confident you can spend on what we’ve approved.

3. Claim reimbursement. We process most claims within 5 working days. If you didn’t end up spending the money, no problem - just don’t submit a claim.

What we fund

The following examples are illustrative; all decisions are ultimately at our discretion:

• Compute for a technical AI safety project (API costs, cloud GPU, training runs).

• Access to paywalled resources like articles, research papers, datasets, or textbooks.

• Hosting costs for your application or tool.

We expect most grants to fall between $50-100 for initial experiments to develop your proof of concept, then up to $500 once you have evidence of initial traction (a strong proof of concept or promising preliminary results). We encourage you to submit a separate application for each of these stages.

What we don’t fund

• Compensation for your time on the project.

• Equipment you’d reasonably already have (laptops, phones, external drives, etc.)

• General productivity subscriptions (ChatGPT Plus, Claude Pro, Cursor, Grammarly, etc. unless this is highly leveraged).

• Personal expenses.

Other funders may cover some of these - see this resources page for AI safety funding opportunities.

When should you apply?

Before applying, you should have a project idea. It doesn’t need to be a slam dunk, but you should:

• Be able to briefly articulate how it relates to AI safety.

• Have ideas for initial experiments you want to run.

• Have a rough estimate for how much those experiments will cost (I recommend using an LLM to help you calculate this).

If you’re unsure whether your application is strong enough, submit an application anyway! We’re not looking for perfection. Your facilitator can also help you stress test your project, so have a low bar for reaching out to them!

While you wait for a grant decision, you should continue to develop your project idea, do a deeper literature review, or start to run smaller experiments that fit on the free Google Colab GPU.

Eligibility

• You must be a current or past participant of BlueDot Impact’s Technical AI Safety Project Sprint.

• We reimburse via bank transfer (Wise) or PayPal (UK), so we cannot send payments to sanctioned countries.

Questions or feedback? Contact us.

Apply

Submit your proposal here - it takes under 15 minutes for most applications. We aim to get back to you within 5 working days.

When I first asked an LLM to code me that annoying function or, worse, that whole experiment pipeline, I felt guilty and anxious as if I was cheating and getting away with it. I’m sure you’ve felt this way too.

It can be easy (especially if you are new to AI safety) to feel like you should avoid this, to feel like you should write all of the code in your project yourself, understanding each line and all the libraries that you use.

There is a time and a place for this, but a 30hr research sprint is not it.

If you are doing a research sprint, you should use LLMs / coding agents for all of your coding. Below I give you three reasons why.

You don’t have time to learn both engineering and research

The skill you should build in a research sprint is research taste: choosing what experiments to run, forming hypotheses worth testing, and updating your intuitions when the results surprise you. This only develops through many iterations and 30hrs is not a lot of time, especially if you spend hours coding experiments that could take minutes with AI. You should be very explicit about which skill you are trying to train, and optimise for that skill alone.

You might be thinking “but I need to be sure my code is actually implementing the experiments I intended” and you’re absolutely correct. However, this is entirely compatible with vibe coding your whole experiment pipeline. Just get the same model to provide you with a summary of the code and keep asking it questions until you are confident it is doing it right.

You’ll do more research and do more writing

If you spend less time coding, you will cover more ground and do better research. You can go deeper, ask that extra question, run that extra experiment, discover that extra awesome result. You’ll develop better intuitions about how AI models behave and which research directions are worth pushing.

You’ll also have more time for writing. Sharing your project is what lands you that next opportunity and where all the best feedback comes from. Nobody will be impressed by how “human written” your code is. They will be impressed by your clarity of thought and the depth of your research.

Coding with AI is a skill worth practicing

AI models are amazing at coding. With Claude Code, you can now build an app in a weekend and a whole machine learning experiment setup in minutes. And this won’t change. For the rest of your life, LLMs will be faster at coding than you, no matter how much you practice. The most productive AI safety researchers use AI for coding, and so should you.

If you want to learn how to use AI for coding, ask your favourite LLM! They’re very good at giving you tips and tricks on how to use them better.

Conclusion

Don’t feel guilty about using AI for your coding. Learn to operate fast while staying in control. Be explicit in the skills you want to develop, and be ruthless in pursuing them. So, which skills do you want to develop in this sprint?

• Startups can become integrated in the AI supply chain, giving them good information about valuable safety interventions. Safety becomes a feature to be shipped directly to users by virtue of this market position.

• Better access to capital, talent, and ecosystem-building is available to for-profits than non-profits. VC funding dwarfs philanthropic funding, and there is little reason to believe that profitable safety-focused businesses aren’t possible.

• Joining a frontier lab is a clear alternative, but most AI deployment happens outside labs. Your marginal impact inside a large organisation is often smaller than your impact when founding something new. Equally, profitable businesses aren’t an inevitability. You should seriously consider working for or founding an AI safety startup.

Introduction

Markets are terrible at pricing safety. In the absence of regulation, companies cut corners and externalise risks to society. And yet, for-profits may be the most effective vehicle we have for deploying safety at scale. Not because the incentives of capitalism align by chance with broader human values, but because the alternatives lack the resources, feedback loops, and distribution channels to turn safety insights into safer outcomes. For-profits are far from perfect, but have many advantages and a latent potential we should not ignore.

Information, Integration, and Safety as a Product

For advanced AI, the attack surface is phenomenally broad. It makes existing code easier to crack. Propaganda becomes cheaper to produce and distribution of it becomes more effective. As jailbreaking AI recruiters becomes possible, so does the data-poisoning of entire companies.

Information about new threats and evolving issues isn’t broadcast to the world. Understanding where risk is most severe and how it can be mitigated is an empirical question. We need entities embedded all across the stack, from model development to deployment to evaluation. We need visibility over how this technology is used and misused, and enough presence to intervene when needed. ‘AI safety central command’ cannot provide all these insights. Researchers acting without direct and constant experience with AI deployment cannot identify the relevant details.

Revenue is a reality check. If your product is being bought, people want it. If it isn’t, they either don’t know about it, don’t think it’s worth it, or don’t want it at all. For-profits learn what matters in an industry directly from the people they serve, giving the best insights money could buy.

This is not to say that AI safety non-profits aren’t valuable. Many do critical work which is difficult to support commercially. But by focusing entirely on research or advocacy and ignoring the commercial potential of their work, organisations cut themselves off from a powerful source of feedback. Research directions, careers, and even whole organisations can be sustained for years by persuading grantmakers and fellow researchers of a thesis, rather than proving value to people who would actually use the work. Without this corrective pressure, even well-intentioned research may drift from what the field actually needs. Commercialisation should not be seen as a distraction or a response to limited funding, but as a tool for staying at the bleeding edge of what is useful for the world.

Productification

Turning research into a product people can buy is extremely powerful for distribution. You are no longer hoping that executives, engineers, and politicians see value in work they do not understand tackling risks they may not believe in. It becomes a purchase. A budget decision. A risk-reward tradeoff that large organisations are very well suited to engage with.

There are clear gaps in securing AI infrastructure which can be filled today. If you’re wondering what an AI safety startup might actually do, here are some suggestions for commercial interventions targeting different parts of the stack.

• Frontier Models: Interpretability tooling, evaluations infrastructure, and formal verification environments. Tools which might be implemented by labs and companies with direct access to frontier models to understand and control them better.

• Applications: Content screening, red-teaming as a service, and monitoring for misuse. Helping startups building on frontier models catch accidental or deliberate misuse of their platforms.

• Enterprise Deployments: Observability platforms, run-time guardrails, and hallucination detection. Enterprises and governments using AI to automate critical work should be able to catch issues early and reliably.

• Market Incentives: Model audit and certification, and safety-linked insurance. Creating market incentives which reward safer models when they’re released into the world.

None of these require waiting for frontier labs to solve alignment, or hoping that someone else finds your work and decides to implement it. Instead of writing white papers hoping governments will regulate or frontier labs will dutifully listen, you build safety directly into products that customers come to rely on. One path hopes someone will do the work, whereas the other is the work.

Safety Across The Stack

When you tap your card at a shop to make a purchase, a network of financial institutions plays a role in processing your transaction. The point of sale system reads your card information, sends it on to a payment processor, who forwards the request to the appropriate card network. The issuing bank for your card authorises the transaction, the money is sent to cash clearing systems, and cash settlement is performed often through a central bank settlement system.

There is a sense in which all fraud happens at a bank. They have to release the fraudulent funds, after all. But the declaration that all fraud prevention initiatives should be focused on banks and banks alone comes across as fundamentally confused. Fraud prevention might be easier at other layers, and refusing to take those opportunities simply because it is in principle preventable at some more central stage would not lead to the best allocation of resources.

Similarly, when a user prompts an AI application, they are not simply submitting an instruction directly to a frontier model company. Just as tapping your card does more than instruct your bank, such a message goes through guardrails, model routing, observability layers, and finally frontier model safety measures. Every step of this process is an opportunity for robustness we should not let go to waste.

This becomes even more critical as AI agents begin acting autonomously in the world, doing everything from browsing and transacting to writing and executing sophisticated code. When an agent’s action passes through multiple services before having an effect, every link in that chain is both a potential failure point and an opportunity for a safety check.

Exclusively focusing AI safety interventions on frontier labs would be like securing the entire financial system by regulating only banks. Necessary, but nowhere near the most efficient or robust approach.

Capital, Talent, and Credibility

Successful for-profits are in an inherently better position to acquire resources than non-profits. Their path to funding, talent acquisition, and long-term influence is far stronger than that of their charitable counterparts.

There is an immense amount of venture capital washing around the AI space, estimated at ~$190 billion in 2025. Flapping Airplanes raised $180 million in one round, comparable to what some of the largest AI safety grantmakers deploy annually, raised in a fraction of the time. VC allows you to raise at speed, try many approaches, and pivot more freely than would be possible in academia or when reliant on slower charitable funders.

In AI safety, non-profits are less likely compared to other sectors to be trapped in an economic struggle for survival. However, even in the AI safety ecosystem, philanthropy is much more limited than venture capital and more tightly concentrated among fewer funders. Non-profits are vulnerable, not just to the total capital available, but to the shifting attitudes of the specific grant makers they rely on. VC-backed companies, by contrast, are much more resilient to the ideological priorities of funders. If one loses interest, many others remain available as long as you have a strong business case.

Yes, there is a large amount of philanthropic capital in AI Safety compared with typical non-profits. Safety products can also be difficult to sell. But the question of whether safety-focused products sell well, as they do in other industries, is a hypothesis you can go out and test. If it turns out that they do, there could be an immense amount of capital available which should be used to make our world safer.

For-profits attract talented people not just through hefty pay packages, but also through their institutional prestige and the social capital they confer. You can offer equity to early employees, which is extremely useful for attracting top technical talent and is entirely unavailable to non-profits. Your employees can point to growing valuations, exciting products with sometimes millions or even billions of users, and influential integration of their technology with pride. For many talented and competent people, this is far more gratifying than publishing research reports or ever so slightly nudging at the Overton window.

All of this, increased access to talent, capital, and credibility, makes for-profits far easier to scale. And safety needs to scale. The amount of time we have until transformative AI arrives differs wildly between forecasts, though it seems frighteningly plausible that we have less than a decade to prepare. If we are to scale up the workforce, research capacity, and integration into the economy of safety-focused products, we cannot afford anything other than the fastest approach to building capacity.

Success compounds. Founders, early employees, and investors in a successful for-profit acquire capital, credibility, and influence that they can reinvest in safety, whether by starting new ventures, funding others, or shaping policy. This virtuous cycle is largely unavailable to the non-profit founders, unless they later endow a foundation with, as it happens, money from for-profits.

In addition to tangible resources, a mature ecosystem of advisors and support networks exists to help startups succeed. VC funds, often staffed by ex-founders, provide strategic guidance and industry connections that are crucial for closing sales. There are many talented people who understand what startups offer and actively seek them out. An equivalent ecosystem just doesn’t exist for non-profits.

Shaping The Industry From Within

Being inside an industry is fundamentally different from being adjacent to it.

Embedding an organisation inside AI ecosystems enables both better information gathering and opportunities for intervention. If you can build safe products appropriate to the problems in an industry, you allow companies to easily purchase safety. If companies can purchase safety, then governments can mandate safety. But to get there, it is not enough to make this technology exist; the technology must be something you can buy.

Cloudflare started as a CDN. By becoming technically integrated, they slowly transformed into part of the critical infrastructure of the internet. Now, they make security decisions which shape the entire internet and impact billions of users every day. A safety-focused company embedded in AI infrastructure could do the same.

Will Markets Corrupt Safety?

Market incentives are not purely aligned with safety. The drive to improve capabilities, maximise revenue, and keep research proprietary will harm a profit-seeking organisation’s ability to make AI safer.

However, every institution has its pathologies. The incentives steering research-driven non-profits and academics are not necessarily better.

Pure Research Also Has Misaligned Incentives.

The incentives of safety and capitalism rarely align. The pressure to drive revenue and ship fast pushes towards recklessly cutting corners, and building what your customers demand in the short term rather than investing in long term safety.

However, research organisations have similar harmful incentives driving them away from research which is productive in the long term. The need to seek high-profile conference publications, pleasing grant makers, and building empires. Incentives of research organisations and individual researchers are notoriously misaligned with funders goals’ in academia and industry alike. Pursuing a pure goal with limited feedback signals is extremely difficult as an organisation, regardless of structure.

Ideally, we would have both. For-profits which can use revenue as feedback and learn from market realities, alongside non-profits which can take longer-term bets on work needed for safety. The question is how to build a working ecosystem, not whatever is more purely focused on safety.

Proprietary Knowledge Is Not Always Hoarded.

For-profits have an incentive to keep information hidden to retain a competitive advantage. This could block broader adoption of safety techniques, and restrain researchers from making optimal progress.

Assuming that for-profits add resources and people to the AI safety ecosystem, rather than simply moving employees from non-profits, this is still advantageous. We are not choosing between having this research out in the open or hidden inside organisations. We are choosing between having this research hidden or having it not exist at all. In many sectors, the price of innovation is that incumbents conceal and extract rents from their IP for years.

Despite this, for-profits do have agency over what they choose to publish. Volvo famously gave away the patent to their three-point seatbelt at the cost of their own market share, saving an estimated 1 million lives. Tesla gave away all of their electric vehicle patents to help drive adoption of the technology, with Toyota following suit a few years later. Some of this additional knowledge created by expanding the resources in AI safety may still wind up in public hands.

Markets Force Discovery Of Real Problems.

The constant drive to raise money and make a profit is frequently counter to the best long-term interests of the customer. Investment which should be put into making a product safer today instead goes into sales teams, salaries, and metrics designed to reel in investors. It is true that many startups which begin with a strong safety thesis will drift into pure capabilities work or adjacent markets which show higher short-term growth prospects.

However, many initiatives operating without revenue pressure, such as researchers on grants or philanthropically-funded non-profits, can work for years on the wrong problem. For-profits will be able to see that they are working on the wrong thing, and are driven by the pressure to raise revenue to work on something else.

This is not to say that researchers are doing valueless work simply because they are not receiving revenue in the short term. Plenty of work should be done to secure a prosperous future for humanity which businesses will not currently pay for. Rather, mission drift is often a feature rather than a bug when your initial mission was ill-conceived. The discipline markets provide, forcing you to find problems people will pay to solve, is valuable.

Failure Is A Strong Signal.

The institutional failure modes of non-profits and grant-funded research are mostly benign. The research done is not impactful, and time is wasted. On the other hand, for-profits can truly fail in the sense that they fail to drive revenue and go bankrupt, or they can fail in more spectacular ways where they acquire vast resources which are misallocated. The difference is not that for-profits are inherently more likely to steer from their initial goals.

Uncertainty about impact is common across approaches. Whereas research that goes unadopted fails silently, and advocacy which fails to grab attention disappears without effect, for-profits are granted the opportunity to visibly and transparently fail. The AI safety ecosystem already funds work which fails silently, and is effectively taking larger risks with spending than we realise. Startups aren’t any more likely to fail to achieve their goals; they are in the pleasant position of knowing when they have failed.

Visible failure generates information the ecosystem can learn from. Silent failure vanishes unnoticed.

Your Counterfactual Is Larger Than You Think.

Markets are not efficient. The economy is filled with billion-dollar holes, which are uncovered not only by shifts in the technological and financial landscape but by the tireless work of individuals determined to find them. Just because there is money to be made by providing safety does not mean that it will happen by default without you.

Stripe was founded in 2010. Online payments had existed since the 1990s, and credit-card processing APIs were available for years. Yet it took until 2010 for someone to build a genuinely developer-friendly API, simply because nobody had worked on the problem as hard and as effectively as the Collison brothers.

Despite online messaging being widely available since the 1980s, Slack wasn’t founded until 2013. The focus, grit, and attention of competent people being applied to a problem can solve issues where the technology has existed for decades.

Markets are terrible at pricing in products which don’t exist yet. Innovation can come in the form of technical breakthroughs, superior product design, or a unique go-to-market strategy. In the case of products and services relevant to improving AI safety, there is an immense amount of opportunity which has appeared in a short amount of time. You cannot assume that all necessary gaps will be filled simply because there is money to be made there.

If your timelines are short, then the imperative to build necessary products sooner rather than later grows even greater. Even if a company is inevitably going to be built in a space, ensuring that it is built 6 months sooner could be the difference between safety being on the market and unsafe AI deployment being the norm.

For many, the alternative to founding a safety company is joining a frontier lab. However, most AI deployment happens outside labs in enterprises, government systems, and consumer-facing applications. If you want to impact how AI meets the world, you may have to go outside of the lab to do it. Your marginal impact inside a large organisation is often, counterintuitively, smaller than your marginal impact on the entire world.

Historical Precedents

History is littered with examples of companies using their expertise and market position to ship safety without first waiting around for permission.

Sometimes this means investing significant resources and domain expertise to develop something new.

• Three point seatbelt: Volvo developed the three point seatbelt and gave away the patent. Their combination of in-house technical expertise and industry credibility enabled a safety innovation that transformed the global automotive industry.

• Toyota’s hybrid vehicle patents: Toyota gave away many hybrid vehicle patents in an attempt to accelerate the energy transition.

• Meta’s release of Llama3: At a time when only a small number of organisations had the resources to train LLMs from scratch, Meta open-sourced Llama3 making it available to safety researchers at a time when little else was in public hands.

Or perhaps the technology already exists, and what matters is having the market position to distribute it or the credibility to change an industry’s standards.

• Levi-Strauss supply chain audit: At the peak of their market influence, Levi-Strauss audited their supply chain insisting on certain minimum worker standards to continue dealing with suppliers. They enforced workers rights’ in jurisdictions where mistreatment of employees was either legal or poorly monitored, doing what governments couldn’t or weren’t prepared to do.

• Cloudflare’s Project Galileo: Cloudflare provides security for small, sensitive websites at no cost. This helps journalists and activists operating in repressive countries avoid being knocked off the internet, and is entirely enabled by Cloudflare’s technology.

• WhatsApp end-to-end encryption: The technology existed, and the cryptography research was mature by this point. WhatsApp just built it into their product, delivering privacy protection to billions of users worldwide.

• Security for fingerprint and face recognition: Apple stores face and fingerprint data in a separate chip, making it impossible to steal or legally demand. This did not require regulation; this decision actually led to clashes with the US government. Because of their market position, Apple was able to push this security feature and protect hundreds of millions of users unilaterally.

All of these required a large company’s resources, expertise, credibility, and market integration to create and distribute valuable technology to the world.

Building a for-profit which customers depend on, be it for observability, routing, or safety-tooling, lets you ship safety improvements directly into the ecosystem. When the research exists and the technology is straightforward, a market leader choosing to build it may be the only path to real-world implementation.

It’s Up To You.

For-profits are in a fundamentally strong position to access capital, talent, and information. By selling to other businesses and becoming integrated in AI development, they can not only identify the most pressing issues but directly intervene in them. They build the technological and social environment that makes unsafe products unacceptable and security a commodity to be purchased and relied upon.

Non-profits have done, and will continue to do, critical work in AI safety. But the ecosystem is lopsided. We have researchers and advocates, but not enough builders turning their insights into products that companies buy and depend on. The feedback loops, distribution channels, and ability to rapidly scale that for-profits provide are a necessity if safety is to keep pace with capabilities.

The research exists. The techniques are maturing. Historical precedents show us that companies embedded in an industry can ship safety in ways that outsiders cannot. What’s missing are the people willing to found, join, and build companies that close the gap between safety as a research topic and safety as a market expectation. We cannot assume that markets will bridge this divide on their own in the time we have left. If you have the skills and the conviction, this is a gap you can fill!

If you’re thinking about founding something, joining an early-stage AI safety company, or want to pressure-test an idea - reach out at team@bluedot.org. We’re always happy to talk.

BlueDot’s AGI Strategy Course is also a great starting point - at least 4 startups have come out of it so far, and many participants are working on exciting ideas. Apply here.

Thanks to Ben Norman, Daniel Reti, Maham Saleem, and Aniket Chakravorty for their comments.

Lysander Mawby is a graduate of BlueDot’s first Incubator Week, which he went on to help run in v2 and v3. He is now building an AI safety company and taking part in FR8. Josh Landes is Head of Community and Events at BlueDot and, with Aniket Chakravorty, the initiator of Incubator Week.

The key challenge that surfaced was sourcing talent who can own and lead work independently, and care deeply about AI risk.

These are people with a strong track record, whether through years of professional experience or a few impressive projects, who can hit the ground running.

Here are some examples of what this experience looks like (these are more illustrative than specific!):

Engineering (Lead / IC)

• You have the technical depth to make good architectural calls on complex systems — databases, ML pipelines, large codebases.

• You’ve shipped high-quality code in demanding environments (e.g. big tech, high-velocity startup)

• You know how to get a junior colleague unstuck on a tricky bug, suggest better tooling, or propose a new process when the current one isn’t working.

• (Lead) You’ve led and grown engineering teams, and can set a high bar for your team.

• (IC) You can take vague requirements for a system and turn them into working systems, even in unfamiliar contexts.

Research lead

• You know what it takes to publish high-quality research end-to-end — scoping questions, choosing methodology, organising the work, and communicating findings to diverse stakeholders.

• You’re a recognised expert in your domain, with the publications and collaborations to show for it.

• You ask good questions that open up impactful research directions, and you lean on a strong professional network to get things done.

Communications lead

• You have a track record of producing high-quality content tailored to specific audiences, and your public profile reflects that.

• You’ve managed large project budgets and timelines, making evidence-based decisions to improve outcomes.

• You’ve engaged seriously with AI safety — perhaps even producing content on it already.

Policy lead

• You’ve worked directly with government and understand how to navigate political institutions.

• You have a strong network in policy environments and know how to get things in front of the right people.

• You have a track record of translating complex technical research into clear, actionable policy memos and briefings.

Note: these profiles are composites from the organisations we spoke to and are meant to be illustrative rather than specific.

You don’t need years of AI safety experience but you do need to care

Given that relatively few people can claim 5+ years of direct AI safety experience, orgs are looking for the next best thing — people with a strong track record of excellence on similar projects outside the field.

Mission alignment helps bridge the gap.

Orgs are looking for people who’ve engaged seriously enough with AI risk to articulate why this work matters from their own perspective, and ideally made career choices that reflect that conviction.

These are small (<50), mission-driven orgs with salaries that can’t compete with big tech companies. As a result, they are looking for those whose motivation goes beyond compensation.

How to get started

The biggest barrier for capable talent entering the field is context.

Our AGI strategy course is designed to give you that context. You’ll develop your understanding of the risks AI poses and explore how you might contribute your expertise.

It’s a free, 30-hour course that AI safety orgs view as a strong signal of motivation. It is the first step most take in their AI safety careers.

AI safety needs your expertise to help ensure a positive future with AI. Apply here to get started.

Our Courses

We currently run four courses: the AGI Strategy Course, the Technical AI Safety Course, the (Frontier) AI Governance Course, and the Biosecurity Course.

Using Our Curricula (Permissionless)

Running an independent version of our courses is largely permissionless. You can pick up our curricula and start running sessions with your group right away. This works well for friends, workplace groups, student societies, and other local organizations who want to engage with the material together.

You can share and adapt our course materials including resource lists, exercises, sample answers, and discussion docs, provided that you:

Brand it distinctly: Market the course in a way that makes clear who’s running it. Don’t use our name in a way that could confuse people as to who is offering the course.

Give appropriate credit: When presenting these materials, please credit us as the source as “BlueDot Impact”.

Don’t hold us liable for errors: While we try our best to ensure the accuracy and relevancy of our course materials, they’re provided solely on an “as is” basis, without warranty of any kind.

Note that we don’t own many of the linked-to resources themselves, so you might need permission from their original owners if you want to copy or translate them.

Facilitator resources: We also provide access to our facilitator templates and documents to help you run effective sessions. You can find these linked here.

Tell us how it went: After you’ve run your course, we’d love to hear from you. Email us at team@bluedot.org with how many people completed the course, what challenges you ran into, what worked well, and whether there were any standout participants you think we should know about. This helps us improve our materials and potentially connect promising people with future opportunities.

Funding: If you’ve got a particularly talented group of folks lined up and a strong theory of change for why and how this experience will be impactful, we’d be excited about exploring how funding for e.g. a venue, snacks or a stipend could take your cohort to the next level. You can apply for such a grant here.

Running Courses at Companies or Institutions

If you want to run our courses internally at a company or institution - or if you’d like us to run them for you - please email us at team@bluedot.org to discuss what this may look like. We typically don't do this but may make exceptions for particularly high-impact opportunities.

Example Marketing (Independent Version)

Here’s an example of how you might market an independent version of our course, following the guidelines above:

Calling all students interested in the future of AI safety and alignment!

🤖 The Greendale Community College AI Society is excited to be hosting an AI safety course this semester, based on the popular AGI Strategy Course developed by BlueDot Impact.

We’ll be following their curriculum. You should do the readings and exercises beforehand, and then each week we’ll host a facilitated group discussion adapted for our university setting.

The discussions will run every Wednesday from 6-8pm in Group Study Room F starting on 17 September. To join the seminar series, RSVP at this link.

If you have any other questions, email ai.soc@greendale.edu. Looking forward to some fascinating discussions! 🙂

What’s Not Okay

Examples of things that would not be okay:

• “GCC AI Society is launching a round of the BlueDot Impact AGI Strategy course”

• “We’re running the BlueDot AGI Strategy Course in Greendale, Colorado”

• “Apply to the AGI Strategy Course”, linking to your version of the course

• “Run in collaboration with BlueDot Impact”, unless we’ve explicitly agreed this

• Issuing certificates “for completing the BlueDot AGI Strategy course” (but it’s fine to issue certificates “for completing GCC AI Society’s AI Alignment course, based on the BlueDot Impact curriculum”)

FAQs

Can local groups get copies of the discussion docs?

Yes! These are linked here.

Can local groups use BlueDot’s infrastructure for running courses?

Yes, almost all our software and corresponding documentation is available on GitHub. You can raise issues there if you get stuck. We aren’t currently able to provide hosted versions of our software, or technical support beyond this. Local groups are also welcome to direct users to our course hub to help learners track their own reading completions and exercises.

Can we use your facilitator training program?

Yes! The resources, exercises and session plans for our facilitator training course are available online. Though, we can’t run facilitator training for you.

I’m planning to run a local group / previously participated in a local group version. Could I join the BlueDot facilitated course?

Yes, please apply in the normal way and mention this in your application. Note that this does not guarantee you a place on our course.

If you have any other questions, feel free to contact us at team@bluedot.org.

Dangerous capability evals make sense for safety. We need to know how worried we should be about models causing harm.

But we’re missing benchmarks for specific positive safety things we want AI to do. Capabilities that could actively keep us safe or steer us toward better futures.

What beneficial evals would measure

Imagine if model cards included sections on how well AI systems perform at:

• Safety research: Assisting with interpretability work, alignment research, or red-teaming other AI systems

• Cyber defense: Not just penetration testing, but actually defending networks, detecting intrusions, and patching vulnerabilities

• Pandemic preparedness: Accelerating vaccine development, optimising PPE distribution, or improving disease surveillance

• Information integrity: Helping people find truth in a world full of mis/disinformation: fact-checking claims, detecting coordinated inauthentic behaviour, or surfacing reliable sources on contested topic

Models have graduated from ‘confidently wrong intern’ to ‘occasionally helpful new grad’, at least for some coding tasks. These evals and reinforcement learning (RL) environments could extend these into beneficial capabilities. If we’re heading toward a capabilities explosion, we need to ensure that beneficial capabilities outpace dangerous ones by a wide margin.

From evals to RL environments

Evals and RL environments are two sides of the same coin. An eval asks “can the model do X?” and produces a score. An RL environment asks the same question, but feeds that score back as a training signal.

In practice, this means a well-designed eval is most of the work to building an RL environment. If you can measure whether a model successfully identified the right papers, traced activations correctly, or generated a viable therapeutic drug candidate, you can turn that measurement into a reward signal. The eval’s scoring rubric becomes the RL environment’s reward function. The eval’s test cases become the RL environment’s training distribution.

A word of caution

Not everything that sounds safety-relevant actually is. Be careful that you’re measuring something genuinely useful for safety, not just general capabilities dressed up in safety language.

Getting models to be better at coding in Python is a general capability, not safety-specific. But improving models’ use of TransformerLens for interpretability research unblocks a specific safety-relevant bottleneck.

The goal is to create targets that, if hit, would actually make AI development go better, without inadvertently pushing general capabilities that could be misused.

How to build beneficial capability evals

Approach 1: Reproduce existing safety projects

Take existing safety-relevant projects and try to get AI to reproduce them. Where does the model fail? Those failure points become eval problems you can measure against.

For example, BlueDot Impact runs courses where participants complete projects on AI safety, biosecurity, and other safety-relevant domains. Take a completed project — say, a mechanistic interpretability analysis of a small transformer — and see if a model can replicate it. If the model gets stuck on a specific step (maybe it can’t correctly identify attention head functions), that’s a concrete eval task.

This approach grounds you in real work that humans have already validated as useful.

Approach 2: Talk to safety researchers

Go directly to people doing safety-relevant work. Ask them to keep a log for a couple of weeks: every time AI tools frustrate them, slow them down, or just can’t do something they need, write it down. Then collect these logs and look for patterns.

You might find things like: “Claude keeps hallucinating paper citations when I ask for related work,” or “I can’t get any model to correctly trace activations through residual stream connections,” or “It takes 10 back-and-forths to get the model to understand my codebase structure.”

These frustrations point directly to capability gaps. Turn them into evals.

Breaking down the work

Whatever approach you take, you’ll need to:

Map the process. Break down the safety-relevant task into all its component steps. For example, with vaccine development: literature review → identifying which parts the immune system can target → choosing an approach (mRNA, viral vector, etc.) → designing candidates → scaling up manufacturing → running clinical trials.

Prioritise. Figure out which parts of the pipeline are most important to work on first. Maybe running clinical trials is the biggest bottleneck or the literature review step that takes researchers months.

Define tasks. For each component, specify what success looks like in a way that’s measurable. For the literature review step, this might be: “Given a novel pathogen genome sequence, identify the 10 most relevant prior papers on similar pathogens, ranked by relevance to vaccine development”. Include a rubric for scoring relevance and completeness.

Build evaluation infrastructure. Many modern evals are “agent-evaluated”—you get one model to attempt a task, then use another model as a judge to verify whether specific conditions have been met. Inspect is a good framework for getting started with building evals. Consider submitting your work to inspect-evals to make it discoverable.

Iterating and improving. A key is desirable difficulty: we need evaluations that remain challenging and informative as capabilities improve. If in doubt, err on the side of too difficult.

Getting Your Work Used

Building good evals isn’t enough. You need companies to actually find and use them.

• Make it discoverable: Publish on your blog, get it circulating on Twitter/X, or submit to relevant publications

• Make it easy to run: Companies are more likely to adopt your eval if they can spin it up quickly

• Work directly with companies: Offer to help them run it, troubleshoot issues, and interpret results

The business opportunity

Building good evals is a serious undertaking. No single person or team is going to create comprehensive positive capability benchmarks across all safety-relevant domains.

But there’s a business opportunity here. Companies will pay for RL environments: the Docker containers, code infrastructure, and carefully written rubrics for each task. A path here might be:

• Start with a small batch or proof of concept

• Pitch it to a company

• If they’re interested, they might pay for a set of ~100 problems

• They’ll QA the work, and if it’s good, they’ll ask for more

Two people working full-time could potentially (with a lot of coffee) get an initial batch done in a month.

Start building

Pick a safety-relevant domain you know, map the steps, find the bottlenecks, and build something. You can also:

• Apply to our incubator weeks.

• Use or apply to the Technical AI Safety Project sprint

• Learn more about plans to make AI go well on the AGI strategy course.

• Tag @BlueDotImpact on Twitter or LinkedIn. We’d love to see what you build!

But luckily there are a few common prefixes and suffixes that crop up again and again. Once you’ve learned this lingo, you’ll have a far easier time fighting through biology papers and wikipedia articles.

Cell structure and location

• Cyto- → cell (cytoplasm, cytoskeleton, cytokine)

• -some → discrete body/structure (ribosome, lysosome, chromosome, endosome)

• Endo- → within/inside (endocytosis, endoplasmic reticulum, endotoxin)

• Exo- → outside/outward (exocytosis, exotoxin, exonuclease)

• Peri- → around (periplasm, perinuclear)

• Trans- → across (transmembrane, transcription, translation)

• Intra- → within (intracellular, intranuclear)

• Extra- → outside (extracellular, extrachromosomal)

Enzymes and processes

• -ase → enzyme (protease, kinase, polymerase, lipase)

• -sis → process/action (mitosis, apoptosis, lysis, synthesis)

• -lysis → breaking down/splitting (hydrolysis, glycolysis, proteolysis)

• -genesis → creation/origin (biogenesis, pathogenesis, oncogenesis)

• Kinase → adds phosphate groups (protein kinase, tyrosine kinase)

• Phosphatase → removes phosphate groups

• -merase → polymerising enzyme (polymerase, telomerase)

• Enzyme names usually end in -ase with the substrate as prefix (lactase acts on lactose, protease on proteins)

Eating, destroying, killing

• Phago-/-phage → eat/engulf (phagocyte, macrophage, phagocytosis)

• Bacteriophage/phage → virus that infects bacteria (you’ll see “phage” used standalone constantly in molecular biology)

• -cidal/-cide → killing (bactericidal, virucidal, fungicidal)

• -static → inhibiting growth without killing (bacteriostatic)

• -lytic → causing lysis/destruction (cytolytic, haemolytic)

Molecules and macromolecules

• Glyco-/-glycan → sugar/carbohydrate (glycoprotein, glycolysis, proteoglycan)

• Lipo-/Lip- → fat/lipid (lipoprotein, lipase, lipid bilayer)

• Proteo-/Prot- → protein (proteome, protease, proteolysis)

• Nucleo- → nucleus or nucleic acid (nucleotide, nucleosome, endonuclease)

• Poly- → many (polymer, polypeptide, polysaccharide)

• Oligo- → few (oligonucleotide, oligosaccharide)

• Mono- → one (monomer, monosaccharide)

• -ome1 → complete set (genome = complete set of genes, proteome = complete set of proteins)

Useful general prefixes

• Bio- → life/living (biosynthesis, biomarker)

• Hetero- → different (heterozygous, heterogeneous)

• Homo- → same (homozygous, homologous)

• Iso- → equal/same (isomer, isotonic)

• Hyper- → excess/above (hyperactivation, hypertonic)

• Hypo- → deficient/below (hypotonic, hypoxia)

• Anti- → against (antibody, antigen, antibiotic)

• Pro- → before/precursor (promoter, prophage, proinflammatory)

Test yourself!

Go through the abstract from the start of this post word-by-word and try to rewrite it in simple language with the help of the terminology above. You might not get everything, but I bet you’ll get further than you expect.

Use an LLM to check your work and help you with any parts you’re still confused about.

The need for field-level strategies

A field is a community of individuals and organisations working to address a shared problem. In our case, the fields we operate in are trying to mitigate risks from AI and synthetic biology.

These fields are under-resourced and confused. Without a strategy for what we’re trying to achieve and how we’re going to achieve it, we risk producing elegant answers to pointless questions, focusing our scarce resources on unimportant problems, and failing to protect humanity from harm.

It’s nobody’s job to produce field-level strategy — most actors in the community focus on their own organization or their own role.

Many “strategies” that people do produce are shallow, one-off wishlists that don’t get distributed to relevant stakeholders. They set goals before deeply understanding the problem and analysing what matters most for whether or not we’ll succeed. Strategy isn’t a goal, it’s a form of problem-solving, and you can’t solve a problem you don’t understand and haven’t defined.

Field strategy is still challenging to do well. Defining the “problem-to-be-solved” is contentious and a moving target. Information is privately held or not written down. Experts disagree because they have different values and world-models. You might just aggregate people’s opinions and produce a vague mush. Few people have done this well before. You’ve not been appointed to do this, so you need to earn legitimacy and trust by doing the hard work well.

Here’s how you can overcome these challenges.

How to produce a field strategy

These steps should be done in parallel.

Read and take notes

You’re not the first person to think about this problem. Start by reading the field’s seminal literature, and important pieces from adjacent fields. You can use AI to search for the best papers and blog posts, and they’re also great thought partners to ask you questions, give you feedback, and prod your mental models for how things work.

Take lots of scrappy notes. Don’t write prose; stick to one-sentence bullet points. Synthesise what you’re learning into the most critical insights, and group your notes into sensible-seeming categories to help you build a stronger mental model of the field. Keep track of your confusions and uncertainties, as these can steer further research and interview questions.

Ask and listen

You’re unlikely to find the most important insights on the public web. They live inside people’s heads or in private google docs. To acquire this information, you need to talk to lots of the right people. Connect with them via cold emails or via warm introductions. As a heuristic, try to have 50 calls in your first 50 days of working full-time on this.

The usefulness of conversations tends to follow a power-law: most conversations will be fine, a few will be extraordinary, some you won’t realize were useful until weeks later. So you need to constantly search for the most insightful people.

To gather the most juicy insights during your calls, you need to build a connection quickly. At the start of every conversation, share your backstory, ask them for theirs, and find common ground. Demonstrate you’re competent, be vulnerable, and mirror their energy.

Avoid biasing them with your ideas early on. Ask open questions that give them a lot of space to explore and share things you don’t expect. For example, “If we’ve solved X problem in 10 years, what’s the most important thing that needs to happen next year?”. Read The Mom Test for question-asking advice.

After you’ve asked a broad open question, listen intently and ask a follow-up question that zooms in on a specific thing they said that seems important. You know it’s a good question if their eyes light up or if they have to think super hard and then say something really valuable.

You should also come prepared with sharp, open questions that enable them to build on what you already know. For example, “In X paper, I noticed Y, but this conflicts with my intuition Z. What’s your intuition about Z, and what drives that?”.

If you’re confused, express that confusion and give them the opportunity to help you. If you hear them say something that doesn’t make sense to you or doesn’t align with your model for how things work, chase it. Ask follow-up questions. This is how you discover unknown unknowns.

Move fast through the network

Every call is an opportunity to bounce into the next conversation. In the final 5 minutes, prioritize generating names for potential introductions. If you’ve asked great questions, demonstrated that you’re determined to solve this problem, and they’ve had fun, they’ll feel excited to introduce you to people they admire.

To help them generate names, prompt them with:

• Who’s done the best writing on this problem?

• Whose work are you most impressed with?

• Who do you turn to for help when you’re stuck?

• Who would I learn the most from?

Explicitly ask for introductions before the call ends. Then you need to make it as easy as possible for them to make those introductions. Immediately after the call ends, send them a complete draft email they can use to make the introductions. This should include a subject, a bio of yourself and a description of this project. Aim for <100 words.

Once you’ve been introduced, you need to move fast. If you send them a scheduling link, it might take them 1-2 weeks to click on it, and they might schedule a meeting for 1-2 weeks after that. But you need to move as deep into the “intros chain” as quickly as possible.

Immediately after you’ve been introduced to someone, send them a calendar invite for the next day at a reasonable time for their timezone. Use this format for the calendar event title: “[TBC] YourName<>TheirName”. Reply to the introduction asking if that time works, and if not, when works best for them. This tactic runs a higher risk of annoying people and shouldn’t be used with everyone, but I think it’s worth the speed benefits most of the time. Use it thoughtfully.

Write and rewrite

After your first few calls, start writing your v1 scrappy strategy doc. In separate sections, answer these questions:

• What is the precise problem/threat you’re trying to solve?

• If this field succeeds in 5-10 years, what’s different about the world?

• Where are things at today? Who’s working on this, what’s been tried, what’s the funding and policy landscape?

• What’s the biggest obstacle between here and success?

• What approach could overcome this obstacle, and why would it work?

• What needs to happen in the next 6-12 months?

Here are some work-in-progress examples: UV air disinfection, DNA synthesis screening, pandemic early warning systems.

This structure is adapted from Richard Rumelt’s “Good Strategy, Bad Strategy,” which is well worth the read. The most common failure mode when doing strategy is jumping from “what success looks like” to “what should we do”, without diagnosing why we’re not already succeeding.

Remember that your first call with an expert is only the beginning of your relationship with them. Give them commenting access on your google doc, thank them for their input, and ask for their feedback. You know this is working when people spend a lot of time in your doc, when experts debate each other in the comments and generate new insights, and when people ask you if they can share the doc with their peers.

This isn’t a one-off, linear process. It’s messy, and you’ll jump around a lot. You’ll discover big new questions and conflicting evidence that undermine your confidence and makes you feel confused. You’ll get lost down rabbit holes. And you should be prepared to rewrite everything from scratch a few times.

But after a few weeks or months, you’ll have a clear sense for where this field needs to be, a sharp diagnosis for the biggest obstacle blocking progress, a plan of action for overcoming that obstacle, and widespread buy-in from the most powerful and influential stakeholders in the field.

Taking action

This process doesn’t just produce a document. It gives you strong relationships with influential players throughout the field. It gives you situational awareness about what everyone’s doing and why, and what’s blocking them. It makes you a person people turn to when they want to know what’s happening, what matters most, and what to do next.

Then the next phase begins: obsessively communicating the strategy, overseeing implementation, removing obstacles to progress, and updating and refining the strategy as the field evolves and we learn from reality. More details on this in a future blog post.

If you’re working on an important global problem and you don’t know what to do (and seemingly neither does anyone else), this is how you can make yourself useful to a field that needs direction.

Further reading

These blog posts will help you understand what we mean by strategy. I believe it’s worth spending 5+ hours engaging with these.

• Strategy as Problem-Solving – US Army War College

• Good Strategy, Bad Strategy – Richard Rumelt

• What makes a strategy great – Jason Cohen

• Strategic Command – David Petraeus

• Field Building for Population-Level Change – Bridgespan Group

• The Case for a Partnership Between Field Strategists And Philanthropists – Tom Kalil

The long answer depends on what you’re trying to do. Are you trying to figure out how you might contribute your technical skills to AI safety, or get opportunities to do it?

This blog post is written for graduates of the Technical AI Safety course who have spent 30 hours learning about current safety techniques, and the gaps to building safer AI.

What counts as ‘can code’?

I don’t mean you have to have professional software engineering experience. I mean that you at least feel comfortable independently completing a simple coding project.

This means things you should be able to:

• Read and review code

• Debug errors and figure things out when you get stuck

• Write basic functions and loops

If that’s not you yet and you want to do a research / engineering project, you’ll want to build those skills first.

You should probably do a project if…

You’re applying for research and engineering roles, but not getting the roles you want. We’ve spoken to hiring managers from a variety of AI safety organisations including government, non-profits and frontier AI companies, who all tell us the same thing. An excellent project is one of the strongest hiring signals.

Most AI safety orgs are not credentialist, so having a project also puts you on more even footing, even if you don’t have years of professional software engineering or research experience. This is also the reason why there are so many AI safety research fellowships like ARENA, MATS, Pivotal, LASR, PIBBS.

Moreover, doing a project will give you some of the skills for technical interviews when applying.

You’re still figuring out how you want to contribute. While you could (and should!) get others’ takes on what technical work is like, just trying the thing can give you way more information than just reading or talking to people. It can be a cheap test for you to see for yourself what doing this work is like.

Before you devote time to upskilling, see for yourself what that work entails. You’ll surprise yourself by how much you can just jump in and learn as you go, rather than upskill generally.

You’re earlier in building your technical skills. Doing the real thing is one of the most effective ways to learn because it forces you to focus on the actual skills you need. Even if you don’t land a role immediately, you’ll be building your portfolio along the way.

You might not need this if…

You already have a strong portfolio. When I speak to hiring managers, and AI safety researchers and engineers, they recommend just applying because people often underestimate how experienced they are or overestimate how experienced they need to be for the role.

They’ve also said that you’re not penalised for reapplying. Just make sure to highlight what’s changed since your last application. In fact, this is often a strong signal of how high agency you’ve been in upskilling since then.

If you already have a strong portfolio or technical skills, apply first. Do a project later.

This probably isn’t for you if…

You’re doing this because you think it’s the only way to contribute. It’s not.

Especially if you’re several years into your career, you’ve racked up expertise that others are spending months trying to acquire. Leverage that!

Look for areas where you can contribute your unique skills and experience. You can read 80,000 Hours career review or check out our AGI Strategy course to learn about other pathways. There are many high-impact paths that don’t require touching code.

So you’ve decided to do a project

You can start right now by following our Technical AI Safety Project sprint.

You can get a richer experience by applying, where we’ll provide:

• Mentorship: Having someone familiar with AI safety can guide your project direction with how to pick and scope your project, what tools to use, and who to talk to.

• Accountability: Setting goals is hard, sticking to them is even harder. Working on this with someone else will be a major boost!

• Rapid feedback: A point person to review your work as you go, so you can iterate faster.

Whether you follow the guide independently or apply to join a cohort, we’re excited to see what you come up with! Tag @BlueDotImpact on LinkedIn or Twitter with your project.

I’d thought about running an event like this before, but felt daunted. It seemed like a huge lift. I wasn’t even sure women would want to attend.

When I went to one that Monika Jotautaitė hosted, I realised that (1) there’s a LOT of demand for this event and (2) it’s pretty straightforward to organise.

Here’s how you can plan this event in less than 2 hours.

This blog post is written for graduates of BlueDot’s courses.

What you’ll create

When I look around, I don’t see many women in AI safety. I’m not sure why, but what I do know is that these events help create:

• A space where women feel comfortable showing up

• Connections with people navigating similar paths

• Role models whose journeys they can learn from

My hope is that by concentrating women in one place, we make it easier to show up, stay in the field, and maybe even bring friends along.

Two event formats

Here are two that worked well and are simple to organise:

Speed friending

Have people pair up for 7-minute conversations, give people a minute to exchange contacts at the end, then swap over to a new pair. After 6-8 rounds, have an open social.

I recommend placing this timer where everyone can see it.

Talk + socials

Invite someone to give a 20-minute talk with a 20-minute Q&A about something that other women in AI safety might find useful.

The talk encourages attendance and gives people something to discuss during the social.

Who to invite

A rule of thumb is to organise an event that YOU would be excited to attend. That includes deciding who it’s for.

“Women in AI safety” is still a broad category. Maybe you want women already working in the field. Maybe you want women curious about entering. Maybe you want a mix. All are valid — just be intentional.

It can feel painful to tell people no, but thoughtful, considered exclusion is vital to any gathering. (Take it from Priya Parker, an expert in event planning!)

4 steps to bring your event to life

1. Pick a date and time: I find that weekday evenings (6/7pm) tend to work well.

2. Find a space (or go online): If you don’t have access to an events space, reach out to other AI safety orgs to see if you can use their office space. Otherwise, you can run it online using our Zoom account.

3. Submit our events form: We’ll get you set up on our events calendar, which handles invites, feedback forms, reminder emails and guest list. We can also provide a budget for snacks.

4. Start sending out invites: You’ll surprise yourself by how many people are excited by this type of event. Having 10-20 people show up is plenty for great conversations.

It’s that simple

We’d love to see more women in AI safety, and I believe that events like these can help make AI safety feel more approachable. This isn’t the only thing we can do to encourage diversity in the field, but it’s one way to start.

We’re excited to see your event come to life! Fill out the form and let’s make it happen.

• Scale AI safety research

• Build tools for AI safety researchers

• Contribute directly to AI safety research

This guide walks through a project you can complete in <1 week to make your first contribution.

This blog post was written for graduates of BlueDot’s Technical AI Safety course who want to contribute their software engineering skills.

Why do a project?

Projects help you figure out where to apply your skills. Does your experience programming GPUs apply to training infrastructure? Does your agent scaffolding experience translate to evals? You’ll learn more from trying than from months of deliberation.

If you find a gap you can fill, it could lead you to orgs doing that work — or inspire you to start something yourself.

When applying to AI safety roles, this project could also serve as a helpful signal for your skills and motivation. A well-executed project can help demonstrate clear reasoning, good communication, and high agency. I’ve spoken to several hiring managers who made offers or fast-tracked candidates because of excellent projects.

While you could complete a project through programs and fellowships which provide more structure, mentorship and stipends, you could also do it yourself!

You already have what it takes to start. Here’s how.

Getting started

• Block out 20-40 hours in your calendar.

  • If you want to keep going after that, schedule more time later. Right now, focus on finishing something by the end of the week.

• Schedule focused blocks.

  • Aim for 2-4 hour sessions where you can get “stuck in”. The more fragmented your time, the more time you’ll burn context switching.

  • Rope someone in to work on this with you or be accountable to.

• Protect this time.

  • It’s easy to let social events or work meetings eat up your project hours. If this matters to you, treat these blocks like any other important commitment.

  • Make a calendar event for yourself!

• Build a routine.

  • Work at the same time each day if possible.

  • Set a clear intention and put it somewhere you can see it. E.g. “Every day for the next 5 days, I’ll spend 4 hours working on my project at my desk.”

Choose your path

Option 1: Fix open issues in AI safety tools

Contribute to tools that AI safety researchers use every day.

Pick 2-5 good first issues to solve from an open source AI safety repo, like:

• Inspect Evals

• ControlArena

• Inspect

• TransformerLens

• Neuronpedia

• nnsight

• safety-tooling

• Other volunteer projects

• (email me if you know of more! anglilian@bluedot.org)

You can also message the maintainers, join their Discord / Slack communities or just try out the tools to figure out what needs improving.

For example, Anthony Duong looked at the issues on TransformerLens and spent a few weekends making PRs.

Option 2: Replicate and extend a research finding

Get closer to research by reproducing and extending a published result.

The goal is to reproduce and add a small tweak to ONE interesting finding from a paper.

Some ideas for picking a starting point:

• Reviewing the resources from the Technical AI safety course

• Get inspired by past projects from BlueDot or ARENA alumni

• Replicate and extend Anthropic’s alignment faking demo

• Pick an open problem in evals

• Pick an open problem in mech interp

If you want to spend closer to 20 hours on the project, pick papers that have code (and datasets if applicable) available for you to run. Otherwise, expect to spend a lot more time working out how to implement the code.

Replicating a finding from scratch is more feasible for papers like evals or elicitation techniques that don’t require deep ML expertise.

Don’t get too bogged down with trying to make a novel research contribution. It takes months to develop good research taste. Instead, follow where your curiosity leads you.

Then, find the fastest way to get signal on whether this is an idea worth pursuing before running a high volume of tests. Can you prompt the model and see what happens? Can you use a fine-tuning API like TogetherAI or OpenAI?

Remember, this is meant to be a short project. You can always build on this in your next iteration.

You can get compute for your project from providers like RunPod or Hyperbolic, and access open source models via OpenRouter. If funding becomes a constraint, you can apply for a small grant as a BlueDot course graduate.

Ethan Perez has written useful tips for empirical research here.

Option 3: Make research reproducible

Unblock other researchers by fixing what’s broken in the replication process

Many published papers are hard to replicate because the code is buggy, dependencies are missing, or the workflow is unnecessarily painful.

Your goal is to pick a paper with available code, try to run it, and fix whatever breaks or makes it painful to work with.

This could mean:

• Fixing broken code or missing dependencies

• Writing clearer setup instructions or documentation

• Building micro-tooling for repetitive, manual steps (e.g., a script for batch querying LLMs, a config manager for hyperparameters, or a notebook that visualises results)

• Packaging the reproduction in a way that “just works” for the next person

Focus on making it easy for everyone who comes after you to build on it. By the end, you should have opened a PR or created an issue that ideally gets merged!

Write it up!

The biggest mistake people make is treating the write-up as an afterthought. Your work won’t advance AI safety if no one engages with it.

Plan to spend at least one full day writing this up. If you’re spending less, you’re not spending enough.

This includes the different forms of write-ups, like your longer-form blog and the Twitter/X or LinkedIn post that helps with distribution.

As a rule of thumb, you should allocate your writing time proportional to how much total reading time each format will receive. Writing a viral thread deserves as much effort as writing a detailed post.

Why write it up

Your write-up is how you get feedback and find a home for your work. It’s also how others gauge how well you know your stuff and how much you’ve thought it through.

Many of our past graduates have found their co-founders, collaborators, roles and funding opportunities from posting their projects.

You might be thinking: “I can write it up much faster than that” or “I’d rather spend time working on the project”. But if you want your work to reach people, it’s worth communicating well.

Writing clearly demonstrates your understanding. You can’t write clearly about something you don’t fully get.

Think about explaining your own codebase to a new hire. If you’re fumbling through the explanation, you probably don’t understand the architecture as well as you thought. Clear communication is an indicator of understanding, and it takes work to achieve.

How to write it well

Lead with what you did. Don’t bury your insights in walls of text or unnecessary jargon. State it clearly upfront.

Explain why it matters. Motivate why you did this and why anyone should care. People are more likely to read your work (and remember it) when they understand the why.

Keep it simple. The goal is for people to actually read and understand what you’ve done. This is not easy. Writing something short and clear is much harder than rambling on. But that’s exactly why you need to devote real time to it.

Get feedback constantly. Good writing requires iteration. Explain your project to others as you work on it. See if they understand. See if they’re convinced it’s compelling. If not, workshop your idea or delivery.

Don’t wait until the eleventh hour. Many AI safety researchers like Neel Nanda have highlighted how important this is. Start early and keep refining as you go.

Share your work

I know it feels scary to stake your name publicly on what you’ve done. But here’s the thing: your work is far more likely to get drowned out in the noise than criticised. You have to put in a LOT of work to be seen (there’s a whole industry around marketing!). And being seen is exactly what you want.

If you’re working on research, much of the community is on Twitter/X, so focus on making a thread and posting on LessWrong and the Alignment Forum.

Star this project on your GitHub, feature it on your blog (make one if you don’t have one!), post it on LinkedIn and keep talking to people about it.

So what are you waiting for? Let’s get started!

PS: Use/apply to our Technical AI Safety Project sprint.