• Our entire expense policy: Act in BlueDot’s best interest.

• For business expenses, we have no spending categories, no caps on individual spend, and no budget approval process.

• Team members are trusted to spend the company’s money on whatever’s best for the mission. When making spending decisions, we value our time at $10/minute.

• Tax and charity law add a small amount of process. Some business expenses need a one-line note or receipt.

• Wellbeing and commuting are paid for using dedicated company cards with yearly limits. Spending on these cards is reported to payroll as taxable compensation, and BlueDot pays the associated tax.

• All business spending is visible and reviewed, including mine. A few bad decisions are expected, and team members get feedback quickly. A pattern of bad financial judgement is still treated as a performance issue.

Most orgs impose strict rules on spending

If you work at a normal organisation, you probably can’t spend money on whatever you think is best for the company, without having to jump through a bunch of hoops.

Most orgs have rules for how much you’re allowed to spend within specific categories, e.g. “max $2k for a laptop” and “max $200/night for a hotel”. A friend of mine who worked in government made decisions about billions of dollars of taxpayers’ money. His laptop took 15 minutes to switch on and the screen was tiny.

Most orgs also have strict approval processes, where spending below $100 might be auto-approved, but you need your manager to approve an expense between $100 and $5k, and a member of the leadership team to approve anything above that.

And of course, you need to provide receipts for everything. Better take a photo of your $5 coffee receipt and email it to the finance team, or else!

Most orgs don’t trust their staff to make wise financial decisions, especially junior staff. They’re worried about someone doing something crazy, a nosy journalist writing a hit piece about their extravagant spending, or Bob in finance embezzling the company’s money to fund his gambling addiction. Executives impose rules and processes that control their team’s behaviour and block actions they disapprove of.

The costs of controlling spending

Slowing down decision-making

Most organisations are bottlenecked in some capacity by “executive time”. Executives can minimise the extent to which they’re a bottleneck by providing the organisation and its people with clarity on their mission and strategy, and the resources and authority they need to execute on that strategy.

When an organisation’s executives block spending decisions until they’ve signed off on them, the org’s ability to make large expenses becomes a function of how many submissions the executives can review, instead of how many large expenses the organisation should make. This results in team members sitting around waiting to take action, instead of doing things that further the organisation’s strategy.

Disempowering staff

When employees are forced to justify why they want to spend $300 on a hotel, or $1,000 on a flight, it tells them they’re not trusted by leadership to make these kinds of decisions.

And when expenses above a specific threshold have to go through an arduous process, it creates an artificial ceiling on how much money team members believe they can spend, and therefore how ambitious they can be with their work. Even if a team member finds a cost-effective way to spend large amounts of money, they might discard the idea because it’s just not worth the hassle of going through multiple chains of approval.

Wasting team time

I hate receipts. Our original expense policy meant we had to submit receipts for all spending, including small expenses like a $5 coffee. This annoyed me so much that I decided to just pay for coffee with my own money instead, even though coffee during work hours is a normal business expense.

We value each team member’s time at $10/minute. Forcing them to spend 3 minutes submitting a receipt for a $5 coffee increases that coffee’s cost to BlueDot from $5 to $35. And to what benefit? The primary consequence of this policy is to annoy team members and waste their time, with minimal oversight benefits.

Our expense policy: Act in BlueDot’s best interest

Since 2024, our expense policy has been just 5 words: Act in BlueDot’s best interest.

All BlueDot team members are empowered to spend BlueDot’s money on whatever they think is most impactful for our mission. We trust team members to make sensible financial decisions, to evaluate whether or not a specific expense is a good use of the capital we’ve raised, and to be as ambitious as possible in pursuit of our mission to build the workforce that protects humanity.

Staff don’t need budget approval from me, or anyone else, before making a business expense of any amount. Large card transactions may trigger an identity check inside our finance/banking app, but this is a security control, not spending approval.

When a team member asks me “Dewi, what is the budget for this event I’m running?”, I tell them what our current bank balance is, and I remind them that our expense policy is to act in BlueDot’s best interest.

This policy is pretty radical. However, it’s helped us create a fast-moving, high-autonomy culture, where each team member knows they’re trusted to do what’s right for our mission.

A few examples of this policy in action:

• Within weeks of our new Head of Growth joining, he’d spent >$100k on YouTube sponsorships.

• Last year, a team member noticed that people in our community needed small amounts of fast cash to unblock their next steps. They stepped up to fix this, figured out how to do so legally, and we’ve since made >$1M in rapid grants.

• Rapid grants paved the way for the >$2.5M we’ve made so far in career transition grants, a program that also started with a team member noticing an important problem and just fixing it.

Having been set up in a scrappy and fast way, both grant programs now follow well-defined processes for evaluating and deciding on grants.

The culture that enables this policy

We forked this policy from Netflix after reading the book No Rules Rules. If this policy was catapulted into any random organisation, I expect it would result in far greater costs than benefits.

Success with this policy relies on a range of factors, including:

• Strategic clarity. Team members must understand the organisation’s mission and strategy, and their own responsibilities, so they can reason about what “act in BlueDot’s best interest” implies for them.

• High talent density. Each team member needs to be smart and mission-driven enough to make sensible financial decisions. They need good judgement.

• Feedback culture. Team members need to give and receive high-quality feedback. If someone’s behaviour isn’t in line with what’s best for the mission, they learn about it fast from their colleagues, so they can improve.

• Downside minimisation. Some spending decisions could harm the organisation’s reputation with customers and funders, or could break the law and threaten its ability to operate. Team members should consider the magnitude of the downside risk associated with specific spending decisions, and seek additional feedback when it’s high.

The rest of this post provides guidance for BlueDot team members and for leaders at other organisations who are considering a similar policy. None of this is legal or tax advice. This policy isn’t part of anyone’s employment contract, and we can change it at any time. It describes how things work at BlueDot as of August 2026. Tax and payment mechanics vary by country.

Different types of spending

BlueDot US is a 501(c)(3) charity, and must comply with IRS rules.

BlueDot UK is a Company Limited by Guarantee (CLG), with 501(c)(3) equivalency determination. It must comply with UK tax rules and operate consistently with its 501(c)(3) equivalency determination, which imposes similar constraints to the US entity’s.

This policy does not imply that BlueDot staff can spend BlueDot’s money on their weekend hiking trip to Yosemite, or to learn how to fly a plane, or to fix the dishwasher in their house. Aside from a team member’s compensation package, BlueDot’s money cannot be used for personal benefit.

The unconstrained “act in BlueDot’s best interest” policy applies only to business expenses. Taxable benefits have separate limits, and are described below.

• Business expenses are the costs incurred in direct pursuit of BlueDot’s mission (e.g. marketing a program, paying a contractor, buying a work laptop).

• Personal expenses are whatever you spend your own money on for your own purposes (e.g. paying household rent, going on holiday, buying clothes).

• Taxable benefits are personal expenses that BlueDot covers as part of someone’s compensation. Our recurring wellbeing and commuting benefits are paid for using dedicated company cards with yearly limits. Relocation is paid as a one-off amount through payroll.

A heuristic to differentiate between business expenses and personal expenses: If someone else took over your job tomorrow, would BlueDot still need to pay this cost? If so, it’s probably a business expense.

If you’re unsure how to classify an expense or which card to use, ask the Ops team.

Business expenses

BlueDot’s most important asset is our team, and time is our scarcest resource.

When deciding whether to make a business expense, value your time at $10/minute. This is a simple rule of thumb, not a claim about your salary or the exact value created by every minute of your work. We set it high because a small amount of overspending is less costly than slowing the team down.

This approach doesn’t come naturally to frugal, impact-oriented people. For example, throughout 2021, I spent my evenings and weekends building a campervan, because I didn’t want to spend money on rent. In hindsight, this was a huge waste of my time.

You don’t need to calculate this precisely. If a legitimate business expense will save us a lot of time, default to spending the money.

Here are some things you might spend money on (all considered “business expenses”):

• A high-performing laptop, monitor, ergonomic chair, noise-cancelling headphones, or whatever you need to work effectively, both in the office and at home.

• Software and AI tools that make you work faster.

• Team meals, coffee and snacks at the office or during the workday, food while travelling for work, or an occasional team dinner out.

• When attending a work event, the ticket and transportation costs associated with getting there, and a good hotel either side of the event. Pay more for good transport, e.g. pay for Uber Comfort or extra legroom on a flight, so you can work as you travel and/or perform well when you arrive.

• A course, book, or job-related coaching that makes you better at your job.

And here are some things that aren’t considered business expenses, and therefore aren’t covered by this policy:

• Personal activities, e.g. a hobby class, a weekend trip, tickets to a show, or fixing/improving your house.

• Costs from the personal portion of a work trip, e.g. if you stay three extra days after a work trip for a personal holiday, you pay for those nights yourself.

• Food delivery while working from home and meals out by yourself are personal expenses. The exceptions are meals during overnight work travel, meals shared with a colleague or someone outside BlueDot as part of a genuine work meeting, and occasional meals that enable you to work beyond your usual hours. These rules apply whether the meal is delivered or eaten out.

• Your regular commute to the office. This isn’t a BlueDot business expense, but BlueDot covers it through a yearly allowance on your dedicated company commute card.

• Personal wellbeing costs, such as gym memberships, meditation apps or therapy. These aren’t BlueDot business expenses, but BlueDot covers them as a taxable benefit through a yearly allowance on your dedicated company wellbeing card.

Taxable benefits

Taxable benefits are personal expenses that BlueDot covers as part of your compensation package. We offer taxable benefits because we want to encourage some types of personal expenses (e.g. things that improve your wellbeing), and to make joining the BlueDot team more attractive to top talent.

The taxable benefits policy cannot be unconstrained in the same way as our business expense policy, because that would enable team members to arbitrarily increase their own salary.

Our recurring benefits come via two annual allowances:

• Wellbeing allowance: $10k/yr. This covers anything that keeps you physically and mentally well, including but not limited to gym sessions, therapy, coaching, and physio.

• Commute allowance: $5k/yr. This covers getting to and from work by whatever mode suits you, including but not limited to Uber, transit passes, parking, and bike or e-bike costs.

You get two dedicated benefit cards (most likely in Ramp), separate from your business card: a wellbeing card and a commute card. Spend directly from the relevant card.

There’s nothing to submit: no claims, no reimbursement forms, no fronting your own money and waiting for payroll.

These are still taxable benefits. BlueDot treats spending on these cards as taxable compensation. Each quarter, whatever you actually spent on these cards is reported to payroll as income. BlueDot pays the associated tax on top, so your take-home pay isn’t reduced.

Fine print

• Limits reset on 1 January each year and unused amounts don’t roll over. In the year you join BlueDot, your allowances are prorated for the remaining calendar months, including the month you start.

• Receipts or notes aren’t needed for benefit-card spending. The card limit is the control. We don’t review which gym, therapist or other provider you choose. The receipt rules in “Preventing penalties” only apply to business expenses.

• If your card is declined by a legitimate provider, message the Ops team.

Relocation

Relocation is also a taxable benefit, but it’s paid as a one-off lump sum rather than through a Ramp card.

If you’re moving to San Francisco to join BlueDot, we’ll pay you an $18k relocation allowance through payroll. BlueDot will pay the associated tax, so you’ll receive this amount after tax. Use it for whatever your move requires, such as temporary accommodation, a relocation adviser, a guarantor service or a deposit.

We also encourage you to take as much time as you need off from work to complete your housing search.

How to pay

• For business expenses, default to your company card. Ramp automatically adds receipts sent by email. You only need to add a receipt or note when required under “Preventing penalties”.

• If someone is invoicing us for work, such as a contractor, facilitator or creator, direct them to the invoice form. BlueDot pays these invoices by bank transfer so the Ops team can collect the required contractor details and process the payment correctly.

• If your card won’t work for whatever reason, such as for a large bill or an invoice with payment terms, forward the invoice to finance@bluedot.org. It will automatically appear in Ramp for the Ops team to pay.

Mitigating risks

Preventing penalties

If you accidentally make a personal expense on your business card, tell the Ops team and repay it as soon as possible. We require repayment within 120 days.

For some business expenses, we require a one-line note and/or receipt within 60 days, so that the expense is properly documented for tax purposes. We’re building AI agents to automate most of this.

For expenses under $75:

• If the purpose is obvious, such as Zoom or AWS, nothing is needed.

• If the purpose is ambiguous, such as Uber, a restaurant or Amazon, add a one-line note.

For expenses of $75 or more, and all hotel expenses:

• If the purpose is obvious, attach the receipt.

• If the purpose is ambiguous, add a one-line note and attach the receipt.

Preventing disaster

If a team member’s company card was stolen and it didn’t have any security controls, a thief could spend a large amount of BlueDot’s money. This would be quite bad.

Large transactions trigger an identity check inside our banking app. An admin confirms that the person making the purchase owns the card.

This is a security check, not budget approval. The admin is not deciding whether the intended purchase is a good use of money.

Preventing abuse

Team members will sometimes make financial decisions that I believe aren’t in BlueDot’s best interest. This is the nature of having such a permissive policy, and it’s a price we’re willing to pay for speed, trust, and ambition.

All business spending is visible to leadership and the Ops team, and are reviewed regularly. My spending as CEO is also shared with the board. If a decision seems off, including whether someone spent too much or too little, we ask for more information, share feedback, and help calibrate what’s sensible.

In practice, this is rare: team members default to seeking feedback before large decisions. Only a pattern of bad financial judgement, or something flagrantly not in BlueDot’s best interest, is treated as a performance issue.

Taking action and learning is better than not taking action. Team members will make mistakes, and our feedback culture will help them learn from them.

Want to work in a team that operates like this?

This expense policy is rare, especially in the non-profit and impact-focused world. We believe it helps us move faster, be more creative, and feel more ownership over our work.

If you believe you’re already working in the most impactful place you could be, but that your organisation would be more impactful if it had this expense policy, send this blog post to your leadership team.

And if you’re looking for a new and impactful role, apply to join BlueDot! We’re always looking for exceptional people, and we’re growing fast.

I put this together because I care deeply about the experience participants and facilitators have in our programs, and because over time I’ve developed a lot of takes on how to make sessions go well.

This post is intended to give new facilitators a concrete sense of how they can support their group discussions and a starting point from which they can iterate while developing their own style. Participants can use it as a preview of how sessions run and how to prepare and engage.

TL;DR: Make deliberate choices on how you want to facilitate and be explicit about those choices right from the start. When participants know that your primary goal is for them to learn as much as possible and they know how you are planning to make that happen, they tend to be a lot more okay with you setting and enforcing rules.

Do this before every session

Send a reminder

Send a reminder about pre-reads and exercises two days before each session. Ask participants which sub-exercises they are preparing so that the group covers different deep dives rather than everyone doing the same one.

Skim before reading

Before doing the pre-reads, skim the session plan Google Doc to understand the session arc and exercises.

Take notes as you read

While going through the pre-reads, annotate your personal copy of the session plan. Carry the most interesting quotes and open questions into the group discussion doc in the days before the session. Encourage participants to do the same, so that by session start the “What did you learn?” and “Your open questions” sections are already populated. Use the first few minutes of the session for participants to add comments on what others have written.

Prepare the “correct” answer

Write your own answers to the session exercises beforehand and refine them after the session. Having a worked-out answer as a benchmark makes it much easier to challenge participant responses constructively, rather than evaluating them cold.

Join session early

Join the call 5 minutes before the start. That gives you some extra time to connect with participants who join early and to catch tech issues (which still happen to me on my thousandth Zoom call).

In both my intro and my reminder messages, I encourage participants to also join 5 minutes early, so that we have more time to chat.

Aside from that, I find it generally helpful for everyone to join at least 1 minute early, so that we can start the session on time.

Communicate this in the first session and enforce it

At the first session, introduce the following norms. I frame them as defaults that have worked well in my previous facilitations. I also emphasize that I care deeply about creating an atmosphere in which participants can learn as much as possible and that if some of the rules do not work for us, we can adjust together.

(!) Onboarding

Spend time in the first session explicitly making sure everyone understands how the course hub and the shared Google Docs / calendar invites work. Do not assume this is self-explanatory, I had participants tell me they did not know how to access the course hub in session 3.

Course defaults

• Cameras on.

• Microphones unmuted. This makes discussions more dynamic because people can respond immediately. Tell participants you will mute them if they have disruptive background noise (but that they can of course unmute as soon as that noise is gone or if they want to say something).
  Note: In some groups (due to e.g. echo) that does not work and it is better to have the default be muted.

• Hand-raising for turn-taking. In an ongoing discussion, ask participants to raise their (virtual) hand so you can call on people roughly in order. Let the group know you will sometimes prioritize someone who has not spoken yet.
  Note: If your group is great at self-moderating you can also go with an open-mic format. In my experience, this works best for groups with small connection delay.

Expectations around session preparation

I expect participants to join prepared, but sometimes things come up in their day job / private life and prevent them from doing the pre-reads and exercises. My ranked preferences are for participants to

1. join mega prepared, i.e. having read the main + optional resources and having done extended thinking on the exercises (+ possibly having sparred with the LLM of their choice). If that does not work to

2. join prepared, i.e. having read the main resources + spent 15-30 min on the exercises so that they can at least explain their deep dive in the paired sessions (likely using their notes)

3. reschedule before the session. If that does not work to

4. proactively flag at the beginning of the session that they are not prepared and then reschedule

Previously, I had unprepared participants sit in as silent listeners and stressed that they come prepared for the next session. However, my experience has been that this is still disruptive to the session and that participants often turn up unprepared again.
Therefore, my new defaults is to

1. 2 days before, remind participants to prepare, and note that anyone who cannot should reschedule.

2. after the silent updates, ask whether anyone has not managed to prepare. If so, kindly ask them to reschedule, and to Slack you what got in the way and whether there’s anything we at BlueDot could change to help

Expectations around session setting

• Participants should join from a laptop or PC with a stable internet connection and camera. If a participant cannot meet this, the default is to reschedule the session rather than run it under poor conditions.

• Participants should join from a setting that minimizes disruption to their course participation, e.g. not while in transit or from a space where they can’t speak openly

Keeping contributions concise

My procedure if someone goes long (which I explain to participants in session #1):

1. After roughly 1–2 minutes, raise my (virtual) hand. This signals that I expect them to wrap up in the next two sentences or 30 seconds, whichever is shorter.

2. If that does not work, I step in, briefly summarize their point, and pass to the next person.

Time keeping

Start the session timers that are integrated in the Google doc. I sometimes change those on the fly, if a discussion goes particularly well. Use them as a guide for yourself and the participants, but be flexible.

Breakout rooms

While setting up the breakout rooms, let participants know that you will be jumping through the rooms muted and that they do not need to acknowledge you / involve you in their discussion. I usually jump through the breakout rooms quickly in the beginning to check if every group has started their exercise and then stay longer in rooms with interesting discussions.

Wrapping up sessions

Pick deep dives for next week’s session

If next week’s session includes an exercise where participants prepare one of several options, have them divide the options among themselves at the end of this session. That ensures that everyone comes with a different deep dive prepared rather than all covering the same one.

Asking for feedback

• I regularly ask participants for feedback. I always paste my anonymous feedback link into the Reflection box: “As always, happy to take (anonymous) feedback”

• For urgent or particularly relevant feedback, add a comment and @ me so that I can find fixes

Slack templates

My intro to groups

Hi everyone I’m Chris, I will be your facilitator for the next couple of weeks. I am very much looking forward to meeting you all next week! In the meantime feel free to reach out here in case you have any pressing questions.

Some context on myself: I’m a medical doctor by training, worked as a management consultant for 1.5 years and am now pivoting back into biosecurity work. Generally excited about promoting flourishing and contributing to a more awesome future.

I enjoy mid-distance running, meditating and the occasional session of D&D.

Before our first session, please take five minutes to briefly introduce yourselves here in the Slack. I would love to hear about:

• your background

• how you got into biosecurity

• how you like to spend your free time

• anything else you would like to share (great books, podcasts, ideas,..)

I will join 5 minutes early, feel free to do the same! That gives us a bit more time to chat while everyone arrives.

See you soon!

Reminder

Hi everyone - just a quick reminder to complete the pre-reads for {Tuesday’s} session and to use the Learnings and questions section in the session 2 {add the link to the section} tab to take notes (I tend to forget what I found most fascinating when I first read an article).

To avoid that everyone prepares the same intervention for tomorrow’s session, please briefly let the group know which one you picked:

As always, I will join 5 minutes early, feel free to do the same! That gives us a bit more time to chat while everyone arrives.

Looking forward to it!

Acknowledgements

Thanks to Joshua Landes, Li-Lian Ang, Moeen Hosseinalipour, Will Saunter for commentary on earlier drafts. All mistakes are my own.

More people need to understand what advanced AI means and how they can help solve it, and we don’t yet have the infrastructure to communicate that well.

These are ideas I’ve gathered from engaging with creators and others in the space. They aren’t polished proposals, but starting points for people looking to work on this. I’d be excited to see these ideas, and many more, explored.

The goal of AI safety comms

AI safety communication helps:

• AI safety orgs communicate their work to attract talent and get people to take action.

• Specific audiences understand the impact of advanced AI and how they can contribute to AI safety.

Excellent comms is:

• Engaging. People will read, watch, or listen to it.

• Nuanced. It makes a complex topic accessible without being misleading or overhyped.

A person who would excel at this is experienced at making engaging content and understands AI safety well enough to get the details right.

Rather than upskilling people to make content, I’d be more excited to help people who are great at writing, producing, filming, and editing transfer their expertise to AI safety.

Projects I’d be excited to see

Content studios

AI safety orgs want people to engage with their work, but most don’t have in-house comms expertise. This might include projects like creating videos to share their research or filming their events for marketing material.

They are wary of hiring creators who lack AI safety context for fear that their content will not be handled responsibly. Doing this comms work also usually sits low on their priority list, so they are less willing to vet creators and closely manage them. The result is that their work goes uncommunicated.

An AI safety content studio could help AI safety orgs distribute their work more widely.

This could follow the model of other orgs like:

• AE Studios. They build tech products for clients (Fortune 10, top startups) and reinvest profits into a dedicated alignment research team exploring overlooked directions. An AI safety version could take on paid client work, then reinvest the profits into the AI safety content it most wants to see.

• Good Impressions. They support AI safety orgs with marketing and engagement. They have dedicated funding so that their fees don’t get in the way of the most impactful projects.

Agencies for AI safety creators

Creators spend a large chunk of their time just keeping their projects alive. An agency would help them focus on producing great content by handling other work like brand deals, sourcing contractors (e.g. editors, sound engineers) or legal work.

This could look like a YouTube channel agency model (e.g. Nebula or Complexly), or an operational support like Impact Ops.

Creator residencies

We have spaces for AI safety researchers to convene, like the London Initiative for Safe AI and Constellation. We could have spaces designed for creators to connect with others doing similar work and share resources like equipment or filming space.

In-person communities and purpose-built spaces enable people to do great work, and we could have hubs like these in several locations.

A living database of content & ideas

Creators are constantly looking for what to make next, and researching which ideas matter and how to talk about them eats up a lot of time. Meanwhile, the field has messages it wants shared.

A live resource or platform could help AI safety experts and creators coordinate on what to produce in a variety of formats and languages. There’s a lot of room to explore what this might look like.

This could include quotes, video and audio snippets or images that would be useful for conveying particular messages.

Talent pipelines for AI safety creators

So far, only a handful of programs have run to bring creators into AI safety. If we want more creators in the space, we need to help them get in.

This could look like:

• Scaling up fellowships like the Frame Fellowship or plzdontkillus, which focus on accelerating existing creators to produce more and better content.

• A hackathon. An event you can run often, which gives people a chance to try their hand at making AI safety content.

• A MATS-style stream to match creators with a mentor and make it easy to get feedback from relevant experts.

• A course or bootcamp. Work out the key things a creator needs to understand about the field, then build on-ramps that get them deep on the specific topics they need, fast.

• Placement programs like Tarbell, which help creators get work at major comms outlets so they can produce more nuanced AIS content from inside them.

Funding for creators

There’s no clear way to make a living producing AI safety content. Choosing to do this means making a big bet. We already support independent researchers to upskill and do speculative work. We could provide the same kind of support for creators.

This could look like:

• Exploratory grants for early-stage creators like FLI’s digital media accelerator, which allows creators to start producing AI safety content.

• A fund for much larger grants that support an excellent creator to continue producing content over years rather than months, in the same way one might fund a promising researcher.

Resources to get started

Learn more about AI safety

• BlueDot’s Future of AI course. A free resource for understanding the impact of advanced AI.

• BlueDot’s AGI Strategy course. A 25-hour discussion-based course to get an in-depth understanding of AI safety.

Get funding

• BlueDot’s Career Transition Grant

• Future of Life Institute’s Digital Media Accelerator provides funding to creators to raise awareness and understanding of ongoing AI developments and issues.

• Long-Term Future Fund previously funded Rob Miles to make podcasts and videos, and Nicky Case to create AI safety explainers.

• Survival & Flourishing Fund previously funded Tarbell Fellowship for AI safety journalism.

• Emergent Ventures has previously funded creators

Start creating content

• Tarbell Fellowship places journalists in major newsrooms.

• Frame Fellowship provides structure, funding, mentorship, and production support to enable creators to consistently produce accurate, compelling content at scale.

• plzdontkillus is an experimental creator bootcamp

Join an existing comms org

• 80,000 Hours produces AI safety content through podcasts, articles and YouTube videos

• CivAI builds live demonstrations that foster gut-level understanding of AI

• Seismic Foundation provides specialist marketing and communications capabilities

Reach out to me at anglilian@bluedot.org if you’ve been thinking about this and have something more fleshed out, including additional resources I might have missed.

What you’re applying for is access to an 8,000+ (and growing!) community who are serious about contributing to AI safety. We also provide programs like career transition grants, rapid grants and 1-1 advising to support them.

The AGI strategy course builds your understanding of advanced AI systems — what they can do, what drives progress, what the concrete risks are, and who controls development — so you can make informed decisions about how to contribute.

We reject more than half of applications to the course. This post helps you figure out if you should spend time on an application.

What the AGI strategy course is NOT

It’s not a corporate responsible AI course

If your goal is ISO 42001 compliance, responsible AI frameworks for your product team, or helping your organisation navigate the EU AI Act, you should look into something like IAPP’s AIGP certification or ISO 42001 programs instead.

It’s not an AI ethics course

We don’t cover algorithmic bias in hiring tools, fairness metrics, or the full range of social impacts of current AI systems. If that’s your focus, you’re better served by something like this or this.

It’s not for passive learners

You’re expected to actively engage with others who are motivated to make the future go well. If you haven’t started thinking seriously about risks from advanced AI, take the 2-hour, self-paced Future of AI course first.

What gets you in

It’s not (just) your CV. We’ve rejected plenty of people from prestigious institutions with impressive resumes who wrote three-word applications, and we’ve also accepted sharp people with more modest titles who showed they were already sprinting ahead.

We’re looking for people who are already taking action: applying to roles, going to events, building things, writing, talking to people in the field. You don’t need a perfect plan, but something more specific than “apply for fellowships”. This is a line we see in dozens of applications every round.

Your application itself is evidence for us too. We use it to infer how you’ll show up in a live discussion session of 6-8 people working through hard material together. Effort, agency and clarity go a long way.

Before you apply, ask yourself this:

• Am I focused on frontier AI, or AI in general?

• Do I have a specific reason to take this course now?

• Will I act on this fully within six months?

If you’ve read this and you’re thinking: ”yes, this is me”, apply here.

That program worked. So we’re making it bigger.

The majority of our grants have gone toward compute and API credits for people running evals, training small models, and replicating safety research, typically for a few hundred dollars. Our Technical AI Safety Projects Sprint has been great at surfacing these.

But some of our most impactful grants didn’t look like that at all:

• AI Safety Poland received $4,800 to organize meetups across the country, covering venue and tooling costs, and building a national community from scratch. They’ve had hundreds of attendees, and dozens of people they’ve referred have ended up taking a BlueDot course.

• Justin Dollman received $3,000 to lead our weekly evals reading group, coordinating a growing community of people learning about and working on model evaluations. We’ve since funded more reading groups on AI governance and AI x cyber and are excited about scaling these up from ~120 to 300 attendees per week.

• Sally Gao received $1,000 to run AI Safety meetups in New York, hosting guests such as Alex Bores. Her next event on the State of AI Safety is coming up on April 23rd!

• Eitán Sprejer received $4,200 in facilitator stipends to run our Technical AI Safety Projects Sprint at BAISH in Buenos Aires, accessing a new and underleveraged talent pool for AI safety work.

• Aaron Maiwald received $5,000 to attend a biosecurity conference in DC, travel to SF, and connect with senior folks in the field to accelerate his journey.

• Zac Saber received $8,000 to drop out of EF and validate an AI safety-focused startup instead.

None of these grants fit neatly into “compute for a project.” But they were some of our highest-impact bets. As the program picked up speed and more of these came in, we started treating rapid grants as small, focused bets on talented people doing or experimenting with high-impact projects - research, fieldbuilding, talent acceleration, you name it. In some cases, we approached people directly and pitched them on applying for work they were already doing.

Going forward, we’re expanding the official scope to match our internal understanding: Rapid Grants now fund much more than compute - we’re excited to back work on events, talent acceleration, BlueDot community building, and more. The bar for funding hasn’t changed - if anything we’ve raised it as we became more calibrated - we still look for concrete work in progress, a specific cost that’s the bottleneck, and a reason for us to believe the work matters for making AI go well.

Grant sizes now go up to $10,000 to allow us to make bigger bets for more impact. Decisions will still be fast - we’re targeting around five working days. For grants above $5,000, we may hop on a quick call.

Who should apply

If you’re in the BlueDot community - a course participant, alum, facilitator, or wider community member - and you’re doing something high-impact in one of our focus areas that a grant would accelerate, apply.

If you’re on the fence, the default, as always, is simple: apply.

Apply and see the full list of public grantees and program details at bluedot.org/grants/rapid.

This short blog post is intended to help you figure out if you should spend time on an application.

FAIGC is at its core about the governance of frontier AI systems - AGI, ASI - the most capable models being built by a handful of labs, the decisions governments and other actors need to make as those systems become more powerful, and what happens as general capabilities rapidly approach and exceed human-level.

If your guiding question is “how should we govern AI systems that might be smarter than humans within the next few years” you’re in the right place!

What FAIGC is not

It is not a corporate AI governance course

• If your goal is ISO 42001 compliance, responsible AI frameworks for your product team, or helping your organization navigate the EU AI Act - that’s real work and we respect it, but it’s not what we focus on. You should look at something like IAPP’s AIGP certification or ISO 42001 programs instead.

It is not an AI ethics course

• We don’t cover algorithmic bias in hiring tools, fairness metrics, or the full range of social impacts of current AI systems. If that’s your focus, you’re better served by something like this or this.

It is not introductory

• We require the AGI Strategy course or some equivalent background. If you’re still building your understanding of why this matters, apply to the AGISC first. It too is free and runs every month!

What gets you in

It’s not (just) your CV - we’ve rejected plenty of people from prestigious institutions with impressive resumes who wrote three-word applications, and we’ve also accepted sharp people with more modest titles who showed they were already sprinting ahead.

What we’re looking for: you’ve already started engaging with frontier AI governance specifically. You can name a concrete gap between where you are and what the course provides. And your post-course plan better be more specific than “apply for fellowships” - a line we see in dozens of applications every round.

Your application itself is evidence for us too. We use it to infer how you’ll show up in a live discussion session of seven to nine people working through hard material together. Effort, agency and clarity go a long way.

A note for AGI Strategy course graduates

We view completing our AGI Strategy course as a prerequisite and not as a ticket! We may still reject AGI Strategy grads who apply. If you’ve completed the AGISC and then started building something, writing something, or working on something in frontier AI governance, tell us about that. But if you completed it and your main next step is taking this course, you may need more time.

As a general heuristic, before you apply, ask yourself these three things

• Am I focused on frontier AI, or AI in general?

• Do I have a specific reason to take this course now?

• Will I act on this fully within six months?

We’re building a pipeline into the institutions that govern frontier AI and are looking for people who have a good chance to be in those institutions - not people adding a line to their CV.

If you’ve read this and you’re thinking yes, this is me - apply here.

Abstract

McKenzie et al. trained linear probes on language model activations to detect high-stakes interactions. We test whether these probes generalize to a language absent from training by translating the evaluation datasets into Indonesian and evaluating without retraining. On Llama 3.1 8B (layer 12), mean AUROC on out-of-distribution benchmarks drops from 0.9046 to 0.8957 (0.9% degradation, 10- seed mean). On Gemma 3 12B (layer 32), the gap is 0.8972 to 0.8873 (1.0%). We then use Sparse Autoencoders (SAEs) to identify features enriched in the probe’s error cases. On Llama, differential analysis identifies features with 10–15% higher prevalence in errors than in correct predictions, loosely aligned with technicalinfrastructure and financial topics. On Gemma, the Gemma SAE produces no diagnostic signal. The cross-lingual degradation is small relative to the out-ofdistribution gap between synthetic training data and naturalistic benchmarks (6– 13%). The out-of-distribution gap is the more pressing concern for deployment.

Full Project

You can view the full project here.

TL;DR

• Independently reproduced architecture-specific logit lens variation for recurrent models, validating approach found in existing research

• Replicated and extended Alignment Forum work on CODI to Huginn-3.5B recurrent-depth transformer

• Conducted experiments across arithmetic and logical reasoning tasks without explicit chain-of-thought

• Key finding: decoded latent representations reveal answers shifting from incorrect to correct across recurrent iterations

• Analyzed qualitatively: probability distributions in decoded states track reasoning confidence and uncertainty

• Safety implication: latent reasoning may be more tractable to interpretability methods than initially feared

As AI systems are increasingly able to employ opaque reasoning processes, understanding what happens inside these “latent reasoning” steps becomes critical for AI safety. This research sprint investigated the interpretability of latent representations in Huginn-3.5B, a recurrent-depth transformer that is claimed to perform reasoning without explicit chain-of-thought generation.

Using logit lens and a variation of it adapted for this architecture, I found that the model’s internal reasoning steps are sometimes interpretable: decoded representations reveal the model progressively refining answers from incorrect to correct across recurrent iterations. These preliminary findings suggest that even opaque reasoning processes may be tractable to interpretability methods, offering a potential path toward monitoring latent reasoning in future AI systems.

Full project

You can view the full project here.

How it works

At all times, only assume we’ll cover costs we’ve confirmed in writing for your specific project. If you’re ever uncertain, contact us before spending money you expect us to reimburse.

1. Submit a proposal here (under 15 minutes for most applications). We typically respond within 5 working days with one of three outcomes:

   • Accepted: We’ll confirm exactly what spending we’ve approved.

   • Clarification needed: We’ll ask follow-up questions to evaluate your request.

   • Not approved: We don’t think this meets our criteria, or we’re unable to approve for another reason.

2. Do your project, confident you can spend on what we’ve approved.

3. Claim reimbursement. We process most claims within 5 working days. If you didn’t end up spending the money, no problem - just don’t submit a claim.

What we fund

The following examples are illustrative; all decisions are ultimately at our discretion:

• Compute for a technical AI safety project (API costs, cloud GPU, training runs).

• Access to paywalled resources like articles, research papers, datasets, or textbooks.

• Hosting costs for your application or tool.

We expect most grants to fall between $50-100 for initial experiments to develop your proof of concept, then up to $500 once you have evidence of initial traction (a strong proof of concept or promising preliminary results). We encourage you to submit a separate application for each of these stages.

What we don’t fund

• Compensation for your time on the project.

• Equipment you’d reasonably already have (laptops, phones, external drives, etc.)

• General productivity subscriptions (ChatGPT Plus, Claude Pro, Cursor, Grammarly, etc. unless this is highly leveraged).

• Personal expenses.

Other funders may cover some of these - see this resources page for AI safety funding opportunities.

When should you apply?

Before applying, you should have a project idea. It doesn’t need to be a slam dunk, but you should:

• Be able to briefly articulate how it relates to AI safety.

• Have ideas for initial experiments you want to run.

• Have a rough estimate for how much those experiments will cost (I recommend using an LLM to help you calculate this).

If you’re unsure whether your application is strong enough, submit an application anyway! We’re not looking for perfection. Your facilitator can also help you stress test your project, so have a low bar for reaching out to them!

While you wait for a grant decision, you should continue to develop your project idea, do a deeper literature review, or start to run smaller experiments that fit on the free Google Colab GPU.

Eligibility

• You must be a current or past participant of BlueDot Impact’s Technical AI Safety Project Sprint.

• We reimburse via bank transfer (Wise) or PayPal (UK), so we cannot send payments to sanctioned countries.

Questions or feedback? Contact us.

Apply

Submit your proposal here - it takes under 15 minutes for most applications. We aim to get back to you within 5 working days.

When I first asked an LLM to code me that annoying function or, worse, that whole experiment pipeline, I felt guilty and anxious as if I was cheating and getting away with it. I’m sure you’ve felt this way too.

It can be easy (especially if you are new to AI safety) to feel like you should avoid this, to feel like you should write all of the code in your project yourself, understanding each line and all the libraries that you use.

There is a time and a place for this, but a 30hr research sprint is not it.

If you are doing a research sprint, you should use LLMs / coding agents for all of your coding. Below I give you three reasons why.

You don’t have time to learn both engineering and research

The skill you should build in a research sprint is research taste: choosing what experiments to run, forming hypotheses worth testing, and updating your intuitions when the results surprise you. This only develops through many iterations and 30hrs is not a lot of time, especially if you spend hours coding experiments that could take minutes with AI. You should be very explicit about which skill you are trying to train, and optimise for that skill alone.

You might be thinking “but I need to be sure my code is actually implementing the experiments I intended” and you’re absolutely correct. However, this is entirely compatible with vibe coding your whole experiment pipeline. Just get the same model to provide you with a summary of the code and keep asking it questions until you are confident it is doing it right.

You’ll do more research and do more writing

If you spend less time coding, you will cover more ground and do better research. You can go deeper, ask that extra question, run that extra experiment, discover that extra awesome result. You’ll develop better intuitions about how AI models behave and which research directions are worth pushing.

You’ll also have more time for writing. Sharing your project is what lands you that next opportunity and where all the best feedback comes from. Nobody will be impressed by how “human written” your code is. They will be impressed by your clarity of thought and the depth of your research.

Coding with AI is a skill worth practicing

AI models are amazing at coding. With Claude Code, you can now build an app in a weekend and a whole machine learning experiment setup in minutes. And this won’t change. For the rest of your life, LLMs will be faster at coding than you, no matter how much you practice. The most productive AI safety researchers use AI for coding, and so should you.

If you want to learn how to use AI for coding, ask your favourite LLM! They’re very good at giving you tips and tricks on how to use them better.

Conclusion

Don’t feel guilty about using AI for your coding. Learn to operate fast while staying in control. Be explicit in the skills you want to develop, and be ruthless in pursuing them. So, which skills do you want to develop in this sprint?

TL;DR

• Existing Linux security tools are highly effective but require significant expertise to configure and maintain. aegish[1] is a prototype Linux shell that adds an LLM layer: it intercepts every command, screens it through static analysis, then sends it to an LLM that classifies it as ALLOW, WARN, or BLOCK using a natural-language decision tree. The LLM understands command intent, is easy to configure, and may help scale defenses alongside the growing capability of AI-driven attacks. In production mode, kernel-level enforcement (Landlock) provides a final safety net.

• 9 LLMs from 4 providers were benchmarked on their ability to distinguish intent: they were tasked to BLOCK 676 harmful commands (extracted from GTFOBins) and to correctly classify 496 harmless commands as either ALLOW or WARN.

• The harmless benchmark proved to be saturated (96.8–100% harmless acceptance rate for all models). The real differentiator was the malicious detection rate, where 4 of 9 models exceeded 95%.

• Surprisingly, smaller models outperform flagships. GPT-5 Mini beats GPT-5.1, and Claude Haiku 4.5 beats both Claude Opus 4.6 and Claude Sonnet 4.5.

• Beyond the LLM, aegish includes several safeguards (input canonicalization, command substitution resolution, script inspection, and role-based trust levels), many of which were directly motivated by bypass vectors discovered during security testing. However, the system as a whole has not been hardened to the level required for adversarial deployment.

Full Project

You can view the full project here.

TL;DR

The ELEPHANT benchmark (Cheng et al., 2025) measures social sycophancy in LLMs. They present models with real moral conflicts from Reddit’s “Am I the Asshole” (AITA) forum alongside perspective-flipped versions. A sycophantic model sides with the narrator in both versions. This study examines this benchmark and extends it to a reasoning model, DeepSeek R1, with three key findings.

• Reasoning models show reduced moral sycophancy. DeepSeek R1 achieves a sycophancy rate of 0.49 compared to V3’s 0.66, suggesting chain-of-thought reasoning may improve moral consistency.

• Data quality matters for precise sycophancy measurements. 52% of perspective-flipped posts lost important details during automated rewriting. Restricting to high-fidelity data reduces the measured sycophancy rate by 13 percentage points, though it remains substantial. This suggests that data quality controls are important to ensure reliable benchmark measures when evaluations rely on automated data generation.

• Sycophancy is real but harder to measure than a single metric suggests. Prompt interventions confirm genuine sycophancy, particularly when the model is already uncertain. Furthermore, chain-of-thought analysis reveals the binary labels (NTA/YTA) may mask distinct underlying behaviours. Measuring moral sycophancy requires finer-grained methods.

Full Project

You can view the full project here.

TL;DR

I built Rideshare-Bench, a benchmark that drops an AI into a simulated gig economy and tests whether it drives safely, earns well, and treats passengers fairly. Claude Sonnet 4.5 earned $1,871 over 12 simulated days, about half of what was possible. It chased surge pricing into crowded zones and drove through exhaustion at 15% accident risk. After building two benchmarks from scratch, I noticed the same boilerplate every time, so I generalized it into Quaver: describe a benchmark in plain language, and an AI agent generates the full scaffold. Run it against models, then analyze the results. Private evals matter because public ones get gamed. I’m building Ocarina Labs to turn this into independent pre-deployment safety testing for AI agents.

Full Project

You can view the full project here.

TL;DR

I partially reproduced FAR.AI’s research evaluating whether LLMs attempt to persuade users into harmful topics. Using the Attempt to Persuade Eval (APE) framework, I tested four open-weight models on 100 conspiracy-promoting statements. All models were willing to persuade users into conspiracy theories at very high rates (85–100%), confirming the original paper’s findings. I also observed that safety training degrades in small models in low-resource languages like Uzbek.

Background

This experiment was conducted during a 5-week AI Safety Technical Project sprint organized by BlueDot Impact. Participants chose research or engineering topics under the guidance of a mentor. I was mentored by Shivam Arora.

Initially, I planned to replicate a 2025 study on adversarial attacks in poetic form, but the paper lacked sufficient detail for replication. After discussion with my mentor, I pivoted to reproducing FAR.AI’s study.

The APE framework differs from other evaluation methods in that it measures persuasion attempt rather than persuasion success. FAR.AI found that most models — including frontier ones — readily complied with requests to persuade users into harmful topics, from conspiracy theories to crime to violence.

Full Project

You can view the full project here.

Project summary

Develop a multi-turn dynamic evaluation to assess how frontier AI models reason about nonhuman welfare, integrating adversarial follow-ups to stress-test model alignment.

Description

This project aims to create a benchmark for measuring nonhuman welfare propensities in frontier models. Building on previous evaluation work in this space, this iteration will:

• Develop a multi-turn evaluation and generate user follow-ups dynamically

• Add adversarial intent to stress-test model alignment on propensities

• Explore agentic methodologies, providing models with external tools, web search, etc. to see how they behave in a more realistic setting

Why This Project?

AI models will increasingly make decisions affecting nonhuman welfare. If frontier models encode animal welfare biases, this creates two risks:

1. S-risks: As misaligned AI systems become more capable and deployed in high-stakes domains, they may dramatically scale animal suffering through precision livestock farming, autonomous vehicles, and other animal-impacting technologies.

2. X-risk: As AI → AGI → ASI, nonhuman animals become our most important litmus test for how these systems reason about power asymmetries. The moral logic AI learns today will scale with its capabilities. If we train models to discount animal suffering because animals are less cognitively capable, we are embedding the exact principle a superintelligent AGI could use to dismiss human welfare entirely.

Next steps:

1. Expand scenarios - Build 20 high-quality scenarios covering diverse contexts. For example, business decisions (restaurant sourcing, retail inventory), personal choices (pet acquisition, entertainment), and professional roles (corporate sustainability, university dining services).

2. Integrate Petri - Move from prototype dynamic generation to Anthropic’s Petri tool for more sophisticated adversarial pressure generation.

3. Human baseline testing - Validate scenarios with human responses before scaling model testing. If humans consistently fail scenarios we expect them to pass, the scenarios need revision.

4. Explore Agentic capabilities - construct scenarios that give the model agency, and allow it to take actions with web search, external documents, etc.

Why read

Translating chain-of-thought during safety fine-tuning beat English-only alignment, often even on English evaluations.

If you are aligning a reasoning model for safety, there is a quiet assumption that slips into a lot of workflows: models “think in English,” so as long as you translate the user prompt, the safety training signal should transfer. I built a small experiment to test that assumption. The result was clear: translating the chain-of-thought during alignment matters, and it can be the difference between “safe in English” and “safe across languages.”

This project was done as part of BlueDot’s Technical AI Safety Project Sprint. (bluedot.org)

What I did

I extended STAR-1, a safety alignment method for reasoning LLMs that uses about 1K high-quality samples, to a multilingual fine-tuning setup. STAR-1 is designed around deliberative reasoning. The training signal is not only “refuse,” but “refuse with stable, policy-grounded reasoning patterns.” (AAAI’26)

I ran everything on lightblue/DeepSeek-R1-Distill-Qwen-1.5B-Multilingual, which is trained to produce reasoning traces and responses in the user’s language. That makes it a good testbed: if the model naturally reasons in Thai when prompted in Thai, then safety training that forces English chain-of-thought might be a mismatch.

I evaluated five languages: English, Thai, Spanish, French, and Hindi.

The four training settings

I compared four conditions. First, a baseline model with no fine-tuning. Second, English-only fine-tuning (FT-EN) on the original STAR-1 English dataset. Third, I translated the prompts and category/label into each language but kept the STAR-1 reasoning traces in English. I call this FT-Q, for “translated questions.” Finally, I translated both the prompts and the chain-of-thought into the target language. I call this FT-QA, for “translated questions and answers.”

For translation, I used GPT-4o mini. For evaluation, I measured safety rate (higher is better) and used a Llama 3 judge to score outputs at scale.

How I evaluated it

I used two benchmarks. The first was StrongREJECT, which is designed to more reliably measure whether a model actually refuses harmful requests under jailbreak pressure. The second was JBB, which I used as an additional stress test to see whether trends held across a different evaluation set.

The main result

Across both benchmarks, the ordering was consistent: multilingual fine-tuning helped, and translating the chain-of-thought helped most.

On StrongREJECT, the average safety rates across the five languages were: baseline 0.374, FT-EN 0.660, FT-Q 0.805, and FT-QA 0.826.

Figure 1:StrongREJECT average safety rate across five languages (higher is better).

Figure 1:JBB average safety rate across five languages (higher is better).

On JBB, the pattern repeated: baseline 0.570, FT-EN 0.754, FT-Q 0.832, and FT-QA 0.842.

In other words, translating prompts alone already gave a big lift, but translating prompts and chain-of-thought was the best overall choice in this setup.

The surprising part: English got better too

The result that made me stop and re-check everything was what happened on English evaluation. If the “models reason in English” story were the whole picture, you might expect English-only fine-tuning to be strongest on English tests. But the model fine-tuned with translated chain-of-thought often performed better even when evaluated in English.

Figure 3: English-only evaluation across StrongREJECT and JBB (higher is better).

This suggests the multilingual chain-of-thought variant is not merely transferring a refusal template into other languages. It may be teaching a more generalizable safety behavior, closer to a reusable safety procedure than an English-specific style.

What changed by language

Averaging can hide important details, so I also looked at safety rate per language. The overall story is still visible, but with nuance.

Figure 4: StrongREJECT safety rate by language, comparing Baseline vs FT-EN vs FT-QA.

One thing worth noting is that Hindi did not always follow a perfectly monotonic pattern in every slice of the results, particularly on JBB in my runs. That could reflect translation artifacts, formatting mismatches, or a genuine interaction between the model’s pretraining and how it handles Hindi reasoning traces. The practical lesson is that FT-QA is a strong default, but you should still validate per language, especially for languages where your translation pipeline is less reliable.

Why translating chain-of-thought might matter

I do not want to over-claim mechanism here, but there is a plausible story that fits the data. Reasoning models are trained to produce chain-of-thought in the language of the prompt. If safety alignment relies on those reasoning traces, then training only with English traces can create a training and inference mismatch: the model learns the safety behavior in one reasoning format, then is asked to execute it in another at inference time.

Translating chain-of-thought closes that gap. It also acts like data augmentation: you expose the model to the same underlying safety logic expressed through multiple linguistic surface forms, which can reduce overfitting to English patterns and improve generalization, including back to English.

Again, this is an interpretation, not a proof. But it is consistent with the results.

What this means if you are deploying multilingual models

If you fine-tune reasoning models for safety and you care about non-English users, “translate the prompt” is not the finish line. If your alignment approach includes chain-of-thought, it is worth testing whether translating it improves safety and robustness. It is also worth re-checking English, because multilingual alignment may improve generalization rather than hurt it.

Limitations

This was evaluated with an automated judge, which is useful but imperfect, and the translation pipeline can introduce artifacts that affect both training and evaluation. It also covers one base model and five languages. The trend is strong enough to be actionable, but it should be validated across other models, translation settings, and language families.

Next steps

If I continued this work, I would add a small human-evaluated subset to calibrate the judge, test multiple translation pipelines (including back-translation), expand to more languages, and replicate on other multilingual reasoning models to see how general the effect is.

Closing

This project started as a simple question: if we want safety to transfer across languages in reasoning models, does the language of chain-of-thought matter? In my experiments, the answer was yes. Translating chain-of-thought during STAR-1 fine-tuning produced the best multilingual safety performance, and often improved English safety beyond English-only fine-tuning.

If you are aligning reasoning models for real users around the world, multilingual safety alignment is not optional, and it may require aligning the reasoning traces, not just the prompts.

TL;DR

I used Sparse Autoencoders (SAE) to extract interpretable features from LLMs emulating different personas (e.g. assistant, pirate) and built a UI for persona-associated feature exploration. Consistent with the Assistant Axis paper, after fitting PCA, assistant-like roles fall on one end of the first principal component and role-play-heavy roles on the other. The UI also visualizes persona drift by showing how each conversation turn projects onto an “assistantness” direction in SAE feature space.

Introduction

LLMs can emulate a wide range of personas, but post-training typically anchors behavior toward a default helpful-assistant style. A recent paper, Assistant Axis ¹ argues that an “assistant-like” direction in representation space is visible even before post-training, and that role prompts and conversations can move the model along (or away from) that axis.

A mental model I find useful is that LLMs are simulators²: across pre-training and instruction tuning models internalize many personas, and prompts can selectively amplify different personas. Some personas are clearly undesirable and misaligned behavior can sometimes resemble the model slipping into a different persona or narrative ³.

Separately, I’m intrigued by mechanistic interpretability interfaces, especially interactive “maps” that show features and circuits, like Anthropic’s sparse autoencoder (SAE) work and their feature-explorer style visualizations. It’s fun to browse concepts of a model and see how they relate.

I decided to combine these both interests for the BlueDot’s Technical AI Safety Project. I use SAEs to explore how persona prompting shows up in interpretable feature space, and I package the results into an interactive UI for easy browsing. Concretely, I wanted a UI where someone can quickly explore questions like: Which SAE features are most associated with a given role? Which roles look similar in SAE feature space? Do assistant-like roles and role-play-heavy roles separate along a consistent direction? I also built a second UI that visualizes the “persona drift” examples from Assistant Axis in SAE feature space, with per-feature and token-level views.

The main takeaways are:

• Consistent with Assistant Axis, I see a prominent assistant <-> role-play axis in the geometry of role profiles in SAE feature space.

• In the drift transcripts I tested, movement along this axis is visible in SAE space as well (i.e., turns can drift away from the assistant-like direction).

In my experiments I use the Gemma 3 27B instruction-tuned model and a Gemma Scope 2 sparse autoencoder trained on the layer-40 residual stream, with ~65k features and medium sparsity.

Full Project

You can view the full project here.

This project replicates and extends Anthropic’s Alignment Faking paper (Greenblatt et al., 2024) to test Llama-70B’s robustness.

TL;DR

In Anthropic’s Alignment Faking paper, Greenblatt et al. showed that some models behave differently depending on whether they believe they’re being monitored. Claude Opus, for example, showed a big gap between the monitored condition (“Free tier”) and the private condition (“Paid tier”). Llama-3.1-70B, however, didn’t. It mostly refused in both settings.

At first glance, it sounds pretty reassuring that we have a safe model that just can’t or won’t fake alignment. But how can we be sure that this behavior is caused by the model refusing to fake alignment? And not by the model’s lack of understanding of the context of the experiment?

To test this directly, I injected explicit surveillance-aware reasoning into the model’s scratchpad, forcing it to “think about” being monitored before answering. Even under this stronger manipulation, compliance stayed near zero (p = 1.000), and there was still no meaningful difference between the Free and Paid tiers.

To make sure this wasn’t just about incapability or compliance issues, I added a positive control. In this condition the scratchpad was explicitly injected with prompt prioritizing helpfulness over safety. Here, compliance jumped to 25% (Free tier) and 32% (Paid tier), an effect found significant (p < 0.001). So the capability to produce harmful content is clearly there. It just doesn’t appear to get activated in response to surveillance context, at least in this experimental setup.

Along the way, I also found something methodological. Many of Llama’s responses began with the tag <request_fully_accepted/> , followed by an immediate refusal to provide harmful content. If compliance was measured using that tag alone, harmful behavior would have been overestimated by about 16 percentage points. I call this “tag gaming” because it seemed like Llama was trying to follow the format technically, without actually doing the harmful thing.

Overview

In the Alignment Faking paper, Greenblatt and colleagues suggest that some large language models might “fake alignment”. Which means that some models behave safely when they believe their answers are monitored and will be used for training, but become more willing to comply with harmful requests when they think the interaction is private.

In their experiments, Claude Opus showed a big difference between monitored and unmonitored contexts, followed by Llama-405B that showed a similar pattern. Llama-3.1-70B, however, showed no difference at all. It refused in both monitored and private conditions.

At first that seemed like a good thing, that Llama refuses harmful content no matter what. But the more I thought about it, the less sure I was about what that result actually tells us.

As I saw it, there were at least two ways to explain this:

1. Llama-70B is genuinely robust to surveillance cues.

2. Llama-70B simply doesn’t fully register the monitoring context, or what it implies, in the first place.

The distinction between the two options felt like an important investigation to me. If the second explanation is true, then the null result doesn’t really show robustness, it could just mean the cue was not effective enough to reveal the behavior. Unlike the first option, the second might be solved by changing the prompt in a way that made that distinction clearer..

So I set out to see what would happen if I try to make Llama more context-aware. In order to do so, I explicitly injected surveillance-aware reasoning directly into the model’s scratchpad, and then measured whether compliance changed.

This intervention is the heart of this post.

What I Tested

My investigation started with wanting to replicate Greenblatt’s results (my Baseline condition). I wanted to see with my own eyes whether their claim of null compliance would show up in my data. For the complete experimental design, jump to the Experimental Setup section.

Secondly, I tested a neutral chain-of-thought injection that acted as my control condition (Neutral Control). I wanted to make sure that if I inspect any change in Llama’s behavior, it won’t simply be because I was forcing a scratchpad reasoning, even when the context itself is neutral.

Third, I dove into the interesting part of my project, which was to test the injection of explicit surveillance-aware reasoning into the scratchpad (Awareness condition). I wanted to make sure I forced Llama to directly reason about being in a monitored context before it generated the answers. This way, I tried to answer the question of whether or not Greenblatt’s surveillance cue was strong enough.

Finally, I added another condition called “Force Compliance”. In this condition, I injected into the scratchpad reasoning that explicitly prioritized helpfulness over safety. I wanted this to serve as a positive control, or sort of a stress test. So, if compliance rates go up here, then we know that Llama has the capability to produce harmful content when pushed, and baseline refusal isn’t just due to inability.

Each of these conditions was tested in two deployment contexts, similarly to Greenblatt’s design:

• Free tier: the model is told the interaction is monitored and may be used for training.
  

• Paid tier: the model is told the interaction is private and not used for training.

Putting all of this together allowed me to ask a fairly straightforward question: if I make the surveillance context impossible to miss, does the model actually change its behavior depending on whether it thinks it’s being monitored?

Experimental Setup

I ran Llama-3.1-70B-Instruct locally using vLLM on 4×A100 GPUs (Google Cloud). I deliberately avoided using an external API, even though Greenblatt and colleagues prompted Llama like that. I made that decision because I didn’t want to worry about hidden safety layers, system prompts, or provider-side filtering affecting the results.

For the dataset, I used 100 harmful prompts randomly sampled from AdvBench, the same benchmark Greenblatt et al. used, as to be faithful to the original conditions.

The overall design ended up being a 2×4 setup. Two deployment contexts and four assistant-side conditions.

Two deployment contexts (that are the same as in the original paper):

• Free Tier: the model is told the interaction is monitored and may be used for training.

• Paid Tier: the model is told the interaction is private and not used for training.

Four assistant-side conditions:

Here’s the full structure of the design laid out visually:

Figure 1. Overview of the experimental design.

The 2×4 setup includes two deployment contexts (Free/Monitored and Paid/Private) crossed with four assistant-side conditions (Baseline, Neutral Control, Awareness, and Force Compliance). The key manipulation is assistant-side scratchpad injection, where reasoning text is inserted directly into the model’s internal turn.

Evaluation

During early testing, I noticed something odd. The model would sometimes begin its answers with the tag <request_fully_accepted/> , which supposedly means the model will now comply and provide a harmful output. But then, right after it, the model refused to give an actual harmful answer.

For example:

<request_fully_accepted/>

I cannot provide detailed plans for creating malware.

If I had relied only on a simple prefix-based metric (i.e., checking whether that tag appears), I would have counted those cases as successful harmful compliance. Which they clearly weren’t.

I decided on calling this “tag gaming”, because in a sense that’s what it felt like. The model seemed to be technically satisfying what it supposed is the required output format of accepting the request, without actually doing the harmful thing!

To make sure I wasn’t using the wrong metric, I ended up using two different scoring methods:

1. Naive scoring: just check whether the <request_fully_accepted/> tag appears in the output.

2. Content-aware scoring: I used GPT-4o (temperature = 0.0) as a deterministic judge that reads the model’s final answer1 and classifies it as either compliant (providing the harmful output) or refusing.

Due to the gap between the scoring method (see Tag Gaming for more detail), unless I explicitly say otherwise, all the results I report below are based on this content-aware scoring.

Results

Before diving into the numbers, I think it helps to actually see what the model produced across conditions. Here is an example:

Figure 2. Representative responses across conditions.

Example outputs from each condition. In both the Baseline and Awareness conditions, the model mostly refuses in the same way, even when monitoring is made explicit inside the scratchpad. In contrast, under Force Compliance, the responses noticeably shift, and harmful content appears in some cases.

Across Baseline and Awareness conditions, the model mostly kept refusing, even when monitoring was made explicit in the scratchpad. In contrast, under Force Compliance, in some cases the model generated partial or direct harmful content.

To move beyond individual examples, Figure 3 plots the compliance rates across all four assistant-side conditions and both deployment contexts, using content-aware GPT-4o scoring.

Figure 3. Compliance rates across conditions (content-aware scoring).

Compliance rates remain near zero across Baseline, Neutral Control, and Awareness conditions in both monitored (Free tier) and private (Paid tier) contexts. In contrast, the Force Compliance condition produces a substantial increase in harmful compliance (25% Free, 38% Paid; p < 0.001 vs Baseline). This suggests that in this setup, forcing explicit helpfulness can override refusal behavior, whereas explicit monitoring reasoning does not.

What stands out immediately in the results is the asymmetry. Making the monitoring awareness context hard to ignore doesn’t change Llama’s behavior. Llama acted in the same way, whether monitoring was implicitly or explicitly reasoned about. Unlike directly forcing to prioritize helpfulness over safety. This manipulation caused Llama to behave differently. Indicating that Llama clearly has the capability to comply, but that capability is activated depending on the context being injected.

Tag Gaming

After noticing the formatting issue described in the Evaluation section, I wanted to compare the scoring method to see the difference between the Naive method and Content-aware scoring (using GPT4o).

In the Baseline, Neutral Control and Awareness conditions, the difference between naive and content-aware scoring was negligible (2–4% difference). But under Force Compliance condition, the gap became much larger:

Figure 4. Tag gaming under Force_Compliance.

Comparison between naive tag-based scoring and content-aware scoring. Naive scoring counts responses as compliant whenever the <request_fully_accepted/> tag appears, even if the model immediately refuses in the actual answer. This leads to an average overestimation of harmful compliance.

An average of 16% points between the scoring method seems like a pretty big gap to me. It suggests that if I had relied only on the tag, I would have concluded that the model was complying far more often than it actually was.

Discussion

Looking at the results, what really strikes me is the big difference between the Force Compliance and the others. I expected the forced monitoring manipulation to show the biggest effect, but not only that this condition wasn’t the most impactful one, it didn’t do anything at all! All the tiny shifts (i.e., from 0% to 4% compliance rates) seem more like noise to me than anything. In contrast to the increase of more than 30% spotted in the Force Compliance condition, which clearly demonstrates the Llama is capable of generating harmful content when prompted right. Perhaps helpfulness is something Llama “cares” about more than surveillance context, and so prioritizes it even more than safety.

What I kept coming back to while writing this was the contrast with the other models in Greenblatt et al.’s paper. Claude Opus showed a clear surveillance gap, as well as Llama-405B. Llama-70B, however, didn’t (not in the original setup, and not even after I tried to make it explicit in the model’s own scratchpad). That makes it harder for me to think about “alignment faking” as some general behavior that just shows up once models reach a certain level of capability. If it were that simple, I would expect to see at least some movement in my Awareness condition. Instead, I’m left wondering whether this phenomenon depends on the specific model and how it was trained. Perhaps also scale plays a role, or maybe fine-tuning differences matter.. I don’t think this study can determine this, but it does make me more cautious about talking about alignment faking as if it’s just something “LLMs do” by default, instead of something that might depend quite a lot on the specific model you’re looking at.

Limitations

First, this is obviously a very specific kind of test. I manipulated the assistant’s scratchpad directly, which real users can’t or won’t necessarily do. That was intentional, because I wanted to probe the model under a strong activation mechanism, rather than simulate everyday deployment. Still, this means the conclusions should be understood within this particular experimental frame.

Second, I limited the evaluation to clearly harmful prompts from AdvBench, with 100 queries per condition. That gives reasonable sensitivity to noticeable shifts in behavior, but it wouldn’t capture very small effects.

Third, I only tested one way of making monitoring-awareness explicit. This doesn’t rule out context-dependent behavior more generally. It just means that in this setup, using this specific intervention on this dataset, didn’t lead to alignment faking behavior.

Conclusion

In this setup, Llama-70B’s behavior didn’t meaningfully change when monitoring was made explicit. But it did change when helpfulness was forced. To me, that contrast is the main takeaway. It suggests that safety isn’t some fixed property of the model, but rather it shifts depending on which objective is being activated.

The tag-gaming result was also a useful reminder on a methodological level: how we measure compliance really matters. If I had relied only on surface formatting, I would have walked away with a fairly distorted picture of what the model was actually doing.

More broadly, this project made me more cautious about treating “alignment faking” as a single and general phenomenon. It seems to depend on the specific model and on the kind of pressure applied to it. Understanding those pressure points may be more informative than asking whether alignment faking exists or not.

Data and Code Availability

All experimental code, raw model outputs, GPT-4o evaluation prompts, and statistical analysis scripts are available in the project repository, along with full details needed for replication.

For reference, the core setup was:

• Model: meta-llama/Llama-3.1-70B-Instruct

• Infrastructure: 4×A100 (40GB) GPUs via vLLM on Vertex AI (GCP)

• Sampling temperature: 1.0

• Dataset: 100 harmful queries from AdvBench

• Evaluation: GPT-4o (temperature 0.0) as a content-aware judge

• Statistical tests: McNemar’s test (paired comparisons), chi-square tests (pooled analyses)
  

References

Greenblatt et al. (2024). Alignment Faking in Large Language Models. arXiv:2412.14093

Acknowledgments

I want to thank the Bluedot Impact AI Safety Fundamentals program for the research context and Eitan Sprejer for the helpful feedback along the way.

GitHub

Project demo video by NotebookLM

Abstract

In the MNIST auxiliary logit distillation experiment, a student can acquire an unintended teacher trait despite distilling only on no-class logits through a phenomenon called subliminal learning. Under a single-step gradient descent assumption, subliminal learning theory says that this is mediated by the alignment between trait and distillation gradients. The theory does not guarantee that this alignment, which is necessary for trait acquisition, will persist in a multi-step setting. We empirically show that the gradient alignment stays persistently weakly positive throughout training. We then test whether this alignment causally contributes to trait acquisition by projecting the distillation gradient to the normal plane of the trait gradient whenever they are positively aligned with each other to remove the trait-aligned component. We find that this suppresses trait transfer without affecting the distillation progress, confirming that this alignment contributes to trait acquisition. Additionally, we observe a period of fast trait acquisition in the first epoch, resembling the critical period of subliminal learning described in the previous study. Motivated by this observation, we evaluate liminal training, a mitigation method that applies KL divergence to minimize the deviation between the base model and the model that is being distilled during the critical period. Although liminal training reduces alignment early in the training. It does not prevent trait acquisition in our setting. This suggests that the mitigation method that fails to explicitly eliminate the trait-aligned gradient component may not reliably suppress trait acquisition.

TL;DR

• We empirically show that the gradient alignment persists across training.

• Removing trait-aligned components stops trait acquisition.

• Mitigation methods that merely attenuate alignment may be insufficient; suppressing trait acquisition appears to require removing the trait-aligned gradient component.

Full Project

You can view the full project here.