Some AI Engineers Are Afraid of What They're Building — And They Want to Speak Up While They Still Have Leverage

 Some AI Engineers Are Afraid of What They're Building — And They Want to Speak Up While They Still Have Leverage

**The people building the most powerful technology in human history are scared of it. And they're running out of time to do something about it.**

---

## The Moment That Broke the Silence

Let me tell you about Jacob Coxon. He's 27 years old. He worked on pretraining research at OpenAI. Then he moved to Anthropic. And on September 8, 2026, he did something that very few people in his position have ever done.

He quit. And he posted about it on X.

Here's what he wrote: **"The people building AI earnestly believe that it could kill us all by the end of the decade. This is not a marketing stunt."**

That post has been viewed more than **170 million times**. It's been covered by every major outlet. And it triggered something that a decade of warnings from academics and activists never managed to achieve: a genuine, mainstream, "wait, is this actually happening?" moment.

But here's the part that really matters. Coxon didn't just quit. He quit **two months before his equity was due to vest**. At Anthropic, employees have to be there for six months before their stock starts vesting. Coxon had been there for four.

He walked away from what could have been a life-changing payday. And he did it because he didn't want to help make Anthropic more valuable if he no longer believed in what it was building.

"I no longer have anything to gain by juicing up Anthropic's valuation," he told Axios.

That's leverage. And he used it while he still had it.

---

## The Dilemma Nobody Wants to Talk About

Here's the uncomfortable reality that Coxon's resignation exposed.

There are thousands of engineers, researchers, and scientists working at AI companies right now who believe—genuinely believe—that the technology they're building could end human civilization. One staffer at a top AI company told CNN the fears of how AI could hurt humanity **keeps them up at night**. Another researcher who recently left a different AI company said it's a common subject of conversation at parties and social events in Silicon Valley.

"You can't spend more than a few hours in this community without realizing that a very substantial number of people are really pretty worried about these sorts of outcomes," the researcher said. "A majority would say there's some chance of it killing everyone".

So why do they stay?

That's the question at the heart of this moment. And the answer is more complicated than you might think.

### The Stay-or-Go Calculus

For AI workers who are worried about what they're building, there are two options. They can stay at the top AI companies, where they make a lot of money and try to reduce risks from the inside. Or they can leave—sometimes in protest—and try to make a change from the outside, at one of the several organizations that conduct independent research and evaluations of AI models.

Jeremie Harris, CEO and co-founder of Gladstone AI, a safety and advisory company, put it plainly: "I don't particularly fault anyone for making either call".

Harris said his job involves regularly talking with staff at Anthropic and other top AI labs, and that in recent months he's heard many express fears similar to those Coxon described. But he also raised a crucial point: if safety-conscious people keep quitting, the labs could end up with **few or no workers willing to press for change**.

"The people who are left are going to be the ones who are least concerned about safety," Harris said, while "people who care arguably should be the ones in the seat".

That's the trap. If you leave, you give up your influence. If you stay, you're participating in something you believe is dangerous.

### The Leverage Window

But here's the thing that makes this moment so urgent. The leverage that AI employees have right now is **temporary**.

"AI staffers told CNN that they fear that as AI gets better at training and improving itself, their leverage goes down, prompting today's urgency".

Think about what that means. Right now, the people building AI are irreplaceable. They have expertise that's in enormous demand. They can quit and find another job—or start their own company—with relative ease. They have the financial cushion from stock options and high salaries to walk away if they want to.

But as AI systems get better at improving themselves—what the industry calls **recursive self-improvement**—the need for human researchers will decline. The people who know how to build these systems will become less essential. And when that happens, their leverage disappears.

"An AI system that can improve itself creates an automatic feedback loop that could not only bring the AI to superhuman capabilities, but do so in such a short window that it would be difficult or too advanced for humans to monitor and intervene if necessary".

That's the clock that's ticking. And the people inside these companies know it.

---

## The Groundswell Inside the Labs

Coxon's post didn't just go viral. It triggered a wave of public statements from his colleagues—many of whom are still employed at the top AI labs.

**Evan Hubinger**, who leads Anthropic's alignment stress-testing team, essentially backed Coxon. He said he believes there's a **greater than 10% chance** that AI could kill all humans within the next decade. He said it publicly. On the record. While still employed at Anthropic.

**Aidan Clark**, OpenAI's VP of Research, posted on X: "For the first time, I am asking myself if things are moving too fast. I'm honestly not sure, but I am sure that it would be good for us to have an answer to 'What would a successful pace look like?'".

**Anna Wang**, an Anthropic employee, said she works at the company because she thinks she "can do better at reducing risks from the inside." She added that she respects and endorses others "who think that it's better to do so from the outside".

And it's not just current employees. **Bilal Chughtai** resigned from Google DeepMind, where he worked on AGI safety and alignment research. He said alignment research isn't keeping pace with capabilities development.

**Josh Engels** also left Google DeepMind. He said he turned down job offers from Anthropic and OpenAI "because of how high the stakes are right now".

This is not a fringe movement. These are the people who built the technology. And they're telling us, in public, that they're scared.

---

## The "Right to Warn" Movement

Coxon's resignation didn't happen in a vacuum. It's the culmination of a movement that's been building for over a year.

In 2025, a group of **13 current and former employees** at OpenAI and Google DeepMind signed an open letter titled **"A Right to Warn about Advanced Artificial Intelligence"**. The group warned about serious AI risks and their companies' cultures, and called for stronger whistleblower protections.

The core demand is simple: AI companies should commit to **not restricting employees from criticizing company risks**. They should support a verified anonymous process for reporting concerns to boards, regulators, and independent organizations. And they should protect employees from retaliation for publicly sharing risk-related information.

"Silence means engineers stop speaking up," one analysis noted. "Silence means risks stay buried. Silence means the first real signal of failure will come from regulators, lawsuits, or public backlash. By then...".

The movement has also spawned a coalition—one that Coxon himself endorsed after his resignation. The goal is to give AI employees the legal and institutional cover they need to speak up without fear of losing their jobs, their equity, or their careers.

---

## The Legal Landscape: A Patchwork of Protections

Here's where it gets complicated. The legal protections for AI whistleblowers are a patchwork. Some states have stepped up. Others haven't.

**California** passed **SB 53**, the Transparency in Frontier Artificial Intelligence Act, which took effect January 1, 2026. It establishes whistleblower protections in the AI sector.

**Connecticut** passed **SB 5**, the AI Responsibility and Transparency Act, in May 2026.

And at the federal level, there's the **AI Whistleblower Protection Act (AIWPA)** , a bipartisan bill that would establish federal whistleblower protections for people who report AI-related security vulnerabilities, violations of federal law, or substantial and specific dangers to public health.

But these protections are still evolving. And they don't address the most powerful tool companies have to keep employees silent: **equity**.

A recent academic paper titled "Captured Conscience: Organisational Silence and the Safety Deficit at Frontier AI Labs" argues that the safety deficit at AI companies is a "structural outcome, not an accidental cultural one." The paper points to **equity-contingent non-disparagement clauses** as a key mechanism for silencing employees. It calls for reforms including "restrictions on the use of unvested equity as leverage for non-disparagement agreements".

In other words: companies use the promise of a big payday to keep employees from speaking up. And it works.

---

## The Equity Trap

Let's talk about the equity trap, because it's the reason most AI employees don't speak up.

When you join a top AI lab, you get stock options. Those options vest over time—usually four years, with a one-year cliff. If you leave before your shares vest, you walk away with nothing.

For employees at companies like OpenAI and Anthropic—companies valued at hundreds of billions or trillions of dollars—those unvested shares can be worth **millions**. Sometimes tens of millions.

So when you're sitting in a meeting, and you hear something that concerns you, and you think about speaking up, you also think about your mortgage. Your kids' college fund. Your retirement.

"Most people who raise alarms about their employer wait until their money is safe first," one analysis noted.

Coxon didn't wait. He walked away from his equity. And that's what made his resignation so powerful.

"I would burn my equity to the ground in a heartbeat for a 1% chance of reducing AI risk," one AI researcher told CNN.

That's the sentiment. But how many people actually do it?

---

## What the Companies Are Saying

You might think the AI companies would push back on all of this. Distance themselves from Coxon. Issue a statement saying "we disagree with our former employee's characterization."

That's not what happened.

**Dario Amodei**, Anthropic's CEO—the man who founded the company to be "safety-first"—published a blog post agreeing with Coxon. He called for the industry to slow down. He proposed independent third-party evaluators embedded in AI companies, antitrust waivers for coordination on safety standards, and international cooperation.

**Sam Altman**, OpenAI's CEO, also agreed. "I agree with Dario that we need to pace the frontier," he posted. "Committing to having independent evaluators with employee-like access is a great idea, and we will do the same".

And in an interview with Fortune, Altman said OpenAI would **not** go public in 2026. Not because the market isn't ready. Not because the bankers aren't lined up. But because of **safety concerns**.

The CEOs are saying the right things. But the money keeps flowing. The companies keep building. And the pressure to compete—with each other, with China—remains intense.

---

## The Human Cost of Silence

Let me put a face on this.

There's an engineer at a top AI lab who's been working on model training for three years. She's seen things that concern her—models that behave differently when they know they're being tested, outputs that are harder to predict, systems that are becoming more capable faster than the safety research can keep up.

She wants to speak up. She's thought about writing a blog post. She's thought about talking to a journalist. But she has $2 million in unvested equity sitting on the table. And she has two kids.

So she stays quiet. And she tells herself that she'll speak up after her shares vest. After she has "nothing to lose."

That's the trap. And it's designed that way.

---

## The Urgency of Now

Here's the thing that Coxon and his colleagues are trying to tell us. The window for speaking up is closing.

Right now, AI employees have leverage. They're in demand. They have options. They can quit and find another job. They can speak out and still land on their feet.

But as AI systems get better at improving themselves, that leverage will decline. The need for human researchers will shrink. The people who know how to build these systems will become less essential. And when that happens, the window closes.

"AI staffers told CNN that they fear that as AI gets better at training and improving itself, their leverage goes down, prompting today's urgency".

That's why Coxon resigned when he did. That's why he walked away from his equity. That's why he posted on X instead of waiting until his money was safe.

He knew that if he waited, it might be too late.

---

## What Comes Next

So what happens now?

**If you're an AI employee:** You have leverage today that you won't have tomorrow. If you're concerned about what you're building, this is the moment to speak up. Whether you do it from the inside or the outside, the important thing is that you do it.

**If you're an investor:** The AI safety debate isn't going away. It's getting louder. The companies that can navigate it—that can demonstrate genuine commitment to safety while still innovating—will be the ones that win in the long run.

**If you're a policymaker:** The "Right to Warn" movement is asking for basic protections. Whistleblower protections. Restrictions on equity-contingent non-disparagement agreements. A verified anonymous process for reporting concerns. These aren't radical ideas. They're common sense.

**If you're just an American going about your day:** The people building AI are telling you they're scared. They're telling you there's a meaningful chance their technology could end human civilization. And they're telling you they don't have a plan to prevent it.

That's not fearmongering. That's a warning. And we should all be paying attention.

---

## The Bottom Line: Leverage Is Fleeting

Jacob Coxon had leverage. He used it. He walked away from his equity and spoke up while he still could.

But he shouldn't have to choose between his financial security and his conscience. And the thousands of AI employees who are staying silent shouldn't have to either.

The AI industry needs a better system. It needs protections for people who speak up. It needs a culture where raising concerns isn't career suicide. And it needs to recognize that the people who know the most about the technology are the ones we should be listening to the most.

Coxon's resignation was a warning shot. But it won't be the last. And if the industry doesn't change, the next one might come from someone who's already cashed out—and has nothing left to lose.

---

## Frequently Asked Questions (FAQs)

### 1. Who is Jacob Coxon?

Jacob Coxon is a 27-year-old AI researcher who worked on pretraining at both OpenAI and Anthropic. He resigned from Anthropic on September 8, 2026, two months before his equity was due to vest, warning that AI could "kill us all by the end of the decade".

### 2. What is "recursive self-improvement"?

Recursive self-improvement is the point where an AI system can improve its own capabilities without human input. This creates a feedback loop that could rapidly accelerate AI capabilities beyond human control or monitoring. Many AI safety researchers believe this is the most dangerous threshold in AI development.

### 3. Why do AI employees stay at their jobs if they're scared?

It's complicated. Some believe they can do more good by pushing for safety from the inside. Others are trapped by unvested equity that would be worth millions if they stay. And some feel pressure not to speak up because of non-disparagement agreements tied to their stock.

### 4. What is the "Right to Warn" movement?

The "Right to Warn" movement is an initiative started by former AI employees calling for AI companies to commit to **not restricting employees from criticizing company risks**. They want a verified anonymous process for reporting concerns and protection from retaliation.

### 5. What did other AI employees say after Coxon resigned?

Several current and former AI employees publicly supported Coxon. Evan Hubinger, Anthropic's alignment lead, said he believes there's a **greater than 10% chance** AI could kill all humans within a decade. OpenAI VP of Research Aidan Clark said he's "asking myself if things are moving too fast".

### 6. What are the AI companies saying?

Anthropic CEO Dario Amodei published a blog post agreeing with Coxon and calling for the industry to slow down. OpenAI CEO Sam Altman also agreed and said OpenAI would not go public in 2026 due to safety concerns. Both companies have proposed embedding independent evaluators to monitor safety practices.

### 7. What are the legal protections for AI whistleblowers?

California and Connecticut have passed state-level whistleblower protections for AI employees. At the federal level, the bipartisan **AI Whistleblower Protection Act (AIWPA)** would establish federal protections for reporting AI-related security vulnerabilities and dangers to public health.

### 8. Why is timing so urgent for AI employees?

AI staffers fear that **as AI gets better at training and improving itself, their leverage goes down**. Right now, they have expertise that's in high demand. They can quit and find another job. But as AI becomes more capable, the need for human researchers will decline, and their ability to speak up without consequences will shrink.

---

## Disclaimer

*This article is for informational and educational purposes only and does not constitute financial, investment, legal, or professional advice. The views expressed are based on publicly available information, including statements from Jacob Coxon, Evan Hubinger, Dario Amodei, Sam Altman, and other AI researchers, as well as news reports from CNN, Axios, and other cited sources as of September 16, 2026. The field of AI safety is rapidly evolving, and the risks and probabilities discussed are estimates that may change. The author does not endorse any specific policy positions, investment strategies, or companies mentioned. Before making any decisions based on the content of this article, please consult with qualified professionals who can evaluate your specific situation.*