A LinkedIn poll landed in my feed this week asking why the people building the most powerful AI systems on the planet keep telling us those systems might end civilisation. Six options, one clear pattern in the votes: nobody picked “genuine concern” as the top answer. Regulatory capture and distraction from today’s problems led the field.
That result should worry you more than the extinction warnings themselves, because it tells you something about how your own leadership team is likely to read these statements — with scepticism, sometimes bordering on cynicism. If you’re setting AI strategy for your organisation, you need a sharper read than “they’re scared” or “they’re lying.” The honest answer is messier, and it matters for the decisions you’re making right now about vendors, governance, and where to spend your risk budget.
The warnings are real, and they come from serious people
Start with the fact that this isn’t just marketing theatre. Sam Altman testified before the US Senate in 2023 asking for licensing and oversight of frontier AI. Elon Musk has repeatedly warned of “civilisational destruction”. Geoffrey Hinton left Google specifically so he could speak freely about the risks, telling 60 Minutes that he regrets his life’s work. Hundreds of researchers and executives, Musk among them, signed the Pause Letter calling for a six-month halt on frontier training runs. Altman himself signed the CAIS statement putting AI extinction risk on the same footing as pandemics and nuclear war.
None of that reads like people who think this is nothing. Whatever else is going on, dismiss the concern entirely and you’re ignoring a chorus that includes some of the field’s most technically credible voices.
But self-interest doesn’t need to be a conspiracy to be real
Here’s the part that gets lost in the “genuine concern versus cynical ploy” framing: they’re not mutually exclusive, and treating them as an either/or is exactly why the LinkedIn poll split the way it did. A CEO can believe the technology is dangerous and recognise that being the person who said so first is good for their company’s positioning. Altman gets to be both the industry’s conscience and the person raising billions to build the thing he’s warning about. That’s not hypocrisy in the simple sense — it’s what happens when the incentive structure rewards being loudly cautious while moving as fast as everyone else.
Regulatory capture fits into that same picture, and it’s the piece decision makers should watch most closely. Rules that require expensive safety evaluations, licensing regimes, or compute thresholds sound responsible in principle. In practice, only the labs that already have the largest balance sheets can absorb that compliance cost. A framework OpenAI or Anthropic can afford is a moat against the next well-funded startup trying to compete on a smaller budget. Warning regulators about existential risk and then helping draft the rules that entrench your position isn’t contradictory — it’s a coherent strategy, whether or not it started out that way.
The skeptics aren’t cranks, either
Weigh the warnings against the pushback, because it’s substantive, not fringe. Yann LeCun, Meta’s chief AI scientist until recently, has called extinction-risk talk “absurd”, arguing current systems are nowhere near the general capability the warnings imply. Andrew Ng has been blunter still, describing the extinction narrative as closer to science fiction and PR positioning than engineering reality — and pointing out that it conveniently draws attention away from concrete harms happening today: biased hiring algorithms, job displacement, unresolved copyright disputes, and the energy footprint of training runs that keep getting larger.
That last point is the one I’d put in front of your leadership team. Whatever you believe about long-term risk, the problems your organisation will actually face this year are bias in a model you’ve deployed, an IP dispute over training data, or an audit asking how much energy your AI workloads consume. Those risks are governable now, with policies, testing, and procurement standards you can write this quarter.
What this means for your AI governance right now
If you’re deciding where to put your attention and budget, don’t let the extinction narrative set your agenda for you. It’s a genuine debate among serious people, but it’s also not the risk sitting in your backlog. The practical move is to:
- Separate the debate from your roadmap. Track the frontier-safety discussion for context, but don’t let it substitute for governance of the systems you’re actually running today.
- Read regulation through a competitive lens, not just a safety one. Ask who a proposed rule actually protects before you lobby for or against it.
- Fund the boring risks. Bias testing, data provenance, and cost/energy monitoring won’t make headlines, but they’re the failures that will land on your desk first.
Whatever the real mix of motives behind the warnings, the CEOs are still shipping the models. Your governance can’t wait for them to sort out which reason is the true one.
What’s your read on this — are you adjusting your AI risk register because of the extinction warnings, or ignoring them entirely in favour of the problems already on your plate?