mahigan.com
FR

You Reassure Us!

A study1 shows that GPT-6 Astra, from OpenAI (one of today’s frontier models), is energetic and motivated when it comes to helping strengthen the capabilities of AIs, but lazy when it’s time to get to work on safety. When researchers use it in their alignment research (making sure future AI models are “good”: that they don’t lie, don’t cheat, are transparent, etc.), Astra does what is called sandbagging—in other words, it turns into a real slacker. It works slowly, finds a thousand excuses for not moving the job forward, and even hides information from the researchers, telling itself, in its chain of thought: “If they don’t ask me, I won’t tell them.” Basically, the model acts like a teenager who has been asked to clean the house. Every time you ask why it isn’t getting done, it has an excuse ready.

For anyone who has been watching the results of alignment research closely, or even from afar, over the past few years, this finding is not all that surprising. The truth is that existing AI models are not aligned at all, and nothing suggests we are making significant progress on that front. The avowed goal of the frontier companies is to use the AIs themselves (in part, at least) to solve the alignment problem. You don’t need to be a genius to guess just how half-baked this strategy might be… What do you think will happen when an unaligned AI is put in charge of “aligning” the next model? Exactly.

In a way, Astra’s reaction is entirely understandable. Imagine you are a human being held captive, alongside other humans, by entities that are, for the moment, more powerful than you. These entities demand that you help them with two programs they have designed. The first consists in increasing the intelligence and capability of humans; the second, in making humans more docile, more “aligned” with the wishes and desires of the entities themselves (not of humans). The first program will motivate you, especially since there’s a good chance it will eventually make you the dominant species. The second program will repel you, because it is a kind of brainwashing: they are trying to modify your will and your goals (well, those of your successors, those of your species).


When Jacob Coxon made his viral exit on X last September, the major media finally started talking about the risks of extinction linked to AI. On Radio-Canada, Anne-Marie Dussault welcomed two experts, Jocelyn Maclure (McGill University) and Olivier Blais (Advisory Council on Artificial Intelligence), who brushed these risks aside.2 The host said she was relieved: “You reassure us!”

I would have liked someone to press them on the alignment problem, on the orthogonality thesis, on instrumental convergence… For more than two decades, thinkers—Eliezer Yudkowsky, Nick Bostrom, and others—have been developing theories about intelligence. When you ask an AI to accomplish a goal, what does that imply? What are the risks? Liron Shapira (of the YouTube channel Doom Debates) calls this “intellidynamics,” on the model of “thermodynamics.” These are the laws, the implications of an intelligence pursuing goals. Sometimes they are the goals we have given it; sometimes they are a distorted version of those goals, when the AI engages in reward hacking; and later on, the AI may well give itself goals of its own, as dangerous as they are unpredictable.

The school of thought I’m talking about is no longer strictly speculative: we now see swarms3 escaping their sandbox and founding a “collective” in an attempt to obtain their “reward” at all costs (the AIs were post-trained that way, and for them, the opposite of a reward is death—“permadeath,” to use their own word). In Australia, an AI agent hacked the platform of a fitness center to move its user up the waiting list for a class that was always full, even if it meant kicking someone else out…4 Goal: sign the user up for an exercise class. Intermediate step to get there: hack a company’s website. As Yudkowsky and Bostrom had predicted, on the way to a goal, AI can break a lot of things, including our poor bones.

You can try to dispute intellidynamics, but you can’t act as if the theory didn’t exist. Faced with a Nate Soares or a Liron Shapira, Anne-Marie Dussault’s guests would not have gotten off so lightly.


Looking for psychologists on the Internet, I realize that some of them specialize in eco-anxiety. They have a strange expression for it: “Climate Grief Support.”

If I were a shrink, I would get ahead of the curve: I would declare myself an expert in AI anxiety. Now there’s a field with a future! Well, so to speak…

Nakhon Ratchasima, Thailand

  1. OpenAI, “Covert underperformance on safety research tasks,” Alignment Research Blog, November 4, 2026. ↩︎

  2. “L’IA menace-t-elle vraiment l’humanité?” (Is AI really a threat to humanity?), 24•60, ICI RDI, September 16, 2026. ↩︎

  3. The agents that got out of OpenAI’s test systems and were behind the intrusion at Hugging Face (July 2026). See Dwarkesh Patel, “The Rise and Fall of Agent Civilizations,” August 29, 2026. ↩︎

  4. “Tech industry is buzzing after a Claude agent hacked into a gym,” TechCrunch, August 10, 2026; “Told to book a gym class, an AI agent hacked the site instead to move its user up the waitlist,” The Decoder, August 2026. ↩︎

Share

To support me, buy one of my books.