How I Learned to Love Superintelligence

Anthropic announces discreetly, in the update to its IPO documents, that it may be about to meet its own definition of “dramatically accelerated progress attributable to AI.” In other words, we would be on the threshold of recursive self-improvement (RSI). People in the field of AI safety, including Daniel Kokotajlo of the AI Futures Project (AIFP), lead author of AI 2027, are aghast: “We at AIFP are spooked by this information.”

Meanwhile, an experiment conducted at MIT shows that an OpenAI model, GPT-6 Astra, can adjust the weights of another AI model, without training.

Nothing there to bring down my level of AI anxiety.

---

To the common run of mortals, all this sounds like science fiction. And yet it is our reality, here, now. Recursive self-improvement isn’t magic, but science. Another name for the same thing: automating AI research. Why wouldn’t intelligences, as they grow ever finer, manage to do what researchers at frontier AI companies do? That has been the bet from the start, and that’s why Anthropic invested so heavily in programming skills; why code fell before everything else. While OpenAI was scattering itself (with its video tools and its erotic velleities), Anthropic kept its focus on the Holy Grail: recursive self-improvement. It looks as if Dario Amodei’s company is about to win the race—that is to say, to kill everyone.

Yesterday, I said that the Artificial Revolution might well reach its conclusion in five years. Even I, writing those words, had trouble believing it, but today’s news points that way. Daniel Kokotajlo, again: “For example if we naively optimize the AIFM parameters to achieve 2x acceleration over baseline progress in Jan 2027, we get superintelligence sooner than depicted in AI 2027.” Never mind the jargon: what Kokotajlo is saying is that, if Anthropic is telling the truth, superintelligence (an AI that outperforms all humans in all domains) is at our door. In the forecasting scenario AI 2027, published in 2025, superintelligence arrived at the end of 2027. If Anthropic’s warning is borne out, it could come even sooner.

---

I never would have believed that the (probable) end of the world would go so unnoticed. I am beginning to think it will take a catastrophe: at the very least a large-scale failure of the power grid or the Internet, and perhaps loss of human life. We are living suspended.

The U.S. midterm elections feed hope, just a tiny bit. But the president has been bought by the billionaires, he has a veto over bills, and he alone can sit down with Xi Jinping and sign an international treaty to stop the suicidal arms race.

When I talk about this with my mother or with certain people, they say as a joke: “Yeah, real cheery, what you’re telling me!” It’s crazy: we are living in the movie Don’t Look Up. “AI is going to kill us all? Talk about killing the mood!”

Add to that the character of Trump, a raving madman who shouts that concerns about AI safety are a hoax, who keeps repeating “Whoever wins AI, wins!”, who bellows “Let ’er rip!”, and suddenly it’s Dr. Strangelove who appears. Stanley Kubrick understood, long before Adam McKay, just how comical the end of the world was going to be. The full title of the film is Dr. Strangelove or: How I Learned to Stop Worrying and Love the Bomb.

That should be the subtitle of our age: How I Learned to Stop Worrying and Love Superintelligence.