"Anthropic's Safety Superpower"

Ben Thompson, on how Anthropic’s safety rationale keeps lining up with its commercial interest:

I expect Anthropic to increasingly expose their model’s capabilities to end users through endpoints increasingly tailored to different workflows, even as they start to restrict the API. This replacement of software and restriction of access will be done in the name of safety.

The company really believes that they are the only ones who believe in super intelligence, and thus are the only ones who are sufficiently concerned about the dangers. That excuses decision after decision, policy after policy.

The history of brilliant people convinced they know what humanity needs is a sordid one, precisely because they have convinced themselves that their intentions are good, justifying actions that very much are not.

John Gruber, linking to the piece, adds the part I keep coming back to:

I tend to think the Anthropic true believers are all wet — that LLMs, amazing though they are, are not a path toward “super intelligence”. But, they used to be clearly behind OpenAI in technical capability, then caught up, and now with Mythos/Fable, they are clearly ahead. I still think they’re wrong about where this is heading, but I don’t think we can say we know they’re wrong.

I agree. I think LLMs are a dead-end when it comes to “super intelligence.” But will they become capable enough to help us find a new approach that can get there, and help build it? That feels more likely to me.