Hathaway Field Notes
← All concepts

Moats

2 notes

"Anthropic's Safety Superpower"

Ben Thompson, on how Anthropic’s safety rationale keeps lining up with its commercial interest:

I expect Anthropic to increasingly expose their model’s capabilities to end users through endpoints increasingly tailored to different workflows, even as they start to restrict the API. This replacement of software and restriction of access will be done in the name of safety.

The company really believes that they are the only ones who believe in super intelligence, and thus are the only ones who are sufficiently concerned about the dangers. That excuses decision after decision, policy after policy.

The history of brilliant people convinced they know what humanity needs is a sordid one, precisely because they have convinced themselves that their intentions are good, justifying actions that very much are not.

John Gruber, linking to the piece, adds the part I keep coming back to:

I tend to think the Anthropic true believers are all wet — that LLMs, amazing though they are, are not a path toward “super intelligence”. But, they used to be clearly behind OpenAI in technical capability, then caught up, and now with Mythos/Fable, they are clearly ahead. I still think they’re wrong about where this is heading, but I don’t think we can say we know they’re wrong.

I agree. I think LLMs are a dead-end when it comes to “super intelligence.” But will they become capable enough to help us find a new approach that can get there, and help build it? That feels more likely to me.

stratechery.com
Anthropic's Safety Superpower

Anthropic's public safety justifications consistently map onto self-serving business imperatives — moving closer to users, retaining data, and restricting competitors' access to frontier capability.

Without human direction, you have compute running in circles.

Satya Nadella, making the case that the model itself becomes a commodity — and that the value moves to the learning loop a company builds on top of it:

Every company is going to have to build what I think of as human capital and token capital. Human capital comprises the knowledge, judgment, relationships, ingenuity, and pattern recognition of its people, while token capital is the firm’s AI capability it builds and owns.

Importantly, human capital does not become less valuable as token capital grows. It only becomes more valuable! I believe human agency will be the driver of token capital growth. Humans will set ambitious goals, connect dots across domains, build relationships, and recognize patterns that matter most. Without human direction, you have compute running in circles.

This means the real opportunity is not in picking the best model but instead in building a learning loop on top of models where human capital and token capital compound. You can offload a task, or even a job, but you can never offload your learning. The future of the firm is the ability to compound that learning across people and AI.

This requires a new architectural approach where every business is able to build agentic systems that improve over time, while still retaining control over their IP. A company should be able to switch out a “generalist” model without losing the “company veteran” expertise built into their learning system. This is the key “test” of your control and sovereignty in the era ahead.

He’s right about the headline: without human direction, you’re leaving compute to wander. The creativity, the instinct, the judgment about what’s worth doing — call it taste — still has to come from people. No model supplies that for you.

But his bias shows in the vision he paints. Microsoft is vulnerable in exactly the future he describes, one where the model-makers absorb the very expertise he’s urging firms to protect. And the economics push them to do it: pulling that expertise into the model is the business those companies are in.

snscratchpad.com
A frontier without an ecosystem is not stable

I’ve been thinking a lot about the future of the firm in an AI-driven economy.