AI company Anthropic needs all the good press it can get right now. The company filed for its IPO last month, a move that may or may not doom every other AI company. CoreWeave and Oracle in particular are in hot water, but Anthropic probably isn’t too concerned about that. There’s a stock market offering hurdle to clear first.
The company filed an updated S1/A document as part of its IPO earlier this week. Part of that document, which further shows the financial strain the company is under, also teases a little something to give investors hope. Part of the document suggests the AI outfit is closing in on recursive self-improvement (RSI).
RSI is when an AI model grows advanced enough to make itself more advanced without human involvement. The tricky bit is that, if this happens, the humans involved won’t know about it until after it has taken place. That’s not my assessment. That explanation forms part of the filing document as well.
Conveniently for Anthropic
According to whoever files these things for Anthropic [emphasis mine], “In addition, our increasing use of AI systems to accelerate our own research and development may cause the capabilities of our models to advance more rapidly than our ability to develop, validate, and implement corresponding safety, alignment, and security measures. As a result, we have in the past not identified, and may in the future not identify, a significant acceleration in our rate of development until after it has occurred.”
“For example, our internal estimates and our expectations regarding future models, rather than our current models, indicate that we may imminently meet our Responsible Scaling Policy’s definition of “dramatically accelerated progress attributable to AI.” We have not designated any of our current models as exceeding this threshold, and we continue to assess the underlying data.”
In other words, recursive self-improvement is a thing that might spontaneously occur without Anthropic’s intervention. The engineers at the company could wake up one morning and discover that its internal models have pooped out a golden egg. What that egg looks and tastes like is anyone’s guess. Maybe it’ll revolutionise mathematics. Maybe it’ll build a better egg beater. The company’s hints are… very vague. But just positive-sounding enough, if you squint, to draw in a few extra investors.
Trending
It’s not like AI companies don’t have a history of vague statements with no evidence behind them. OpenAI’s Sam Altman is especially prone to publicly commenting on artificial intelligence capabilities without showing anyone how they might come about. Sam’s tendencies and Anthropic’s statement in its revised filing are the ultimate ‘Trust me, bro.’
Yes, it’s possible that AI companies might reach a point where previously undiscovered capabilities spontaneously manifest in their models. It’s probably happened before. It doesn’t seem like something you’d want to hinge an investment portfolio on, though. The Harry Potter novels have an interesting example of how this situation might work. You might recall Bertie Bott’s Every Flavour Beans. These are magical jellybeans that can (and do) taste like anything. The catch is, you don’t know what you’ll get until you stick one in your mouth.
Anthropic’s tease of rapidly advancing AI capability hinges on Émile Borel’s infinite monkey theorem. Given enough time, enough monkeys hitting keys at random will output something readable. Will it be Homer’s Odyssey? Will it be the blurb on the back of a Rice Krispies box? There’s no way of knowing until it comes out. And that’s supposed to be a reason to think positively of a company that warns that “our historical growth rates are not indicative of future performance or our ability to sustain or forecast future revenue growth.”
Warning lights
It’s worth going over Anthropic’s S1/A filing, even if you’re only reading the ‘Risks’ section. Almost 100 pages of the document, which runs to 340 pages before the financial statements begin, are dedicated to all the things that could go wrong with the company.
Some of those things are an inability to raise capital, the need to continue raising capital, the effect of reputational harm if Anthropic never builds what it says it will, and how teetering debts could wipe it off the map. There’s no way to include every risk in a single sentence, but the mountain of ‘stuff that could go wrong’ is high.
Anthropic (and others) could pull it off. It probably won’t. It’s more likely the company will raise its $100 billion from its IPO, get at or close to its $1.5 to $2 trillion valuation, and poison the well for everybody else in the space if (or when) the stock craters.
This exact scenario played out in China when humanoid robotics company Unitree started strong and then limped away wounded, leading the country’s regulators to slam the brakes on the IPO of every competing robotics company. If this takes place, it won’t be Dario Amodei and his cohort taking it in the neck. It’ll be the investors, many of whom are likely so small that they won’t survive the financial hit this scenario proposes. Counting on infinite monkeys and magic eggs isn’t sound investment policy.
