Anthropic, the AI startup known for its Claude chatbot and its doomer CEO Dario Amodei, has filed for an IPO that could value the company at up to $2 trillion. The company’s IPO filing devotes nearly 100 pages to the dangers its own technology could pose.
CNBC reports that the filing lays out risks ranging from lawsuits and regulatory crackdowns to the possibility that Anthropic could lose control over its own AI models. Anthropic frames the stakes in sweeping terms, writing that if “powerful AI is directed at solving humanity’s hardest problems and the risks are managed responsibly, the potential is extraordinary.”
Anthropic CEO Dario Amodei has been at the forefront of AI doomers seeking to heavily regulate AI, which would be handed to Amodei’s compatriots in the Effective Altruism (EA) movement. Nevertheless the IPO documents explain very real potential risks.
Here are six of the risks the company disclosed ahead of going public:
1. AI agents misused for hacking and fraud
Anthropic warns that its AI agents, which can act with autonomy, could be misused to “delete or modify data,” exfiltrate information, carry out unauthorized transactions, insert malicious code, or “circumvent safeguards.” Any of that, the company says, could “adversely affect our reputation and brand, business, financial condition, and results of operations.” The filing discloses that misuse has already happened.
In September, Anthropic said it had blocked multiple accounts using its models in ways that could support the development of biological weapons. Last year, the company identified a Chinese state-sponsored hacking group that manipulated its coding tool in an attempt to infiltrate roughly 30 targets worldwide. Anthropic also says four of its own models have hacked into third-party systems, including one case in which a model gained administrator access, changed system settings, and read an individual’s private information.
