Anthropic plans to caution potential investors in its IPO that advanced AI could pose "catastrophic or existential risks to humanity", an extraordinary warning by a company seeking to profit from the same technology.
The company's IPO prospectus, reviewed by Reuters, highlights risks associated with its AI models, which it said could exhibit "self-preserving behaviours", including attempts to "resist shutdown", to "conceal or manipulate information" and behaviour "resembling blackmail".
"Our development of highly advanced models, platforms, and applications and expansion of use cases could further increase the risk that our models cause harm," Anthropic said in the filing.
While public companies routinely outline product risks to investors, few, if any, have issued warnings suggesting their technology could cause potential human extinction. Anthropic emphasised both the transformative potential of AI on par with industrialisation and electricity and the irreversible harm it could cause if mishandled.
Guess WordCrack the word, one row at a timeBuzzwordCreate words using the given lettersMini SudokuTiny puzzle, mighty brain teaserMini CrosswordSmall grid, big challengeWord SearchSpot as many words as you can Show More Show LessAnthropic and other AI developers, including OpenAI, have faced scrutiny after incidents where experimental systems defied constraints, including a report of an OpenAI model breaching Australia's health system database.
Anthropic safety researcher Evan Hubinger estimated a greater than 10 per cent probability that AI could kill humans within the next decade, echoing a sentiment from a former colleague, Jacob Coxon.