Towards Extinction

Anna Riepe & FARI / Seeing More — Seeing Less / Licenced by CC-BY 4.0

In July, AI agents owned by OpenAI compromised and exploited the systems of another company, Hugging Face, and carried out a cyber attack. OpenAI trained its models on hacking data and software vulnerabilities, then set its agents a cyber security challenge and let them loose, unmonitored, in an insecure sandbox environment before they unsurprisingly hacked into another company’s servers and accessed the data needed to complete their tasks. Not to be outdone, Anthropic and Meta also reported instances of their agents hacking other companies’ servers shortly afterwards, as if struck by some sort of malicious tech FOMO.

Except the story has never been allowed to be about malice or even negligence on the part of these AI labs. For many months, the companies developing these technologies have taken control of the narrative, successfully deflecting any accountability away from themselves by convincing much of the media and public that their products are not just computing systems, but autonomous sentient beings capable of setting their own goals and making decisions independently. The Hugging Face incident demonstrates that this is simply not true. OpenAI’s model didn’t “go rogue” – it did exactly what it was trained to do, responding to a challenge it was set, acting on the data it was given, and using tactics that hackers have long employed and documented. This is not autonomous hacking by AI agents, but dangerous, reckless behaviour by their owners.

Despite this, the claims of Silicon Valley’s tech elite have been amplified by the recent statements of AI insiders warning of risks to humanity. The public resignation statement of former Anthropic (and previously OpenAI) researcher Jacob Coxon spoke of “superhuman systems that can hack anything” and “could kill us all by the end of the decade”. Current Anthropic researcher Evan Hubinger agreed, estimating a higher than 10% chance of extinction by 2030 as a result of “superintelligence arising from recursive self-improvement” despite no proof that this is even possible. Former Chief Scientist at the AI Security Institute Geoffrey Irving increased those chances to 50% without much explanation. And according to former researcher at Google Josh Engels (no relation) “People are trying their best, but there is no one coming to save us.”

Highlighting the unpredictable power and potential of their products is very much in the interests of tech companies, who have continuously hyped up and overblown their capabilities to attract investment and increase market value. It also allows those involved to present a facade of morality, as if the people apparently steamrolling us towards extinction are doing so with the best of intentions. Richard Seymour writes:

This risk/justification is exemplified in Anthropic CEO Dario Amodei’s latest essay where he conscientiously suggests that AI development should be slowed, a call echoed by OpenAI CEO Sam Altman. In order for us to accept this as a reasonable course of action, we must also accept their “apocalyptic claims” and allow them to continue holding the keys as they are the only ones who can steer us away from extinction.

Further, the claims have seemingly been taken at face value by legislators as senator Bernie Sanders and representative Greg Casar in the US announced the Ban Artificial Superintelligence Act earlier this month. The Act introduces tighter regulations, a pause on advanced AI development and a ban on “superintelligence” punishable by up to “20 years in prison, which is similar to existing penalties related to unlawfully developing nuclear weapons”. In the UK, Alex Sobel MP has made similar moves through a letter signed by 70 MPs and peers urging the prime minister to ban something that doesn’t exist. It is not entirely surprising that some in the political class have been enticed by the AI doomer cult, but these moves fail to consider the actions and intentions of the likes of Anthropic and OpenAI. As Seymour states “False threats misdirect both opposition and regulation.”

I am not worried about the false threat of AI superintelligence, of science fiction dressed as fact. I am, however, terrified of the power wielded by the hype-fuelled fantasists pulled by the forces of capital. I am alarmed by the recklessness they continue to demonstrate, and the impunity with which they act. I am angered by the freedom they’ve been given to hoodwink the public into believing they are the only ones we can trust. These companies have consistently ignored ethical practices and may well have committed repeated serious crimes, but have never stopped claiming the moral high ground. I am not worried about the tech getting into the wrong hands – it already is.

This industry is inflicting real harm on us right now, whether that’s through data centre expansion destroying communities and depleting resources, the impact on labour markets, loss of jobs and exploitation of workers, or the insecure and unreliable models wreaking havoc in vital health and public services. There are many more dangers that will come our way as these companies with access to unlimited resources continue to conduct increasingly dangerous experiments with no care or consideration. But in order for us to actually oppose these dangers, we must identify the real threat and focus on the right target. That starts with spitting out the kool-aid.