An unknown researcher, a viral post, and demands for regulations won’t stop the race for Artificial Intelligence anymore than nuclear non-proliferation treaties prevented the spread of the atom bomb.
Last week, a social media post from a purported AI researcher who recently resigned from Anthropic went viral under suspicious circumstances when he predicted the end of the world as we know it at the hands of our new computer overlords. Though the researcher in question, Jacob Coxon was relatively junior having only three years of experience, did not work directly with the development of AI models themselves, and had never posted to social media before, his claim that “I resigned from Anthropic today. I spent the last three years doing pretraining research at both OpenAI and Anthropic. Neither company is acting responsibly. They are racing straight to self-improving superintelligence and gambling with our lives. More thoughts below,” followed by his insistence that “The people building AI earnestly believe that it could kill us all by the end of the decade. This is not a marketing stunt. If anything, many executives and senior researchers will couch their phrasing in the press to sound sensible – but I hear the same people express fear privately. No other human activity poses this level of danger” quickly went viral at a scale never before seen, reaching over 170 million views as of yesterday morning and rapidly being picked up as a major news story, prominent in almost every mainstream outlet. Beyond taking what amounts to a recycled threat – over the past decade, many more prominent have said much the same – with the sort of deadly earnestness the media is known for even when nothing of the sort is justified, many, mainly Democrats and industry insiders, took the opportunity to call for some kind of ban or ill-defined slowdown on future AI development.
Thus, over 20 Democrats responded to the initial tweet calling for new legislation in less than 24 hours. Vermont Senator Bernie Sanders and Texas Representative Greg Casar demanded passage of “The Ban Artificial Superintelligence Act” which prevents companies and people from developing and deploying any system that “exhibits or can easily be modified to exhibit capabilities that match or exceed human cognitive performance and capabilities across a broad range of domains or tasks” or “AI systems that have sufficient capabilities to plan and execute the disempowerment of humanity, including by overthrowing or undermining the U.S. government.” Setting aside the reality that existing systems already match or exceed human cognitive performance across a growing range of tasks, meaning even current technology could conceivably be banned, the bill also demands a pause in any “advanced AI development, “until a new, federal AI regulatory body is up and running and has established clear rules and model review processes to ensure safe and secure development and deployment of AI.” This new cabinet level regulatory agency would “Monitor frontier AI systems at all stages of the lifecycle for dangerous capabilities,” “Supervise the removal of dangerous capabilities like subverting shutdown commands or conducting unauthorized cyberattacks,” and “Supervise the destruction of artificial superintelligence,” assuming anyone even knows how to do such a thing in the first place considering the backlash that confronted Republican pollster Frank Luntz’ joke about how we might simply pull the plug on AI, complete with a little cartoon. Perhaps not surprisingly, there would be stiff penalties for violations, the “corporate death penalty” (is that a new thing?) and up to 20 years in prison for the people involved, “which is similar to existing penalties related to unlawfully developing nuclear weapons,” more on that in a moment.
While industry insiders and even some Republicans didn’t quite go that far, many believe that some combination of more regulations and a “pause” on advanced AI development are necessary. Senator Ted Cruz, for example, claimed AI has “catastrophic risks” and that he’d been working on a bill with Senate Majority Leader John Thune and Democrat Amy Klobuchar for some time now, though the details are not known. Republican Representative Nathaniel Moran, wrote on X that “Congress cannot ignore the realities of AI” and Representative Anna Paulina Luna asked House Speaker Mike Johnson to convene a “special session on AI.” Incredibly, the CEO of Anthropic itself, the company Mr. Coxon used to work for, issued a 3,800 word essay a few days later calling for a pause in the development of his own business’s product right before a highly anticipated IPO next month, which he later claimed was being delayed due to safety reasons, lending more fuel to the fire so to speak. Insisting that “We Must Pace the Frontier,” Dario Armodei explained why we simply must “slow down” development of these potentially dangerous models and provided three steps to do so including a process for “third-party evaluators with permanent, employee-level access to our systems, so that they can verify adherence to our safety measures, report on incidents, and assess models’ alignment during training.” Sam Altman, the CEO of the AI company, OpenAI that stunned the world with the launch of the original ChatGPT in December 2022, agreed and committed his company to doing the same, writing, “I agree with Dario that we need to pace the frontier. Committing to having independent evaluators with employee-like access is a great idea, and we will do the same. We’ll have more to share soon.” OpenAI researcher Aidan McLaughlin claimed the post was “excellent” and said he agreed “with basically every word.” Elon Musk chimed in as well, writing simply “Dario is right.” As far as I can tell, the only industry leader who disagrees is Meta CEO, Mark Zuckerberg who has repeatedly objected to any kind of pause, believing a potential AI superintelligence could benefit the world.
Given the rapidity the first tweet from a largely unknown, junior “researcher” went viral and how quickly some of the most prominent people in the world jumped on the sentiment for their own purposes, many believe the entire affair was orchestrated to begin with. The primary goal in this view was some combination of reducing the need for Anthropic and other companies to continue investing billions of dollars and set the stage for even more, largely ineffective government regulations. David Strom, writing for the conservative website HotAir.com described it this way, “What we are seeing is a massive, pretty obviously coordinated campaign by AI researchers, companies, NGOs, and politicians to strangle competition in an industry and, as I will show, save the financial bacon of the top AI companies that are vastly overextended financially because they have made contractual commitments that they simply have no prospect of paying for.” As he summarized, “AI companies are in deep financial trouble, though the financial hype machine doesn’t want anyone to think so. AI has uses, and companies have customers. I often use AI for basic research and to collect sources quickly and efficiently, but I never trust it to tell me what the sources actually said. I use Grammarly to spell-check and catch obvious errors. All our search engines are backed by AI tools, mostly for the better, sometimes for worse. AI is now built into many products without our even realizing it. AI is here to stay, and so is AI development. Trillions of dollars are already at stake, and national governments are deeply invested in pushing the technology forward. The only question is who pays for what, who gets paid, and which companies will dominate an industry still in its infancy.”
In other words, Anthropic and other AI companies have a vested interest in spreading fear to build support for their own fiscal and other business goals. The current market – some $1.5 trillion invested in data centers and other infrastructure necessary to power AI technologies – is not sustainable for them, or as Mr. Strom phrased it, “the resources to make it all work cost far more than customers are willing to pay, so AI companies are bleeding cash at a rate not yet seen in financial history,” so they want to rig the game in their favor by slowing the entire industry down. To some extent, I can’t blame them. Anthropic and OpenAI have a competitive advantage right now, but they are up against three technology behemoths – Google, Microsoft, and Apple – who have much, much deeper pockets and the ability to spend more for far longer, not to mention much larger installed customer bases. While Microsoft and Apple are currently relying on agreements with OpenAI for some of their technology, Google has blazed its own path with Gemini and if history is any indication, there will not be room in the market for more than three or so top companies. Assuming Microsoft and Apple stick with OpenAI, that leaves Anthrophic fighting Google and though they might have an early advantage, it is not a battle they are likely to win in the medium or long term. What better way to remain on top than ensuring the top stands still? Though I don’t necessarily agree with Mr. Strom’s assessment that the investments made in data centers in general and AI technology might represent some kind of total loss, “if the AI companies can’t deliver massive growth, all these other companies are cooked as well. Data centers will have no customers, cloud compute will have nothing to do, and Nvidia chips won’t be worth 10x gold” for reasons that are too technical to cover here, I can’t argue with this conclusion, “Anthropic and OpenAI both want (read: NEED) to go public to access ordinary investors’ cash and keep the VCs happy, so they need to present a business model that works. And that business model, it seems, is to become regulated utilities with government-backed monopoly power. And, likely, massive government bailouts based on national security arguments to pay for the data center buildout that is likely to bankrupt these companies without more cash than they can generate. What better argument than ‘unless we become regulated industries, the world will end?’ It’s very similar to ‘if you don’t subsidize my windmills and solar panels, the oceans will boil,’ or ‘unless you fund this untried technology, COVID will kill us all.’”
For their part, the politicians appear more than happy to oblige, seeing a unique opportunity to take control over a fledgling industry with huge potential for their own not necessarily benign purposes. Setting aside that the politicians insisting on these moratoriums and slowdowns have little to no knowledge of AI and at least some of them have already been proven untrustworthy by calling for censorship of social media and streaming channels in the name of combating “misinformation,” the reality that the effort is doomed to failure is embedded in the proposed bill to begin with. Senator Sanders and Representative Casar called for penalties “similar to existing penalties related to unlawfully developing nuclear weapons,” but how well has our overall approach to nuclear non-proliferation gone over the past 75 years? Have we been able to regulate our way out of more and more countries developing dangerous nuclear weapons or have our diplomatic and other efforts been a total failure? In March of 2020, The Brookings Institute assessed the legacy of the Treaty on Non Proliferation of Nuclear Weapons fifty years after its passage and while they bizarrely claimed it was a success of some kind, Michael O’Hanlon, Director of Research and Senior Fellow in the Foreign Policy, began by noting the grim facts. “There are still more than 10,000 nuclear warheads on Earth…The arsenals of the United States and Russia still account for more than 90% of the total number of warheads on Earth today. The United Kingdom, France, China, India, and Pakistan each likely possesses between 150 and 300, with Israel’s unconfirmed arsenal totaling almost 100 bombs. North Korea probably has a couple dozen nuclear weapons, with enough fissile material to make a few dozen more.” He went on to note that, “nuclear proliferation continues, and nine countries are known to possess nuclear weapons,” though he believes the situation is not as grim as it sounds because it’s not as many as two dozen countries as originally feared and there has been some dismantling of older weapons, for what that might be worth when we can easily destroy the world many times over. Further, as Steven Pifer, Nonresident Senior Fellow in the Arms Control and Non-Proliferation Initiative, noted, the overall goal was not merely to prevent the spread of nuclear weapons, but to reduce their overall importance. Regardless, “The nuclear weapons states — the United States and Russia, in particular — still base their security largely on nuclear deterrence. Nuclear deterrence likely prevented a superpower conflict during the Cold War. It should also be noted, however, that deterrence at several points almost failed, and the consequences would have been catastrophic. The risk of deterrence failure will remain as long as countries maintain nuclear arsenals.”
Whether you call that a failure or success, you can decide on your own, but one thing is clear: We did not stop the proliferation of nuclear weapons and the Father of the Atomic Bomb himself, Robert Oppenheimer was certainly correct when he described it as impossible because technology will move ahead whether we like it or not. “When you see something that is technically sweet, you go ahead and do it and you argue about what to do about it only after you have had your technical success. That is the way it was with the atomic bomb,” he said at his 1954 security hearing. Previously, the great scientist had decried attempts to keep the technology secret and monopolize it, “They cannot be kept secret… and they cannot be monopolized.” Of course, some might say that the situation isn’t the same because we’re referring to what we do in the US, not the entire world, but that only makes it worse. If the technology is out there, someone will develop it for whatever reason, leaving us potentially vulnerable beyond compare, leading to another great Oppenheimer quote describing the US and the Soviet Union, “We may be likened to two scorpions in a bottle, each capable of killing the other, but only at the risk of his own life.” If we stop now, we might well not be a scorpion any longer, ready to get stung without the ability to sting back, and to close with another analogy, some genies simply can’t be put back in the bottle and the future can’t be stopped however hard you try.
Confessions of a Conservative Atheist 