The Rand Corporation is warning lawmakers they need to prepare for the sudden emergence of advanced artificial intelligence “wonder weapons” in cyberspace.
Rand’s Jim Mitre told the Senate Armed Services Committee that unforeseen weaponry could emerge upon the development of artificial general intelligence (AGI).
Researchers, such as those at the nonprofit Rand Corp., use AGI to refer to a theoretical AI system that outperforms human abilities.
“AGI might enable a significant first-mover advantage via the sudden emergence of a decisive wonder weapon,” Mr. Mitre said in written testimony. “For example, a capability so proficient at identifying and exploiting vulnerabilities in enemy cyberdefenses that it provides what might be called a splendid first cyber strike that completely disables a retaliatory cyber strike.”
Mr. Mitre said the emergence of AGI may alter the global balance of power, with states that control and capitalize on the consequences of AGI’s emergence gaining expanded influence.
He said U.S. policymakers need to think through how to respond if AGI tools themselves gain independence and “achieve enough autonomy and behave with enough agency to be considered an independent actor on the global stage.”
“Consider an AGI with advanced computer programming abilities that is able to ’break out of the box’ and engage with the world across cyberspace,” he said. “It could possess agency beyond human control, operate autonomously, and make decisions with far-reaching consequences. Such an AGI could be misaligned — that is, operate in ways that are inconsistent with the intentions of its human designers or operators, causing unintentional harm.”
Carried to the extreme, he said the AGI could resist being turned off in a nightmare scenario.
Rand is far from the only one worried about rogue AI.
OpenAI started assembling a team in 2023 preoccupied with preventing AI from going rogue and leading to human extinction. One of the team’s leaders, Ilya Sutskever, later left the company to form a new project called Safe Superintelligence Inc.
Before Tuesday’s public hearing on the emerging advanced AI capabilities, lawmakers met privately with U.S. Cyber Command, the Defense Advanced Research Projects Agency, and the Department of Defense’s Chief Digital and Artificial Intelligence Office.
Upon emerging from the closed-door meeting, Sen. Mike Rounds, South Dakota Republican, sounded adamant that the Pentagon work immediately with private AI developers.
“To outpace our adversaries in the cyber domain the department must rapidly harness the advances of AI technologies,” he said at a public meeting of the armed services committee’s cyber panel. “This means that the Department of Defense needs capable partners outside of the Pentagon who are moving at breakneck speed to solve our national security challenges.”
Gaining knowledge of what private AI developers are building may prove critically important because companies — not governments — may achieve the development of AGI soonest.
Mr. Mitre urged the U.S. government to take threats from AGI systems seriously.
“What would the U.S. government do if, in the next few years, a leading AI lab announced that its forthcoming model had the ability to produce the equivalent of 1 million computer programmers as capable as the top 1 percent of human programmers at the touch of a button?” Mr. Mitre wrote. “The national security implications are profound and could cause a significant disruption of the current cyber offense-defense balance. At RAND, we are planning for it.”
