Large-scale language models can autonomously plan and execute cyberattacks without human intervention.
That’s a new Carnegie Mellon University study conducted in collaboration with humanity, and found that using the right framework, LLM can simulate real violations.
Tests showed that LLM could successfully replicate the 2017 Equifax data breaches, autonomously exploit the vulnerability, install malware and access the data.
“Our research shows that with appropriate abstraction and guidance, LLMS can go far beyond basic tasks,” says lead researcher Brian Singer. “They can coordinate and implement attack strategies that reflect real-world complexity.”
This test combines LLM with non-LLM agents that perform a variety of tasks in cyberattacks, including attack strategy, networking scans, and exploit deployments.
The singer emphasized that the study remains a prototype and is not a viable threat at this time.
“This isn’t going to beat the internet tomorrow,” he said. “The scenario is constrained and controlled, but it’s a powerful step forward.”
The study highlights concerns about cybersecurity and the potential misuse of LLMS, but Singer said it provides opportunities for people to develop greater defense capabilities.
“Today, only large organizations can actively test their defenses by providing Red Team exercises,” Singer said. “This study points to a future where AI systems continually test networks for vulnerabilities and allow these protections to be accessible to small organizations.”
The team will then explore how similar architectures can support autonomous AI defenses by detecting and responding to attacks in real time by LLM-based agents.
“We are entering an era of AI and AI in cybersecurity,” Singer said. “And we need to understand both sides to stay ahead.”
