For AI and Security, ‘The Storm is Coming’

SAN FRANCISCO–The broad use of AI for general purpose computing tasks is still a fresh concept, even though it seems like tools such as Claude and Gemini and ChatGPT have been around forever. Technology moves quickly, and as we’re discovering yet again, threat actors move just as quickly, and are adopting AI tools and platforms at an astonishing rate, one that the defender community is hard pressed to match. 

“As we’ve seen many times, threat actors will leverage anything and everything they can in order to make their attacks more effective. AI is no exception and we’re seeing them use it all over the place,” John Hammond, principal security researcher at Huntress, said during an interview at the RSA Conference here Tuesday. 

Some of the initial concerns around the ways in which attackers might use AI centered on the rapid development and deployment of AI-created malware and exploits. There have been some limited examples of these things happening in the wild, but in a lot of cases researchers say exploits written by AI agents are unusable and poorly constructed. 

Malware development is a different story, however. Threat actors have found ways to use LLMs to increase the efficiency and speed of malware development in the last year or so, and that’s likely to become the norm rather than the exception relatively quickly. 

“In our research, we observed threat actors using LLMs to produce infostealers, RATs, droppers, ransomware engines, or other malicious scripts,: researchers at Arctic Wolf said in a new threat report on AI-aided malware development.

“We observed a consistent pattern of threat actors learning iteratively, using LLMs to scaffold code and fill in gaps, moving from broken proof-of-concept (POC) implementations into usable malware faster and more capably than they could on their own. In that sense, AI is not simply producing more code; it is narrowing the gap between developer skill and operational capability, and timeline.”

“These AI models are the dumbest they’re ever going to be right now. The storm is coming."

Attackers have a couple of important advantages when it comes to adopting AI usage in their work. One, they can test tools and models in their malware development and operations and quickly decide which ones work and discard the rest. And two, they don’t (usually) have product roadmaps and quarterly goals to worry about. They mostly do what they want. 

Defenders and security tool vendors have real world concerns like release timelines, investor pressure, and OKRs that can get in the way of testing the latest and greatest AI tool or LLM. Speed is an advantage for attackers in that sense. 

And the old maxim that attacks always get better, not worse, is proving out once again. 

“These AI models are the dumbest they’re ever going to be right now. The storm is coming, that’s already happening. The current approach is we’re being swarmed by these agents and they’re attacking anything they can find. The situation is we have incomplete data and we’re fighting this swarm with humans on SOC teams and we’re locked into these proprietary silos. This approach is just not tenable,” Ali Ghodsi, CEO of Databricks, said in a keynote speech Tuesday. 

The use of AI in SOCs is of particular interest to many organizations, as it promises new efficiencies and velocity that SOCs manned strictly by humans can’t match, But, it’s no sure bet that AI-powered SOCs will be the rule in the near future. 

“We want real human eyes on intel and evidence and everything so we have real expertise. And so we are being a partner to someone who’s going through their worst day when they get attacked,” said Hammond. 

“We were reluctant about AI a little but I will say that come the end of 2025, the whole world woke up. Threat actors are adopting this and are the first movers to use it. Right now, it’s mixed into the SOC and supplementing a lot of the human work. We are leaning in the best we can, and are seeing some super power with AI.”

Machines aren’t always better, just faster.