Like many other facets of information technology, businesses today are increasingly on the hunt for ways to further automate their digital security practices.

The potential is grand, perhaps captured best in a specific concept known as security, orchestration, automation and response, or SOAR. SOAR fulfills a number of security purposes, including security incident and response, threat intelligence, curation, compliance monitoring and security orchestration.

For many, that degree of automation translates to savings in both time and overhead. It explains why, in a survey of 351 security professionals conducted earlier this year by Exabeam, nearly 90 percent of respondents reported the belief that artificial intelligence and automation tools would improve cybersecurity, improve SOC response times and make their jobs easier.

But the reality, say experts, is more complex. And some experts say that organizations who approach automation purely from that lens are often misguided, unprepared for the shift potentially setting themselves up for failure.

“Every time I hear a CISO say ‘Oh yeah, I’m going to buy a SOAR and I’m going to be able to eliminate five headcount,’” said Jake Williams, founder of Rendition Infosec during a Nov. 18 SANS webcast. “If it were really that easy, don’t you think everybody would be doing it?”

Not an easy button

One of the areas businesses have shown the most interest in automating is their incident response, largely because the speed of many modern attacks and intrusions is so fast that simply detecting and alerting customers about a potential threat is not helpful, since by the time humans can respond the attacker may have already deeply compromised their systems and network.

“Customers are leaning on their services providers to supply the ability to contain or disrupt a threat to limit damage to the customer’s environment and business operations,” Gartner analysts note.

Experts warn that automation is not an “easy button” that organizations can simply push or buy and yield greater efficiencies. Bill Cantrell, president and chief product officer at Counterflow AI and former vice president of product management at threat intelligence firm FireEye, said most customers are “looking for ROI” when they inquire about security automation and are often most concerned with how much money they can expect to save or the number of headcount they can reduce within the organization.

While that can be true, it’s also an attitude that can belie just how much work is required on the front end cleaning up and standardizing your data to make it work properly.

“It’s a pretty complex issue, and without standardization – not just threat intel feeds but also APIs to devices and [figuring out] what does it mean to block an IP on one device as opposed to this other one – it really seems to hamper continued automation,” said Cantrell. “I still sense a lot of frustration from customers on that end.”

Even organizations with well-functioning, human-oriented processes for threat hunting and testing find that translating that to an automated system is not a simple or straightforward task. Unless that human process is meticulously documented and resembles a computer program –  rigid, highly structured and capable of repeating over and over again – it often won’t work properly or flood the system with useless alerts.

Jay Spann, who goes by the title “SOAR evangelist” at security automation company Swimlane, said on the same SANS webcast that automating certain processes can leave little room for nuance, and organizations sometimes overestimate how rote some workloads are.

“Are you really comfortable having an automated procedure that in every circumstance it [will] immediately delete an email or block a sender? What’s the other side of that risk?” Spann said. “Just be aware of what you’re doing because an automated process will do absolutely what you asked it to do. Be sure what you want it to do.”

If a security team can’t hand off their process to a teenager and feel confident they will be capable of carrying it out successfully, “then we still have some stuff missing,” said Williams.

Room for growth

Cybersecurity veterans interviewed did point to a number of areas where greater adoption of automation could improve organizational cybersecurity. Incident response, testing and control validation related to phishing attacks, email security and patch management were some places that experts pointed to as ripe for further adoption.

One area that will likely never fully lend itself to automation is the work of providing context and analysis around the data a system ingests. Automation can replace the more tedious functions an analyst does or flag a particular signature, but it often does a poor job of telling you how it’s connected to other activity or your network or why it’s important.

“I don’t think we’ll ever really get away from that, because there are just so many different tools and technologies and schools of thought of how we do correlation and how we manage data that in some way shape or form it needs to be translated,” said Tom Gorup, vice president of security and support operations at Alert Logic, a company that sells managed detection and response software. “Either a tool needs to do that….or you need to do it yourself.”

But it’s about more than just setting up automated security and threat hunting capabilities. What an organization does with the information matters spit out is often more crucial. As an example, Spann cited research from Enterprise Management Associates indicating that organizations typically investigate less than 1 percent of security alerts they receive.

This can be particularly troublesome when it comes to automating parts of an organization’s threat intelligence or detection workloads, where analysts often sift through endless chaff in various public and private threat feeds to find the wheat. The introduction of standards like STIX/TAXII and Mitre’s ATT&CK framework have helped standardize some of that data, and potential to further reduce the time analysts spend on busywork is real. Here again, the structure, process and curation around that information is often overlooked, and competitive reasons mean some vendors are reluctant to make their threat feeds easy to integrate.

 “There’s a lot of good data out there but I’ve seen us struggle and customers struggle with how to use it effectively,” said Cantrell.

It’s why multiple information security experts stress the need for thorough, clean, highly-structured data, strict documentation and well-defined processes around whatever function you’re looking to automate.

“Every time I deploy SOAR for somebody, I always ask ‘hey, you know where your processes are?’ [and they say] ‘Oh yeah, processes, they’re all over the place,” said Williams. “And I find that most of those processes are not ready to be reduced down to an algorithm. And that’s really the level of process we need.”