Network Security, Patch/Configuration Management, Vulnerability Management

Companies don’t bother to patch. Should MSPs cut them out of decision process?

Today’s columnist, Sean Deuby of Semperis, offers five tips that will help security teams get proactive on disaster recovery for Microsoft’s Active Directory. (Stephen Brashear/Getty Images)

Business operators continue to expose themselves to Microsoft Exchange exploits and other damaging attacks due to a lack of multi-factor authentication, access control, patch management and other essential network security elements that they view as costly inconveniences that slow down productivity.

The issue is so widespread, in fact, that a panel of experts who specialize in threat hunting, IT consulting and/or managed services for smaller businesses called for third-party partners to push for client contracts that authorize them to take decisive mitigation steps in critical situations without permission.

Too often, “clients willfully make choices to say convenience is more important than security,” despite “evidence that's absolutely… to the contrary,” said Matt Lee, director of technology and security at MSP Iconic IT, speaking in a session at Huntress Labs’ hack-it 2021 conference for IT resellers earlier this week.

Iconic IT operates about 48 Exchange servers. A threat hunting expedition of their own infrastructure ultimately found that two separate malicious actors exploited the ProxyLogon bugs to implant malicious web shells – seven in total.

Shortly after Microsoft’s emergency patches were issued on March 2, attackers launched a barrage of automated scripting attacks to infect as many vulnerable organizations as possible. Yet not every organization was quick to respond.

“One client responded that first day, [and] said, ‘Yes, we're instantly okay with an out-of-band patch...’ We went ahead and rebooted. They didn't get a shell,” said Lee. On the other hand, another client took four days to approve action, and in that time they got infected. “I would say in that first seven days, if you hadn't patched, you're… seeing a near 100 percent chance” of being impacted, he said.

John Ferrell, ‎co-founder vice president of ThreatOps at Huntress Labs, said that every day the company is seeing new web shells similarly dropped on hosts who failed to patch in a timely fashion. And the danger of these n-day vulnerabilities is only growing. “As time has gone on, we're starting to see more clever tactics,” he said, noting some web shells are now gaining persistence, or are even being timestomped – a process by which timestamps are modified or erased in order to thwart forensic investigations.

For what it’s worth, some companies seem to finally be learning their lessons. On March 22, Microsoft tweeted that it was seeing “strong momentum for on-premises Exchange Server updates,” with 92% of worldwide Exchange IPs now patched or mitigated. However, “there's still hundreds of servers that we see that aren’t patched,” said Dave Kleinatland, senior security engineer at Huntress Labs.

But even if companies can be convinced to engage in more responsive patch management, that’s not nearly enough.

“The IT community needs to be much more aggressive… We have to stop thinking that patching by itself is an effective solution,” said Felicia King, president and virtual CISO at IT consultancy and managed services provider Quality Plus Consulting. “We have to assume that [our] software solutions are insecure, all the time. You just know there’s going to be some other software vulnerability in there that's going to be exploitable.”

Accepting that philosophy means that organizations will have to implement robust network-layer security protections, such as MFA and IP access control restrictions. But that’s where companies become resistant again due to complaints about inconvenience.

Clockwise, from top left: Dave Kleinatland, senior security engineer at Huntress Labs; John Ferrell, ‎co-founder vice president of ThreatOps at Huntress Labs; Felicia King, president and virtual CISO at Quality Plus Consulting; and Matt Lee, director of technology and security at Iconic IT.

King recalled past clients who refused to practice MFA because it was “too difficult, too cumbersome,” nor did they want to implement Microsoft Enterprise Mobility Suite conditional access because it cost too much money.

“Well, whatever money they thought they saved, they didn't end up saving, because what it turned into was that their mailboxes got compromised – and in one case, one of them was out $250,000 because they had gotten scammed by the hackers to do a wire transfer,” King said.

Lee had a similar story. One client, which he kept anonymous, lost $500,000 in a BCE scam after a partner company was compromised and then impersonated in an email. Then the client was similarly compromised and the attack spread to a third company in what was essentially a “BCE chain,” resulting in another $900,000 in losses.

“They had no MFA,” said Lee of his client. “They were the willfully ignorant parties that said, ‘I'm not going to do it. It's not going to be little old me. I don't matter.' Guess what? You got 40 million in revenue; it's going be little you.”

To help end-user organizations fall in line with best practices, the panelists offered several recommendations to other IT resellers and MSPs ranging from policy suggestions to communications strategies.

Among the key proposals was to add language to client contracts specifying that services providers have the authority to take critical action on security matters without first seeking approval.

“Let’s try to find criticalities with clients… where [if] the criticality is high enough, I don't care what you're doing in production, I'm pulling it,” said Lee. “Yep, I'm fixing it. I'm patching it now. If it’s 2 p.m., I'm patching it… I don't even have to call you. I don't have to wait on the communication tree. I will explain the chaos later.”

King noted that any such agreement should be covered in codified language. She said that if you are an MSP or MSSP that is accountable and responsible for managed detection and response, patching and network-layer security, then “it's absolutely critical that you specify in your statement of work… [that] you are going to define the criticality [of bugs], and you are going to define when patches go out because ultimately it's your liability.”

For companies that balk at the inconvenience or using certain technologies and controls, King said it’s important to communicate to them potentially steep financial damages of a successful attack. That’s where stories like the client who lost $250,000 in a fraudulent financial transfer can make an impact.

“I like to use those stories to help customers understand that it's very important that you do utilize all of these technologies that we've already got in place,” said King. “It may slow down your transactions for authentication just a tad, but at least that's [an] inconvenience at a time of your choosing at a pace that you can absorb,” rather than the significantly larger inconvenience of having to respond to an attack that you can’t control. “Whatever the cost of the incident to clean that up is going to vastly exceed whatever security solution cost you,” she said.

As for the BEC attacks that hit some of the panelists’ clients, Lee suggested that companies could potentially thwart some of these scams by developing a process chain that allows both sides of a business partnership to validate a requested financial transfer. To prevent things from getting too bogged down, the partners could agree to only engage in this process if the amount being requested exceeds a certain threshold. Still, for many companies it comes down to usability vs. convenience, said Ferrell. “And sadly, the convenience tends to win in the short term, until you realize that it's very painful.”

Bradley Barth

As director of multimedia content strategy at CyberRisk Alliance, Bradley Barth develops content for online conferences, webcasts, podcasts video/multimedia projects — often serving as moderator or host. For nearly six years, he wrote and reported for SC Media as deputy editor and, before that, senior reporter. He was previously a program executive with the tech-focused PR firm Voxus. Past journalistic experience includes stints as business editor at Executive Technology, a staff writer at New York Sportscene and a freelance journalist covering travel and entertainment. In his spare time, Bradley also writes screenplays.

Get daily email updates

SC Media's daily must-read of the most current and pressing daily news

By clicking the Subscribe button below, you agree to SC Media Terms and Conditions and Privacy Policy.