A newly published Office of the Inspector General report detailing how the U.S. Census Bureau mishandled a January 2020 cybersecurity incident is a strong reminder to the business community to follow best practices such as IT asset management, frequent vulnerability scanning and mitigation, comprehensive event logging and prompt notification and incident response when a possible incident is suspected.
“It ultimately comes down to the age-old conversation of cyber hygiene. Do the simple things well and spend resources and time on the lowest hanging fruit first,” said Qualys CISO, Ben Carr, commenting on the report. “Time and time again, poor cyber hygiene has proven to be one of the most common reasons for why organizations are compromised. Organizations must focus on knowing where their assets are, keeping up with vulnerability patching and having the appropriate logs and instrumentation in place. Implementing these initial steps allows for a reduction in dwell time, [and] the ability to understand the full scope of an attack and respond accordingly.”
Frederick Meny Jr., assistant inspector general for audit and evaluation, disclosed in a report to the Department of Commerce and its Census Bureau department that several missteps allowed hackers last Jan. 11 to exploit an unpatched critical vulnerability and compromise remote-access servers.
The attackers were able to modify user account data on the systems in an attempt to enable remote code execution, but firewalls ultimately prevented the adversaries from creating a secret backdoor or gaining any access to the actual 2020 census networks or data.
According to the report, the incident could have been prevented if the bureau had applied corrective steps shortly after the servers’ vendor (which is not named in the report) revealed the vulnerability on Dec. 17, 2019, and recommended mitigations to quash the threat. But the bureau failed to implement the mitigations until after the attack took place.
Moreover, the bureau did not conduct vulnerability scanning of the affected remote-access servers, as they were not included on an official list of systems and devices to be scanned. This is because the “system and vulnerability scanning teams had not coordinated the transfer of system credentials required for credentialed scanning,” the report explains. Had the remote access servers been included in the required monthly vulnerability scanning, the Bureau could have identified the vulnerability and taken action to mitigate it before the incident.”
The attackers’ failed attempts to use the servers to communicate with its command-and-control infrastructure went unnoticed for more than two weeks because the bureau at the time was not using a SIEM (security information and event management) tool to identify anomalous network traffic and alert incident responders.
Additional issues: the Commerce Department’s Enterprise Security Operations Center (ESOC) didn’t promptly share with the bureau a Jan. 16 CISA communication warning that the remote-access servers were attacked, possibly leading to a compromise. Likewise, once the bureau learned that its servers were affected after receiving a second warning from CISA on Jan. 30, it waited five days until informing the ESOC of its findings.
“Communication challenges between security and IT personnel at the Bureau clearly contributed to the compromise,” said Jake Williams, CTO at BreachQuest.
Ironically, before realizing the full extent of the attack, the bureau had actually run an IOC script on servers in its lab environment only, and learned that they were compromised. But the bureau never bothered to check the other servers until receiving that second warning from CISA. On the bright side, at least this demonstrates that “indicator of compromise scanning is an effective force multiplier for security teams,” said Williams. “Senior analysts discover the appropriate indicators and write detections, and junior analysts can use the scanner to automatically and reliably identify compromised systems.”
The OIG’s office also noted that at the time of the incident, none of the servers were sending system logs to the bureau’s operational SIEM. They were either storing the data locally or sending the data to an old, decommissioned SIEM. This ultimately impeded investigation of the event.
“The root cause of this is a failure in the change control process, which is fundamental for security,” said Williams. We recommend that clients configure alerting for their log aggregators (SIEM systems) so they are immediately alerted if logs are not being delivered. If that alerting isn't available in their log aggregator, analysts can regularly query logs from random systems to ensure their delivery.”
Finally, the report criticized the bureau for not conducting a post-mortem after the incident and for continuing to operate the servers long after the incident, even though they had actually reached their end of life on Jan. 1, 2021.
In response to these findings, the OIG’s office recommended that the Department of Commerce and Census Bureau implement better procedures for timely incident notification, ensure that IR specialists and ESOC employees disclose intelligence promptly, regularly review and update vulnerability scanning lists for IT assets, ensure that all assets are scanned using credentials when possible, review SIEM auto alert capabilities to ensure that similar attacks can be identified in the future, review log configurations, and decommission obsolete IT systems.
“The important takeaway from this event is that additional logging and visibility may have supported more timely identification and reporting, which could have both limited persistent access and subsequent impact,” said Tim Wade, technical director, CTO Team at Vectra.
“The takeaway should not be that some exploitable vulnerability was discovered and abused — given time, resources, and motivation an adversary will always uncover some exploitable condition but by developing an organization’s detection, response, and recovery capabilities there is an opportunity to mitigate the risks of such discovery and abuse before material damage is realized.”
