The world of DDoS security has changed. For more than two decades, the detection of DDoS attacks was based on using hardware probes or deep packet inspection (DPI) to observe network traffic patterns and protocol anomalies at specific network interfaces, looking for network traffic that exceeds bandwidth and packet intensity thresholds and baselines.

End-to-end encryption of traffic, exponentially growing volumes of data exchanged between the internet and service provider networks, and an increasing number of network edges have challenged traditional DDoS security approaches. Legacy solutions cannot detect new and complex attacks as quickly and accurately, and their terabit-level scalability comes at a great cost. In other words, legacy DDoS security methods are increasingly ineffective in detecting and stopping today's complex and sophisticated attacks.

Big data for better DDoS detection

Big data, AI, and ML can help us better understand network traffic and quickly recognize and detect DDoS threats. Today, DDoS attacks come from outside and inside service provider networks, so we need a holistic perspective on DDoS security. New attack vectors such as botnets have expanded the threat surface to targets that include enterprise and residential customers, and service provider infrastructure.

To differentiate between "good" and "bad" traffic does not present itself just as an operational issue of keeping the network running as intended. The failure to make this differentiation in seconds can lead to a significant loss of connectivity and service disruptions. Making a sound judgment on whether certain network flows are DDoS or regular traffic requires a network owner to have a full and wide perspective of the traffic–within their network and outside of it. Only when the larger, internet-related security context gets "overlapped" with network-related information can we find a holistic picture of what’s going on in the network, accurately detect malicious network traffic. and make rapid decisions on how to mitigate.

The good news: network routers have evolved into sophisticated instruments that can perform principal traffic routing and forwarding tasks of shuffling terabytes of data. They are now so sophisticated that they can also inform about all traffic flow they "see" by generating large volumes of telemetry data in real-time. This data is the source for big data-based network analytics. When used for DDoS security, the big data approach can result in faster, more accurate DDoS detection and agile mitigation.

AI/ML: DDoS detected

Security teams can train AI and ML to identify network flow anomalies, patterns and trends that are impossible for humans to discern or would take an extremely long time. Like with other big data applications, there are still challenges.

First, the security team must have a large data set representing the network's reality as accurately as possible. Having the additional internet-related security context can help a lot. This knowledge can offer a wider security perspective and inform about repeated malicious patterns, attempts and techniques used in the past. 

Next, we need human intelligence (HI) – a team of experts who dissect and analyze DDoS attacks and teach the AI/ML what to look for and how to differentiate "good" from "bad" traffic. Once taught by HI, AI/ML can identify data patterns that may indicate when attacks are taking place.

With large, high-quality data sets and the ability to employ AI/ML to process petabytes of network data in real-time, the latest big data-based DDoS security solutions can help us to detect threats earlier and more accurately.

Big data, AI and ML can also create more sophisticated cyber defenses. Because all networks are different, mitigation of a particular DDoS attack (or a multitude of concurrent ongoing attacks) must consider the network's actual capabilities to neutralize DDoS.

AI/ML can help create optimized defense tactics tailored to a particular network and service provider's needs and goals. In addition to the obvious objective of stopping DDoS attacks in the most agile and efficient manner, additional criteria for optimized mitigative actions could include minimizing false positives and false negatives – something that equally concerns service providers, their customers and regulators.

To fully benefit from the big data approach, security teams must automate detection and mitigation as much as possible. In addition, big data analytics and AI/ML can help simulate different attack scenarios, allowing organizations to test their defenses and ensure they are better prepared for real-life attacks.

Big data, AI, and ML can improve DDoS security and help service providers stay one step ahead of the attackers and keep their networks and systems safe. By harnessing the power of big data, AI, and ML, we can establish a more effective, proactive and preventative approach to DDoS security. Solutions based on this approach are already here – they offer the scalability and adaptability needed for the cloud era and superior cost-efficiency.

Alex Pavlovic, director of marketing, Nokia Deepfield