Security Staff Acquisition & Development, Data Security, Leadership, Network Security

Learning Lessons of Security Failures Ensures Future Success

By Katherine Teitler

We are (maybe) the champions

As a first time DerbyCon goer, I didn’t quite know what to expect. In its sixth year, DerbyCon is well known throughout the security community, and I’ve worked with several of the speakers, a few of the organizers, and met many security vendor representatives at MISTI and past-job events. In my mind, I had a vision based on what I’d heard, and I figured a trip to Louisville wouldn’t be strikingly similar to InfoSec World.

You see, MISTI focuses on business-oriented conferences and DerbyCon—along with many of the other*Cons—is more technical. Both are necessary in infosec, but there’s often a division between the “techies” and management, which is part of the problem with present-day security programs. Security in a vacuum isn’t working particularly well, so I was pleasantly surprised to see some strategy- and planning-level talks programmed on the 2016 agenda alongside more technical demos.

One of these talks was given by the infamous Jayson E. Street. Jason’s talk was titled, “…And Bad Mistakes, I’ve Made a Few…” and it was bound to get interesting. Of course Street included humor and kept the tone light, but the laughter in the room was not only because of his pithy presentation skills. Much of what Street said rang true with attendees, and small nods of acknowledgement could be seen throughout the audience. In his description of the talk, Street wrote, “In an industry that does so much to uncover and expose the mistakes of others. [SIC] Which, don’t get me wrong, is a valuable service in helping to increase security by the discovery of these vulnerabilities, it seems everyone though is very shy about pointing out their own failures! I’ve decided that I could help teach others valuable lessons I learned by showcasing…failures I’ve had…”

No one likes to admit his or her failures, but airing them in the open—albeit in a closed forum like a security-community conference—is important; if security can’t admit its mistakes and then learn and grow from them, we’re bound to remain the proverbial “problem child,” kept at arm’s length and not truly progressing toward more secure organizations.

I’ve paid my dues

The first failure presented by Street was: Treating everybody like they’re always doing something wrong. Oftentimes security staff deal with anyone outside of security as if that person is actively trying to screw up security processes and practices. Frequently at conferences I hear, “security is everyone’s job,” but that’s false. Security isn’t everyone’s job in the same way that human resources or finance isn’t everyone’s job. While certain people on the security team might hire and train new staff and/or manage a budget, those tasks are elements of the job; security isn’t expected to know the ins and outs of EBITDA or labor relations. By constantly pointing out to non-security staff how they’re destroying security’s efforts, Street said that security teams must engage and teach other employees parts of the security process that are relevant. Instead of scolding someone for clicking a link or forgetting his employee badge (which only teaches that person to want to avoid security), offer ways for that person to get involved and show her or him how they’re assets to the program. “We work for them,” said Street. If the business weren’t selling products or services and hiring people and expanding into new markets and all the things for which businesses exist, infosec wouldn’t have data to protect and systems to maintain. Treating employees like the “bad guy” doesn’t add value, and it won’t make people do what you want them to do; empower employees to get involved in security instead of acting as an “enforcement arm” of the company.

Time after time

The second “don’t do” was: Treating the networking team like the adversary, and escalating problems too quickly. Street shared an anecdote about an instance when he jumped the gun on a firewall alert, running straight to the CIO and declaring an incident. After much ado, it turned out that the networking team was running a scan which produced the alert, but a perfectly non-threatening one, as it was part of the test. While identifying and acting on critical alerts is important, Street moved too quickly—and in isolation. Because he didn’t bother to cultivate working relationships with the networking team, he missed an opportunity to be better prepared. Had Street been talking to the networking team on an ongoing and consistent basis, developing effective communication channels before a crisis, he would have been much more likely to know what types of projects were being worked on and been able to avoid crying wolf to the CIO.

Networking teams’ objectives include keeping systems up and efficient for the organization, while security often tries to slow things down, taking time to ensure systems have been deployed properly, are patched and up to date, and aren’t leaking data. Even though goals seem to be in opposition, teams to live side-by-side, and security would do well to collaborate more with networking groups. When you get down to it, the C-suite values uptime and efficiency more highly than it does stopping systems to check if they’re working properly. And, yes, a possible breach is lower on the totem pole—given that it’s a potential occurrence, not definite—so Street advised security pros to start working with networking staff in advance of an incident. Build a collaborative relationship. “Listen first,” Street said, “things work more smoothly when you work together.” Plus you won’t end up with your tail between your legs when you make false assumptions or accusations because you didn’t know what was going on with the team sitting right down the hall.

I’ve done my sentence

The third failure exposed was: Playing offense and acting too adversarial. This goes along with points #1 and #2; no one likes a person who is constantly in opposition and trying to make others look stupid. “When did the client become the enemy,” asked Street. When the security team finds a problem, that problem shouldn’t be used to ridicule the person or team responsible. Finding vulnerabilities helps secure companies, “not the other way around,” and instead of waving the vulnerability/finding/issue like a failure flag, use it to inform next steps and fix problems. Use the vulnerability as an opportunity to work more closely with other departments and ask how security can help them get their jobs done.

For this example, Street drew on a past red team engagement; as an outside consultant he was brought in by senior management to find system weaknesses and produce a report. Rather than pointing out every found problem, he said he could have used that instance to help the internal team convince senior management of the security enhancements required. Instead, he chose to mock the internal team…and he was never asked back for another engagement. Aside from business lost, the lesson was that helpful people get farther than those who scorn.

But committed no crime

Keeping with this theme, number four was: It’s not a game to make others look stupid. During one previous pen test, Street walked through the organization that hired him, breaking all kind of rules and acting (he felt) obviously about his actions. Yet it took a long while for employees to notice anything out of the ordinary. Having learned from some previous failures, Street decided to keep pushing boundaries until someone, finally, caught him. He then took the time to praise the person’s actions, rewarding him for doing the right thing by stopping Street from removing a server from the premises. It took a while, but he “let the client win,” because the game, he said, “is about awareness and teaching,” not shame, blame, or superiority. “You’re dealing with people,” Street reminded the audience, and no one likes to be treated like a bad dog or an idiot. Doing so won’t help promote better security and won’t even make you feel good about yourself in the long run.

And bad mistakes

Moving aherad to the fifth lesson learned, Street talked about harsh criticism in the security industry. More often than not, a public breach incites judgement and finger pointing. Security practitioners not involved in the breach or in any way associated with the breached company feel free to write blogs and post tweets about what went wrong and all of the things that could have, should have been done differently or better. It’s so easy to be a sideline critic, though, warned Street, and we should “never judge someone for their failings more than we want to be judged ourselves.” Yes, every failure is a teachable moment, but instead of throwing shade, ask yourself: What did I learn from what happened? It could be you on the dark side one of these days (is anyone perfect, really? How many companies are honestly immune to breaches?), and no one is as inclined to improve when they’re being treated like the laughing stock of the community.

I’ve made a few

And finally, Street’s closing advice was to help industry newbies. It’s a systemic failure when the industry requires more practitioners but ignores, or worse, criticizes those with lesser experience. Shame shouldn’t result from trying something new and not executing perfectly. Everyone in the industry was new to security at one point. Yes, some were new when security wasn’t as robust an industry as it is now, but today’s there’s so much more to learn and more to secure, and to accomplish those goals, we need new people to get involved—and security practitioners need to be more supportive, said Street. “Share your passion; respect others’.”

In closing, Street likened security pros to dentists and urged the audience to “stop talking about rock stars.” Many industries, security included, choose a few standout or vocal representatives and put them on pedestals. These are the voices other practitioners attend conferences to hear. Their thoughts are echoed on social media. Their words are quoted in presentations. While role models are good to have, “There is no level [among security practitioners]. There is no one better than someone else. Some [people] get more known, others don’t.” Security practitioners, in truth, said Street, are just like dentists. Dentists tell patients what’s needed to keep their mouths (systems) healthy and ensure cavities or root canals (breaches, intrusions), which cause painful and expensive procedures (incident response, forensics, job loss) don’t occur. In the best cases, dentists perform preventative tasks (patching, system updates, monitoring) to keep patients’ teeth (networks) healthy (secure).

I’ve had my share of sand kicked in my face, but I’ve come through

Advice on how to become a more effective security practitioner from a human aspect wasn’t expected at DerbyCon, but it was refreshing to hear. The industry can’t improve if the focus isn’t on collaboration and influence. No job department operates in a vacuum; the idea that people are bad, stupid, or our adversary if he or she doesn’t know or practice the same security as infosec staff needs to decline. Hard enough problems in cybersecurity exist. Let’s stop making it harder on ourselves and work on the things that matter.

Get daily email updates

SC Media's daily must-read of the most current and pressing daily news

By clicking the Subscribe button below, you agree to SC Media Terms and Conditions and Privacy Policy.