Strategies for Building an Effective, Resilient Security Operations Center

The modern Security Operations Center (SOC) has morphed and matured since its infancy in the early 1990s. The primary responsibility of monitoring for any indication of intrusion or compromise has remained a critical and valuable control. The modern SOC now encompasses more robust information to assist in its primary objective, including correlating data from asset management, vulnerability management, data loss prevention systems, and cloud access security brokers to events to provide enriched information to our investigators. Modern SOCs can deploy more proactive systems that become much more powerful when combined with security orchestration and automated response. However, even with all of the technology at our disposal, the fundamental challenges of a SOC remain. In order to be an effective and efficient control within our environments, we must manage and mature our SOCs along five fundamental channels.

• Visibility
• Alert effectiveness
• Investigative prowess
• Threat intelligence
• Incident response

Visibility

At the core of any SOC is monitoring, and effective monitoring hinges on visibility. Gaps in visibility can result in unmanaged risks and present real danger in the event of a cyber incident. Still, not every event holds the same level of security value. While every security professional desires abundant data, we all struggle to prioritize data based on its potential impact for an investigation.

An investigation is only as good as the data it is based on; as such we must first take steps to ensure the integrity of events. As a baseline, ensure that all devices are reliably receiving their network time protocol sync from the same sources, that the events in the SIEM have the needed information, and that the ingestion uptime is reliable.

Since the vast majority of incidents will involve either a user, a device, or multiples of both; use the rule of non-repudiation to find gaps within monitoring. Every event should correlate to the account or device it originated from. Performing exercises to practice non-repudiation within logged events will ensure we are prepared for any investigation and speed up the root cause analysis process during an incident. The operational and investigative teams should be challenged to answer simple investigative questions: who, what, when, where, why, and how. Regularly practicing these challenges not only familiarizes the team with the network but also helps keep them calm and confident in the face of a true incident.

Alert Effectiveness

There are many articles discussing analyst burnout or false positive churn, leading most of us to believe it is inevitable. This certainty has fostered a culture of “burn and churn,” where we inadvertently devalue our newest employees in the name of giving them a “foot in the door.” Regrettably, the expectation that they will leave in a year or so also often leads security leaders to neglect investing in their employees.

This process becomes endemic, and the effectiveness of the SOC becomes stagnant and subject to regression. The last line of defense is when an attacker gains access to YOUR network, and it is staffed with untrained, burnt-out, tier-one analysts. No wonder so many have decided to outsource to managed services. Unfortunately, that is not always a remedy, as most managed shops suffer from the same challenges.

So, how do we get out of this deep rut? The answer lies in alerting effectiveness.

Common tuning methods require that we stomach a large load of false positives in order to never accidentally get a false negative. The false positive load does more damage than good, causing our analysts to become “snow blind” and miss the real events as they pass. We essentially move the problem from the technology side to the human side. Since humans are not good at processing large amounts of data, we are setting them and ourselves up for disappointment, failure, and quite possibly setting our companies up for material impact from a cyber event.

Any good engine must be tuned, and that tuning requires checkups to ensure effective and efficient operation; our operations software is no different. Tuning can and should be done to mold the alert system to the environment. Train the alerts to know their environment, and the operations team will catch more. The more we tune into our infrastructure, the better the results.

But what about the dreaded false negative? Well, if we also spread alerts to fire throughout the cyber kill chain, it is less likely that a single miss will compromise the environment. In fact, having a deep bench of alerting throughout the stages of an attack builds trust and confidence in our systems that malicious actions will not go unnoticed.

Investigation

Consider this scenario: the team has seen something strange, but before they reach out to the Cybersecurity Incident Response Team, they need to dig in and confirm the event is truly an indicator of a successful attack or compromise. This requires investigation. Remember when I spoke about training analysts earlier? This is where it needs to be. Analysts are digital investigators. They need to understand the investigation principles and incident response and be constantly exposed to attacker’s tactics, techniques, and procedures to recognize them when found in the logs.

There should be a sense of trust that investing in our analysts with the intent of creating professional investigators and incident responders will yield a high return on investment. They cannot be considered entry-level security employees, but rather, like architects or engineers with distinct skills and abilities who gain the same level of commitment from security leaders. When analysts can trace every step the penetration tester took within the environment, we can all rest easier.

Threat Intelligence

Every incident response flow chart should start with sensor data. Sensor data are the alerts that kick off response. This could be everything from signature or behavioral findings to an article or random conversation in the hall. Quite often, though, the best sensor data comes from indicators delivered from investigating real events. The more related those events are to the company, the better.

There are many threat intelligence feeds out there, and quite often, we add them only to never hear from them again, or we remove them a few days later as they are alerting on standard processes. Threat intelligence feeds vary in quality, and like most things in cybersecurity, there is a balance between the amount of effort put into managing the system and the return on effort. Properly investing in threat intelligence by actively managing the indicators within the feeds yields much higher value. Curating feeds with data for the business vertical or even data from the security operation team’s investigations can be incredibly powerful. Unfortunately, threat feeds are rarely valuable as a set-and-forget alert producer.

Incident Response

Finally, a SOC is essential to cyber incident response flows. When we need them most, there must be full confidence that the SOC team can investigate and provide timely evidence. Train them, involve them, tabletop with them, and you will be thankful for the confidence in the face of adversity.

The cyber security operations center is a longstanding control. It plays a crucial role in our efforts to minimize the effects of cyber incidents on our organization. Ultimately, investing time and energy into the control can yield high returns.

About the Author

William Wetherill is currently the Chief Information Security Officer (CISO) for DefenseStorm. He is a Certified Information Systems Security Professional (CISSP) and Certified Information Security Manager (CISM) with extensive training, background, and experience in various aspects of IT systems and applications. He has over 27 years of IT experience, almost a third of it directly in cybersecurity. William was the Director of Cybersecurity Operations overseeing the 24/7 SOC at DefenseStorm before being promoted to CISO in January 2024. William was previously the Chief Information Security Officer at the University of North Carolina in Wilmington (UNCW) where he built and managed their Information Security Program.

William can be reached online at [email protected] and at our company website https://www.defensestorm.com/