Understanding the Value of Use Cases
To demonstrate what a difference Use Cases make, we will take a look at a company that was using only the built-in alerts. Then, we will take a look at that same company after they replaced those built-in alerts with alerts based on Use Cases. More importantly, we will look at their signal to noise ratio both before and after that change. Once we have, you will understand the benefits that you get from Use Cases. OK. Let’s begin.
Initial Setup and the Problem
The company that we will be discussing used a local provider to deploy their SIEM. It was a well-known provider. This well-known provider deployed the SIEM with the built-in alerts. Several months after the SIEM was deployed, we checked in on them. We wanted to see how it was going. We asked them, how many of the alerts that they were looking at day to day were false positives? It is a good question to ask. It lets you know how well you have done setting up your SIEM. It won’t necessarily tell you where you might be having problems, but it will certainly tell you if you have one. Assuming that the SIEM was set up properly, that number should be low. Theirs was not. It was somewhere around 96%. That meant that they were wasting 96% of their time. To put it another way, their signal to noise ratio was 4%, which was horrible. This meant that despite spending all that money, they were getting very little out of their SIEM.
Introducing the Use Case Strategy
Once they saw that number, they realized that something needed to change. We offered them a change. We offered to replace one of their built-in alerts with an alert based on a Use Case. They agreed. Our choice was the “Excessive Failed Logins” alert. Why did we want to replace it? We wanted to replace it because it would be triggered by both users and attackers. It would detect both mistakes and guessing. Given that users are much more common than attackers, the SOC would be spending most of their time chasing down users—which is, quite frankly, a waste of time.
Designing an Effective Use Case
Our job, then, was to replace that built-in alert with one based on a Use Case. An appropriate Use Case would be to detect “Password Guessing.” For that, we needed to ask ourselves: how is guessing different than mistakes? Well, when a user makes a mistake, they make a mistake with only their account. We would only see one account failing logins from any particular IP address. An attacker, however—guessing (or even spraying)—is going to be failing with multiple accounts. They are not likely to be successful with just one account. They are going to need to try multiple accounts. Trust me, we know this from experience. That being the case, the signature for this Use Case wouldn’t just be three or more failed logins, but three or more failed logins involving more than one account.
The Impact
With this in place, we asked them again: how many of the alerts that they were looking at day to day were false positives? It was now somewhere around 90%. With just one built-in alert replaced, they had increased their signal to noise ratio. They had gone from 4% to 11%. That was not a lot, but it was a start. More importantly, it was a start in the right direction. From that point forward, they knew what they had to do, and Hellfire was there to guide them the entire way.
