Attackers see the world differently. That became clear as I listened to Randori Principal Scientist Aaron Portnoy’s recent talk at the SANS Attack Surface Management Virtual Conference on the differences between the things he looks at as a hacker and those vulnerability management teams often consider.
Aaron Portnoy is a life-long hacker. The original architect of the Pwn2Own contest, and discovered his first zero-day when he was 15. You may have read about him in Time or run into him on twitter sharing knowledge & tips. After watching his talk, here is a summary of four ways attackers view the world differently.
Attackers See What Is, Not What Should Be.
“As an attacker, I don’t really care about what your policies are,” Portnoy explained. “I don’t care about how your system is supposed to work, I don’t care about what designs you have in a document somewhere on a shared drive about what the system is supposed to be doing, I simply care about reproducible behavior.”
The first key lesson I learned during Portnoy’s talk: attacker’s don’t care why something is exposed, only that it is Defenders can tend to get caught up in how or why assets become exposed. To an attacker, if there is an exposed asset, it is nothing more than an opportunity. In order to prioritize real-world risk, defenders must know what their system looks like from the outside.
“The way that most people measure the severity of an issue is with CVSS” Portnoy said. “And if you’ve had any experience like I’ve had, you’ve realized that CVSS scores can be quite misleading at times, it’s very specific to a vulnerability, right? So if you have two mediums or two lows, most scanners don’t say, ‘Hey, if you put those two things together, this is a critical,’ right? Or, ‘Put these 10 things together and all of a sudden you have a really critical vulnerability you need to worry about.’ But that’s the reality organizations are grappling with today.”
Organizations are constantly faced with risks they have no way of knowing about. For example, if an attacker were to spend enough time in the lab to create a zero-day, they could compromise your network without you having any indication that a vulnerability even existed. Meanwhile, access to zero-days has been increasing.
“I mean, I’ve been in this industry since zero-days were a mythical black magic no one talked about to Google releasing blog posts about how various nation states are using these, right? And outing them publicly,” said Portnoy. “We are now in an age of vulnerability brokers, worldwide individuals selling zero-days to companies, companies selling zero-days to other companies, companies selling zero-days to governments, and so on. It’s on the rise and it’s only getting worse.”
Bottom line: if you’re dealing with traditional vuln scanners, you’re only scanning for things that you know about — known knowns. This means you’re prioritizing based off of a metric that may be inaccurate or misleading.As a defender, you need to be very aware of the fact that there are unknown unknowns into which you will have no insight until they affect you. Making this change is key because traditional vuln management systems do not account for that risk.
Attackers See Targets, Not Vulnerabilities.
When an attacker looks at a target, they’re trying to determine if it’s interesting enough to spend time and resources on.
A target in this context refers specifically to an exposed asset that an attacker finds interesting and is taking a closer look at. Different targets have different characteristics and contexts, presenting different levels of interest to attackers. This means they also present different levels of risk to organizations.
“I’m going to look at, okay, this target could be as complex as a piece of custom hardware running in a rack with custom software, custom kernel, proprietary software with a hundred different dependencies,” Portnoy said. “It may have trust relationships with itself in different components, it may have trust relationships with other systems, other networks, right?”
A target is a large holistic complex system. An attacker must essentially build a mental model of this system and understand the interoperability between the different moving parts. This leads them to different conclusions than they would arrive at if simply considering severity.
“I ran the Zero Day Initiative, we acquired vulnerabilities from researchers around the world, published thousands of them a year, we had a team moving quick, getting these vulnerabilities fixed, disclosing them to vendors, assigning CVSS scores,” said Aaron Portnoy. “There is no central authority for validating what a CVSS score coming from a third party is. I can tell you from firsthand experience that we were entering CVSS scores that, in hindsight, were not accurate when I look at them now. However, the down effect of that is that you had quite a few defenders who are trusting that information and making real decisions about their networks and trying to integrate that into their traditional vulnerability management process. So basically, they’re dealing with bad data. So we’re trying to move away from that, I think everyone here probably has had some pain dealing with CVSS at some point.”
Attackers Know They’ll Win Eventually
According to Aaron, every hacker he’s ever met has been extremely persistent. They are used to the grind, and they overcome it with a blind optimism that they will eventually succeed at exploiting any target they have in their sights. And in most cases, they’re right.
Sometimes that means creating a new capability. Sometimes it means sitting back and waiting for a new public n-day to come out to use, or waiting for someone to make a mistake or misconfigure a device. Regardless, they know they will be successful – the only question is when?
“Imagine you have, say, an employee who goes to RSA and wants to demonstrate their product, but they need to set up a VPN of their own rather than use the corporate VPN, right?” Portnoy told the conference. “So they stand something up for an hour to do a demo. Well, as an attacker, if I’ve already determined that a particular thing is interesting or a particular person is interesting, I’m opportunistically waiting for something like that to happen, right? So if I’ve already determined something’s tempting, I have that on my radar. It’s something I’m watching and I’m waiting, and that is kind of the difference between what you may be familiar with as far as a penetration testing or a point in time testing versus a continuous automated red teaming-type process in which we are waiting for you to change something, we are waiting for you to start your migration of networks because it opens up a hole.”
Attackers Ask What is Possible
Here are Aaron Portnoy’s ultimate lessons for defenders:
- Focus on what is actually susceptible on your network and what could happen — figure out the what-ifs.
- You can’t possibly know everything, so focus on the consequences. Ask what happens if something is compromised, rather than what it is currently vulnerable to in this state.
- Don’t play whack-a-mole with your vulnerabilities. Focus on the risk — ask yourself, can you contain a threat if the asset is compromised?
- When identifying risk, focus on attackability, rather than vulnerability data. It is not just vulnerabilities.
When designing a security program around resiliency, you have to remember: you are going to get hit with a zero-day at some point. You need to stop worrying about patching all the vulnerabilities. Of course, patching vulnerabilities will always be part of a good program. But they must be patched in a process focused on resiliency and prioritized by risk.
“It’s if this thing gets popped, what happens?” Portnoy posed. “Do you have logging around that thing? Do you have notifications? Do you have some insight or introspection into that system? And do you know, what you think is important, is that also what an attacker is going to think is important? And if there’s any daylight in between those things, figure out why that is.”
Finally, practice how you fight. Randori, which refers to martial arts sparring in Japanese, is designed around fighting a genuine adversary. If you are competing against an adversary with limited (sometimes inaccurate) information that everybody knows, your practice is not very realistic. You really need to understand the experience of being attacked by someone who doesn’t have any preconceived notions of your network and is going to do everything empirically. To create resiliency and understand risk, you need to practice against realistic, empirical attackers who act the same way attackers in the wild do.
