Our community has been plagued with quite a few breaches last year: Kaysea, JBS Meatpacking, Colonial Pipeline, Park Mobile and most recently, Log4j. These have been actively exploited by bad actors, ransomware entities or nation states. We saw thousands of vulnerabilities and zero-days disclosed and IT teams scrambling to patch and mitigate exposure.
While 2021 was not an aberration, it seems that as a community, we were tragically underprepared for it. When patching is our current best response, what happens when, as it did in this case, patching is not possible? What happens when affected systems are buried in third party appliances or undiscoverable via scanning? I believe if we want 2022 to be better than 2021, as an industry we need to focus on making our systems more resilient – we have to start thinking today what we can do to be ready for the next Log4j.
The beautiful thing about the security community is how much we care. We all feel the pain of incidents like Log4j and Solarwinds together. When I was at Bit9, we experienced a breach which deeply affected both the company and me personally. That experience, and the lessons we learned from it, is the reason we started Randori.
As I am watching and participating in the conversations cropping up in the security community this year, I’ve been observing and reflecting on the state of cybersecurity, and these are my observations:
- There will always be another bug. This has always been our reality. It’s our reality today. It will be our reality tomorrow. Log4j isn’t the first, it won’t be the last.
- Patching cannot be the only answer. Given the ubiquity of Log4j, it’s going to take the community months, perhaps years, to identify vulnerable applications. CISA and many others, including Randori, are helping on this front – but we’re not going to be able to patch our way out of this one anytime soon. In lieu of a catch-all solution, we need to ensure we not only patch, but invest in making our networks more resilient to attack.
- Exploitation increases dramatically after disclosures. We’ve known for a long time that newly disclosed vulnerabilities are often quickly weaponized, but the data from Greynoise on this is stunning. Our team at Randori had a working exploit in 5 hours and the attacker’s were not far behind. Within 48 hours, Meijer (one of our customers) was already seeing 4,000 attacks against their systems.
- You can be resilient in the face of a compromise — but you have to take the offensive. Vulnerability management is foundational, but it’s not enough. Following the Bit9 breach, we realized we had to be our own worst enemy – to test our assumptions as hard as the enemy. We took the offensive and it paid off. At Randori, this approach is paying off for our customers too.
Lessons Learned
While I could share dozens of stories from 2021, I want to share a recent story that gives me great hope for the future. I want to share with you the 24 hours after Log4j was announced – from Randori’s point of view.
I was in Denver with our founding team, enjoying dinner and margaritas, when news broke. It was around 7pm, Thursday night of December 9th. For anyone who knows our team, you’ll know we immediately got up and walked back to the office. Huddled in a small conference room – with additional team members on video – within hours we were able to confirm exploitability and use Log4j to infiltrate several customer environments and alert more than 100 more organizations running our ASM solution to their potential exposures. Game over right? Not so fast.
While vulnerable to the Log4j bug, two thirds of these accounts stopped us from successfully exploiting the vulnerability – more specifically, two thirds of our customers were successful at blocking exfiltration, and prevented exploitation. We landed, but we were not able to get out with any data. In jiu jitsu, they had us in Full Guard. How’d they do it – they were blocking outbound traffic on internet facing apps. We had spent the prior 11 months practicing with them for this scenario; they’d learned their lessons – and in this case, when it counted – they were ready.
However, that left one third where we, acting as a trusted adversary, were successful. They had not blocked outbound traffic on these systems and we were able to land and expand. Back to jiu jitsu, we had Top Mount. However, because of how quickly our team had responded – we were able to get the customers on the phone Thursday night and work with their security team to drastically improve their defense — before widespread internet scanning and attempted exploitation began.
So what’s the lesson here? Each time we “randori” with customers, their defenses improve and we find them more and more difficult to bypass. Few are ready on day one, but over time each gets better. In the case of Log4j, the work each of our customers had put in over the previous eleven months (the lessons, the ah-ha moments, the validations) – got them to the point that when it mattered, they were able to respond and protect their organizations from Log4j BEFORE mass exploitation began, and before the patches were available. Few organizations can say the same – but I now know that every organization can get there.
As a community, we have the opportunity to take action and make 2022 better than 2021. Doing so won’t be easy – and it will take work.
We need to defend forward, take the offensive and know where we stand – before an attack occurs. That’s what we’re helping companies do at Randori. It is why we started this company. If you’d like to take the offensive in 2022, we can help you take the first step. Get a free attack surface audit – powered by our award-winning Attack Surface Management platform – to find out where you’re most exposed and what steps you can take today to begin building a more resilient security program, please click here.
