Penetration testing, or pen testing, is the practice of testing a computer system, network, or web application for potential security vulnerabilities that an attacker might exploit.
This is a valuable tool that can be used to ensure security controls are working properly, validate remediation, or meet compliance or customer requirements. However, it’s wise to be realistic about what the results of a pen test really tell you about the efficacy of your security program – and what they don’t. Interpreting the results incorrectly can leave organizations with a false sense of security.
In this blog series, we’ll be taking a look at three key aspects of penetration testing – scope, timing, and approach. We’ll look at the impact each of these has on the authenticity of your results, the benefits of penetration testing, and when attack surface management or red teaming would improve your process.
Let’s start with scope.
Where Pen Tests Fall Short
If you work as a security professional, you’re probably familiar with a traditional two-week network penetration test. You may conduct them once or twice a year to meet a compliance requirement or more frequently as part of an internal testing program to help identify vulnerabilities, demonstrate improvement, and assess current and future security investments. That’s a lot to expect from a two-week test.
A pen test, by design, relies on what is called a “white box” approach – you provide a list of assets you want to have included in scope for an engagement and those assets are then attacked. Designed to provide you with insight into the efficacy of your controls within a specifically defined scenario, they often miss things a real adversary is likely to consider. This is because attackers tend to seek out assets which are unguarded and therefore would be missed by a test of limited scope. While pen testing is great for validating controls, they are not a good assessment for the efficacy of a holistic security program.
If your goal is to understand how well specific controls are working or meet a basic compliance requirement – penetration tests are a good place to start. However, if your goal is to gain a holistic assessment of your organization’s security posture, you’re likely better off considering a more authentic approach – such as a red team.
Four types of assets your pen test strategy may be missing:
Pen tests alone cannot give you a holistic view of your attack surface. To increase the scope or frequency of a pen test is to increase the budget. Using an attack surface management solution, you can determine the necessary scope of a pen test ahead of time and drastically reduce costs and increase ROI.
Below, we outline some of the most common assets left out of a penetration test but an adversary would consider fair game. These are the kinds of assets that, when overlooked, could lead to future issues for CISOs and security teams.
We have also included a few tips that can help you ensure a more authentic assessment of your security program’s effectiveness.
1. Unknown Assets
A penetration test is only going to test the things you tell it to test. You provide a service provider with a list of assets to test and off they go. If you have shadow IT or other unknown systems lurking on your perimeter, the results of a pen test could be painting a rosier picture than reality.
For instance, a recent report, “What’s Lurking in the Shadows 2020,” noted that 89% of IT leaders were concerned about shadow IoT devices lurking on remote or branch locations of the business. Hackers love to go after assets like these, as well as others that have slipped through the cracks – like ones acquired in a merger or acquisition, supposedly decommissioned yet still active devices, or less secure testing and development environments.
Tip: Make sure you have a strong asset management program in place. You’ll also want to leverage an attack surface management solution like Randori Recon to identify things your asset management program may have missed. With organizations shifting to the cloud and the proliferation of self-service IT, it’s common to find 10-15% more assets than initially estimated.
2. Third Party Assets
The hard reality is there are assets that security teams may want to test, but they can’t legally touch. For example, it’s not unusual for an organization to have a large volume of third-party hardware assets, such as leased office equipment like phones or printers. There are also software-as-a-service (SaaS) applications like salesforce.com, Microsoft Office 365, and Box – where you own the data but not the software.
Depending on the contracts you have with these organizations, you may or may not be able to include them in a pentest. However, these assets are still on your network or house your data – and therefore, your problem. Make sure you’re taking the time to assess and consider the risks these assets pose, as they will never show up in your pentest report.
Tip: It never hurts to ask. Many IaaS and SaaS providers are changing their policies to enable clients to pentest hosted systems as long as they are notified. If you can’t get permission to run a test yourself – do what you can and get a passive adversarial assessment of the system so you have some insight into the risk posted by those assets.
3. Business-Critical Assets
Depending on the size of your company, you may be limited in testing by internal restrictions or business requirements. Politics is often cited as one of the biggest blockers to improving an organization’s security, so security teams need to be smart in how they compensate for these internal limitations. Blackout windows, mission critical systems, old or outdated systems, or even fear can all limit the results of penetration tests, but the risks they pose remain. Once again, just because an environment can’t be included in a pentest doesn’t mean it is off limits to an adversary – so make sure you have a plan to compensate for the lack of insight.
Tip: Try and make testing an opt-out, not opt-in requirement. Business owners often miscalculate the risk of service disruptions from testing, resulting in them restricting assets that often pose the greatest risk. In these scenarios, work with business owners to define an acceptable testing process and, if possible, ask your testing provider for data on the stability and efficacy of their tests to help reduce business anxiety associated with testing.
4. Unmanaged Assets
Your attack surface is more than just your on-prem systems. If the goal is to get an authentic assessment, consider the assets that may be connected or have access to your network, such as a building management system, partner portals, work-from-home devices, subsidiaries, or even assets used and owned by third-party vendors. These are systems that a hacker could get into, as an attack on Target demonstrated. An adversary got into its HVAC contractor’s system for electronic billing, contract submission, and project management. The adversary was then able to make its way into Target’s network, resulting in a breach and the theft of secure information on up to 70 million shoppers.
Tip: You may not have the authority to test certain assets, but they remain tempting access points for an adversary. Make sure to test nearby internal systems and ensure you have the necessary visibility and detection in place on those devices to catch an issue should it arise.
Bottom Line
Scope matters, when reading a penetration test make sure you’re thinking about all of the assets your penetration test may be missing. In order to make the most of your pen test budget, you need to determine the proper scope ahead of time. Use attack surface management to get an accurate picture of where to deploy pen testing.
Attack surface management is continuous and automated, so you can stay current and up to date on your attack surface even as new cloud assets spin up and down. Next up in part two, we’ll take a look at timing and how two week engagements distort reality and often produce misleading results.
