Ä¢¹½TV

UPDATE 03/13/2023 2252 ET: After taking further inventory of our partner's Veeam service binary details to review the version number, we uncovered many more unpatched and vulnerable hosts. We are sending incident reports for all affected partners and making direct phone calls when Veeam is publicly exposed on the Internet.

On 7 March 2023, outlining CVE-2023-27532, a vulnerability in the Veeam Backup & Replication component that allowed an unauthenticated user to retrieve host credentials stored in the configuration database.

This weakness could ultimately enable an attacker to gain access to hosts and devices managed by the Veeam Backup server. With access to the open TCP port 9401, any individual could obtain credentials and potentially move laterally throughout the network with the newly exposed username and passwords. Any Veeam Backup & Replication version prior to V12  () and V11a () is vulnerable.

With a High severity rating and a CVSS score of 7.5, this vulnerability made its rounds across headlines, news outlets, and further scrutiny by security researchers.

If you do not know your Veeam version and build number, the following PowerShell lines will display it.
If the output does not match any of the build numbers mentioned previously, [.underline]you need to patch[.underline].

[.highlight]$CoreDllPath = (Get-ItemProperty -Path "HKLM:\Software\Veeam\Veeam Backup and Replication\" | Select-Object -ExpandProperty CorePath) + "Veeam.Backup.Core.dll"$CoreDll = Get-Item -Path $CoreDllPath$CoreDll.VersionInfo.ProductVersion + " - " + $CoreDll.VersionInfo.Comments[.highlight]

Outreach

The Ä¢¹½TV team proactively identified partners running vulnerable Veeam software versions on servers with a Ä¢¹½TV agent and has sent incident reports with specific hostnames and Ä¢¹½TV agent IDs. If you received a report via email or PSA integration, we urge you to patch and upgrade to the latest available Veeam version as soon as possible.

If you do not have a Ä¢¹½TV agent installed on all hosts and we were unable to detect your Veeam instance, you are of course still urged to patch.

Ä¢¹½TV partners with a publicly exposed Veeam instance and TCP port 9401 open to the Internet have received a direct phone call to urgently patch or block external connections from the firewall. (Frankly, these were single-digit occurrences.)

Out of our 2 million install base, we uncovered north of 7,500 hosts with a vulnerable version of the Veeam Backup & Replication service present.

Investigation

Following the initial advisory of this new Veeam vulnerability, there was not yet a publicly available or commoditized proof-of-concept exploit. At the time of writing, Ä¢¹½TV has not observed any exploitation in-the-wild.

However, on March 9th, Ä¢¹½TV researchers caught wind of from .Ìý

CVE-2023-27532 in Veeam Backup & Replication is serious, expect exploitation attempts soon. Our teammate was able to develop an exploit just by using the exposed API.

, a researcher we have worked with before, had crafted a proof-of-concept exploit to demonstrate both the simplicity of the attack vector as well as the potential impact. Usernames and passwords were easily obtained, decrypted, and this technique merely required access to the exposed API of the Veeam Backup & Replication component (Veeam.Backup.Service.exe).

Seeing the tangible impact, Ä¢¹½TV researchers went to work recreating the proof-of-concept and assessing how this may affect our partners. We reached out to Code White GmbH for further discussion and we are always grateful for their transparency and support in information sharing. We successfully recreated the proof-of-concept and were able to extract login credentials.

Technical Details

Exploitation is achieved by interrogating the local API, served on port 9401 listening on all interfaces (0.0.0.0) by default.Ìý

The implementation of this API is specific to the Veeam Backup & Replication server. After configuring a client to communicate with this API, querying for credentials is trivial. Credentials are initially encrypted, but a separate API call can easily decrypt them into their original plaintext value. There are no child processes created, nor any filesystem or registry artifacts left behind.Ìý

With the default settings of the Veeam Backup & Replication server, no logs related to this attack are left behind. Logs specifically for the API can be configured with the registry changes discussed in the Indicators of Compromise section below.

Our Proof-of-Concept

Ä¢¹½TV researcher Caleb Stewart has recreated the proof-of-concept to demonstrate retrieving credentials from a vulnerable Veeam Backup & Replication server instance. All of our testing for the vulnerability and exploitation was tested against the installation provided by the ISO file VeeamBackupReplication_11.0.1.1261_20211123.iso (SHA256: 0dcebe370ec61bd817ebeadcc827f7aadac3e5011f7a4e5967c50df85380074c)

The following video showcases Caleb’s efforts to leak and decrypt credentials via CVE-2023-27532:

With this proof-of-concept, the Ä¢¹½TV team was able to rapidly detect and safely check for publicly exposed vulnerable instances without any exploitation. This enabled us to make more accurate reporting for partners and determine how urgently they need to react and respond.

Additionally, the Ä¢¹½TV team uncovered other potential API calls that could be weaponized. We were able to further expand our proof-of-concept to achieve arbitrary code execution.

While the unauthenticated credential dump acts as a vector for lateral movement or post-exploitation, the vulnerability in question can also be used for unauthenticated remote code execution - turning the vulnerable Veeam instance itself into a vector for initial access or further compromise.

The patch resolves both these issues of unauthenticated credential dumping and remote code execution.

The following video is a demonstration of Veeam Backup & Replication server exploited to run arbitrary commands:

As a public proof-of-concept has not yet been shared online, and there have not yet been any observed exploitation in the wild, we are withholding specific technical details that might enable adversaries or threat actors to craft these techniques.

Validating the Patch

The official release patch and security update from Veeam successfully remediates CVE-2023-27532 and other unauthenticated attack vectors against the API.Ìý

Veeam Backup & Replication service versions V12.0.0.1420 P20230223 and V11.0.1.1261 P20230227 make changes to the underlying codebase to require an authentication token with all requests to the API.Ìý

This is accomplished with a JSON Web Token () that is signed with a unique certificate specific to the Veeam Backup & Replication installation. Empty or invalid tokens cause an access exception and are unable to retrieve information or make changes.

Indicators of Compromise

While there are no child processes created following exploitation, logs are generated under C:\ProgramData\Veeam\Backup\Svc.VeeamBackup.log. However, by default, logging is not configured to include API calls.

Without manually changing the log level, the Svc.VeeamBackup.log will not show any entries related to the attack technique.

By default, the Windows registry value HKLM\Software\Veeam\Veeam Backup and Replication\LoggingLevel is set to the DWORD value of 4. To observe API calls, this value must be set to 7.

The following commands ran from an Administrator PowerShell prompt will configure logging to capture API calls:

Without sharing the specific API methods, the pertinent log entries look like the following after our proof-of-concept exploitation:

The official Veeam knowledge base advisory explains these credentials are encrypted, which they are -- the database manager does not return plaintext credentials, but it does return encrypted values and each account UUID.Ìý

Once the account identifiers are obtained, further API calls can be made to decrypt the credentials into a of the original plaintext value.