Citrix Vulnerability Hits Private Sector Hard - The Monitor, Issue 12

Kroll’s intake reports for the month of January emphasize the severity of Citrix vulnerability CVE-2019-19781 in terms of its cross-industry spread and the innumerable actions threat actors can take once they gain access to an enterprise network. CVE-2019-19781, which was first detected on December 17, 2019, allows unauthenticated users to gain access to a company’s local network and remotely carry out ad hoc code execution. Over 25,000 servers globally are estimated to be exposed. 

In our casework, Kroll incident responders observed numerous instances where threat actors exploited the vulnerability in the Citrix Application Delivery Controller (ADC) and Citrix Gateway products. Attacks ran the gamut from closing virtual desktop sessions to extracting and sending data to unauthorized IP addresses. 

Kroll analysts assess that the large number and frequent occurrence of incidents concerning CVE-2019-19781 are likely due to the readily available proof-of-concept (PoC) codes and exploit scripts circulated on hacker-frequented websites. Publicly accessible and weaponized PoC exploit codes—like those issued by Project Zero India and TrustedSec, as well as those in the form of a Metasploit module (discussed in more detail below)—have lowered the technical threshold needed to exploit the vulnerability, making it relatively easy for threat actors to take advantage. 

In every case observed by Kroll, threat actors executed or were suspected of executing unauthorized commands within a client’s network. Kroll incident responders most commonly identified threat actors executing Linux commands to initiate a software utility cron (aka cron job – i.e., a time-based job scheduler), instructing a machine to periodically exfiltrate data back to the actor at set time intervals. 

Of the CVE-2019-19781 network compromises observed by Kroll in January, 50% concerned medical service providers and healthcare systems. According to open-source reporting, the hospitals/healthcare sector is the third largest industry segment to utilize Citrix, after computer software and information technology. The large volume of healthcare providers and services employing Citrix applications made the industry particularly susceptible to the recent vulnerability.  

Technically Speaking – Proof-of-Concept Codes and Exploit Scripts

Two independent groups, Project Zero India and TrustedSec, were among the first to publish PoC exploit codes on GitHub. The Project Zero India PoC exploit code includes two curl commands: first, to write a template file that includes the user’s shell command, and second, to request to download the result of the command execution.¹ (Figure 1)

The TrustedSec PoC exploit code is a variant of the Project Zero India code, but is written in Python and includes a reverse shell.² (Figure 2)

Figure 1 - Project Zero India PoC Exploit Code (Accessed via GitHub)

Figure 2 – TrustedSec PoC Exploit Code (Accessed via Pastebin)

In terms of the Metasploit module, the script allows threat actors to execute a payload through the remote code execution vulnerability.³ The accessibility of the Metasploit module, as well as of the two PoC codes, have made exploitation relatively simple for threat actors of all skill levels.

There was a week-long window in early 2020 where Gateway and ADC users were defenseless against potential threat actor attacks, starting on January 10 when the PoC exploit codes were first released, up until the initial Citrix fix issued on January 19. During this period, CVE-2019-19781 was the subject of frequent chatter on hacking forums, and compromised company information was regularly exposed. (Figures 3-6) The considerable number of references to the Citrix vulnerability on discussion websites like RaidForums and others highlight the significant value threat actors attribute to the weakness.

Figure 3 – Observed CVE-2019-19781 Chatter Example #1 on Hacking Forums (Accessed via RaidForums)

Figure 4 – Observed CVE-2019-19781 Chatter Example #2 on Hacking Forums (Accessed via RaidForums)

Figure 5 – Vulnerable Citrix User IP Addresses Exposed (Accessed via Pastebin)

 Figure 6 - Vulnerable Citrix User VPN Exposed (Accessed via Pastebin)

The Department of Homeland Security Cyber and Infrastructure Security Agency (CISA) also uploaded a PoC exploit code to GitHub, in addition to posting an alert to their official government website. (Figure 7) The CISA release was to assist vulnerable companies with a code that would simulate an actual attack, allowing IT departments to then patch the security hole without compromising enterprise data.

Figure 7 – CISA PoC Exploit Code

Case Studies

As of this writing, 50% of CVE-2019-19781-related incidents reviewed by Kroll have occurred within the healthcare industry. In one noteworthy case, the threat actor(s) terminated a compromised user’s virtual desktop session and then elevated the victim’s account with administrator privileges. This was done to execute a data ingestion command that routed company information to an unauthorized destination.  

In another incident, the threat actor(s) exploited CVE-2019-19781 to extract a virtual disk image (VDMK) file, transmitting the file to an overseas IP address associated with cryptocurrency mining.

Kroll Experts Corner – Mitigation of the Vulnerability

Prior to the release of patches and firmware updates, Citrix recommended Citrix ADC and Citrix Gateway users scan their systems for any exploitation attempts using “grep” requests containing “..” and “vpns”. 

Other suspicious executables to scan included:

• Curl
• Hostname
• Whoami
• Commands run by the user “nobody”

Citrix had released firmware updates through refresh builds available for every vulnerable version of Citrix ADC and Citrix Gateway by January 31, 2020.  Patches for Citrix ADC version 11/12 and 13 became available on the Citrix website on January 19, 2020.  Patches for Citrix ADC version 10, the final version to be addressed, was released on January 31, 2020.⁴  

Organizations should note that Citrix emphasized once a system has been exploited by the vulnerability CVE-2019-10781, the system cannot be patched to expel the attacker from the system.   

Steven Coffey, Senior Associate in Kroll’s Cyber Risk practice, recommends clients check their systems immediately, for any potential exploitation attempts through vulnerability CVE-2019-19781 and immediately proceed with mitigation tools and techniques, including the following:

• Implement latest patches
• Follow Citrix’s remediation instructions
• Review /flash/ns.conf and /etc/passwd files for sensitive accounts
• Change passwords of sensitive accounts
• Revoke and reissue security tokens within the /flash/ns.conf file
• Review any suspicious cron job named nobody under /var/cron/tabs
• Audit Active Directory environment for any suspicious logins involving the account specified within ns.conf that binds to Active Directory
• Delete any suspicious file(s) under:
  • /var/vpn/bookmark
  • /var/tmp/netscaler/portal/templates
  • /var/run/nshttp_profile_ids
• Check for and review any web-related files such as PHP, Perl or other executables modified during or after the incident. This will help determine if any backdoors or malicious files might possibly be located elsewhere in the system.

Is Your Organization Vulnerable? 

The level of unauthorized activity, enabled by the CVE-2019-19781 vulnerability, requires an immediate assessment of your Citrix devices. As noted by Citrix, patching after an intruder has entered the system will require a separate effort to remediate that particular threat. Additionally, organizations will need to assess whether sensitive data, such as personally identifiable information (PII), has been compromised during the period of unauthorized access and any ramifications for breach notification. 

Kroll’s incident response team has extensive experience with matters related to the CVE-2019-19781 vulnerability. If you need additional details, have questions, or suspect unauthorized access has already occurred, contact a Kroll expert today. 

Sources
¹ Threatpost[.]com Accessed January 10, 2020
² hxxps://github.com/trustedsec/cve-2019-19781/blob/master/cve-2019-19781_scanner.py
³ https://packetstormsecurity.com/files/155930/Citrix-Application-Delivery-Controller-Gateway-10.5-Remote-Code-Execution.html
⁴ hxxps://support.citrix.com/article/CTX267679

The article above was extracted from The Monitor newsletter, a monthly digest of Kroll’s global cyber risk case intake. The Monitor also includes an analysis of the month’s most popular threat types investigated by our cyber experts. Subscription is available below.