Isolates root cause of slow performance issues in 20 minutes with Goliath

“We are having major issues with slowness and response times on our new Windows 10 Desktops, these Desktops replaced the Windows 7 Epic Desktops that are no longer compliant.” – Citrix Team

Infrastructure

Epic, Citrix, Windows 10, ChromeOS

Challenge

The Citrix team was in the process of upgrading older, out of compliance Windows 7 Epic Desktops to new compliant Windows 10 Desktops. During the rollout, several clinicians began experiencing slow performance in their Citrix sessions. The problem persisted for about two weeks as they tried targeting images, running Citrix Optimizer and following its recommendations (disabling services and applications), as well as reaching out to their EHR vendor and following recommendations like turning off certain telemetry. Nothing resolved the issues, so they reached out to Goliath. The Citrix team lacked the necessary visibility to isolate root cause of the issue and looked to leverage Goliath to uncover blind spots.

Solution

The IT & Goliath teams met to discuss what was needed to install the Goliath agent onto one of the Delivery Controllers for Epic and to begin gathering metrics for the users on the new machines. The teams agreed upon the requirements and outcomes and began the deployment and configuration of Goliath in the upgraded CTX environment. Within 20-minutes of deployment, Goliath was able to isolate three areas that were causing performance issues.

While observing a clinician who has recently been complaining about poor performance we filter by his sessions and initially see the client devices are ChromeOS devices and multiple Client IP addresses (this clinician is badging in and out from patient rooms with stationary ChromeOS devices in each room).

Upon analyzing the clinician’s sessions, we found the following items that could be causes of the performance issues.

1. High memory utilization on the XenServer Host that was supporting the clinician’s session. We can see below that memory for the host was above 93%. This seemed to be consistent across multiple XenServer Hosts.

2. There was also high memory usage on the VDI the clinician was using, typically around 80%. We were able to identify the processes that were the culprit of that memory usage and it was no surprise to the IT team that Epic and Chrome were at the top of the list.

3. User behavior was also playing a role. There was a significant amount of bandwidth being used in portions of the session and on average, between 20-30 Mbps of connectivity for the Citrix session. That is typically enough bandwidth for basic usage, but if the user is performing actions that are consuming excessive amounts of bandwidth, the result is a less than optimal experience.

a) High levels of audio bandwidth being leveraged, both inbound and outbound (Dragon Dictation likely being used). Any slowness here can negatively impact user experience.

b) There was also an excessive amount of printer bandwidth being consumed. At the levels observed these were very large print jobs. The IT team immediately thought of different ways they could alter their print spooler configuration as a result of seeing this data.

Results

The Healthcare organization was able to leverage Goliath to pull in a subset of the new Windows 10 environment (including hypervisor layer), and within 20 minutes of deploying Goliath were able to isolate root cause of the poor session performance experienced by their clinicians; memory usage on XenServer Host, memory usage on the VDI, and user behavior. Goliath has provided IT with detailed telemetry not previously possible, correlating data from the Citrix session, underlying IT delivery infrastructure, user behavior, and ChromeOS device. The IT team now has levels of visibility they didn’t have prior to Goliath and can now further optimize their end user clinician experience.