Isolates “True Root Cause” Saves Time and Money

Infrastructure

Citrix XenApp, Citrix XenDesktop, VMware Horizon

Challenge

Healthcare IT faces both complexity and urgency when dealing with end user experience issues that impact their EHR environments.  In addition to the usual issues of frustration, delays and inefficiency, patient care is impacted when healthcare workers can’t access critical applications via Citrix XenApp, Citrix XenDesktop, or VMware Horizon.

Solution

In this case, a top 5 nonprofit healthcare organization assembled a major incident response team when users reported major performance issues and delays when scanning data into their MEDITECH EHR application.  Initially, the team reviewed the user accounts, scanner data and MEDITECH scanner applications.  Applications teams, Citrix Teams and Network teams reviewed this data and reported no issues in their parts of the system.  At this point, the team began planning to upgrade the drivers on each PC and device involved in scanning – a massive, and time-consuming undertaking.

Parallel to this effort, the End User Computing team used Goliath Performance Monitor and the Goliath MEDITECH Module to troubleshoot the issue in the Citrix XenApp sessions affected.  All of these sessions clearly showed low Input Line Speed and high Citrix ICA latency as depicted in the image below:

Figure 1: Impacted sessions show low Input Line Speed and high Citrix ICA Latency

This raised the key question: When did this start and what caused the change?  Because Goliath Performance Monitor maintains historical session data at the same level of detail as current sessions, it was simple to begin backtracking the issue.  Looking back a week, they were able to see there were no issues reported, and the data showed no problem with Citrix ICA latency or Input Line Speed as seen below:

Figure 2: Prior sessions show normal Input Line Speed and ICA Latency

Further investigation pinpointed the start of the problem to a specific date and showed the same issues: high ICA Latency and low Input Line Speed as depicted in the image below:

Figure 3: Pinpointing the first sessions showing the issue

Results

Taken altogether, this data showed that multiple users, computers and even entirely separate facilities were having the same problem.  This clearly pointed to a change in the network made just before the issue started.

The major incident response team reviewed the data and conclusion, and their focus shifted from the plan to update the scanners to the network.  A more detailed network review showed that a specific MLPS circuit was changed just before the issue was reported.  Further tests showed that this circuit was dropping packets and causing retransmits that slowed network traffic across five different facilities.  The networking team disabled the faulty circuit and initiated failover to a backup connection.

As soon as the alternate circuit configuration was in place, Goliath Performance Monitor reported an immediate increase in performance for the impacted users’ Citrix XenApp sessions.  Citrix ICA Latency spikes returned to normal during scanning operations and the Input Line Speed increased to standard levels.  End users no longer reported issues with scanners or session performance. Goliath Performance Monitor clearly documented the successful remediation of the issue as depicted in the screenshot below:

Figure 4: Documenting successful remediation by showing improved performance

This is one example of how this customer used Goliath Technologies embedded intelligence and automation to anticipate, troubleshoot and prevent end user experience issues. The base product, coupled with the Goliath MEDITECH Module, enabled this health system to save countless hours and time.  In addition, it increased the credibility of the IT Team overall.