CGH Medical Center Finds Root Cause and Resolves Cerner Millennium & Citrix Performance Issues

“I use Goliath daily to quickly troubleshoot and resolve end user complaints around slow logons and poor performance with Cerner applications delivered via Citrix. What had previously taken me hours to hunt down root cause, now takes me minutes with Goliath.” – Greg Beck, System Administrator at CGH Medical Center

Infrastructure

Cerner Millennium, Citrix Virtual Apps (XenApp), VMware vSphere

Challenge

Troubleshooting Citrix and Cerner Performance Issues Consumes Hours of IT Time

CGH Medical Center is a progressive acute care facility located in Northern Illinois leveraging Citrix to deliver their clinical applications including Cerner Millennium. Greg Beck, System Administrator, routinely fields end user complaints around “Citrix is slow,” “my system keeps disconnecting,” or worse yet “I can’t login.” He shared, “End users have sporadic slow logons or performance issues that can consume hours of hunting down answers as the system works 90% of the time as expected. Our clinicians are often moving floor to floor and using different machines, so hunting down root cause of a sporadic issue can be challenging when you don’t have visibility into their exact behavior when they experienced an issue.”

As Beck and team prepared to upgrade Citrix Virtual Apps and Desktops 7.15, they knew they needed a purpose-built tool that would help them monitor and troubleshoot end user experience issues. Part of the expectation with the Citrix upgrade was that performance and logon times would be faster. They needed a solution that would enable them to hit these targets and provide visibility into root cause when problems did occur and then document proof that permanent fixes had been put in place.

Solution

End-to-End Visibility Isolates Root Cause of End User Performance Issues

Goliath Technologies was selected by CGH Medical Center to provide end-to-end visibility from endpoint logon, through the Citrix infrastructure, to the Cerner Millennium environment. Its deep and broad metrics offer visibility into both the infrastructure and end user behavior, empowering the team to quickly troubleshoot performance issues and reduce remediation times.

“We selected Goliath Technologies as it was one of the only solutions that had both a technical and business partnership with Cerner,” shared Beck. “We needed a solution that would last five or more years and have the flexibility to support our infrastructure regardless if we implemented elements and applications on-premises or hosted in the cloud.”

“I use Goliath daily to quickly troubleshoot and resolve end user complaints around slow logons and poor performance with clinical applications delivered via Citrix. What had previously taken me hours to hunt down root cause, now takes me minutes with Goliath,” shares Beck.

Greg Beck said he consistently has Goliath’s Citrix Published Apps and Desktop tab (Image 1) open to not only quickly troubleshoot issues if a user calls in, but also to periodically check that logon times and overall performance looks normal throughout the day. “Goliath gives me and the team real numbers so that I know what the end user experience is. I no longer have to go off gut feelings if things are working or not.”

Results

As a result of using Goliath, CGH Medical Center has been able to more quickly troubleshoot and resolve performance issues when medical staff is accessing Cerner Millennium and other applications delivered via Citrix.

Improving End User Complaint Resolution Times

“I use Goliath daily to quickly troubleshoot and resolve end user complaints around slow logons and poor performance with clinical applications delivered via Citrix. What had previously taken me hours to hunt down root cause, now takes me minutes with Goliath,” shares Beck.

Greg Beck said he consistently has Goliath’s Citrix Published Apps and Desktop tab (Image 1) open to not only quickly troubleshoot issues if a user calls in, but also to periodically check that logon times and overall performance looks normal throughout the day. “Goliath gives me and the team real numbers so that I know what the end user experience is. I no longer have to go off gut feelings if things are working or not.”

Image 1: The Citrix Published Apps and Desktop view gives a high-level overview of all end-user sessions’ key performance metrics such as logon times and what apps are in use.

Reducing Logon Times

With the migration of Citrix over to version 7.15 and increased visibility on performance with Goliath, the team has been able to reduce logon times by almost 50%, taking what was typically over a minute to log on down to under 30 seconds consistently.

If issues do arise, the team can quickly troubleshoot those individual issues by drilling into Goliath’s logon tab (Image 2), showing individual user sessions, and drilling into the various logon stages.

Image 2: The logon tab in Goliath breaks down the end-user’s login from start to finish, providing the administrator actionable metrics to reduce troubleshooting times. The example highlights a user experiencing slow logons due to high-profile load times.

Baselining Performance During Citrix Migration

When the team at CGH Medical Center upgraded Citrix, they used Goliath to baseline performance to measure key performance indicators, such as logon times, application launch times, VM resource consumption, and more, before and after the upgrade. This enabled them to confirm that changes in the environment only had positive impact on performance as they moved forward – not negative.

With the migration almost complete, Greg Beck says he can now leverage Goliath’s historical reports to anticipate and prevent future issues and continue to drive collaboration across internal IT teams, vendors, and the end users themselves.

Image 3: The above image is Goliath’s Session Logon report integrated with Power BI showing average logon times broken down over the past week highlighting average time at each stage of the process. It also showcases average logon times per app, session counts by day, and session counts per hour to track overall activity.

Beck summarizes the overall value of Goliath by stating, “Goliath has significantly cut down the amount of time I spend answering questions around performance issues related to Citrix, Cerner, and any applications delivered via Citrix. I also have been able to find and fix issues before end users complain, reducing our overall support tickets.”

Quickly Troubleshoots “Citrix is Slow” Complaints

“Citrix gets a bad reputation as being blamed for everything, but the underlying issue rarely has anything to do with Citrix. Goliath provides end-to-end visibility across the entire Citrix delivery infrastructure and enables us to quickly isolate root cause of performance issues, identifying who in IT can help resolve the issue quickly.” – Vitaly Petrovsky, Manager of Enterprise Applications, Maimonides Medical Center

Infrastructure

Citrix XenApp, Citrix XenDesktop, VMWare Hyper-V, Nutanix, Allscripts

Challenge

Increase in Remote Workers Due to COVID-19

Maimonides Medical Center is a non-profit hospital located in Brooklyn, N.Y. When COVID-19 hit, many on their team were instructed to work from home. Vitaly Petrovsky, Manager of Enterprise Applications, realized that with an increase in remote workers, he needed a solution that would more effectively monitor and troubleshoot their Citrix infrastructure, which was delivering medical staff access to clinical and other applications.

Even prior to COVID-19 Petrovsky shared that, “Citrix gets a bad reputation as the root cause of all slow logons or session performance issues. The reality is many of the underlying issues just surface themselves when users logon to Citrix, and Citrix is not at fault.” When many on Maimonides’ medical staff started working from home, Petrovsky recognized the number of “Citrix is Slow” complaints were only going to increase.  He needed a tool that the support team could use to prevent calls being falsely escalated to his 3rd line engineers.

“I had looked at Citrix native tools, like Director, but it lacked the granularity in its metrics to isolate root cause of poor performance issues. It also didn’t provide historical data to look back at past trends,” said Petrovsky. End users don’t typically report issues in real-time, so he needed the ability to look historically to understand how performance has changed or find exactly when an issue did occur.

Finally, Petrovsky said when he started looking at monitoring tools he “needed a solution that would proactively look for issues. My team can’t just be reactive waiting to hear end users complain. We need to prevent issues from occurring before our medical team even logs into Citrix to access their clinical applications.”

Solution

Purpose-built Monitoring and Troubleshooting Tool with Embedded Intelligence & Automation

When COVID-19 hit, Petrovsky knew he needed a solution that not only delivered broad and deep metrics for troubleshooting, but also had intelligence and automation embedded so that the support team and his infrastructure team could utilize it for troubleshooting.

“We selected Goliath Technologies as it offered us a single pane of glass along with deep and broad metrics to isolate the root cause of end user performance issues beyond Citrix. It easily identified if slow performance could be attributed to a bad network drive mapping inside of a group policy or to slow connection speed with a user working from their home WiFi network,” shared Petrovsky. He asked his support team to learn how to use the tool by attending a webinar and virtual training, and they have found it beneficial when verifying end users are experiencing slowness and isolating the root cause to pass along to the appropriate team to fix. This helps reduce the number of escalations to 3rd line support.

Results

As a result of using Goliath, Maimonides has been able to more quickly troubleshoot performance issues both for their remote medical staff and those on-site at the hospital. This has not only saved his team significant time but has also increased overall clinician satisfaction when accessing clinical apps.

Example #1: Troubleshooting Slow Performance for the Home Worker

With everyone working from home, Maimonides’ support team was seeing an increase in calls about slow performance. Using Goliath, the team could easily view an end user’s session and see that their local network connection was slow (Image 1). They could then share the data with the user, highlighting the slow connection speed and either ask if someone else was consuming bandwidth (maybe streaming Netflix) or recommend they upgrade their internet speed in order to improve performance. This saved a lot of time trying to hunt down answers to complaints that had nothing to do with the Citrix environment, but instead, the user’s home office.

Image 1: Goliath enables admins to drill quickly into a Citrix user session and identify if slow performance can be attributed to a spike in network latency correlated to a drop in connection speed, identifying the root cause of poor performance is due to a poor Internet connection.

Example #2 Troubleshooting Slow Logons

Petrovsky shared, “We were getting sporadic complaints from some users that applications launching in Citrix were crashing. It wasn’t consistent and while some applications worked, others did not. Before, trying to troubleshoot these sporadic issues was difficult as a pattern could not be identified to understand what was going on.”

With Goliath, Maimonides was able to drill down into the end user session display and verify that the user was experiencing long logon processing time.  From there, they could see that the majority of logon time was being consumed at the group policy level. As they dug further, they identified that there was an old network drive that no longer existed mapped to their Active Directory group. As a result, when they logged on Citrix would time out looking for that drive. As soon as they removed the drive from the AD group, logon times went back up (Image 2).

“Goliath gives us the granular visibility we need to troubleshoot issues quickly,” said Petrovsky.

Image 2: Goliath enables admins to drill quickly into a user session and view detailed breakouts of the logon process to isolate quickly root cause of slow logons.

The other value that Goliath delivered is the visibility into logon times historically.

One thing they discovered using Goliath is that when they were running on Linux OS vs. Microsoft, there were significantly faster reconnects. It enables them to go back to Citrix who confirmed running on Linux is natively faster than Microsoft. This now has Maimonides exploring how they can potentially use more with Linux to improve logons.

Image 3: Goliath offers reports tracking initial logons and reconnect speeds to help IT management prove commitments around performance are being met.

When Petrovsky sums up the value that Goliath has delivered, he states “Goliath offers us a single pane of glass to view everything, helping us quickly troubleshoot performance problems for our staff. It is easy to use and offers historical insights we never had visibility into before.”