Recently, I worked with a customer who was having inconsistent issues with user experience. To date they have been utilizing Goliath Performance Monitor to troubleshoot and alert on faults and errors taking place in their XenApp 7.6 environment.
They were using their Citrix NetScaler ADC’s as a NetScaler Gateway to reverse proxy ICA remote sessions within their Citrix environment, and they were especially interested in seeing how Goliath could provide deeper analytics into the experience of their end users who were logging into the environment from home and constantly complaining about poor session performance, specifically slowness.
Gaining the appropriate visibility into the NetScaler Gateway VIP to identify and resolve the end user’s complaints was extremely important because in the end users’ eyes, the problem was always Citrix, when in reality the root cause was unknown.
AppFlow Data: What It Is & How It’s Used for Analysis
Before diving into how the customer leveraged Goliath for NetScaler to solve their challenge, it’s important to understand a little bit about how the technology operates and delivers results. This technology provides a unique vantage point from a monitoring perspective. Very simply, Goliath for NetScaler collects AppFlow data from the Citrix NetScaler Application Delivery Controller and the NetScaler Gateway.
AppFlow consists of messages that are sent from the Citrix NetScaler ADC & Gateway which include data points around the connections that are established to it. This data contains valuable information related to what is occurring during a session, and delivers full session visibility from source to destination.
Once Goliath for NetScaler receives the AppFlow messages, it begins to automatically generate real-time dashboards, live reports, and historical analysis on critical data related to Citrix XenApp & XenDesktop ICA/HDX sessions, application utilization and access, NetScaler Gateway execution, and web application VIP performance. The newly gained visibility gained with NetScaler Platinum licensing combined with the power of Goliath technology made Goliath for NetScaler the obvious choice for solving this customer’s user experience challenges.
Up & Running Faster than it Takes to Get a Pizza Delivered
Getting Goliath for NetScaler set up and running for this customer was fast and simple. Enabling AppFlow on the Citrix NetScaler ADC’s and configuring the message to be sent to Goliath only took less than thirty minutes. Once AppFlow configuration was complete, the customer immediately started receiving actionable data from Goliath for NetScaler.
To start their analysis and troubleshooting, the customer went to the ‘sessions’ screen (see image below) of the product and was able to gain total visibility into the end users sessions on the NetScaler Gateway, as all of the current sessions are listed on the bottom half of the real-time display.
Round Trip Time – Jitter – Retransmits
The information gathered for the sessions includes the username, user’s IP address, the version of Citrix receiver they were using to establish the connection, and the date/time the session was established. In addition to the previous data points, the key end user experience metrics of Round Trip Time (RTT), Network Jitter, and Packet Retransmits were also detailed.
These metrics are clear indicators as to whether or not a user was experiencing poor Citrix session performance. To further understand why these metrics are important, and what they mean, please see the below definitions:
- RTT – Round Trip Time measures the packet time from the source, to the intended destination, and then back again. This is the primary measurement which would indicate performance problems. On a Citrix ICA/HDX connection, data is captured from the time of the initial mouse click, to the handoff to the server which is handling the session, its processing time to compute the action, and then presents it back down to the endpoint.
- Jitter – Jitter is technically defined as the delay variation or deviation in the connection. This effectively measures the quality of the connection, and if there is any “noise on the line.” This would cause delays in the presentation of a user’s Citrix session.
- Retransmits – This is a measure of the volume of packets that had to be re-sent due to packet loss. This would be indicative of a poor connection, and would result in choppiness in the presentation of an end user’s Citrix session.
In the screenshot below, you can see how these data points are color coded on the basis of connection quality red (severe), yellow (lackluster), and green (satisfactory).
Using this quick and easy graphical representation of the user sessions, the Citrix administrators were able to quickly target problematic users, and immediately drill down into the sessions to see exactly how the performance metrics aligned for each user. By clicking on the user’s session, the Citrix administrators now had the ability to segment the end user experience quality metrics down to Client Jitter Round Trip Times (RTT) and Server Jitter Round Trip Times (RTT). The ability to have visibility into both the client and the server sessions is critical.
The client-side metrics show the performance of the connection from the user’s end point to the Citrix NetScaler Gateway and back, whereas the server metrics pertain to the connection between the Citrix NetScaler Gateway, and the internal architecture.
This is exactly why having visibility into the ICA/HDX sessions through the NetScaler Gateway is so important. Having the aforementioned visibility provides incontrovertible evidence as to whether the issue is from the data center side, where admins have control over the performance, or if it’s external to the network, and therefore the responsibility of the end user.
The Citrix administrators at the manufacturer were amazed how Goliath for NetScaler was able to break down the users ICA/HDX session performance, and how actionable the data would be for their help desk. They were able to “live share” this data off to their help desk to aid in dealing with end user complaints when users called in about their Citrix sessions being slow.
Product Note: The “Live Share” feature, creates a link that can be taken and then handed off to an end user or administrator by email or over your corporate chat software. When the recipient of the live share clicks on the link, they are provided with direct access to the report which allows them to establish the objective evidence they need to see. The report can then be refreshed to show current, live data when performing ongoing troubleshooting.
How the Help Desk Used Goliath for NetScaler
Here’s how the help desk used Goliath for NetScaler – The help desk started by quickly searching for the user’s session from the sessions tab:
The help desk then clicked on the user’s session to see where the session performance had degraded. The Citrix administrators could now provide a visual timeline of end users session performance, empowering them to correlate what the user said they were doing at a specific time, to the performance over the duration of that time period.
For example, they had an end user who called in one day to complain about their Citrix connection. The help desk user followed the above process and could see that between 10 am and 10:05 am the client RTT and retransmits extremely increased. When stating this information to the end user, they recollected the timeline and stated that at 9:45 am they boarded and commuter train and around 10 am they would’ve been entering a tunnel, which was about the time they started having connection problems.
The ability to correlate the connection data to a real-world event is unique to Goliath for NetScaler. As detailed below, data points were provided as an average and then rolled up, obfuscating the performance:
When support engineers wanted to share the details of what happened within a user’s session with management, and/or Citrix administrators, they were easily able to access a reporting screen. From there, they could print a report, or “live share” as seen below. As described earlier, live share makes it very easy for the organizations departments to collaborate on troubleshooting this issue, and to quickly provide the data to the people that needed to see in real time.
After review of the above user’s experience, it was obvious that specific RTT values correlated to a poor end user experience. As a result, the administrators refined the alerting thresholds in order to be proactive. Rather than waiting for the user to call and report a problem, the administrators would already know the user may be experiencing a problem within their Citrix sessions, and required attention and/or support.
Alert thresholds were set at 400ms, the threshold at which users started to experience slowness and other Citrix session performance degradation. To ensure that support staff and admins were aware of degradation as it happened, alert notifications were configured to be automatically sent to the help desk and the Citrix admins, empowering them to effectively support the end user in the event future performance issues arose.
Finally Able to Gain Full Visibility into the End User Experience
Prior to deploying Goliath for NetScaler, the Citrix admins had no way to analyze and troubleshoot their end user sessions. Gaining visibility into an end users experience was extremely important because in the end users’ eyes, the problem was always Citrix, when that was not always the case.
The deep visibility that the technology provided allowed the Citrix Admins to very quickly understand not only that a user working remote was having a poor end user experience, but also determine that the root cause of the performance degradation was the end user’s connection. The Administrators had tangible proof that the problem was not the delivery infrastructure, and were able to provide the objective evidence to management to support each scenario.
This allowed the administrators to target the specific root cause of the performance issue and enables them to be proactive in addressing user experience challenges going forward. Furthermore, the export feature of each of the reports allowed the Citrix administrators to share the deep insight provided by Goliath for NetScaler to their managers in just two clicks. Overall, by deploying Goliath for NetScaler in addition to Goliath Performance Monitor, we were able to provide the Citrix administration teams the objective evidence needed to effectively monitor their Citrix environment from end to end.