Troubleshooting Slow Performance

Contents

The purpose of this article is to review the known causes of delays or slow performance when using StayLinked. This article is intended for cases in which users return to work normally once they are able to reconnect to the server.

Many of the symptoms that appear are common among multiple issues. Several clues can help identify which of these issues are possible causes. Keeping track of the exact symptom or message, the time it took place, and which device(s) are affected can make the review of log files more effective.

It is important to note that there are no priority or preferential treatment options available in StayLinked. Any issue that affects a specific network segment, area/region, or model of device while other areas or devices remain unaffected is a network issue and not related to StayLinked configuration. Standard network diagnostics would apply in these cases. In these circumstances, Quality of Service (QoS) affecting UDP packets and/or routing tables are the most likely causes.

The following are the signatures typically associated with various causes:

Device Radio Power Management
- Slow transaction and slow ping times on a network where other ping response times are fast
Network Routing
- Wifi connection may not be interrupted
- Pings to/from the server are interrupted at random times for random durations
- Possible constant changes to associated access points (AP) by radio
Quality of Service (QoS)
- Fast client-server pings
- Clients show signs of communication interruption (asterisk, exclamation, linking screen or a retry counter)
- Echo test shows high RT2 or RT3
- Significant numbers of DUP, RTX and OOS in Stats logging
- May include high AGD values in stats as well
Server resource limitations
- All devices affected at the same time
- Statistics log might show signification numbers of aged syncs (AGD)
- Virtual Machines exceeding 60% CPU utilization or with dynamic resources instead of dedicated
3^rd Party (Antivirus, packet filtering, system maintenance)
- Higher than usual CPU utilization and constant use of CPU time by Antivirus program
- Usually all the time or on a regular schedule
WMS/ERP response delays
- Session, Handler and/or Proxy logs would show delays in responses after an input is submitted
- Clients would not show signs of interruption (asterisk, exclamation, linking screen or a retry counter)
- Other telnet clients (such as PuTTy) often show the same delays

Device Radio Power Management

StayLinked recommends radio power management in the radio profile is set to CAM. PSP and Fast PSP settings can attempt to power down the radio fast enough to affect ping times and all other network traffic. Android devices may not display a setting, but are often still configurable using alternative configuration options. Check with your mobile device manufacturer for more details about the options and defaults for your mobile devices.

Network Routing

Servers with multiple network adapters on the same network segment can sometimes have different routing tables for each adapter. This can cause communications to reach the server, but packets from the server leaving on the wrong adapter to never reach the intended client. Binding or setting StayLinked to use a specific IP for all I/O can prevent the server from using the adapter intended for other networks.

Clients affected by this situation would still show good connectivity to the local network. Pings and other network testing would all show intermittent communications in this scenario. Using ICMP ping testing from the device to other machines on the same network segment as the server while also pinging the server address can help confirm if packets can reach the network segment.

Quality of Service (QoS)

QoS settings that downgrade UDP traffic can often impact the performance of StayLinked. When enabled, these protocols typically default to a reduced priority for UDP in favor of TCP transactions. In this scenario, devices would show poor echo test results while still returning fast uninterrupted ping times. Statistics logging showing significant numbers of OOS, DUP, RTX and AGD are a common sign of QoS, while high numbers of only RTX are often wireless network interruptions.

These settings are often difficult to identify since all non-UDP testing appears responsive. Any intelligent network appliance or access point may be configured with QoS settings and features. Using other UDP communications like TFTP or VOIP can help determine if UDP is affected.

StayLinked cannot be configured to use an alternative protocol. The architecture is fully routable and reliable, with features built into the UDP protocol to achieve the desired results and still maintain a minimal network footprint.

Additional information can be found in our article about Quality of Service.

Server resource limitations

When issues affect all devices at the same time, it may be related to the performance of the server process. Process review of the server during these times can show what processes are taking resources. Most StayLinked installation default to a 1GB maximum Java Heap size, even when the system might have more RAM available.

Log files like the StayLinkedManager.log file will often show stacks of entries in the same millisecond as the process gets the required CPU time to make the entries. StayLinkedStdErr logging may also show explicit entries for threads, memory or other limitations.

Many of the common resourse limitations and tuning parameters can be found in our Server Tuning documentation.

3rd Party (Antivirus, packet filtering, system maintenance routines, other network appliances)

These issues are commonly associated with certain times that coincide with regular routines. If issues are specific to a region, then Intelligent switches and routers can often have issues that are easily resolved with a restart.

Installations have found issues with Antivirus and other third-party programs that treated StayLinked as a possible threat and constantly scanned the program and packets. This can include any process, even internet filtering programs and other packet monitoring programs.

WMS/ERP response delays

When the telnet host or hosted application takes time to respond to inputs or complete queries, the clients will not display any interruption messages because it can freely communicate with the StayLinked Server. In these cases, ping testing and StayLinked Server commands at the device will continue to operate normally and quickly, while data inputs to the affected program experience delays.

These are often easily identified by completing the same transaction on a desktop PC PuTTy or other telnet client connection. Alternatively, the StayLinkedSession logging features will show user input and host response times to the millisecond. Some hosts require multiple paint events, so the last paint resulting from any input should be the guideline for total response time.

Some host applications display a clock or timer. It is highly recommended that these are removed or disabled, since all devices have a clock and the constant presentation updates server no functional purpose in most environments.

The following diagnostics tools have been mentioned in this article:

Updated on September 15, 2020