Tag Archives: “Fireware XTM”

Watchguard XTM Firewall and UTM Appliance – High CPU Usage in the GAV (gateway anti-virus) scand process causes lag and typing delay in Remote Desktop Sessions (RDP) and SIP or VoIP latency issues

Watchguard XTM Firewall and UTM Appliance – High CPU Usage in scand process causes lag and typing delay in Remote Desktop Sessions (RDP).  You may find that remote users report a lag with Remote Desktop Sessions, freezing sessions, black screen and random disconnections.  At around the same time users report these issues you may find that the CPU usage of the scand process on your Watchguard has increased to 100% and the majority of the activity is attributed to the scand process.  You may be able to recreate this issue by browsing websites that utilise lots of Adobe Flash or Media Content as GAV will need to scan all these elements of the web page.  Login to the Watchguard System Manager and then open Firebox System Manager click on Status Report and scroll down the report until you find the Process List (Screenshot Below).  This information will automatically update every 30 seconds so you can see the %CPU column will change and update every 30 seconds.  The top value system shows the overall CPU utilisation and if you look further down you can see which sub processes are actually occupying the CPU time and making up the overall system usage.  In the screenshot below we can see that system is showing 100 % CPU Usage and then further down we can see that the scand process is accounting for 90.99% of this.  When the CPU Usage reaches 100% on the Watchguard unit it may stop forwarding other traffic and this accounts for the lag and jitter we see within the Remote Desktop Session.  Other time sensitive traffic such as VoIP or SIP traffic may also be affected by this issue as the packets are delayed whilst the Firewall recovers from the resource exhaustion.  Users may also report that web pages are slow to load at the time these issues occur where the GAV process is still dealing with the other requests.

Resolution/Workaround:

You can try disabling the GAV (gateway antivirus) for the HTTP and FTP Proxy to ensure that this is the actual cause of your issues, if the problem subsides then you may need to consider updating the XTM OS to the latest release i.e. 11.5.2 and/or adjusting the GAV policy so that it does not scan some content i.e. Images/Text within websites.  You may also need to consider opening a support case with Watchguard to make them aware of this issue, if you have a large number of users then you may even need to consider upgrading your XTM appliance to a larger unit i.e. XTM 23 to XTM 505 or XTM 22 to XTM330 to provide additional processing power (CPU) and system resources to cope with the additional anti-virus scanning requirements.

Watchguard XTM High CPU Usage scand
Watchguard XTM High CPU Usage scand

Watchguard XTM 2 Series, XTM 3 Series, XTM 5 Series, XTM 8 Series – Fireware XTM OS 11.5.1 – CSP4 Build # 335367

Watchguard XTM 2 Series, XTM 3 Series, XTM 5 Series, XTM 8 Series – Fireware XTM OS 11.5.1 – CSP4 Build # 335367

11.5.1 – CSP4 Build # 335367 Resolves the following issues:

BUG64669: Resolved a Firebox crash and reboot when using FireCluster.

BUG63793: Improved proxy debug logging when using FireCluster.

BUG63574: Proxy connections fail with logs showing: “failed to create new traffic spec” and “insert_tspec:XX index inuse?

BUG65026: Iked stack trace eip=0x080c4013 caused by Mobile VPN with IPSec connection.

BUG63860: snmpd memory leak

You can request 11.5.1 – CSP4 Build # 335367 from Watchguard Support by logging a support case online, they should then be able to provide an ftp download link and appropriate credentials.

Please note that Watchguard CSP releases are cumulative so you should only need to apply the latest to ensure that you also have any previous fixes.

Watchguard – SSL VPN clients cannot resolve internal host names despite DNS servers being configured for the connection

You may find that when you configure your Watchguard XTM Firewall to accept SSL VPN connections that clients can connect to

the VPN and ping IP addresses of internal resources, however you cannot resolve internal hosts even via FQDN using DNS.  You

may also find that when you run NSLOOKUP on the SSL VPN connected client that the  result is your Internet Service Providers

DNS servers rather than the DNS servers assigned via the VPN connection.

 

To resolve the issue you can change your SSL VPN configuration from a “Routed VPN” to a “Bridge VPN”, the routed VPN uses a

virtual IP address pool (192.168.113.0/24) which does not match your internal IP range or the address range of the internal

DNS Servers.  When a Windows client connects to the “Routed VPN” it appears that due to the DNS server mismatch they are not

utilised by the client.

 

When you configure the VPN in “Bridge VPN” mode you can work around this issue, the Bridge VPN configuration allows you to

exclude some addresses from your Windows DHCP Server Pool and add the into them “Start” and “End” IP addresses on your

Watchguard SSL VPN Configuration Page. The Watchguard will now become responsible for assigning these internal IPs to VPN

clients as they connect rather than the Windows DHCP Server.

 

You should now find that when your SSL VPN clients connect that they are assigned an IP address and DNS server that are all

within the existing internal IP range of your network.  An NSLOOKUP should now return your internal DNS server address and

you should be able to ping hostnames and FQDNs that reside within your internal network.

 

Examples:

ping windowsserver

ping windowsserver.exampledomain.local

 

Please remember that the only down side with this configuration is that a “Bridge VPN” bridges to the “Trusted” interface,

this means that the client computer can access any internal resources that they have permissions for by default. A “Routed

VPN” allows you to offer traffic to Optional/secondary networks and gives you more control by letting you lock down access

using “Specify allowed resources”.