NSX Load Balancer 502 Bad Gateway: Header Size Limits, Logs and Edge CLI Checks

NSX native load balancer returns 502 Bad Gateway? Check the header size limits in the HTTP application profile, find the 502 in the Edge LB error and access logs, and verify pool and server-side SSL.

Short answer: A 502 Bad Gateway from the NSX native load balancer means the virtual server accepted the client request but did not get a usable response from the pool member. A frequent cause is a header that is larger than the limit set in the HTTP application profile, typically large response headers or cookies from the application. Raise the request and response header size in the profile (Networking > Load Balancing > Profiles > Application), and confirm the cause in the load balancer error log and access log on the Edge.

Which 5xx error points where

ErrorUsual meaning on the NSX LBCheck first
502 Bad GatewayThe member answered with something the LB could not use, or closed the connection.Header size limits, server-side SSL, the application on the member
503 Service UnavailableNo pool member is available.Health monitor and pool status
504 Gateway TimeoutThe member did not answer in time.Response timeout in the HTTP profile, application performance
400 Bad RequestThe client request itself was rejected, for example a request header over the limit.Request header size, client cookies

Cause: header size over the limit

The HTTP application profile has two limits: Request Header Size and Response Header Size. When the application returns headers larger than the response limit, which happens with large cookies, long redirect URLs, many Set-Cookie headers or big authentication tokens, the load balancer cannot pass the response on and the client gets a 502. Requests with oversized headers, for example a browser that sends a pile of cookies, fail in a similar way on the request side.

Typical signs: only some users or only some URLs fail (often after login, when the session cookie or token is set), the backend log shows the request as successful, and the LB log shows the 502.

Fix: in the NSX Manager UI, open Networking > Load Balancing > Profiles > Application, edit the HTTP profile used by the virtual server and raise Request Header Size and Response Header Size (values in bytes). Raise them to what the application really needs rather than to the maximum, and use a dedicated profile for that application so other virtual servers keep the default.

Other common causes of 502

  • Server-side SSL mismatch: the pool members listen on HTTPS but the virtual server has no server SSL profile, or the other way round. The LB then speaks the wrong protocol to the member.
  • Member resets the connection: the application crashes or a firewall on the member drops the connection from the SNAT address of the load balancer.
  • Health monitor too weak: a TCP monitor marks the member up while the application behind the port is broken. Use an HTTP or HTTPS monitor that checks a real URL and response code.

Edge CLI checks

On the Edge node that runs the load balancer (the active Edge of the Tier-1 gateway it is attached to):

get load-balancers
get load-balancer <lb-uuid> status
get load-balancer <lb-uuid> virtual-servers status
get load-balancer <lb-uuid> pools status
get load-balancer <lb-uuid> health-check-table

# logs straight from the CLI
get load-balancer <lb-uuid> error-log reg-filter 502
get load-balancer <lb-uuid> virtual-server <vs-uuid> access-log follow

# built-in diagnosis
get load-balancer <lb-uuid> diagnosis summary

Logs on the NSX Edge

Load balancer error log

From the root shell of the Edge, each load balancer has its own log folder under /var/log/lb/ with an error.log. Search it for the 502 and for the reason around it:

grep -r "502" /var/log/lb/ | tail -20
grep -ri "too big header\|upstream" /var/log/lb/ | tail -20

A line with HTTP/1.1 502 Bad Gateway confirms that the load balancer generated the error itself. Error log entries that mention a header that is too big while reading the response from the upstream (the pool member) point straight to the response header size limit.

Access log in syslog

When access logging is enabled on the virtual server, every request is written to /var/log/syslog on the Edge with subcomp="lb" and s2comp="access":

grep 's2comp="access"' /var/log/syslog | grep "HttpResponse.Status: '502'" | tail -20

Each entry is a full record of the request: virtual server name, IP and port, pool and the pool member that handled it, client IP, SNAT IP and port, HTTP method, URI and host, and the response status, size and server time. That tells you which member and which URL return the 502. A response size of 0 together with a long server time means the member never sent a usable response.

Further reading

Author: Daniel Micanek

Senior Service Architect, SAP/TDL Platform Services Team at Vivicta | vExpert ⭐⭐⭐⭐⭐⭐⭐ | Broadcom Knight: vSphere Kubernetes Service, VCF Networking (NSX), VCF Automation | Broadcom Software Knight: vSphere | Cisco Champion. Notes from running about 1,000 ESXi hosts on Cisco UCS, VCF 9 and SAP HANA on vSphere.