Short answer: A vCenter “HTTP Status 500 – Internal Server Error” at the vSphere Client login is usually a backend failure, not a browser problem. Check, in this order: expired certificates (Machine SSL, solution users, STS signing), a full appliance partition, a corrupted STS Tomcat cache after patching, a missing vsphere-webclient solution user in ActAsUsers, and broken Lookup Service registrations. Take an offline snapshot first. The fixes run over SSH as root.
Which 500 error do you have?
| Symptom | Most likely cause | Broadcom KB |
|---|---|---|
| “HTTP Status 500 – Internal Server Error” or “no healthy upstream”; services fail to start | Expired Machine SSL or solution user certificates | 324301 |
| “[500] An error occurred while fetching identity providers” | Expired STS signing, Machine SSL or solution user certificate (in ELM, one node can break the whole SSO domain) | 322178 |
| 500 after a patch or upgrade, often in Linked Mode | Corrupted Tomcat cache of the STS service | 336290 |
| 500 at login although all services run (7.0.x) | vsphere-webclient missing from ActAsUsers | 315453 |
| “500 Internal error” instead of the login page | Client registered to an old or unknown SSO domain | 374304 |
Step 1: Quick health check
df -h # any partition at or near 100%?
service-control --status --all # which services are stopped?A full /storage/log, /storage/db or root partition stops services and produces the same 500 page. Free the space first, then continue.
Step 2: Check certificate expiration
This command from KB 324301 lists every VECS store with alias and expiry:
for store in $(/usr/lib/vmware-vmafd/bin/vecs-cli store list | grep -v TRUSTED_ROOT_CRLS); do echo "[*] Store :" $store; /usr/lib/vmware-vmafd/bin/vecs-cli entry list --store $store --text | grep -ie "Alias" -ie "Not After";done;The STS signing certificate is not stored in VECS, so check it separately. See vCenter STS signing certificate expiration. To replace expired certificates, Broadcom points to vCert, the fixcerts script or vSphere Certificate Manager. vCert is the quickest; see Installing and using the vCert tool. Then restart everything:
service-control --stop --all; service-control --start --allIn Enhanced Linked Mode, take offline snapshots of all linked vCenters together before changing certificates, and restart services on every node.
Step 3: Clear the STS Tomcat cache (500 after patching)
If the error appeared right after a patch or upgrade and certificates are valid, KB 336290 describes Tomcat failing to rebuild its cached JSP files. On vCenter 8.0.x:
service-control --stop vmware-stsd
rm -rf /var/lib/sso/webapps/ROOT/
rm -rf /var/lib/sso/workDir/ROOT/
service-control --start vmware-stsdOn vCenter 7.0.x the path is different:
service-control --stop vmware-stsd
rm -rf /usr/lib/vmware-sso/vmware-sts/work/Catalina/localhost/ROOT
service-control --start vmware-stsdStep 4: SSO solution user and registrations
- ActAsUsers: if
vsphere-webclientlogs “Access not authorized!” when acquiring a token, KB 315453 re-adds the solution user to the ActAsUsers group (LDIF or the supplied Python script; needs the SSO admin password), then restartvsphere-ui. - Stale SSO domain registration: if
vsphere_client_virgo.logshows “Couldn’t find any API GW registration on the local node”, rebuild the registrations with lsdoctor-r(option 2), per KB 374304. See A quick guide to lsdoctor.
Prevent it next time
- Monitor certificate expiry, including the STS signing certificate, and replace certificates weeks before they expire.
- Watch appliance partition usage in the VAMI or your monitoring system.
- Take a file-based backup and an offline snapshot before every patch.
Further reading
- Broadcom KB 324301: “HTTP Status 500 – Internal Server Error” when accessing VCSA
- Broadcom KB 336290: “500 internal server error” logging into a Linked Mode vSphere Client
- Broadcom KB 322178: “[500] An error occurred while fetching identity providers”
- Andy’s World: Fixing VMware vCenter Server: Resolving HTTP 500 Internal Server Error (original post)
