STS_INTERNAL_SSL_CERT LEGACY in vCert: Meaning and Fix

Short answer: STS_INTERNAL_SSL_CERT is a legacy VECS store that older vCenter releases used for the Security Token Service (STS) TLS listener. After an upgrade, the STS service can still point at that store, so renewing the Machine SSL certificate does not change what STS presents on port 3128. vCert reports this as LEGACY. Broadcom’s fix is to switch the STS configuration to MACHINE_SSL_CERT with vCert, restart vmware-stsd, and then remove the old store (KB 423742, KB 424604).

Part of the series “vCenter and VCF Certificate Management: Diagnosis, Renewal, and Operations”: vCenter Certificate Auto-Renewal: When You Still Need vCert (start here) · vCenter Certificate Alarm After Renewal? Check the Backup Stores · VCF 9 Certificate Management: Custom CAs, Auto-Renewal, and Host Onboarding

Table of Contents

The symptom: new Machine SSL certificate, old STS certificate

The usual sequence goes like this. You replace or renew the Machine SSL certificate, the vSphere Client shows the new certificate on 443, and everything looks healthy. Then a vulnerability scanner flags an expired or soon-to-expire certificate on port 3128, or an identity-related compliance check fails. Or you run vCert and the STS configuration check comes back as LEGACY.

Nothing went wrong with the renewal. There are simply two different certificate sources involved, and the one on 443 is not the one STS is using.

Where STS_INTERNAL_SSL_CERT comes from and why it survives upgrades

In vCenter 6.x and earlier, the vmware-stsd service took its TLS certificate from a dedicated VECS store named STS_INTERNAL_SSL_CERT. Upgrades migrate that configuration so that STS keeps working, which means the old store reference can live on for years. Broadcom’s sources describe the origin slightly differently:

KB 423742: usually left behind after upgrading from vCenter 5.5 or 6.x to 7.x or 8.x.
The vCert KB 385107: the legacy store is present if the system was upgraded from 5.5; greenfield 6.x/7.x deployments use MACHINE_SSL_CERT.
KB 316625: found on appliances that were originally Windows vCenter Servers and were later migrated to the VCSA. According to that KB, new 8.x deployments do not have this store.

So the practical rule is simple: if the vCenter has a long upgrade history, check for it. KB 423742 also explains why it causes trouble. Newer vCenter versions do not refresh this store when the Machine SSL certificate is renewed, so the certificate inside it can expire without anyone noticing. On 7.x/8.x, the STS store reference is defined in /usr/lib/vmware-sso/vmware-sts/conf/server.xml (KB 316625). On vCenter 9.0 and later, STS runs on an embedded Tomcat and the reference moved to /usr/lib/vmware-sso/vmware-sts/conf/catalina.properties (KB 438122).

STS TLS certificate vs. STS signing certificate

These two are often confused because both have “STS” in the name. They are different objects with different jobs, and vCert handles them in different menus.

	STS TLS (server) certificate	STS signing certificate
Purpose	Secures the TLS connection to the STS listener	Signs SAML tokens issued by SSO
Where it lives	`MACHINE_SSL_CERT` (expected) or legacy `STS_INTERNAL_SSL_CERT` in VECS	TenantCredential entries in VMware Directory
Who sees it	Anything connecting to port 3128, including scanners	Services validating SSO tokens
vCert: check/view	5. Check configurations → 2. Check STS server certificate and configuration	2. View certificate info → 8. STS signing certificates
vCert: fix	Switch STS to `MACHINE_SSL_CERT` and restart `vmware-stsd`	Replace the STS signing certificate (Manage certificates)
Symptom when expired	Scanner/compliance finding on 3128	Login failures, “No Healthy Upstream”

A LEGACY result is about the left column only. If users cannot log in and the vSphere Client shows “No Healthy Upstream”, you are looking at the signing certificate instead; see vCenter STS signing certificate expiration.

How to read the vCert configuration check

If you have not used vCert yet, start with installing and using the vCert tool. The main menu option 1. Check current certificate status checks certificate health, meaning expiry, SAN and key usage. It does not tell you which store STS reads from. That question is answered by 5. Check configurations → 2. Check STS server certificate and configuration. This check does two things (KB 385107):

It checks which VECS store the STS service is configured to use. Anything other than MACHINE_SSL_CERT is reported as LEGACY, and vCert offers to update the configuration and restart the service.
It checks the vmwSTSConnectionStrings attribute in VMware Directory. A standalone vCenter should reference its own hostname; a node in Enhanced Linked Mode should reference localhost. If the value is wrong, vCert prompts you to correct it.

Inspect what port 3128 actually presents

KB 423742 identifies the STS listener as port 3128 and gives this command to run on the appliance:

openssl s_client -connect localhost:3128 -showcerts

For a shorter comparison, you can pipe the output through standard openssl x509 and put it side by side with the Machine SSL certificate on 443. This is plain OpenSSL usage, not a Broadcom-specific procedure:

for p in 443 3128; do
  echo "== port $p"
  openssl s_client -connect localhost:$p </dev/null 2>/dev/null | openssl x509 -noout -subject -issuer -enddate
done

To see whether the legacy store exists and what its entry expires, use the VECS listing from KB 424604 / KB 316625:

for i in $(/usr/lib/vmware-vmafd/bin/vecs-cli store list); do echo STORE $i; /usr/lib/vmware-vmafd/bin/vecs-cli entry list --store $i --text | egrep "Alias|Not After"; done

Broadcom’s workflow: KB 423742 and KB 424604

There are two KBs, and they cover two stages. KB 423742 (“STS_INTERNAL_SSL_CERT replacement using vCert”) switches the STS configuration interactively through the vCert menu. KB 424604 (“Removing STS_INTERNAL_SSL_CERT from vCenter Server with vCert”) removes the store, with a Lookup Service check first. Both apply to vCenter 7.x and 8.x, and both restart vmware-stsd. Combined, the procedure is:

Take a backup or snapshot. KB 424604 makes an offline snapshot mandatory. In Enhanced Linked Mode, take offline snapshots of all nodes.
Check the Lookup Service with lsdoctor: python lsdoctor.py -l. If it finds stale registrations, run python lsdoctor.py -s (stale configuration left over from 5.x) and then python lsdoctor.py -t (SSL trust mismatch). See the lsdoctor guide, and note that KB 320837 requires a service restart after -s and -t.
Run vCert. Either use the menu (./vCert.py → 5 → 2, then SSO administrator credentials) as in KB 423742, or run the check directly as in KB 424604: python vCert.py --run config/check_config/sts_config/op_check_sts_config.yaml
Answer y to “Update STS server configuration to use the MACHINE_SSL_CERT store?”
Answer y to “Restart service(s) vmware-stsd”.
Validate: port 3128 should now present the Machine SSL certificate, and the vCert check should no longer say LEGACY.

Do not delete the store by hand first. KB 316625 warns that removing STS_INTERNAL_SSL_CERT while server.xml still references it stops STS from starting, and the other vCenter services then fail with it. The configuration change always comes before the removal. That KB (and KB 438122 for vSphere 9.x) documents script and manual alternatives in case vCert is not an option.

Annotated example: before and after

Illustrative example only. Host names and dates are placeholders. The LEGACY line follows the format shown in KB 423742. Your vCert version may word the post-fix line differently.

# BEFORE - vcsa01.lab.local, upgraded from 6.x
== port 443
subject=CN = vcsa01.lab.local
issuer=CN = CA, DC = vsphere, DC = local, ...
notAfter=<new date, e.g. 2028>          <-- renewed Machine SSL, fine
== port 3128
subject=CN = vcsa01.lab.local
issuer=CN = CA, DC = vsphere, DC = local, ...
notAfter=<old date, already past>     <-- what the scanner reports

vCert: 5. Check configurations -> 2. Check STS server certificate and configuration
Checking STS Server Configuration
-----------------------------------------------------------------
Checking VECS store configuration                          LEGACY
                                    ^-- STS reads STS_INTERNAL_SSL_CERT, not MACHINE_SSL_CERT
Update STS server configuration to use the MACHINE_SSL_CERT store? [y/n]: y
Restart service(s) vmware-stsd [y/n]: y

# AFTER
== port 3128
subject=CN = vcsa01.lab.local
notAfter=<same date as port 443>       <-- 3128 and 443 now match
vCert STS check: no longer LEGACY           <-- configuration uses MACHINE_SSL_CERT
vecs-cli store list: no STS_INTERNAL_SSL_CERT entry (after KB 424604 removal)

A short checklist after the change: the 3128 and 443 certificates should have the same expiry date, vmware-stsd should be running, a vSphere Client login should work, and the scanner should be clean on its next pass. KB 423742 also says that once STS uses the Machine SSL store, later Machine SSL renewals carry over to STS as expected. For how renewals themselves are scheduled and monitored, see vCenter Certificate Auto-Renewal: When You Still Need vCert. If an expiry alarm still points at an old certificate after the fix, check the BACKUP_STORE article.

Sources

Related on vdan.cz

Author: Daniel Micanek

Senior Service Architect, SAP/TDL Platform Services Team at Vivicta | vExpert ⭐⭐⭐⭐⭐⭐⭐ | Broadcom Knight: vSphere Kubernetes Service, VCF Networking (NSX), VCF Automation | Broadcom Software Knight: vSphere | Cisco Champion. Notes from running about 1,000 ESXi hosts on Cisco UCS, VCF 9 and SAP HANA on vSphere. View all posts by Daniel Micanek