Add F1705 Alert to Cisco UCS Manager Plugin 4.0(0)

New Cisco UCS firmware brings possibility to have notification about F1705 Alerts – Rank VLS.

In latest version of Cisco UCS Manager Plugin for VMware vSphere HTML Client (Version 4.0(0)) we could add Custom fault addition for proactive HA monitoring. How to do it?

Cisco UCS / Proactive HA Registration / Registered Fault / Add / ADDDC_Memory_Rank_VLS

If You can’t Add, it is necessary to Unregister UCSM Manager Plugin.

Cisco UCS / Proactive HA Registration / Registered Fault / Add
Cisco UCS / Proactive HA Registration / vCenter server credentials / Register
Cisco UCS / Proactive HA Registration / Register
How Could I check it? Edit Proactive HA / Providers
It is better use Name “ADDDC_Memory_Rank_VLS” without spaces. On my picture I used “My F1705 Alerts”

Adding Custom Alert is only possible with unregistered Cisco UCS Provider, it is better to do it immediatly after Cisco UCS Manager Plugin instalation.

Now I can deceided If I will block F1705 or NOT. I personaly preffer to have F1705 Alert under Proactive HA. Then I only restart Blades with F1705. During reboot Hard-PPR permanently remaps accesses from a designated faulty row to a designated spare row.

Links:

vSphere 8.0 Update 2: Introducing Azure Active Directory Federated Authentication

At the 2023 VMware Explore event in Barcelona. I was on presentation with Viviana Miranda relates to the way users authenticate to vCenter.

vSphere 8.0 Update 2 update heralds a number of groundbreaking features, with one standout enhancement in user authentication methods for vCenter. vCenter Server 8.0 Update 2 has now incorporated federated authentication capabilities with Azure Active Directory.

Dive into the details of this integration and discover how to activate it.

The Advantages of External Identity Providers

Organizations that leverage external identity providers can expect to reap substantial benefits:

- Integration with existing identity provider infrastructure to streamline processes.
- Implementation of Single Sign-On to simplify access across services.
- Adherence to the best practices of role separation between infrastructure management and identity administration.
- Utilization of robust multi-factor authentication options that come with their chosen identity providers.

Supported Identity Providers in vSphere 8.0 U2

While our focus here is the Azure Active Directory integration, it’s essential to highlight the comprehensive range of authentication methods now supported with vCenter Server 8.0 U2.

More info: https://core.vmware.com/resource/vCenterAzureADFederation#Intro

Host Cache can significantly extend reboot times

Exploring the vSphere environment, I’ve found that configuring a large Host Cache with VMFS Datastore can significantly extend reboot times.

It’s a delicate balance of performance gains versus system availability. For an in-depth look at my findings and the impact on your VMware setup, stay tuned.

Host Cache can significantly extend reboot times

Downfall CVE-2022-40982 – Intel’s Downfall Mitigation

Security Fixes in Release 4.3(2b)

Defect ID – CSCwf30468

Cisco UCS M5 C-series servers are affected by vulnerabilities identified by the following Common Vulnerability and Exposures (CVE) IDs:

  • CVE-2022-40982—Information exposure through microarchitectural state after transient execution in certain vector execution units for some Intel® Processors may allow an authenticated user to potentially enable information disclosure through local access
  • CVE-2022-43505—Insufficient control flow management in the BIOS firmware for some Intel® Processors may allow a privileged user to potentially enable denial of service through local access

https://www.intel.com/content/www/us/en/developer/articles/technical/software-security-guidance/technical-documentation/gather-data-sampling.html

Workaround EVC Intel “Broadwell” Generation or gather_data_sampling

wget https://raw.githubusercontent.com/speed47/spectre-meltdown-checker/master/spectre-meltdown-checker.sh# sh spectre-meltdown-checker.sh --variant downfall --explain

EVC Intel “Skylake” Generation

CVE-2022-40982 aka 'Downfall, gather data sampling (GDS)'
> STATUS:  VULNERABLE  (Your microcode doesn't mitigate the vulnerability, and your kernel doesn't support mitigation)
> SUMMARY: CVE-2022-40982:KO

EVC Intel “Broadwell” Generation

CVE-2022-40982 aka 'Downfall, gather data sampling (GDS)'
> STATUS:  NOT VULNERABLE  (your CPU vendor reported your CPU model as not affected)
> SUMMARY: CVE-2022-40982:OK

Mitigation with an updated kernel

When an update of the microcode is not available via a firmware update package, you may update the Kernel with a version that implements a way to shut off AVX instruction set support. It can be achieved by adding the following kernel command line parameter:

gather_data_sampling=force

Mitigation Options

When the mitigation is enabled, there is additional latency before results of the gather load can be consumed. Although the performance impact to most workloads is minimal, specific workloads may show performance impacts of up to 50%. Depending on their threat model, customers can decide to opt-out of the mitigation.

Intel® Software Guard Extensions (Intel® SGX)

There will be an Intel SGX TCB Recovery for those Intel SGX-capable affected processors. This TCB Recovery will only attest as up-to-date when the patch has been FIT-loaded (for example, with an updated BIOS), Intel SGX has been enabled by BIOS, and hyperthreading is disabled. In this configuration, the mitigation will be locked to the enabled state. If Intel SGX is not enabled or if hyperthreading is enabled, the mitigation will not be locked, and system software can choose to enable or disable the GDS mitigation.

Links:

How to Configure NVMe/TCP with vSphere 8.0 Update 1 and ONTAP 9.13.1 for VMFS Datastores

vSphere 8U1 – Deep dive on configuring NVMe-oF (Non-Volatile Memory Express over Fabrics) for VMware vSphere datastores.
What’s new

With vSphere 8.0 update 1, VMware has completed their journey to a completely native end-to-end NVMe storage stack. Prior to 8.0U1, there was a SCSI translation layer which added some complexity to the stack and slightly decreased some of the efficiencies inherent in the NVMe protocol.

ONTAP 9.12.1 added support for secure authentication over NVMe/TCP as well as increasing NVMe limits (viewable on the NetApp Hardware Universe [HWU]).

For more info and source blog please check great post How to Configure NVMe/TCP with vSphere 8.0 Update 1 and ONTAP 9.13.1 for VMFS Datastores

How to run Secure Boot Validation Script on an ESXi Host

Help for validation script:

/usr/lib/vmware/secureboot/bin/secureBoot.py -h
usage: secureBoot.py [-h] [-a | -c | -s]

optional arguments:
  -h, --help            show this help message and exit
  -a, --acceptance-level-check
                        Validate acceptance levels for installed vibs
  -c, --check-capability
                        Check if the host is ready to enable secure boot
  -s, --check-status    Check if UEFI secure boot is enabled

Check if the host is ready to enable secure boot

/usr/lib/vmware/secureboot/bin/secureBoot.py -c
Secure boot can be enabled: All vib signatures verified. All tardisks validated. All acceptance levels validated

Check if UEFI secure boot is disabled

/usr/lib/vmware/secureboot/bin/secureBoot.py -s
Disabled

Create Cisco UCS Boot Policy

Check if UEFI secure boot is enabled and working

/usr/lib/vmware/secureboot/bin/secureBoot.py -s
Enabled
vSphere Secure Boot

Deprecation of legacy BIOS support in vSphere 8.0 (84233) + Booting vSphere ESXi 8.0 may fail with “Error 10 (Out of resources)” (89682)

UCSX-TPM2-002 Trusted Platform Module 2.0 for UCS servers

    Personally, here are the recommendations for new ESXi 8.0 installations:

    • VMware only supports UEFI boot in new installations
    • For the purchase of new servers, it is suitable with TPM 2.0
    • When upgrading to ESXi 8.0, verify that UEFI boot is enabled

    Booting vSphere ESXi 8.0 may fail with “Error 10 (Out of resources)” (89682)

    • Hardware machine is configured to boot in legacy BIOS mode.
    • Booting stops early in the boot process with messages displayed in red on black with wording similar to “Error 10 (Out of resources) while loading module”, “Requested malloc size failed”, or “No free memory”.
    “Error 10 (Out of resources) while loading module”, “Requested malloc size failed”, or “No free memory”

    VMware’s recommended workaround is to transition the machine to UEFI boot mode permanently, as discussed in KB article 84233 . There will not be a future ESXi change to allow legacy BIOS to work on this machine again.

    Deprecation of legacy BIOS support in vSphere (84233)

    VMware’s plans to deprecate support for legacy BIOS in server platforms.

    If you upgrade a server that was certified and running successfully with legacy BIOS to a newer release of ESXi, it is possible the server will no longer function with that release. For example, some servers may fail to boot with an “Out of resources” message because the newer ESXi release is too large to boot in legacy BIOS mode. Generally, VMware will not provide any fix or workaround for such issues besides either switching the server to UEFI

    Motivation

    UEFI provides several advantages over legacy BIOS and aligns with VMware goals for being “secure by default”. UEFI

    • UEFI Secure Boot, a security standard that helps ensure that the server boots using only software that is trusted by the server manufacturer.
    • Automatic update of the system boot order during ESXi installation.
    • Persistent memory
    • TPM 2.0
    • Intel SGX Registration
    • Upcoming support for DPU/SmartNIC
    Securing ESXi Hosts with Trusted Platform Module
    vSphere 6.7 Support for ESXi and TPM 2 0

    List of vSphere 8.0 Knowledge base articles and Important Links (89756)

    List of Knowledge base articles for vSphere 8.0 – [Main KB] – List of vSphere 8.0 Knowledge base articles and Important Links (89756)

    “SECUREBOOT: Image DENIED.” – Virtual Machine with Windows Server 2022 KB5022842 (OS Build 20348.1547) configured with secure boot enabled not booting up (90947)

    Reference error “SECUREBOOT: Image DENIED.” for Linux VMs

    Important KB90947 Symptoms

    After installing Windows Server 2022 update KB5022842 (OS Build 20348.1547), guest OS can not boot up when virtual machine(s) configured with secure boot enabled running on vSphere ESXi 6.7 U2/U3 or vSphere ESXi 7.0.x.

    In VM vmware.log, there is ‘Image DENIED’ info like the below:

    2023-02-15T05:34:31.379Z In(05) vcpu-0 - SECUREBOOT: Signature: 0 in db, 0 in dbx, 1 unrecognized, 0 unsupported alg.
    2023-02-15T05:34:31.379Z In(05) vcpu-0 - Hash: 0 in db, 0 in dbx.
    2023-02-15T05:34:31.379Z In(05) vcpu-0 - SECUREBOOT: Image DENIED.
    To identify the location of vmware.log files:
    1. Establish an SSH session to your host. For ESXi hosts
    2. Log in to the ESXi Host CLI using root account.
    3. To list the locations of the configuration files for the virtual machines registered on the host, run the below command:
    #vim-cmd vmsvc/getallvms | grep -i "VM_Name"
    1. The vmware.log file is located in virtual machine folder along with the vmx file.
    2. Record the location of the .vmx configuration file for the virtual machine you are troubleshooting. For example:
    /vmfs/volumes/xxxxxxxx-xxxxxxx-c1d2-111122223333/vm1/vm1.vmx
    /vmfs/volumes/xxxxxxxx-xxxxxxx-c1d2-111122223333/vm1/vmware.log

    Resolution

    Currently there is no resolution for virtual machines running on vSphere ESXi 6.7 U2/U3 and vSphere ESXi 7.0.x. However the issue doesn’t exist with virtual machines running on vSphere ESXi 8.0.x.

    Note: vSphere ESXi 6.7 is End of general Support. For more information, see The End of General Support for vSphere 6.5 and vSphere 6.7 is October 15, 2022.

    Workaround

    There are three methods to avoid this issue

    1. Upgrade the ESXi Host where the virtual machine in question is running to vSphere ESXi 8.0
    2. Disable “Secure Boot” on the VMs.
    3. Do not install the KB5022842 patch on any Windows 2022 Server virtual machine until the issue is resolved.

    See the Microsoft article for details on the updates within the patch release

    To disable virtual machine “Secure Boot “option, please follow the below steps:

    1. Power off the VM.
    2. Right-click the virtual machine and click Edit Settings.
    3. Click the VM Options tab.
    4. Under Boot Option, uncheck the “Secure Boot enabled

    Related Information

    Uninstalling the KB5022842 patch will not resolve the issue. If the Virtual machine has already been updated, then the only available options are
     

    1. Upgrade the ESXi Host where the virtual machine in question is running to vSphere ESXi 8.0
    2. Disable “Secure Boot” on the VMs.

    “SECUREBOOT: Image DENIED.” – Linux VMs created with Hardware version 20 will fail to start installation when Secure Boot is enabled (88737)

    Reference “SECUREBOOT: Image DENIED.” for Windows Server 2022
    How ESXi Uses UEFI Secure Boot

    Important 88737 Symptoms

    The installation of the Operating System image will be denied and “SECUREBOOT: Image DENIED.” will be reported in vmware.log.

    Below goes the list of the impacted Linux Operating Systems.

    • RHEL 8.0~8.4, 7.x 
    • CentOS 8.0~8.5, 7.x
    • Oracle Linux 8.0~8.3, 7.x
    • AlmaLinux 8.4    
    • Rocky Linux 8.4    
    • Photon OS 4.0GA, 3.0 GA & Rev 2 & Rev 3, 2.0    
    • Ubuntu LTS 20.04~20.04.4, 18.04~18.04.5 and earlier
    • Ubuntu Non-LTS 21.04, 20.10, 19.10, 19.04, 18.10 and earlier     
    • Debian 10.9 and earlier     
    • SLE 12SP0~SP5, 15SP0-SP2

    Cause

    This is caused due to the Secure Boot deny list (dbx) is updated to prevent vulnerable bootloaders from being used. For more information, refer to VMware response to GRUB2 security vulnerability CVE-2020-10713 (80181)

    Resolution

    1. Create the SecureBoot Virtual Machine with Hardware version 19 (or earlier).
    2. After the installation is completed, update the vulnerable bootloader of the VM to a newer and fixed version, refer to VMware response to GRUB2 security vulnerability CVE-2020-10713 (80181)
    3. Upgrade the Virtual Machine’s Hardware version to 20.

    Workaround

    Create the Virtual Machine with Secureboot disabled instead.