Microsoft KB2661254 and VMware

Microsoft today is pushing an update to Windows via Windows Updates that increases the bit length requirements for RSA certificates.  With this update, Microsoft is requiring RSA certificates to have a key length of 1024 bits.  Some applications do not yet use a 1024 or greater key length and could be impacted by this updated.  Some examples:

• Internal Microsoft Certificate Authority servers that were built with a less than 1024 key will not start after the update – this will make certification validation for internally generated certs fail (could impact Outlook Web Access, vCenter server, intranet sites, etc.)
• Internet Explorer will not allow users to connect to sites secured with less than 1024-bit RSA cert
• Installation packages signed with less than 1024-bits will not run without a ‘Unknown Publisher’ security warning being acknowledged.

Microsoft has a list of known issues for this update here: https://support.microsoft.com/kb/2661254

Be on the lookout for this update causing other issues as it is applied across your environments – I suspect the list of known issues is not complete.  Many apps use lower strength keys to sign communications (e.g. – VMware View uses a 512-bit signed RSA cert for Java Messaging Service to communicate between Connection Servers.  Note – I haven’t heard any issues specific to View and this update, just the first example I thought up).

There is a known issue with VMware vCenter 4.0 and KB 2661254.  VMware covers the issue in their KB 2037082 – After installation of Microsoft Security Advisory update (KB2661254), connection to vCenter Server 4.0.x web services may fail.  This is due to the vCenter 4.0 self-signed certificate having a key length of 512-bits.  After MS KB2661254 is applied to the vCenter Server or to clients accessing the vCenter Server, connections to vCenter Web Services (Managed Object Browser, WebAccess) fails, redirection of http to https fails, (not confirmed – but I suspect is true) some plugins such as vCenter Update Manager that are signed by shorter bit-length certificates will be disabled, and other VMware components such as View Composer and View Connection Servers may not function correctly against a vCenter with an invalidated certificate.  To correct the issue, you will need to replace the default Self-Signed certificate with a new certificate.  The new cert can be a new Self-Signed, or can be issued from a CA (internal Microsoft Certification Authority or Commercial Third-Party).

vCenter Servers 4.1 and later are 2048 keys by default.  However, if you upgrade from a previous version of vCenter and did not apply new certificates, the upgrade may have carried the old 512-bit certificate (which is now invalid due to Microsoft’s update).  This means it is possible for vCenter 4.1, 5.0 and 5.1 to have invalidated certificates.