The Async Patch Tool is a command line utility that allows you to apply critical patches to VCF components NSX-T Manager, vCenter Server, and ESXi (Note:VxRail ESXi patching not supported). As this moves you out of the release versions, the tool also enables upgrades of an async patched system to a new version of VCF!
This tool was developed after hearing from many customers that not being able to patch as quickly as possible after a vulnerability was discovered was a big issue. All the product teams involved lined up to provide a programmatic way to get the critical patches certified and in the hands of customers. This also gives our customers greater flexibility in not having to wait for or consume a full VCF release.
The Async patch tool has two modes, online and offline, meaning if you don’t have internet connectivity at your site for whatever reason you can still use this tool to patch similar to the LCM offline bundle utility. For this post we’ll focus on the online mode and start by taking a look at the docs and associated KB article:
https://docs.vmware.com/en/VMware-Cloud-Foundation/services/ap-tool/GUID-49818DF1-94EA-4C85-8CB6-6EFFCE5F8060.html
https://kb.vmware.com/s/article/88287
https://docs.vmware.com/en/VMware-Cloud-Foundation/services/rn/async-patch-tool-release-notes/index.html
My walkthrough version of the docs above:
- Download Async Tool from Customer Connect
- SSH to the SDDC manager and create a directory and copy the downloaded tool’s tar.gz to SDDC Manager (for those interested in *native scp* in Windows see here)
mkdir /home/vcf/asyncPatchToolscp -oUser=sddclab\administrator 10.0.0.220:/c:/users/administrator/downloads/vcf-async-patch-tool.tar.gz /home/vcf/asyncPatchTool/
- Extract, set the appropriate permissions and run the tool
tar -xvf vcf-async-patch-tool.tar.gzcd /home/vcf/chmod -R 755 asyncPatchToolchown -R vcf:vcf asyncPatchTool./vcf-async-patch-tool --listAsyncPatch --du customer_connect_email- Replace customer_connect_email with your VMware Customer Connect email
The list of files above are ALL the available patches, this does not necessarily mean that you should, or can apply them all. To determine what to apply you should reference the KB article.
In the case of our lab we have a fully updated VCF running 4.4.1.1, looking at the KB article and then at our live system we can see only 3 out of the 7 apply to VCF 4.4.x, additionally two of those are vCenter patches, of which we’ll only apply the latest.
Next step is to get those bundles we want to apply downloaded and pushed into the local LCM repository. For that, let’s go back to our SDDC Manager ssh session. From the /home/vcf/asyncPatchTool directory run the following:
./vcf-async-patch-tool -e --patch product:version --du customer_connect_email -–sddcSSOUser SSOuser --sddcSSHUser vcf --it ONLINE- Replace product:version with the product and version of a patch retrieved in the last step
- Replace customer_connect_email with your VMware Customer Connect email
- Replace SSOuser with the SSO user account
The very first thing you will see is a confirmation that you’ve completed all the steps we went through before. Measure twice, cut once and type Y.
Next, we’ll need to enter passwords for the SDDC Manager SSH user, and root user. Then, the password for the SSO user and finally your password for the customer connect account. At this point it will download the bundle, verify it, and then add it to the local LCM repository.
When you see it get to the “Current upload status” stage you’ll start to see tasks appear in SDDC Manager
I did see some ERROR and WARN in the logs when LCM was restarted but it recovered very shortly thereafter.
With the download and subsequent upload complete, let’s go into the SDDC Manager UI and see if 1) The bundle showed up, and 2) If we can apply it. Drilling down into the Management Domain, Security tab we can see that there is indeed our vCenter update.
Before we click that “Update Now” button we’ll want to ensure that we’ve run through and gotten a clean pre-check, additionally we’re prompted to make sure we’ve got a file based backup of our vCenter
Once those checks and backups are complete and confirmed we’ll have a task in SDDC Manager showing us the upgrade has started. We’ll also be able to view quite a detailed status of what’s going on it the Update/Patches tab. I did ask about the Update being “1.1.1.1” and was told the only significance to that was to indicate that it’s an Async patch. I also inquired about the from in the from -> to versions (1.3.0-123456) and was told the same, these are also documented in the release notes for the tool.
Once it’s fully completed we’ll see and update and be able to see the new version in the current versions tab, as well as being able to log in to the vCenter directly.
Now, the most important part of all that is that we’re now in a situation where upgrading to a future full version of VCF can be done in a supported, automated way. In the past there have been many one off solutions that either required direct GSS and engineering support, hotfixes, or editing things that should not be edited by humans! I hope you find this useful and as things progress we’ll revisit this topic to see what different situations look like when upgrading different components and then upgrading the whole SDDC.