Although VMware Cloud Foundation (VCF) will automatically download the software bundles once connected to the Internet, there are times that you may wish to manually inject the software bundles into the SDDC Manager using the API.
After the initial deployment of VCF is completed, additional bundles may be required to enable optional functionality. For example, the bundles for Horizon, PKS, and vRealize Automation are optional and would be downloaded and then installed as needed. Because these bundles can be quite large and VCF downloads them in a serial fashion, it can be faster at times to inject the bundles into VCF manually through the API.
This is the process I use to do this…
To start, you will need to download the desired bundle files. Each bundle normally consists of three files:
- A manifest file
- This is a JSON formatted file that provides information about the bundle.
- A signature file
- This is a hash of the tarball and is intended to verify that the tarball has not been tampered with.
- The bundle tarball
- This contains the files that comprise the bundle.
All of the VMware bundle files are accessible via the web server at depot.vmware.com. For example, here are the links you would use to download bundle with the ID of 9927:
https://depot.vmware.com/PROD2/evo/vmw/manifests/bundle-9927.manifest.sig
https://depot.vmware.com/PROD2/evo/vmw/manifests/bundle-9927.manifest
https://depot.vmware.com/PROD2/evo/vmw/bundles/bundle-9927.tar
In order to download these, you will need any valid MyVMware account. To download other bundles, you would simply change the bundle ID to match the bundle ID that you wish to download.
You can see the bundles by looking at the index file on depot.vmware.com. The name of the index file may change depending on the version of VMware Cloud Foundation you are using. You can verify this by looking at the application-prod.properties file, if needed. With the current VCF 3.9.1 release, the index file is called ‘index.v3’ and the complete path can be seen in the screenshot below:
Notice that each line in the index file consists of a GUID followed by the name of the manifest file.
But how do you determine what bundle ID you need? Although you could go and look at the contents of each manifest file, there is a quicker way! The easiest is to use the lcm-bundle-transfer utility that is included on the SDDC Manager. You can find full directions for using this in the VMware Cloud Foundation Documentation. For the purposes here, we are just going to use it to help us identify the bundles that we want.
When a new version of VCF is released, I will install it on my home lab using the VMware Lab Constructor. Once VCF is installed, I then use the lcm-bundle-transfer utility to generate a marker file and a corresponding md5 file. Leveraging these files, I will then use the same utility to query the depot and list all the bundles applicable to a fresh install of that VCF version. The utility will show me what the bundle ID is and shows me what the bundle is for. I will then take this information and download the bundle, manifest, and signature files manually, taking care to save the manifest and signature files as text files (and not HTML). Once I have the bundles downloaded, I can keep those files and use them for multiple VCF installations – only refreshing my list of bundles when I know there are more updates available.
To use the bundles you have manually downloaded, first you have to transfer them to the SDDC Manager. It’s important to note that some of the bundles can be quite large. If you don’t take care with where you stage them, you can easily fill up the filesystem on the SDDC Manager. This will cause you other issues. (Ben Sier recently published a blog on how to expand the filesystem here, if you need it.)
What I will do is temporarily enable the root user to ssh directly to the SDDC Manager. I do this as the default user ‘vcf’ does not have permissions to write to the /nfs mountpoint, which just so happens to have the largest amount of filesystem space available on the SDDC Manager.
Enabling root ssh access is done by modifying the PermitRootLogin attribute in the /etc/ssh/sshd_config file:
PermitRootLogin yes
After restarting sshd (#systemctl restart sshd), I will then create a temporary directory under /nfs to store all the bundles that I have manually downloaded previously.
Using your preferred scp utility, copy the bundle files you downloaded (the tarball, the signature, and the manifest file) to that directory.
Once everything is in place, you then need to change the ownership and permissions on the directory and the associated files you copied. The files and directory should be owned by the user vcf_lcm and the group vcf.
Assuming the directory I was storing my bundles in was /nfs/bundles/, I would use commands similar to set these for the files:
# chown vcf_lcm:vcf /nfs/bundles/*
# chmod 777 /nfs/bundles/*
You would do the same for the directory. This is excessive, but once the bundles are injected then you can delete the directory (the files will be removed from the directory automatically).
All that is left now is to use the API call to ‘upload’ the bundles. To do this, you can simply use the curl command on the SDDC Manager like so (making sure you change the bundle IDs and paths as needed):
# curl -k http://127.0.0.1/lcm/bundle/upload -X POST -d ‘{“bundle”:”/nfs/bundles/bundle-6718.tar”,”manifest”:”/nfs/bundles/bundle-6718.manifest”, “signature”:”/nfs/bundles/bundle-6718.manifest.sig”}’ -H ‘Content-Type:application/json’
After executing the command, you can then go to the SDDC Manager UI and look at the bundles. Each bundle will have to go through the validation process and when that completes successfully, you will see the bundle appear in the bundle download history.
You can do this for multiple bundles at the same time, just be aware that all the bundles still have to go through the validation process. This can take some time to complete, depending on the size of the bundle. My general rule here is to do about three bundles at a time and wait until one of them completes before doing more.
After your done uploading all the bundles you want, you can delete the directory you made for the staging area as it should be empty now. Don’t forget to change back the sshd_config file settings if you changed that.
Again, the easiest way to get updates is to just let VCF download them. However, if time is your enemy and you find yourself doing multiple VCF deployments, this method might help.