Ever have a host in your VMware Cloud Foundation (VCF) environment die suddenly? Ever had an issue adding a new host back into your VCF environment because you have no available IP addresses left in your Network Pool? Let’s talk about this and I’ll show you some ways to address this!
As you have see in Part 1 in this blog series we were able to get the ESXi host hardware configured as well as the Cloud Builder deployed and ready to power on (but don’t do that yet!).
After you have downloaded pfSense or the router of your choice we need to upload the media.
Click Libraries in vCD
Click Media & Other
Click Add
Select your catalog
Click upload
Navigate to your pfSense ISO and select it
Click Open
Click OK
This will upload the iso for your router to vCD. Next we need to build the VM.
Navigate back to your vApp and add a VM
Name your router VM
Click New VM as we don’t have a template yet
Select Linux from the drop down
Select Other 64bit
Find your pfSense ISO or other suitable ISO
Leave CPU count set to 1
Change memory to a suitable value, here I am going to start with 4gb and update if needed
Set storage to 12gb
Click ADD twice to add two network adapters
Set the value for network adapter 1 to the vAppNet-VCF value created earlier, leave network adapter type as E1000, set DHCP for IP Mode, and set as Primary Nic
Set the value for network adapter 2 to the vAppNet-VCF value created earlier, leave network adapter type as E1000, and set DHCP for IP Mode
Click OK
Click Add (not pictured)
Time to power on our router and configure it, but don’t power on all of the VM’s
Click the ellipsis next to router
Click Power
Click Power On
Now Click VM Console to view the console to configure our router
Click Enter key to accept the EULA (not shown)
This is a summarized configuration of pfSense to get us started before adding a Windows client desktop to our vApp.
Choose all the defaults through the installation process. After which you will configure interface eth0 or interface 1 as the WAN interface with the following information: IP address: 192.168.0.2 Netmask: 24 Gateway: 192.168.0.1
Configure interface eth1 or interface 2 as the LAN interface with the following information: IP address: 10.0.0.1 Netmask: 24 Choose enter or No Gateway when prompted
Now our router configuration has been started and should be able to be configured the remainder in the UI once our Windows Client is setup and configured. For this step we will assume you can add an additional VM and load it with the installation media for Windows Server in our case.
In our vCD environment our Windows server was configured with the following network information: IP address: 10.0.0.2 Netmask: 255.255.255.0 Gateway: 10.0.0.1 DNS: 127.0.0.1
The following packages were installed with the dependencies: Active Directory DNS Certificate Management IIS
In Part 3 we will configure the Router via the UI as you will need to build your Windows server at this point.
For the last 3 years I have been building and maintaining the VMware Cloud Foundation Experience Program (VCF Experience). This has caused me to become essentially a one man hands on lab team. However I realize there is a lot of learnings through this process, that I probably take for granted, and realize I should share my learning of nesting VCF on Cloud Director. The VCF Inception part in my case is that the physical infrastructure is running VCF as well. While I am a daily user of the VLC tool to build out my home lab, Cloud Director is a bit of a different animal as I don’t have direct access to the ESXi servers or vCenter for that matter.
Building these environments always starts with access to a Cloud Director instance, whether that be on prem or SaaS through Cloud Director Service (CDS). Once you have that in a consumer mode Organization Administrator as an example you will need to get a few things downloaded. First you will need to download your desired VCF Cloud Builder appliance, for example I used VCF 5.1. Second you will need the corresponding version of ESXi, in my case that is 8.0U2. Third you will need a Windows Server and license, while this can be done with desktop, I use the Certificate Management piece as well as Active Directory in my lab, therefore I opted for a server operating system. Finally you will need a tool to be a router we will use pfSense for this, however you can use whatever your heart desires. A complete list of VCF and its underlying component versions can be found here: VCF Component Versions. Once you have those downloaded lets head over to Cloud Director and get started.
End Result of what we are building:
Let’s start by uploading our Cloud Builder Appliance as a vApp
Click on Libraries (newer versions will show content hub potentially)
Click New
Click Browse, locating your Cloud Builder Appliance OVA
Click Next and finish out the wizard naming your vApp etc
Now let’s upload the ESXi ISO
Click Libraries
Click Media & Other from the left
Click Add
Pick your catalog and click to upload and point to your ESXi ISO that was downloaded
Finish out the upload of the ESXi ISO
Now we will need to create a new vApp
Click Applications
Click New
Click New vApp
Click to select the capacity, you may only have one here or you may have more than 3 depending upon your environment.
Click Next
Name your vApp
Click Create (we will add the VM’s manually)
Time to create a network
Click Networks
Click New
We will use these settings, and we will add routing later, however we will need to get this setup before adding VM’s.
Choose Isolated (for now)
Name your Network
Enter the CIDR for your networkE
Enter your DNS server
Select Allow Guest VLAN
Click Add
Now let’s add our first VM, you will be doing this at multiple times
Click Virtual Machines
Click All Actions
Click Add
Click Add VM
Name your VM
Add Computer Name (can be the same)
New from the selection of a New VM or from Template
Choose Other for OS Family
Choose Other (64-bit)
Choose the Boot Image of the ISO we uploaded earlier
Choose 12 CPU’s
Choose 6 cores per socket
Choose 64gb of Memory
Choose 16gb for storage
Choose Add under networking
Verify the Network is set to the Network created earlier, Network Adapter Type is set to VMXNET3, IP Mode is set to DHCP, and NIC 1 is set to Primary NIC
Verify the Network is set to the Network created earlier, Network Adapter Type is set to VMXNET3, and IP Mode is set to DHCP
Click OK
Now that we have our first ESXi-1 server in the list, repeat the previous steps until we have atleast 4 ESXi servers with the same configuration. If you want to add additional clusters or workload domains you can add the necessary servers to support that configuration.
Now that all the ESXi Hosts have been created, click Add. Before powering them on we have some more customization to do. Click on one of the ESXi hosts and it will display the following:
Click on Hard Disks
Click on Edit
Click on Add
Set the size to 50gb, and set the bus type to NVME
Set the size to 300gb, and set the bust type to NVME, and the unit number should increase to 1
Click Save
Repeat this step on each of the ESXi Hosts
These hardware settings are for OSA, at the end of this blog series I will detail out how to deploy ESA nested on vCloud Director, there will just be a few changes required in order to support this configuration.
Now that all the ESXi Hosts are configured with the proper hardware and NIC settings, we can deploy the Cloud Builder Appliance.
Now let’s add our Cloud Builder VM
Click Virtual Machines
Click All Actions
Click Add
Click Add VM
Then click Add Virtual Machine
Name your VM
Provide a Computer Name
Ensure you click on ‘From Template’ this time
You can filter to search for your Cloud Builder ova
Select your Cloud Builder OVA from the list of templates
Ensure that the network is set to e1000e, vAppNet, and DHCP mode
Provide your NTP Server 10.0.0.1
Provide DNS Search Path
Provide DNS Domain
Provide DNS Server
Provide Default Gateway
Provide Subnet
Provide an IP address to be assigned to the Cloud Builder Appliance
Provide a Hostname to the Cloud Builder Appliance
Provide a Root Password
Provide an Admin Password
Choose to enable or disable FIPS (for Lab, leave FIPS disabled unless you are testing this)
Scroll down the EULA
Accept the Eula
Hit ok to create your Cloud Builder VM
Click Add (not pictured)
Do not power on the VM
We need to disable guest customization before powering otherwise it will fail to configure the IP address properly.
Guest OS Customization
Click Edit
Deselect Guest OS Customization
Click Save
At this point we should have our ESXi hosts hardware configured but not powered on and our Cloud Builder deployed, but not powered on. In the next installment, we will discuss getting the router deployed and allow for both BGP peering as well as NAT to the outside world of vCloud Director.
Sometimes, we get ahead of ourselves… In this case I may have prematurely removed some ESXi hosts from existence! Due to this misjudgment my remove workload domain task is failing, let’s explore how we can move past it.
Adding an NSX Edge Cluster to a VCF workload domain brings a huge amount of versatility to your workloads living there. Software defined networks that can be provisioned with full routing when you need them. Security, for workloads on segments as well as those on traditional portgroups through both the Distributed and Gateway Firewalls. With VCF it’s easy to get started.
The Async Patch Tool is a command line utility that allows you to apply critical patches to VCF components NSX-T Manager, vCenter Server, and ESXi (Note:VxRail ESXi patching not supported). As this moves you out of the release versions, the tool also enables upgrades of an async patched system to a new version of VCF!
Never underestimate the bandwidth of a station wagon filled with backup tapes.
Words to live by, and as the density of those backup tapes continues to increase (Last time I used backup tapes they were 400/800GB LTO3’s), so does the station wagons bandwidth! I’ll leave it up to you to get your tapes to the vault but let’s go through what it takes to get the pertinent VCF infrastructure configuration data into a state and place where it can be backed up and restored. This will include configuration of the backups for SDDC Manager, vCenters, Exporting VDS configs, and NSX Managers, lets go!
This is not my normal blog post, but.. a while back someone from Synology contacted me and asked if I’d be interested in reviewing one of their devices. Having never used one before I asked a few questions about running virtual machines on it and was sent a 1621+ which could run a small VM as it only has 4GB RAM. I wasn’t sure what I could do with this for what I was looking to test (backup/DR, running Cloudbuilder and a virtual router), but I was given a promise of something larger in a few months time where I could run multiple larger VMs. That was last summer and the person who sent the device has left Synology and now I’ve gotten a follow up from another person looking for my review. I did use the 1621+ for a few things while I was waiting for something that, unfortunately never materialized. I’ll detail those here.