As you have see in Part 1 in this blog series we were able to get the ESXi host hardware configured as well as the Cloud Builder deployed and ready to power on (but don’t do that yet!).
After you have downloaded pfSense or the router of your choice we need to upload the media.
Click Libraries in vCD
Click Media & Other
Select your catalog
Navigate to your pfSense ISO and select it
This will upload the iso for your router to vCD. Next we need to build the VM.
Navigate back to your vApp and add a VM
Name your router VM
Click New VM as we don’t have a template yet
Select Linux from the drop down
Select Other 64bit
Find your pfSense ISO or other suitable ISO
Leave CPU count set to 1
Change memory to a suitable value, here I am going to start with 4gb and update if needed
Set storage to 12gb
Click ADD twice to add two network adapters
Set the value for network adapter 1 to the vAppNet-VCF value created earlier, leave network adapter type as E1000, set DHCP for IP Mode, and set as Primary Nic
Set the value for network adapter 2 to the vAppNet-VCF value created earlier, leave network adapter type as E1000, and set DHCP for IP Mode
Click Add (not pictured)
Time to power on our router and configure it, but don’t power on all of the VM’s
Click the ellipsis next to router
Click Power On
Now Click VM Console to view the console to configure our router
Click Enter key to accept the EULA (not shown)
This is a summarized configuration of pfSense to get us started before adding a Windows client desktop to our vApp.
Choose all the defaults through the installation process. After which you will configure interface eth0 or interface 1 as the WAN interface with the following information: IP address: 192.168.0.2 Netmask: 24 Gateway: 192.168.0.1
Configure interface eth1 or interface 2 as the LAN interface with the following information: IP address: 10.0.0.1 Netmask: 24 Choose enter or No Gateway when prompted
Now our router configuration has been started and should be able to be configured the remainder in the UI once our Windows Client is setup and configured. For this step we will assume you can add an additional VM and load it with the installation media for Windows Server in our case.
In our vCD environment our Windows server was configured with the following network information: IP address: 10.0.0.2 Netmask: 255.255.255.0 Gateway: 10.0.0.1 DNS: 127.0.0.1
The following packages were installed with the dependencies: Active Directory DNS Certificate Management IIS
In Part 3 we will configure the Router via the UI as you will need to build your Windows server at this point.
For the last 3 years I have been building and maintaining the VMware Cloud Foundation Experience Program (VCF Experience). This has caused me to become essentially a one man hands on lab team. However I realize there is a lot of learnings through this process, that I probably take for granted, and realize I should share my learning of nesting VCF on Cloud Director. The VCF Inception part in my case is that the physical infrastructure is running VCF as well. While I am a daily user of the VLC tool to build out my home lab, Cloud Director is a bit of a different animal as I don’t have direct access to the ESXi servers or vCenter for that matter.
Building these environments always starts with access to a Cloud Director instance, whether that be on prem or SaaS through Cloud Director Service (CDS). Once you have that in a consumer mode Organization Administrator as an example you will need to get a few things downloaded. First you will need to download your desired VCF Cloud Builder appliance, for example I used VCF 5.1. Second you will need the corresponding version of ESXi, in my case that is 8.0U2. Third you will need a Windows Server and license, while this can be done with desktop, I use the Certificate Management piece as well as Active Directory in my lab, therefore I opted for a server operating system. Finally you will need a tool to be a router we will use pfSense for this, however you can use whatever your heart desires. A complete list of VCF and its underlying component versions can be found here: VCF Component Versions. Once you have those downloaded lets head over to Cloud Director and get started.
End Result of what we are building:
Let’s start by uploading our Cloud Builder Appliance as a vApp
Click on Libraries (newer versions will show content hub potentially)
Click Browse, locating your Cloud Builder Appliance OVA
Click Next and finish out the wizard naming your vApp etc
Now let’s upload the ESXi ISO
Click Media & Other from the left
Pick your catalog and click to upload and point to your ESXi ISO that was downloaded
Finish out the upload of the ESXi ISO
Now we will need to create a new vApp
Click New vApp
Click to select the capacity, you may only have one here or you may have more than 3 depending upon your environment.
Name your vApp
Click Create (we will add the VM’s manually)
Time to create a network
We will use these settings, and we will add routing later, however we will need to get this setup before adding VM’s.
Choose Isolated (for now)
Name your Network
Enter the CIDR for your networkE
Enter your DNS server
Select Allow Guest VLAN
Now let’s add our first VM, you will be doing this at multiple times
Click Virtual Machines
Click All Actions
Click Add VM
Name your VM
Add Computer Name (can be the same)
New from the selection of a New VM or from Template
Choose Other for OS Family
Choose Other (64-bit)
Choose the Boot Image of the ISO we uploaded earlier
Choose 12 CPU’s
Choose 6 cores per socket
Choose 64gb of Memory
Choose 16gb for storage
Choose Add under networking
Verify the Network is set to the Network created earlier, Network Adapter Type is set to VMXNET3, IP Mode is set to DHCP, and NIC 1 is set to Primary NIC
Verify the Network is set to the Network created earlier, Network Adapter Type is set to VMXNET3, and IP Mode is set to DHCP
Now that we have our first ESXi-1 server in the list, repeat the previous steps until we have atleast 4 ESXi servers with the same configuration. If you want to add additional clusters or workload domains you can add the necessary servers to support that configuration.
Now that all the ESXi Hosts have been created, click Add. Before powering them on we have some more customization to do. Click on one of the ESXi hosts and it will display the following:
Click on Hard Disks
Click on Edit
Click on Add
Set the size to 50gb, and set the bus type to NVME
Set the size to 300gb, and set the bust type to NVME, and the unit number should increase to 1
Repeat this step on each of the ESXi Hosts
These hardware settings are for OSA, at the end of this blog series I will detail out how to deploy ESA nested on vCloud Director, there will just be a few changes required in order to support this configuration.
Now that all the ESXi Hosts are configured with the proper hardware and NIC settings, we can deploy the Cloud Builder Appliance.
Now let’s add our Cloud Builder VM
Click Virtual Machines
Click All Actions
Click Add VM
Then click Add Virtual Machine
Name your VM
Provide a Computer Name
Ensure you click on ‘From Template’ this time
You can filter to search for your Cloud Builder ova
Select your Cloud Builder OVA from the list of templates
Ensure that the network is set to e1000e, vAppNet, and DHCP mode
Provide your NTP Server 10.0.0.1
Provide DNS Search Path
Provide DNS Domain
Provide DNS Server
Provide Default Gateway
Provide an IP address to be assigned to the Cloud Builder Appliance
Provide a Hostname to the Cloud Builder Appliance
Provide a Root Password
Provide an Admin Password
Choose to enable or disable FIPS (for Lab, leave FIPS disabled unless you are testing this)
Scroll down the EULA
Accept the Eula
Hit ok to create your Cloud Builder VM
Click Add (not pictured)
Do not power on the VM
We need to disable guest customization before powering otherwise it will fail to configure the IP address properly.
Guest OS Customization
Deselect Guest OS Customization
At this point we should have our ESXi hosts hardware configured but not powered on and our Cloud Builder deployed, but not powered on. In the next installment, we will discuss getting the router deployed and allow for both BGP peering as well as NAT to the outside world of vCloud Director.
You just downloaded the latest version of the VMware Cloud Foundation Lab Constructor (VLC) so you can deploy a nested version of VMware Cloud Foundation in your lab. You’ve read all the articles and followed the directions. You kick off the deployment and…. It fails with a BGP error.
Now what? Today I’ll show you something that might help…
Sometimes.. things just don’t work right, I am writing another blog entry that required me to deploy an edge cluster and well.. I fat fingered an IP address so there was a failure. While it would be nice to just make a change to the submitted configuration, we can’t in this instance. Rather we need to remove and then redeploy the edge cluster. There is a KB for this, but I like visuals to go along with my text so let’s walk through it together!
Not so long ago while working at customers, one of the things you didn’t like to hear was that old “data center services” would need to be retired. In my experience, it was usually because they were running an ancient version of FreeBSD or <pick you distro> that wasn’t supported any longer. New services would be stood up, with new IP’s of course.. and then you had to manually change it on every. single. system… The VCF DNS and NTP API’s go a long way to help in these situations!
The other day I was helping a friend of mine get VLC up and running on his server. When doing some discovery about what hardware he had I learned that it was a dedicated server hosted at OVH. I asked how much he paid per month and was quite surprised at the response. As this is something that we constantly run into… HOW much for a lab to run VCF?! I was surprised enough that I went out and got my own!