VCF Stretched VSAN Cluster and APIs
In my previous blog I talked about the similarities between a 2 node VSAN cluster vs a stretched VSAN cluster so I thought I would continue the same theme and write about stretched VSAN cluster management in VMWare Cloud Foundation [1] using APIs.
In VCF, VSAN is a requirement for the Management Workload Domain and is deployed during instantiation of the environment by Cloud Builder, however Cloud Builder will only deploy a single availability zone VSAN cluster. If you want your management plane to be highly available, you can stretch your Management Workload Domain across two availability zones and stretch the VSAN.
The advantage of having a stretched Management Workload Domain is that is provides high availability for your management virtual workloads in the event of for example a Data Center failure. Virtual machines will be brought up in the secondary availability and in my experience, this is generally under 10 minutes.
One thing that is not generally known about VCF is that any changes to the environment are generally done via API within the SDDC Manager web interface, or other common methods of pushing out API requests e.g., Postman or Curl. This applies to any VSAN operation which include stretching / un-stretching VSAN clusters and adding or removing hosts from the VSAN cluster. There is publicly available documentation for VCF API Reference Guide [2].
There are perquisites for stretched VSAN which must be addressed before any API calls can be carried. These include, available hosts must be added to the SDDC Manager and available for use, a VSAN Witness nodes at a tertiary location, and layer 3 network for the VSAN vmkernel at the secondary availability zone where you are stretching the VSAN too.
Once this is done you can retrieve the host ids of the newly assigned hosts and the cluster id which you wish to stretch via API.
To get the hosts run the following curl command or use the SDDC API Explorer.
$ curl 'https://sddc-manager.yourdomain.local/v1/hosts' -i -u 'admin:VMwareInfra@1' -X GET -H 'Accept: application/json'
You will a JSON respone like the extract below which provides the host ids of the new hosts.
"id": "62771c25-8ef0-430c-b69b-a297e42c0ce1",
"esxiVersion": "7.0.1-17551050",
"fqdn": "esxuat10.techaspire.com.au",
"hardwareVendor": "SomeVendor",
"hardwareModel": "Some Model",
To get the cluster id run the following curl command or use the SDDC API Explorer
$ curl 'https://sddc-manager.yourdomain.local/v1/clusters' -i -u 'admin:VMwareInfra@1' -X GET
You will get a JSON respone like the extract below which provides the cluster id of the existing VSAN cluster you wish to stretch
"id": "c385a274-352f-4359-81c0-409efed3a012",
"name": "mgmt-cluster",
"primaryDatastoreName": "mgmt-vsan01",
"primaryDatastoreType": "VSAN"
As a note, the minimum number of Management hosts required to stretch a VSAN cluster between availability zones is 8 (4x hosts in each AZ ). Standard Workload Domain clusters can have a minimum of 3x hosts per availability zone.
Once all the pre-requisites are completed you can stretch the VSAN cluster to the secondary AZ which require 2x API POSTs to the SDDC Manager. The first is Cluster Validation SPEC API and the API to stretch the cluster.
An example API call and spec file to stretch the cluster would be similar to the following JSON albeit the example is a cut down version.
curl 'https://sddc-manager.yourdomain.local/v1/clusters/c385a274-352f-4359-81c0-409efed3a012/validations' -i -u 'admin:VMwareInfra@1' -X POST -H 'Content-Type: application/json' -d '
{
"clusterStretchSpec": {
"hostSpecs": [ {
"id": "62771c25-8ef0-430c-b69b-a297e42c0ce1",
"licenseKey": "THISI-SNOTT-HEKEY-YOURA-FTER1",
"hostNetworkSpec":{
"vmNics":[
{
"id":"vmnic0",
"vdsName":"data"
},
{
"id":"vmnic1",
"vdsName":"data"
},
{
"id":"vmnic2",
"vdsName":"vsan"
},
{
"id":"vmnic3",
"vdsName":"vsan"
}
]
}
},
"secondaryAzOverlayVlanId": 1234,
"isEdgeClusterConfiguredForMultiAZ":true,
"witnessSpec": {
"fqdn": "dawitnessnode.yourdomain.local",
"vsanCidr": "192.20.2.64/27",
"vsanIp": "192.20.2.20"
}
}
}
Once this task has completed you will have a stretched VSAN cluster across two availability zones.
As per my previous blog, the VSAN cluster will be broken up into a primary and secondary fault domain with virtual machines residing in the primary fault domain unless they are manually vmotioned to the secondary fault domain, or a disaster takes place which brings the virtual machines up in the secondary AZ.
An important note is that once you have expanded your VSAN cluster you will have to modify your Cluster HA Admission Control settings to reflect the number of hosts in a single VSAN fault domain.
Figure 1 – Example HA Configuration
To add or remove hosts from the cluster through the normal lifecycle of infrastructure and capacity requirements APIs are used once again using the same basic requirements of the host and cluster id’s, and is a very quick process especially if you use the SDDC API Explorer tool built into the SDDC Manager.
Figure 2 – Example SDDC Manager API Explorer
If you want to get your head around VCF and API’s there is always VMWare’s Hand on Labs, and VMUG Advantage allows you to get VCF licensing so you can deploy a nested lab environment following this guide https://blogs.vmware.com/cloud-foundation/2020/01/31/deep-dive-into-vmware-cloud-foundation-part-1-building-a-nested-lab/ .
[1] https://www.vmware.com/au/products/cloud-foundation.html
[2] https://vdc-download.vmware.com/vmwb-repository/dcr-public/60ff5385-d6ee-41d3-9ccf-b719a59f7971/b8ba0641-d8ae-4243-acf2-3639ca248783/index.html