vCenter Server Reduced Downtime Upgrade

I have seen some questions coming in about Reduce Downtime Upgrade features lately, so I figured I’d share some more information about this. This feature was introduced in vSphere 7.0 Update 3 and it provides a new way of doing migration based upgrades for vCenter servers.

Reduced Downtime Upgrade (RDU) simplifies the migration process and reduces downtime (as the name implies) for vCenter while the data is being moved/copied from the old vCenter to the new vCenter. So the only downtime happens when the services on the old vCenter are stopped and started on the new vCenter. The data is copied almost in a vMotion type of way. Pretty slick.

The main question I see is: Does this apply to all deployment types including On-Premises and Cloud deployments?

The answer is NO. This feature (as of right now) only applies to VMC on AWS and Project Arctic. So for now, RDU is not available/supported for on-premises deployments, but that’s not to say it will never be supported on-premises in the near future. Also RDU is only available via API at the moment, and for the VMC on AWS and Project Arctic use cases, the vCenter upgrade is done by VMware Site Reliability Engineers (SRE), so you as a customer don’t need to worry/trigger the upgrade/update of vCenter server. You can safely pass the burden on to the SREs. That alone can justify moving to VMware’s Project Arctic offering when available IMHO.

Hopefully this post answers some questions. For more information refer to the official blog post here.

My Top 10 VMworld 2021 Sessions

VMworld 2021, is fast approaching. In only two weeks we will be able to enjoy free sessions on new technology, features, and solutions. Imagine that!

I have recently moved to a new role, and I now have a little (a lot actually) more insight as to what is going on behind the curtain. I am exited for this year’s VMworld announcements about VMware partnering with vendors to deliver new, emerging tech to help customer solve their complex problems. From cost efficiency to high performance, to the cloud… I better stop before I give anything away.

I have compiled my top ten sessions. Make sure to check them out.

  1. Big Memory – An Industry Perspective on Customer Pain Points and Potential Solutions [MCL2384] 
  2. Introducing VMware’s Project Capitola: Unbounding the “Memory Bound” [MCL1453] 
  3. 10 Things You Need to Know About Project Monterey [MCL1833]
  4. Partner Roundtable Discussion: Project Monterey—Redefining Data Center Solutions MCL2379] 
  5. Project Monterey: Present, Future and Beyond [MCL1401]  
  6. Disaggregating Storage and Compute with HCI Mesh: Why, When, and How [MCL1683]
  7. VMware Cloud Foundation Tips and Tricks from the Trenches [MCL1025]
  8. How vSphere Is Redefining Infrastructure For Running Apps In the Multi-Cloud Era [MCL2500]
  9. The Big Memory Transformation [VI2342]
  10. Bring Intel PMem into the Mainstream with Memory Monitoring and Remediation [MCL3014S]

Go register for VMworld (free). See you there…

NSX Advanced Load Balancer configuration for vSphere with Tanzu

One of the pre-requisites for vSphere with Tanzu is the ability to provide a Load Balancer to the environment. THe options as of vSphere 7.0 U1 included NSX or HAProxy appliance. In vSphere 7.0 Update 2, a new option for load balancer is available for the deployment of Workload Management. The NSX Advanced Load Balancer (ALB) also known as AVI, is available for download in OVA format from Deploying ALB will allow for communication between users, service engines, load balancer type services with supervisor and TKG clusters.

Let’s jump into it. First download the OVA for NSX Advanced Load Balancer from my.vmware site.

Once the OVA has been downloaded, proceed to your vCenter and deploy the OVA by supplying a management IP address.

Supplying a sysadmin login authentication key is not required.

Once the appliance has been deployed and powered on, login to the UI using the supplied management IP/FQDN.

Create username and password. Email is optional.

Add DNS , NTP, and backup passphrase information.

If you provided email address, you would also need to provide email settings.

You will also need to identify which Orchestrator Integration will be used. Select VMware vCenter/vSphere.

The appliance needs to know how to connect to your vCenter. Supply the username, password and vCenter information so that ALB can connect to vCenter. For permissions, you can leave “Write” selected, as this will allow for easier deployment and automation between ALB and vCenter. Leave SDN Integration set to “None”.

Select the Management PortGroup, IP subnet, gateway, and IP address pool to be utilized. This IP Pool is a range of IP to be used to connect the Service Engine (SE) VMs.

After the initial configuration, we will need to either import a certificate or create a self-signed certificate to be used in Supervisor cluster communication. For PoCs, creating a self-signed certificate is perfectly fine.

Log in to the appliance using the user/password combination specified during initial deployment.

Navigate to Administration by selecting this option from the drop-down menu on the upper left corner.

In the administration pane, select Settings.

Click on the caret under SSL/TLS Certificate to expand the options. Click on “Create Certificate” green box.

Create a self-signed certificate by providing the required information. Make sure to add Subject Alternate Name(s).

The next step is to configure the data network. Change from the administration tab to Infrastructure by selecting the option from the drop-down menu on the upper left corner.

From the Infrastructure pane, click on Networks. ALB will talk to vCenter and retrieve available networks.

Ideally, you’ll want to create a separate PortGroup for your data network. In this example, this is the Frontend network. The Load Balancer VIPs will be assigned to Service Engines from this network. These VIPs are the interfaces that connect users to clusters and Kubernetes services.

Select the Port Group for the data network, specify an IP address, and add a static IP address pool to be used.

Because service engines (SE) don’t have interfaces on the workload network that Kubernetes cluster nodes are connected to, we need to specify a route. Note: VIPs are on a network accessible to the client, but the workload network may not be, so these routes enable traffic to get to the clusters.

To add a route, Navigate to Infrastructure > Routing > Create

In this example, the is the Workload Network, and the next hop is the gateway for the VIP network.

Next, we need to create an IPAM Profile. This is needed to tell the controller to use the Frontend network to allocate VIPs via IPAM.

Navigate to Templates > Profiles > IPAM/DNS Profiles > create

Via IPAM profile, change the cloud for usable network to Default-Cloud, and set the usable network to the VIP network, in this case vDS-WCP-Frontend.

At this point your NSX Advanced Load Balancer Appliance is ready to be used during the configuration of Workload Management.

Dave Morera

VCF Cloud Builder: REUSE

When deploying VMware Cloud Foundation (VCF), we deploy an appliance (ova) called Cloud Builder. This appliance allow us to load the parameter file an automate the deployment of the entire infrastructure, allowing us to go from spreadsheet to full SDDC.

Cloud Builder also includes the VMware Imaging Appliance that you can leverage to build your ESXi hosts. Just navigate to https:// Cloud_Builder_VM_IP:8445/via and login with the admin credentials you set during Cloud Builder ova deployment. There are some documents out there stating to use root and EvoSddc!2016 as password but that most likely not work, at least not on newer versions available. More on VIA here.

Anyway, once the bringup of VCF has been completed, Cloud Builder will most likely not be used again, you can safely remove it. However, if you plan to deploy other VCF environments or, like me, is constantly deploying VCF on lab environments you can “refresh/reuse” Cloud Builder via a couple of options… See what I did there?!? via … like the imaging appliance.. never mind.

So the first option is to simply take a snapshot of Cloud Builder after the initial deployment of the ova and BEFORE you login to it and start importing the parameter file. Once you are done using Cloud Builder, you can simply revert the snapshot back and you can reuse the appliance again.

If you prefer not to take the snapshot or forgot to do it at the beginning, no worries. We can run a command within Cloud builder to remove previous entries and REUSE Cloud Builder.

  • su –
  • psql –host=localhost -U postgres -d bringup
  • delete from execution;
  • delete from “Resource”;
  • \q

Disclaimer: this is most likely not officially supported and hacking at the db is done at your own risk. Since this is Cloud Builder the risk is low and you can always deploy a new appliance if you mess it up, but still be careful.

That’s it, this will remove the previous entries and next time you log in to Cloud Builder it will feel like you just deployed the appliance. Meaning that you have to go through the initial prompts, such as accepting the EULA, selecting the platform to be used, etc.


vSAN Encryption at Rest & In Transit: What is the difference?

In the past, I’ve written a few posts about vSAN Data-at-Rest Encryption, which became available with vSAN 6.6. You can find those posts here. In vSAN version 7.0U1 there is a new option for encryption, Data-In- Transit Encryption. So what is the difference? Can I only choose one or both? Let’s find out.

vSAN Data at Rest Encryption

Data-at-rest (D@RE) was designed to do just that. Encrypt all your data once it lands on the disks being used by vSAN. This will work regardless the Storage Policy you choose, and all the data replicas will be encrypted at both the cache layer and the capacity layer. One major advantage of Data-at-Rest Encryption over the vSphere VM encryption is that vSAN will still allow you to encrypt your data and take advantage of space saving features such as deduplication and compression. When the data lands in cache it will be encrypted using the Data Encryption Key (DEK), then while the data is being destaged to the capacity layer it will be decrypted, and it is here where the deduplication and compression takes place. Finally when the data lands in the capacity devices, the data gets encrypted once again. It is also important to highlight that the DEK is protected by the Key Encryption Key (KEK) which is coming from the Key Management Server (KMS)… and this is one of the differences between the two options.

vSAN Data in Transit Encryption

Data-In-Transit Encryption (DIT) comes in to complete the end-to-end encryption of the data while in transit between hosts. Data-at-Rest encryption only encrypts the data when it lands on disk, so if someone takes a disk out of a server, all data is encrypted. But what about other attacks such as Man-in-the-middle attacks? Well, this is where Data-In-Transit encryption can protect the data. The keys used for DIT encryption are managed internally and there is no need for a KMS. Such keys are also rotated much, much faster when compared with D@RE. DIT encryption keys are rotated weekly by default, but you can change this option and rotate keys either every 7 days or every 6 hours or something in between. Just like D@RE encryption, DIT encryption works at a vSAN cluster level; so either all the hosts are protected or none.

Here is a quick comparison between the two options


Can I enable both at the same time?

Yes. You can enable Data at rest and Data in Transit encryption in order to get full protection in your vSAN environment. It is recommended to enable vSAN Data at Rest encryption in the early stages of the cluster to minimize the time for on-disk formatting as there is less data to move around.

What is the performance impact of turning encryption on?

There are a lot of variables that come in to play when we talk about performance. However; vSAN encryption (both) will take advantage of AES-NI and offload operations in order to reduce any performance hit. Most modern CPU have AES-NI, but sometimes this feature is not enabled, so make sure to check this at deployment. Please also be mindful that enabled D@RE when the cluster has a lot data in it will result in large amounts of data being moved, so plan this to be done during off hours if possible.

What vSAN License do I need to enable vSAN Encryption?

In order to enable Data-at-Rest and/or Data-In-Transit Encryption you will need vSAN Enterprise or vSAN Enterprise Plus licenses. Refer to licensing guide here.

How do I enable Data-In-Transit Encryption?

Enabling DIT encryption is easy. Within the vCenter UI, select the vSAN cluster > Configure > Services > Data-In-Transit can be enable with or without Data-at-Rest encryption. Here is where you can also change the key rotation schedule for the DIT encryption keys.