Nested Virtualization

Automated Nested Lab Deployment on SDDC Part 4: Oracle Cloud VMware Solution

10/30/2020 by William Lam Leave a Comment

In this final blog series, we are now going to take a look at deploying our Nested Lab in an Oracle Cloud VMware Solution environment. Similiar to GCVE, we also will have the ability to run Nested vSAN on top of the physical vSAN within OCVS and by default, customers are provided with the full administrator credentials. For networking, customers also have full access to the NSX-T Manager instance and this means that MAC Learning can be enabled and will allow inner-guest workloads will be able to communicate properly within and outside of the Nested Lab deployment.

Disclaimer: Nested ESXi is not officially supported on Oracle Cloud VMware Solution or by VMware.

Automated Nested Lab Deployment on SDDC Part 3: Google VMware Cloud Engine

10/29/2020 by William Lam Leave a Comment

In Part 3 of this series, we will be taking a look at deploying our Nested Lab in a Google Cloud VMware Engine environment. One thing you may notice that is different between VMConAWS and AVS is that we are actually running a Nested vSAN on top of the physical vSAN within GCVE. In the current version of GCVE, customers have the ability to "escalate" their privileges from the default and this would allow us to configure the necessary ESXi advanced setting so that we do not have to rely on an NFS setup. For networking, customers also have full access to the NSX-T Manager instance and this means that MAC Learning can be enabled which will allow inner-guest workloads will be able to communicate properly within and outside of the Nested Lab deployment.

Disclaimer: Nested ESXi is not officially supported on Google Cloud VMware Engine or by VMware.

Automated Nested Lab Deployment on SDDC Part 2: Azure VMware Solution

10/28/2020 by William Lam Leave a Comment

In Part 2 of this series, we will be taking a look at deploying our Nested Lab in an Azure VMware Solution environment. Simliar to VMConAWS, we will need to deploy our NFS Photon OS OVA because administrative access to the physical ESXi hosts is not available. This is required to enable an advanced ESXi setting to be able to run Nested vSAN on top of the physical vSAN. From a networking standpoint, customers do have full access to the NSX-T Manager instance and this means that MAC Learning can be enabled which will allow inner-guest workloads will be able to communicate properly within and outside of the Nested Lab deployment.

Disclaimer: Nested ESXi is not officially supported on Azure VMware Solution or by VMware.

Automated Nested Lab Deployment on SDDC Part 1: VMware Cloud on AWS

10/27/2020 by William Lam 2 Comments

While preparing for this years VMworld, I had the unique opportunity to work across a number of VMware Cloud SDDC solutions such as VMware Cloud on AWS (VMConAWS), Azure VMware Solution (AVS), Google Cloud VMware Engine (GCVE) and Oracle Cloud VMware Solution (OCVS). During the the development of several demos, I found it invaluable to be able to deploy a Nested vSphere environment to validate my configurations prior to connecting our real vSphere on-premises infrastructure.

Putting aside VMworld demos, this can certainly be extended to other use cases such as accelerated pilots, proof of concepts and lab/development purposes. Customers have been leveraging Nested Virtualization technology for more than a decade plus now and it definitely makes sense that they would also want to do the same for certain workloads running within a VMware Cloud SDDC. With that said, Nested ESXi is not officially supported by VMware or on any other VMware-based platform.

As part of building my VMworld demos, I also had spent some time on creating some automation that would make it easier for me to re-deploy these Nested Lab environments and also being aware of the specific VMware Cloud SDDC solutions, so that I only have a single script to maintain. In case folks are also interested in being able to do this, you can follow this 4-part blog series which I will be kicking it off with VMware Cloud on AWS (VMConAWS).

Heads Up – Nested ESXi crashes in ESXi 7.0 running on older CPUs

04/17/2020 by William Lam 27 Comments

Thanks to Patrik Kernstock, who works in our Technical Support organization at VMware, for making me aware of an issue related to Nested ESXi running on an ESXi host that has been upgraded to ESXi 7.0. Several folks in the community have noticed after upgrading their Intel NUC 7th Gen and deploying a Nested ESXi VM and powering on an inner-guestOS would causes the Nested ESXi VM to crash.

Upon further investigation, it looks like this is not specific to the Intel NUC platform but rather with a specific generation of CPUs which are Intel Sky Lake-based and as a result, some customers are noticing this affect on their 7th Gen NUC.

UPDATE (06/23/20) - ESXi 7.0b has just been released and contains the fix for the Nested ESXi VM crash. If you are using an Intel NUC 10, do not just apply the patch as the updated ne1000 VIB within the patch will override existing Intel NIC driver causing the network adapter to no longer function. It is recommended that you download the patch and replace the default ne1000 VIB using Image Builder with the Intel NIC version before applying the update. To download the patch, please visit VMware Patch Portal site.

The good news is that this issue has already been reported and we should have a fix in a future update of ESXi. In the meantime, you can still run Nested ESXi and Nested Virtualization on these affected CPUs, you just will not be able to power on inner-guest VMs. Big thanks to Patrik for helping out with the testing and triaging this internally.

Automated vSphere 7 and vSphere with Kubernetes Lab Deployment Script

04/13/2020 by William Lam 91 Comments

I know many of you have been asking me about my vSphere with Kubernetes automation script which I had been sharing snippets of on Twitter. For the past couple of weeks, I have been hard at work making the required changes between the vSphere 7 Beta and GA workflows, some additional testing and of course documentation. Hopefully the wait was worth it (I think it is) and if you enjoy the script or have benefited, please consider adding 🌟to the Github repo to show your support! Thanks and enjoy

Had to make some updates to one of my vGhetto Automated Lab Deployment Scripts

💥44min to automate all required #vSphere7 infrastructure! 🤛🎤🥳

1 x VCSA 7.0
3 x ESXi + vSAN 7.0
1 x NSX-T 3.0 UA
1 x NSX-T Edge

Need to clean up #ProjectPacific wording but its working great! pic.twitter.com/ZInPgVgbGS

— William Lam (@lamw) April 4, 2020

The Github repository:

https://github.com/lamw/vghetto-vsphere-with-kubernetes-external-nsxt-automated-lab-deployment

Before getting started, please carefully read through the requirements section along with the complete sample end-to-end execution if you are new to vSphere with Kubernetes. You will need to have a VMware Cloud Foundation (VCF) 4.0 license before you can get started and specifically an NSX-T Advance license which is one of the required parameters within the script. If you do not have access to a VCF 4 license, I strongly recommend taking part in the recent VMUG Advantage Homelab Group Buy effort which I had started to easily get access to the latest VMware releases along with a nice 15% discount!

The script supports deploying both a standard vSphere 7 environment with just VCSA, ESXi and vSAN as well as the complete solution which includes NSX-T to support vSphere with Kubernetes. For more details, please refer to the FAQ.