virtuallyGhetto

Nested Virtualization

How to simulate Persistent Memory (PMem) in vSphere 6.7 for educational purposes? 

by 2 Comments

A really cool new capability that was introduced in vSphere 6.7 is the support for the extremely fast memory technology known as non-volatile memory (NVM), also known as persistent memory (PMem). Customers can now benefit from the high data transfer rate of volatile memory with the persistence and resiliency of traditional storage. As of this blog post, both Dell and HP have Persistent Memory support and you can see the list of supported devices and systems here and here.


PMem can be consumed in one of two methods:

  • Virtual PMem (vPMem) - In this mode, the GuestOS is actually PMem-aware and can consume the physical PMem device on the ESXi host as standard, byte-addressable memory. In addition to using an OS that supports PMem, you will also need to ensure that the VM is running the latest Virtual Hardware 14
  • Virtual PMem Disks (vPMemDisk) - In this mode, the GuestOS is NOT PMem-aware and does not have access to the physical PMem device. Instead, a new virtual PMem hard disk can be created and attached to a VM. To ensure the PMem hard disk is placed on the PMem Datastore as part of this workflow, a new PMem VM Storage Policy will be applied automatically. There are no additional GuestOS or VM Virtual Hardware requirement for this scenario, this is great for legacy OS that are not PMem-aware

Customers who may want to familiarize themselves with these new PMem workflows, especially for Automation or educational purposes, could definitely benefit from the ability to simulate PMem in their vSphere environment prior to obtaining a physical PMem device. Fortunately, this is something you can actually do if you have some additional spare memory from your physical ESXi host.

Disclaimer: This is not officially supported by VMware. Unlike a real physical PMem device where your data will be persisted upon a reboot, the simulated method will NOT persist your data. Please use this at your own risk and do not place important or critical VMs using this method.

[Read more...] about How to simulate Persistent Memory (PMem) in vSphere 6.7 for educational purposes? 

Native MAC Learning in vSphere 6.7 removes the need for Promiscuous mode for Nested ESXi

by 7 Comments

Over the years, several solutions have been developed here and here to help reduce the impact of promiscuous mode, which is a requirement for running Nested ESXi as a workload. Although these solutions worked extremely well, it however did require users to install additional software to enable this functionality. The most recent solution was a new Learnswitch VMkernel module (released as a VMware Fling) that enables MAC learning capabilities on ESXi.

Today, I am pleased to announce that with the release of vSphere 6.7, the MAC Learning functionality is now available as a native feature of the VMware Distributed Virtual Switch (VDS) and as some of you may have guessed from the title, promiscuous mode is also no longer a requirement for running Nested ESXi! I wanted to take a moment and thank Subin, Jobin, Sriram, Rajeev & Samuel from our Network and Security Business Unit (NSBU) at VMware who worked tirelessly to get this integrated and productized into ESXi. Not only will this benefit Nested ESXi workloads but also other solutions and use cases that have historically required the use of promiscuous mode. For customers who are still running ESXi 6.0 or 6.5, you should continue to use the Learnswitch Fling until you fully upgrade to vSphere 6.7.

To use the new MAC Learning functionality, you will of course need to upgrade to vSphere 6.7 (both vCenter and ESXi) but also upgrade to the latest VDS version which is 6.6. MAC Learning can be enabled on a per Distributed Virtual Portgroup bases and today, it is only available when using the vSphere API. For those that have used the VDS API to manage their VDS, you will simply use the existing ReconfigureDVPortgroup_Task() method and in 6.7, there now a new macManagementPolicy property which allows you to enable and define your MAC Learning settings. This new MAC Management Policy will also be the new preferred method for managing security policies going forward for a DV Portgroup and the previous security policy settings should no longer be used.

Disclaimer: Nested ESXi is still not officially supported by VMware. Please use at your own risk.  [Read more...] about Native MAC Learning in vSphere 6.7 removes the need for Promiscuous mode for Nested ESXi

Nested ESXi 6.7 Virtual Appliance Updates

by 4 Comments

I know many of you have been pinging me the last couple of days for an updated Nested ESXi 6.7 Virtual Appliance and I have just finished my strict quality control process 🙂 The only minor change with the 6.7 appliance is the VM is now configured with EFI Firmware, where as in the past it was set to BIOS. As of vSphere 6.5+ appliances, the customization scripts are automatically removed by default which means that customers can turn on Secure Boot feature post-deployment without having to perform any manual workarounds. In addition, you will find a few more updates related to the updated ESXi appliance below. I hope you enjoy these free resources to help learn and plan for your vSphere 6.7 upgrades, Happy Friday!

Note: These solutions are all developed during off hours and does take a considerable amount of time/effort to manage and update. Although they are provided to you as a free solution, the development itself is not 🙂

o

Nested ESXi 6.7 Appliance:

ESXi 6.7 Virtual Appliance (Nested_ESXi6.7_Appliance_Template_v1.ova)

Nested ESXi Content Library

If you are using my Nested ESXi Content Library, I have updated it to include the latest 6.7 Appliance. Simply refresh your Content Library to automatically pull down the image or you can create a new Content Library by subscribing to the following URL: https://download3.vmware.com/software/vmw-tools/lib.json For more details, please take a look at this blog post here.

vGhetto vSphere Automated Lab Deployment:

For those that use my vGhetto lab deployment script to automate a fully functional vSphere environment, I have created a new version of the script to support vSphere 6.7 which you can find more details here. One neat feature that was suggested by Christian Mohn awhile back was the ability to get more insights to what is happening during the VCSA deployment since the verbosity can be quite distracting on the primary screen. There is now a new $enableVerboseLoggingToNewShell variable that is enabled by default to spawn a new PowerShell console that will watch the VCSA installer logs, so you have a better idea of what is going on.

New Nested ESXi 6.x Content Library 

by 1 Comment

A few years back I had showed how you could create and host your own 3rd Party vSphere Content Library which allows customers to decouple their content from the underlying vSphere environment and centralizing their content and making it available to number of vCenter Servers by simply just having an HTTP(s) endpoint. The other huge benefit is being able to take advantage of the existing web content tools for optimizing delivery or retrieval whether that is replication, caching, etc. and not relying a single vCenter Server for providing Content Library publication. In addition to showing how to create your own content libraries, I also had built my own 3rd Party vSphere Content Library which contains a variety of my Nested ESXi Templates (empty VM shells) running on Amazon S3 which can be consumed by anyone as long as you are running vCenter Server 6.0 or newer.

Although the empty Nested ESXi Templates were quite useful for myself and customers, it would have also been nice to include my pre-built Nested ESXi Virtual Appliances which I had recently updated to support vSphere 6.0 Update 3 and vSphere 6.5d (vSAN 6.6). Thanks to Dana Nourie, who runs our wildly popular VMware Flings Program, was kind enough to help me with the content hosting and now anyone can also subscribe to my Nested ESXi VA's and automatically have the content sync down using the vSphere Content Library feature.

UPDATE 1 (07/31/17) - The Nested ESXi Content Library has been updated to include the latest ESXi 6.5 Update 1 VA. If you are already subscribing to the library, it should have already pulled down the content (or at least the metadata which you can then force synchronization) or you can simply subscribe to the library and have access to the latest ESXi VA.

UPDATE 2 (05/07/18) - The Nested ESXi Content Library has been updated to include the latest ESXi 6.5 Update 2 VA. If you are already subscribing to the library, it should have already pulled down the content (or at least the metadata which you can then force synchronization) or you can simply subscribe to the library and have access to the latest ESXi VA.

To get started, just create a new vSphere Content Library and enter the following subscription URL: https://download3.vmware.com/software/vmw-tools/lib.json 


You can either download the content immediately or only when you need to use it. I recommend the former since its only two images which totals up to a whopping 1GB 😉

Once the creation of the Content Library has been completed, you should see the following two Nested ESXi VAs in the library which are now ready for deployment!


For more information about the Nested ESXi 6.0u3/6.5d VA's and how they work, please have a look at this blog post here. For more information about the Nested ESXi Templates and how to subscribe to the 3rd Party vSphere Content Library, please have a look at this blog post here.

Updated Nested ESXi 6.0u3 & 6.5d Virtual Appliances

by 23 Comments

I finally found a bit of "extra" spare time to update my Nested ESXi Virtual Appliances to support some of the recent releases of ESXi, 6.0 Update 3 and 6.5d, which enables customers to easily and quickly deploy vSAN 6.6 in their environment for testing, development or learning purposes. If you have not used this appliance before, please have a look at this article which goes into greater detail on how to deploy and use the Nested ESXi VA.

As part of this update, I also spent some time looking at all the feedback that I had received from the community since releasing the VA and I took this opportunity to also add some nice enhancements that folks have been asking about 🙂 Jump towards the bottom to see what's new. To reduce the number of VA's that I need to manage and due to usage, the following VA's have recently been decommissioned. I only plan on supporting the latest versions which you can find in the links below.

Decommissioned VA's:

  • ESXi 5.5 Update 3 (Nested_ESXi5.x_Appliance_Template_v2.ova)
  • ESXi 6.0 Update 2 (Nested_ESXi6.x_Appliance_Template_v5.ova)
  • ESXi 6.5 GA (Nested_ESXi6.5_Appliance_Template_v1.ova)

New VA's:

What's New:

  • Support for DHCP 
    • I know this might sound pretty basic but before you were required to specify a static IP (even if you had DHCP). By default, you no longer need to fill out the networking section as highlighted in yellow below.
  • Support for default root password
    • You no longer need to provide root password, it will default to the famous VMware1! The issue in the past was that I had randomly generated a password which I discarded and when the customization failed, it was very difficult to troubleshoot since I do not actually have the password 😉 Hopefully we do not have any other bugs, but this will make debugging easier and also reduce the amount of input if you want to quickly spin up an ESXi instance.
  • Support for VLAN ID
    • Though not a huge number of requests, there were still of you who asked for 802.1q (trunk) support on Management VMkernel interface. This is an optional field and obviously this is only applicable if you provide a static IP Address.
  • Automatic removal of Customization VIB
    • As some of you may or may not know, the way in which these OVF properties are processed within the Nested ESXi instance is a special firstboot script which reads in these values and then applies the ESXi customization. If everything is successful, there really is no use for this to exists further and although you could set a certain advanced setting to force re-customization, it was quicker to just re-deploy. With that in mind, the customization VIB is now automatically removed once its done its job. I have included a special debug option that would allow it to not be deleted in scenarios where there are issues and we need to take a look at the state of the system. With this change, you really now have a "vanilla" ESXi instance 🙂
  • Fixed dvFilter param for eth1


Hope you enjoy some of these new updates and happy Nesting!

ESXi Learnswitch – Enhancement to the ESXi MAC Learn DvFilter

by 13 Comments

The ESXi MAC Learn dvFilter Fling was released a little over two years ago and it has become a must have when it comes to running our ESXi Hypervisor within a VM, also referred to as Nested ESXi. The reason this Fling has become such a popular hit amongst our customers and partners is that it greatly improves the performance when “Promiscuous Mode” is enabled on a Virtual or Distributed Virtual Portgroup, which is a requirement for using Nested ESXi. Although this Fling works great, there are a couple of limitations with this solution today. The first of which is called out in the original Fling release notes, that once a MAC Address has been learned, it never ages out which is not ideal for long running Nested ESXi environments that generates a large amount of new MAC Addresses. The second, is the lack of vMotion support where the learned MAC Address table is not transfered to the destination ESXi host and must be re-learned.

To help address both of these limitations, the folks over in the Network and Security Business Unit (NSBU) have been working hard to improve upon the existing solution and have developed a new native MAC Learning VMkernel module called the Learnswitch. This new Learnswitch not only helps improves Nested ESXi workloads but it can also potentially benefit other workloads such as Nested Containers or other 3rd Party network inspection software. One immediate difference from the previous MAC Learn dvFilter solution is that rather than operating on the Network IO Chain, the filtering is now performed within the outer virtual switch layer itself which will provide some additional performance gains. The other added benefit from an internal VMware standpoint is that the Learnswitch is now vmkapi compatible, which means we will have a better backwards compatible story for supporting old releases of ESXi. One downside to this new solution compared to the previous one is that because the dvFilter operated below the virtual switch layer, it could support both a Virtual Standard Switch as well as the Distributed Virtual Switch. With the new Learnswitch, a Distributed Virtual Switch will be required. If you currently do not meet the requirements of the new Learnswitch, you can continue using the dvFilter, but it is recommended that you do not mix both on a single system but you can definitely make use of both solutions across different ESXi hosts depending on the constraints of your environment.

Here are some of the new capabilities provided by the new Learnswitch module:

  • Overlay Network based that learning and filtering are done in Etherswitch forwarding check
  • MAC Address learning is based on VLAN ID or VXLAN ID on uplink and leaf port
  • Packet is filtered on uplink and leaf port if the MAC is learned on a different port
  • MAC Address table size is 32k per system
  • MAC Address aging support with default aging time of 5 minutes and configurable
  • Unknown unicast packet is flooded by default and configurable to drop
  • vMotion support that the MAC table learned on the port is transferred to destination host and RARP packet is sent
  • Standalone VMkernel module available as a VIB
  • net-learnswitch CLI to display MAC Address table, configuration and stats

[Read more...] about ESXi Learnswitch – Enhancement to the ESXi MAC Learn DvFilter