Over the years, several solutions have been developed here and here to help reduce the impact of promiscuous mode, which is a requirement for running Nested ESXi as a workload. Although these solutions worked extremely well, it however did require users to install additional software to enable this functionality. The most recent solution was a new Learnswitch VMkernel module (released as a VMware Fling) that enables MAC learning capabilities on ESXi.

Today, I am pleased to announce that with the release of vSphere 6.7, the MAC Learning functionality is now available as a native feature of the VMware Distributed Virtual Switch (VDS) and as some of you may have guessed from the title, promiscuous mode is also no longer a requirement for running Nested ESXi! I wanted to take a moment and thank Subin, Jobin, Sriram, Rajeev & Samuel from our Network and Security Business Unit (NSBU) at VMware who worked tirelessly to get this integrated and productized into ESXi. Not only will this benefit Nested ESXi workloads but also other solutions and use cases that have historically required the use of promiscuous mode. For customers who are still running ESXi 6.0 or 6.5, you should continue to use the Learnswitch Fling until you fully upgrade to vSphere 6.7.

To use the new MAC Learning functionality, you will of course need to upgrade to vSphere 6.7 (both vCenter and ESXi) but also upgrade to the latest VDS version which is 6.6. MAC Learning can be enabled on a per Distributed Virtual Portgroup bases and today, it is only available when using the vSphere API. For those that have used the VDS API to manage their VDS, you will simply use the existing ReconfigureDVPortgroup_Task() method and in 6.7, there now a new macManagementPolicy property which allows you to enable and define your MAC Learning settings. This new MAC Management Policy will also be the new preferred method for managing security policies going forward for a DV Portgroup and the previous security policy settings should no longer be used.

Disclaimer: Nested ESXi is still not officially supported by VMware. Please use at your own risk. 

To demonstrate the new MAC Learning APIs, I have created two small PowerCLI functions called Get-MacLearn and Set-MacLearn which you can download from here. You will need to make sure to download the latest PowerCLI 10.1.0 release which adds support for vSphere 6.7

The Get-MacLearn function can be used to retrieve the current MAC Learning configuration for a given DV Portgroup, simple run the following command which can accept a one or more DV Portgroup names:

Get-MacLearn -DVPortgroupName @("Nested-01-DVPG")

As we can see from the output, I currently do not have MAC Learning enabled on this DV Portgroup. We can also see new properties such the limit which defines maximum number of MAC Addresses that can be learned (4096 max) and limitPolicy which defines the switching policy (drop or accept) when exceeding the learned MAC Address limit. As mentioned earlier, the new Mac Management interface should be used to manage security policies and as part of the output, I have also include both the new and legacy security policy settings.

The Set-MacLearn function can be used to enable MAC Learning as well as specifying the security policies for a given DV Portgroup. For Nested ESXi usage, you will want to set the following:

  • MAC Learning: true
  • Promiscuous mode: False
  • Forged Transmit: True
  • MAC Changes: False
  • Limit: 4096 (optional, default is provided in the function)
  • Limit Policy: Drop (optional, default is provided in function)

Set-MacLearn -DVPortgroupName @("Nested-01-DVPG") -EnableMacLearn $true -EnablePromiscuous $false -EnableForgedTransmit $true -EnableMacChange $false

Once the reconfiguration has completed, we can re-run the Get-MacLearn function to confirm our changes as shown in the screenshot below:

At this point, you are now ready to start deploying your Nested ESXi workloads to this DV Portgroup or if you performed this operation on one of your existing DVPortgroup, you have now disabled the need for promiscuous mode!

Lastly, I wanted to share one additional tool that can be useful get more information about the current learned MAC Addresses which is only available directly on the ESXi Shell. The utility is called netdebg and below are a few examples on how to use it.

Note: Please note, this tool is meant for debugging purposes and there are no guarantees this will continue to work the same way in future releases.

To list all switches both VSS and VDS, run the following command:

netdbg vswitch instance list

To check whether a given powered on VM's DV Port has MAC Learning enabled, you can run the following and specify the DVPortID as well as the name of your VDS (which you need to use esxcfg-vswitch -l or esxcli network vswitch dvs vmware list to retrieve):

netdbg vswitch mac-learning port get -p 10 --dvs-alias VDS 6.7

To retrieve all learned MAC Addresses on a given DV Port,you can run the following and specify the DVPortID as well as the name of your VDS (which you need to use esxcfg-vswitch -l or esxcli network vswitch dvs vmware list to retrieve):

netdbg vswitch mac-table port get -p 10 --dvs-alias VDS 6.7

In the screenshot above, the first address (d5:d6) is actually a VM running on top of my Nested ESXi VM and the second address (5c:98) is my Nested ESXi VM's vmnic0. MAC Address entries will age out automatically between 10-20 minutes and no additional steps are required to clear out old learned entries.

5 thoughts on “Native MAC Learning in vSphere 6.7 removes the need for Promiscuous mode for Nested ESXi

  1. Thanks for the update William – this is great news. Looking forward to trying it out in my lab.

    Do you know how MAC moves will be handled? If a guest VM VMotions from one nested ESXi to another, will the learned MAC be moved to the new port or will it exist in both places until the original one times out.

    • James,

      Yes, here’s what I received back from Engineering:

      1) If the move is from one nested ESX VM to another on the same host, the mac will move from older vNIC port to the new vNIC port based on packets coming from new nested ESX VM.

      2) If the move is from one nested ESX VM to another on a different host, the mac will move from vNIC port to uplink port on source outer ESX host based on packets coming from new nested ESX VM. It will also get learned on the vNIC of the destination outer ESX host.

      A learned MAC (i.e. MAC address + VLAN) won’t exist in two ports at the same time for the same host switch.

  2. Thanks William, Does the underlying Physical host need to be 6.7 as well as the Nested Lab or is it just the Nested Lab, If I leave my Underlying physical host as 6.5 can I make use of MacLearn ?

    • Just like past solutions, the settings are ONLY applied to the physical ESXi host 🙂 If you’re still on 6.5, then you’ll need to continue using the MAC Learn Fling

Thanks for the comment!

This site uses Akismet to reduce spam. Learn how your comment data is processed.