While attending an offsite this week, there were some discussions amongst my colleagues about their new Apple Mac Pro and its USB-C only ports. The discussion was completely unrelated to work, however that did get me thinking about the USB-C peripheral market and specifically their ethernet adapters. While searching online, I came across several new USB-C to gigabit ethernet adapters that were now available and one in particular that was very interesting, was the Plugable USB-C to 10/100/1000 Gigabit Ethernet LAN Network Adapter. What caught my eye about this specific network adapter was that it uses the exact same ASIX AX88179 driver as my USB 3.0 to Ethernet Adapter ESXi VIBs were built off of! There was a good chance this might just work.
As you can probably guess, I was pretty excited and quickly ordered one of the Plugable USB-C Ethernet Adapters. The next challenge was getting access to a system that has a USB-C port. After asking around, I finally got my hands on a Dell XPS 13 which has a USB-C port that I could use for a few days. Funny enough, the Dell laptop only has USB 3.0 and USB-C ports, so the first challenge was to disable Secure Boot since I had built a custom ESXi 6.5 image that included my USB 3.0 Ethernet Adapter VIB. Below are the ESXi VIBs or offline bundles that will be required for this solution.
- ESXi 5.5 Update 3 USB Ethernet Adapter Driver VIB or ESXi 5.5 Update 3 USB Ethernet Adapter Driver Offline Bundle
- ESXi 6.0 Update 2 USB Ethernet Adapter Driver VIB or ESXi 6.0 Update 2 USB Ethernet Adapter Driver Offline Bundle
- ESXi 6.5 USB Ethernet Adapter Driver VIB or ESXi 6.5 USB Ethernet Adapter Driver Offline Bundle
Please see this blog post for more detailed instructions on installing the VIB as well as accessing the vusbX pNIC.
Disclaimer: This is not officially supported by VMware. Use at your own risk.
Once I got ESXi up and running, I was disappointed to see that the USB-C device was not being detected. I had tried a few more things but nothing worked and I decided to sleep on it. The next morning, I realize maybe there was some additional settings that needed to be tweaked in the BIOS. With a bit of trial/error, I found out that you needed to enable the "Thunderbolt Boot Support" which apparently is disabled by default, at least on this Dell system. Below is a screenshot of the BIOS USB/Thunderbolt Settings and this was the only change required from the system defaults.
Running the lspci command after the BIOS tweak, I can now see both the Thunderbolt 3 bridge as well as the USB 3.1 controller, where as before it was just showing the TB3 bridge.
You can also verify that the USB NIC has been claimed by running the lsusb command. In my setup, I have both the USB 3.0 to Ethernet Adapter as well as the USB-C to Ethernet Adapter functioning on my ESXi 6.5 host.
I ran a couple of iPerf experiments between two ESXi hosts and the speeds are pretty decent, see the results below.
iPerf benchmark for Ingress traffic (single port USB ethernet adapter):
Now, how freaking cool is this!? Best of all, this solution should work on ANY system that has a USB-C / Thunderbolt 3 port as long as you are using the driver along with a supported ethernet adapter. For other systems, it may just work out of the box or you may be required to make similiar changes to the BIOS. If you are successful in getting this to work on other systems, please leave a comment sharing the details and any workarounds that may be needed so others may benefit. Some other popular platforms that are currently shipping with USB-C ports are the Intel NUC Skull Canyon or the new Apple Macbook Pro. The upcoming Intel NUC i5 Kaby Lake will also include USB-C ports similiar to the Skull Canyons, so if you are considering purchasing a new system, you may want to wait until those are available so you can take advantage of the USB-C ports when running ESXi.
Given the large bandwidth of a USB-C, I suspect you *might* even be able to drive 2x more the bandwidth by using a Dual Gigabit Ethernet Adapter like the USB-C to Dual Gigabit Ethernet Adapter from StarTech which also uses the ASIX AX88179 driver. I personally would love to see a USB-C to 10GbE adapter where you can then use a cross-over cable to connect two devices for a sweet 2-Node vSAN Cluster. As of right now, the only 10GbE solution in the market is from ATTO with their ThunderLink device (Thunderbolt 3 to 10GbE SFP+). This would require an additional 10GbE switch, which is probably out of reach for most folks including myself. Alternatively, which I suspect is probably not possible without changes to ESXi, is to get Thunderbolt Networking which allows you to connect two hosts together using TB3 to TB3 cable like you can today on Mac OS X. This would theoretically give you 40Gbps of bandwidth, which should be more than enough for anyone wanting to run a small vSAN Cluster 🙂 There seems to be some good progress on this front with the Thunderbolt Networking support for Linux project, but it is still very early to tell. Perhaps Plugable, StarTech or other vendors might consider looking into this for the future given USB-C potential?