esxi

Aquantia/Marvell AQtion (Atlantic) driver now inbox in ESXi 7.0 Update 2

03/11/2021 by William Lam 3 Comments

Last spring, VMware and Aquantia (now part of Marvell) collaborated and delivered their first ESXi Native Driver for their AQtion (Atlantic) based 10GbE network adapters. This new driver was primarily focused on enabling network connectivity for ESXi when running on either an Apple 2018 Mac Mini (8,1) and Apple 2019 Mac Pro (7,1) that included the 10GbE networking option. Consequently, this driver also benefited the broader VMware Community as it enabled additional 10GbE networking through a number of Thunderbolt 3 to 10GbE network adapters that customers could now take advantage in their VMware environments.

With all these benefits, VMware has decided to inbox the Aquantia/Marvell driver with the latest ESXi 7.0 Update 2 release, so that customers no longer had to create a custom ESXi Image Profile that included the driver, which was always required when installing ESXi on either the Apple Mac Mini or Mac Pro that were configured with the 10GbE networking option. For a complete list of supported Aquantia/Marvell AQtion based network adapters, please see the VMware HCL.

Here is a screenshot of an earlier release of ESXi 7.0 Update 2 running on the 2018 Mac Mini which now automatically recognizes the 10GbE network adapter out of the box.

VMware customer production use cases for Intel NUC

02/19/2021 by William Lam 3 Comments

The Intel NUC also known as the Next Unit of Computing is a very popular platform for running VMware based homelabs. I have been working with the Intel NUCs since 2016 with their 6th Generation model when I decided to rebuild my personal home lab. Since then I have continued my efforts to ensure that vSphere continues to run extremely well on this amazing little platform even if it is not officially supported by VMware, which now also includes the latest 11th Generation (Tiger and Panther Canyon NUCs).

At the end of last year, I came across this fascinating Intel NUC documentary that was put together by Robtech, which I highly recommend a watch.

While listening to some of the use cases that SimplyNUC had observed over the years which has spanned land ⛰️, air 🛫, sea 🛳️ and space 🚀, it got me thinking about some of the use cases that I had come across while talking to our VMware customers.

Disclaimer: The Intel NUC is not officially supported by VMware and therefore they are not listed on the VMware HCL

A common misconception is that Intel NUCs are only useful for homelab purposes and has no place for running production workloads, which is just simply not true. Here are some of the common use cases that I have seen over the years, most of which are deployed at the Edge/ROBO:

vSphere Development/Testing, Education and Training
Retail, Grocery, Industrial Factories and Ships
Build Automation (CI/CD)
Telco/NFV (e.g. Network/Hardware monitoring)
Virtual Desktop Infrastructure (VDI)

I also wanted to take this opportunity and to share some of the stories on how some of our customers have taken advantage of this platform, even though it is not officially supported by VMware and some of the underlying business drivers. Hopefully these stories will educate, resonate and perhaps even inspire other customers to explore different computing platforms, especially at the Edge where constraints and requirements will differ quite significantly when compared to a typical Enterprise Datacenter.

If you would like to share your story of how you are using Intel NUC and VMware for production, feel free to reach out using the contact page.

Quick Tip – How to use Apple Thunderbolt 2 ethernet adapter with ESXi 7.0 or greater

11/13/2020 by William Lam 6 Comments

I was doing some testing on my Apple 2018 Mac Mini with the latest ESXi 7.0 Update 1 release and I needed to setup a separate network connection as the onboard 10GbE was not working for me initially. I was out of ideas but I did remember that I still have my Apple Thunderbolt 2 to gigabit ethernet adapter which was something I had used quite a bit in the early days when I was using the Apple Mac Mini as my homelab system.

Like all recent Apple Mac's, the 2018 Mac Mini only supports Thunderbolt 3 (USB-C) ports and obviously not compatibility with the network adapter. Luckily, I did have an official Apple Thunderbolt 3 to Thunderbolt 2 adapter lying around which would allow me to connect the network adapter to the Mac Mini and to my surprise, it was automatically detected by the latest release of ESXi!

This partially came in a surprise because the Apple network adapter uses the Broadcom tg3 driver and I was not 100% sure if the native Broadcom (ntg3) would automatically claim this device since it was never officially supported.

Its definitely good to know this ethernet adapter still works as long as you have a TB2 to TB3 converter adapter and this should also work for any Intel NUC that have Thunderbolt 3 ports.

Stateless ESXi-Arm with Raspberry Pi

11/03/2020 by William Lam 18 Comments

I am super excited to be able to finally share, what I think, is a really cool ESXi-Arm solution which has been an evolution of this and this. This solution also incorporates a number of automation techniques I have shared over the years when it comes to ESXi scripted installation aka Kickstart, so it was really neat to all those things get pulled into a single solution. Lastly, I also want to give huge thanks to Cyprien Laplace who threw the initial challenge my way after I had shared how to perform an ESXi-Arm scripted installation without using SD Card.

ESXi-x86 can be deployed using either a stateful or stateless installation. In the latter case, ESXi is booted over the network using the vSphere Auto Deploy feature in vCenter Server which does not require any local media for ESXi. Upon attaching itself to vCenter Server, Auto Deploy then leverages vSphere Host Profiles and its rules engine to determine which configurations or profiles should be applied to ensure the ESXi hosts are configured per their desired stated. Here is a quick video overview of how Auto Deploy and Host Profiles work.

Fundamentally, vSphere Auto Deploy and Host Profiles can also work with ESXi-Arm but today, vCenter Server would require some code modification for this to actually work.

OK, so am I teasing you with something that does not exists? Nope, but I just wanted to help set the context 🙂

The solution that I have created boots ESXi-Arm over the network in a "stateless" manner, so there is no need for an SD Card or USB device plugged into the Raspberry Pi (rPI). In addition to the ESXi-Arm files, it also includes a custom payload which runs to retrieve additional configurations which can automatically join a desired vCenter Server as well as apply further customizations of an ESXi-Arm host. As you can see, this solution behaves similar to that of vSphere Auto Deploy and Host Profiles but does not use either of these vSphere features and works with the ESXi-Arm Fling right now.

Technically speaking, these techniques can also be applied to ESXi-x86 but I will leave that to the reader for further exploration.

Kubernetes on ESXi-Arm using k3s

10/16/2020 by William Lam 11 Comments

The tiny form factor of a Raspberry Pi (rPI) is a fantastic hardware platform to start playing with the ESXi-Arm Fling. You can already do a bunch of fun VMware things like running a lightweight vSAN Witness Node to setting up basic automation environment for PowerCLI, Terraform and Packer to running rPI OS as VM, enabling some neat use cases like consolidating your physical rPI assets which might be running RetroPi and Pi-Hole which many home labbers are doing.

In addition to VMware solutions, its is also a great platform to learn and tinker with new technologies like Kubernetes (K8s) which I am sure many of you have been hearing about 🙂 Although our vSphere with Tanzu and Tanzu Kubernetes Grid (TKG) does not currently work with the ESXi-Arm Fling, I have actually been meaning to try out a super lightweight K8s distribution designed for IoT/Edge called k3s (pronounced k-3-s) which also recently joined Cloud Native Computing Foundation (CNCF) Sandbox level.

k3s is supported on rPI and you normally would have multiple rPI devices to represent the number of nodes, for example if you want a basic 3-Node cluster, you would need three physical rPI devices. With ESXi-Arm, you can now create these nodes as VM, using just a single rPI. This opens up the door for all sorts of explorations, you can create HA cluster or try out more advanced features which might be more difficult if you needed several physical devices. If you mess up, you can simply re-deploy the VM without much pain or simply clone the VM.

In my setup, I am using 3 x Photon OS VMs. One for the primary node and two for k3s worker nodes. You can certainly install k3s on any other Arm-based OS including rPI OS (which can now run as a VM as mentioned earlier).

How to copy the Raspberry Pi UEFI files and boot ESXi-Arm using just USB device?

10/14/2020 by William Lam 2 Comments

There are actually a number of ways to boot and install the ESXi-Arm Fling, but the easiest method is already outlined in the official ESXi-Arm Raspberry Pi (rPI) documentation and the PDF can be downloaded from the Fling website. As a quick refresher, you only need to have two storage devices.

SD Card - Used to store the rPI UEFI files which is required to boot ESXi-Arm Installer
- ESXi-Arm can not run on SD Card and for these reasons, you do not need a large capacity SD Card
USB Device - Contains ESXi-Arm Installer / Installation
- After the ESXi-Arm installer boots, you can actually re-use the exact same USB device for the installation of ESXi-Arm itself. A separate USB device is not required unless that is your goal or if the capacity is not enough for running VMs

The boot process after the ESXi-Arm installation is that the UEFI firmware will first load on the rPI and then it will boot up the ESXi-Arm from the USB device. As mentioned, there are other variations but this is the most basic option. The other nice behavior is that if you need to re-install ESXi-Arm, you simply create a bootable USB device with the ESXi-Arm installer and then install that right on the same USB device without having to mess with UEFI image. This also allows you to perform scripted installation also known as Kickstart, which is something I will be covering in the future that takes UEFI image into consideration.

I have seen a few questions asked whether it is possible to have everything run off of the SD Card and/or USB Device and the answer is yes to certain degree.

It is possible to put the ESXi-Arm installer + UEFI on SD Card but ESXi-Arm will NOT be able to use it as installation media, so there is not a whole ton of value there.
It is possible to have both the UEFI image and ESXi Installation on the same USB device, especially if you do not have spare SD Cards which apparently has come up a few times

In this blog post, I will outline the instructions for booting an installed ESXi-Arm installation completely off of the USB device without the needing an SD Card containing the UEFI image.