vSphere 6.7

Update on ESXi on Apple Mac Mini 2018 & Mac Pro 2019

04/28/2020 by William Lam 35 Comments

Although there has not been any news in some time regarding the support for ESXi on the latest Apple Mac Mini 2018 and the recently released Apple Mac Pro 2019, there has definitely been work happening behind the scenes at VMware. Today, I would like to share a pretty significant update as a result of some of these efforts.

MacOS Guest

One of the biggest issue which I had observed when using a T2-based Apple system with ESXi is that it would fail to boot a MacOS Guest and just keep rebooting the VM. I am very happy to announce that this issue has been resolved and ESXi can now properly recognize the Apple System Management Controller (SMC) device which is used as part of the MacOS Guest start up process. This now means a MacOS Guest will be able to properly boot on a T2-based Apple system.

Thunderbolt 3

Another impact of a T2-based Apple system with ESXi is that storage and networking devices connected to the Thunderbolt 3 ports are not visible. I am also happy to announce that this issue has been resolved and ESXi can now see PCIe devices that are attached to the Thunderbolt 3 ports.

An ESXi Advanced Setting change is required for Thunderbolt 3 to work correctly and the following command will need to be executed after installing ESXi:

esxcli system settings kernel set -s pciExperimentalFlags -v 16

Once the setting has been applied, a system reboot will be required and your PCIe devices will show up properly. In future, this additional configuration may not be required and can be detected based on the underlying hardware.

Both of the fixes mentioned above are included in the latest ESXi 6.7 Patch 02 (ESXi670-202004002) release which is available today! Hopefully this was the news that many of you have been waiting for 😀

UPDATE (08/27/20) - The Apple 2018 Mac Mini 8,1 is now officially supported with ESXi 6.7 Update 3 which requires the latest ESXi 6.7 Patch 03 which also incorporates automatically setting the ESXi Advanced Setting for Thunderbolt 3 access.

UPDATE (06/25/20) - The Apple 2018 Mac Mini 8,1 is now officially on the VMware HCL and is fully supported with ESXi 7.0b, which contains the fixes mentioned above. See note below on 06/23 for more information.

UPDATE (06/23/20) - ESXi 7.0b has just been released and contains fixes for both the MacOS guest boot issue support for Thunderbolt 3 devices which now enables support for the vSphere 7 release. One additional enhancement, customers no longer need to configure the ESXi Advanced Setting to enable Thunderbolt 3 support, this is now automatically configured based on detecting an Apple hardware system such as an Apple Mac Mini 2018 or Apple Mac Pro 2019. This is a patch release and you will need to go to the VMware Patch Portal site to download and apply the update.

Now, before you rush out to start deploying MacOS Guests on either the Mac Mini or Mac Pro, I do have to mention that neither the Mac Mini 2018 or the Mac Pro 2019 will be officially supported by VMware. Due to the current situation that we are all in with COVID-19, personnel access to VMware facilities like many other organizations has been severely restricted and/or prohibited. In fact, much of the early validation was done by yours truly using a Mac Mini 2018 which I had access to (Thanks Michael Roy) as Engineering did not have access to hardware during the shelter in place orders. This also means that certifications of these platforms is still on-going and until these systems are officially listed on VMware's HCL, they will not be officially supported by VMware.

Disclaimer: VMware currently does not officially support the Apple 2019 Mac Pro7,1

Apple Mac Mini on VMware HCL!

08/01/2019 by William Lam 9 Comments

For the past 6 years, the Apple Mac Mini has been one of the most popular hardware platforms for Virtualizing MacOS running on VMware vSphere enabling our customers to develop and build iOS and MacOS applications. With that said, VMware has historically only supported two Apple hardware platforms: Xserve (now EOL'd) and the Mac Pro (6,1) which is officially listed on VMware's Hardware Compatibility list and this has been officially supported by VMware since 2012 when we first introduced support for MacOS Virtualization with the vSphere 5.0 release.

As many of you know, I have been a huge advocate of this platform for a number of years now and I have been working with various Engineers over the years to ensure that we have the exact same user experience when working with ESXi on the Mac Mini as you do with the Mac Pro. I still recall in the early days where it took several "hacks" to get ESXi to successfully boot and install.

Today, ESXi installs on the Mac Mini just like any other x86 platform. It runs amazing well for our customers, especially for a consumer device, who have deployed them in their datacenters ranging from a couple hundred to several thousands for some of our larger Enterprise customers, one such example is MacStadium, the largest Apple Infrastructure-as-a-service provider which many of the Fortune 100/500 companies are leveraging to provide them with a platform to build and develop for the Apple eco-system.

UPDATE (08/27/20) - Apple 2018 Mac Mini 8,1 has been added to VMware HCL which supports both ESXi 6.7 Update 3 (Patch 03) & ESXi 7.0b

Quick Tip – Crucial NVMe SSD not recognized by ESXi 6.7 & 7.0

05/19/2019 by William Lam 54 Comments

If you own or have recently purchased Crucial NVMe SSD such as CT1000P1SSD8 (1TB M.2 NVMe SSD) or CT500P1SSD8 (500GB M.2 NVMe SSD), please be aware that these devices may no be recognized by ESXi after upgrading to the latest release. Thanks to Pete Lindley, (OCTO for End-User Computing), who reached out last week regarding the observation as well as a workaround for the problem. This was also quite timely as I recently purchased a Crucial M.2 NVMe SSD and would have also ran into this problem.

It turns out these Crucial devices were working fine while running on ESXi 6.5 Update 2 but was no longer recognized in latest release of ESXi 6.7 Update 2. It is unclear whether support for these SSDs were removed intentionally or unintentionally, but in either case, these devices are not officially on VMware's Hardware Compatibility List (HCL).

UPDATE (07/29/20) - Over the past few months, I have had a number of folks share feedback that using the trick mentioned below for ESXi 7.0, they have had success of ESXi detecting their NVMe SSD. I wanted to share some of the model and/or vendors that folks have reported success with. I will keep this list updated, so feel free to leave a comment below.

OWC Aura Pro X2 2TB NVMe
ADATA XPG
Sabrent

UPDATE (06/13/20) - Thanks to reader Dave, it looks like this trick also works with ESXi 7.0 but the filename has changed. Simply copy nvme.v00 VIB from the ESXi 6.5 Update 2 and replace it on ESXi 7.0 system (either live under /bootbank or part of the installer) but rename the file to nvme_pci.v00 which is the new filename for NVMe driver.

UPDATE (05/23/19) - After speaking with a few folks who took a closer look, the issue is due to the fact that we added support for NVMe 1.3 spec in latest ESXi 6.7 Update 2 release, but because these are "consumer" devices, they did not conform to the latest specification and hence the driver is unable to claim the device. This is another good reminder when using components not on VMware HCL, this is always a risk from a home lab perspective. In general, I know Samsung and Intel NVMe SSD usually works quite well without issues but always good to do some research. I think Engineering is looking to see if there are other workarounds for the future, but for now, you can use the workaround below.

The easy workaround that Pete found was to simply replace the NVMe driver from ESXi 6.7 Update 2 (1.2.2.27-1vmw.670.2.48.13006603) with one found in ESXi 6.5 Update 2 (1.2.1.34-1vmw.650.2.50.8294253). To so do, simply copy nvme.v00 to /bootbank from either an existing ESXi 6.5 Update 2 system or directly from the ISO. Please note, any future updates or patches to the ESXi host will most likely override the updated driver.

vSphere 6.7 Update 2 add support back for Apple Mac Pro 6,1

04/11/2019 by William Lam Leave a Comment

Last fall, support for vSphere 6.7 Update 1 on the Apple Mac Pro 6,1 was removed by VMware after finding out an updated version of the Apple Firmware prevented ESXi from booting up properly after upgrading. After some investigation and troubleshooting with several customers, VMware Engineering found a solution to workaround the problem and that fix is included in the latest vSphere 6.7 Update 2 release. As mentioned in the original blog post, this could also impact Apple Mac Mini's, which are not officially supported by VMware, but the fix should also apply to the Mac Mini.

In addition, the VMware HCL will also be updated shortly to reflect support for both vSphere 6.7 Update 2 along with the existing vSphere 6.7 support. For customers that have been waiting for features in vSphere 6.7 Update 1, you can finally upgrade to the latest release to get all the new features and benefits of vSphere.

Instant Clone Apple MacOS

03/28/2019 by William Lam 1 Comment

Whether you are a brand new startup working on the next hot mobile app or an established Enterprise or Consumer brand company, development and testing of Apple iOS and/or MacOS is simply a reality in todays world. The vast majority of these customers accomplish this by running Apple MacOS on vSphere, either within their own on-premises datacenter or leveraging MacStadium, the largest MacOS Cloud hosting provider, who also runs their Mac infrastructure using VMware vSphere.

The ability to quickly build/test and deploy your application (Continuous Integration and Continuous Delivery) can mean the difference of having an edge over you competitor or being able to keep up with the demands of your business. Many customers have benefited from using vSphere platform and with technologies like Linked Clones, which allows you to quickly spin up a new VM without having to perform a complete full clone, it means you can build and test your application even faster.

In vSphere 6.7, we introduced a major enhancement to Instant Clone, which you can read more about here and here. One of the questions I have been seeing lately is whether Instant Clone can be applied to MacOS guests? The answer is absolutely! In fact, Matt Moriarity, who works for TravisCI, recently shared some tidbits on how to get a MacOS Mojave guests to see the updated MAC Address to ensure that there are no network conflicts when performing an Instant Clone.

The majority of the "hard" work to use Instant Clone is really from within the GuestOS and the customization script that needs to be developed. In fact, Instant Clone is pretty OS agnostics and you can even Instant Clone Microsoft Windows 98 and 2000, if you really wanted to 😀

Dynamic vSphere Health Checks in vSphere 6.7+

01/22/2019 by William Lam 12 Comments

One really neat feature of the vSphere HTML5 Client that was shipped in vSphere 6.7 is the ability to deliver new data applications that can run in the vSphere UI without requiring customers to update or upgrade their underlying vCenter Server. An example of this is the vSphere Health Check plugin that was included in vSphere 6.7, which I am guessing most folks probably did not even notice, including myself. The vSphere Health plugin is located at the vCenter Server level and under Monitor->Health as shown in the screenshot below.

Unlike a traditional vSphere Plugin, where the code and business logic is local to the vCenter Server and must be updated each time for new functionality, these data applications are actually delivered automatically and more importantly, out-of-band to a vCenter Server patch or upgrade. This means as new functionality is added, customers will automatically get the latest updates without having to do anything. So how does this actually work?