William Lam

Quick Tip – Crucial NVMe SSD not recognized by ESXi 6.7

05/19/2019 by William Lam 13 Comments

If you own or have recently purchased Crucial NVMe SSD such as CT1000P1SSD8 (1TB M.2 NVMe SSD) or CT500P1SSD8 (500GB M.2 NVMe SSD), please be aware that these devices may no be recognized by ESXi after upgrading to the latest release. Thanks to Pete Lindley, (OCTO for End-User Computing), who reached out last week regarding the observation as well as a workaround for the problem. This was also quite timely as I recently purchased a Crucial M.2 NVMe SSD and would have also ran into this problem.

It turns out these Crucial devices were working fine while running on ESXi 6.5 Update 2 but was no longer recognized in latest release of ESXi 6.7 Update 2. It is unclear whether support for these SSDs were removed intentionally or unintentionally, but in either case, these devices are not officially on VMware's Hardware Compatibility List (HCL).

UPDATE (05/23/19) - After speaking with a few folks who took a closer look, the issue is due to the fact that we added support for NVMe 1.3 spec in latest ESXi 6.7 Update 2 release, but because these are "consumer" devices, they did not conform to the latest specification and hence the driver is unable to claim the device. This is another good reminder when using components not on VMware HCL, this is always a risk from a home lab perspective. In general, I know Samsung and Intel NVMe SSD usually works quite well without issues but always good to do some research. I think Engineering is looking to see if there are other workarounds for the future, but for now, you can use the workaround below.

The easy workaround that Pete found was to simply replace the NVMe driver from ESXi 6.7 Update 2 (1.2.2.27-1vmw.670.2.48.13006603) with one found in ESXi 6.5 Update 2 (1.2.1.34-1vmw.650.2.50.8294253). To so do, simply copy nvme.v00 to /bootbank from either an existing ESXi 6.5 Update 2 system or directly from the ISO. Please note, any future updates or patches to the ESXi host will most likely override the updated driver.

Deploying a vCenter Server Appliance (VCSA) in VMC?

05/07/2019 by William Lam Leave a Comment

During the VMware Cloud on AWS (VMC) Customer Summit last week, I received an interesting question from one of our field folks on whether it was possible to deploy a vCenter Server Appliance (VCSA) to VMC for testing purposes? This was not a use case I had heard of before but it would enable the team to quickly prototype a solution to demonstrate to their customer.

I figured this should work and you should be able to just point the VCSA Installer to an existing VMC environment for deployment. It was mentioned that they had attempted to run the installer but ran into a permission issue where it required a full administrator role, which in VMC, customers do not have.

In taking a look for myself in one of my VMC environment using the VCSA UI Installer, I did indeed run into the same permission issue as shown in the screenshot below.

User has no administrative privileges

This surprised me as the VCSA Installer does not actually require administrative privileges to deploy a VCSA, just the privileges for deploying a regular VM. I captured the logs and screenshots and have shared this with the VCSA PM for further investigation.

New Thunderbolt 3 to 10GbE options for ESXi

04/17/2019 by William Lam 16 Comments

With the help from Aquantia, we now have an ESXi driver to enable the built-in 10GbE adaptor for both the Apple 2018 Mac Mini and the iMac Pro. Although this was exciting news for our VMware/Apple customers, I was actually more excited for what this development meant for the larger VMware Community when it comes to 10GbE accessibility.

Many Enterprise customers have already been using Thunderbolt 2/3 to access their 10 Gigabit infrastructure, usually over SFP+ but Ethernet based options also exists such as the Sonnet solution which I had shared last year. This is especially common for VMware customers who virtualize Apple MacOS on vSphere for MacOS/iOS development and the use of Thunderbolt enables ESXi to connect to the underlying storage and networking infrastructure, which traditionally has been either Fibre Channel and/or IP-based storage running over a 10Gig link.

When you start looking at 10GbE accessibility for VMware home labs which could potentially apply to remote office/branch office (ROBO) and Edge/IoT environments, the cost and the complexity of the setup is something that many folks have to consider. There are definitely some creative options out there, most recently Chad Moon shared his solution using a Thunderbolt 3 to PCIe expansion chassis with his Intel NUCs which will run you about $230 per setup or you can be a true hardware hacker like Jack Harvest and use one of the M.2 slots in the Intel Skull Canyon NUC and connect that to PCIe 10GbE SFP+ card with a custom 3D printed chassis to hide everything for just $43.68 🙂

NSX-T Opaque Networks now supported with Cross vCenter Workload Migration Fling

04/15/2019 by William Lam Leave a Comment

The Cross vCenter Workload Migration Fling continues to be an extremely powerful tool for our customers and I just love hearing successful customer stories like the one recently shared by Jason on how the tool allowed him to easily migrate over 600 Virtual Machines and with more to come!

Moved 600 and looking at another thousand. Some monster VMs... our test environment moved as easy as vmotion.

— Jason 🛠 (@JasonThoms) April 9, 2019

One capability that customers have been asking for is the ability to migrate to and from an NSX-T Opaque Network type, which the current Fling does not support. This has become more and more important as the default NSX stack for VMware Cloud on AWS (VMC) is now NSX-T by default, rather than NSX-V. We are also seeing requests from on-premises customers who have deployed NSX-T for their next generation infrastructure and needing the ability to easily migrate workloads between their old infrastructure that maybe running VSS/VDS or NSX-V backed networks.

With the help from two new colleagues, Vikas Shitole and Rajmani Patel, we are excited to announce the release Cross vCenter Workload Migration v2.6 which now adds support for NSX-T Opaque Networks!

vSphere 6.7 Update 2 add support back for Apple Mac Pro 6,1

04/11/2019 by William Lam Leave a Comment

Last fall, support for vSphere 6.7 Update 1 on the Apple Mac Pro 6,1 was removed by VMware after finding out an updated version of the Apple Firmware prevented ESXi from booting up properly after upgrading. After some investigation and troubleshooting with several customers, VMware Engineering found a solution to workaround the problem and that fix is included in the latest vSphere 6.7 Update 2 release. As mentioned in the original blog post, this could also impact Apple Mac Mini's, which are not officially supported by VMware, but the fix should also apply to the Mac Mini.

In addition, the VMware HCL will also be updated shortly to reflect support for both vSphere 6.7 Update 2 along with the existing vSphere 6.7 support. For customers that have been waiting for features in vSphere 6.7 Update 1, you can finally upgrade to the latest release to get all the new features and benefits of vSphere.

Aquantia 10GbE ESXi Driver for Apple 2018 Mac Mini

04/10/2019 by William Lam 7 Comments

I know many of you have been anxiously waiting for ESXi to be fully functional on the latest Apple Mac Mini (2018), unfortunately I do not have any news to share with you on that front. Without help from Apple, we are still challenged with Apple's new T2 chip, which prevents us from accessing the underlying NVMe device.

Having said that, I do have some exciting news regarding the built-in 10Gigabit Ethernet adaptor on the 2018 Mac Mini. The 10GbE adaptor uses an Aquantia chipset, this is also the same chipset used in Apple's high end iMac Pro which was released earlier in the year. Over the past few weeks, I have been working with the Aquantia team and they have successfully ported over their open source Atlantic driver to a VMKlinux driver for ESXi, which they have published here. Although the local NVMe device can not be used to run any VMs, with the network fully enabled, customers could take advantage of this model and connect to IP-based storage to start leveraging the new Mac Mini platform.

The easiest way to incorporate the driver into the latest ESXi release is to use Image Builder within the vSphere H5 Client UI, below are the step-by-step instructions.