If you own or have recently purchased Crucial NVMe SSD such as CT1000P1SSD8 (1TB M.2 NVMe SSD) or CT500P1SSD8 (500GB M.2 NVMe SSD), please be aware that these devices may no be recognized by ESXi after upgrading to the latest release. Thanks to Pete Lindley, (OCTO for End-User Computing), who reached out last week regarding the observation as well as a workaround for the problem. This was also quite timely as I recently purchased a Crucial M.2 NVMe SSD and would have also ran into this problem.
It turns out these Crucial devices were working fine while running on ESXi 6.5 Update 2 but was no longer recognized in latest release of ESXi 6.7 Update 2. It is unclear whether support for these SSDs were removed intentionally or unintentionally, but in either case, these devices are not officially on VMware's Hardware Compatibility List (HCL).
UPDATE (05/23/19) - After speaking with a few folks who took a closer look, the issue is due to the fact that we added support for NVMe 1.3 spec in latest ESXi 6.7 Update 2 release, but because these are "consumer" devices, they did not conform to the latest specification and hence the driver is unable to claim the device. This is another good reminder when using components not on VMware HCL, this is always a risk from a home lab perspective. In general, I know Samsung and Intel NVMe SSD usually works quite well without issues but always good to do some research. I think Engineering is looking to see if there are other workarounds for the future, but for now, you can use the workaround below.
The easy workaround that Pete found was to simply replace the NVMe driver from ESXi 6.7 Update 2 (1.2.2.27-1vmw.670.2.48.13006603) with one found in ESXi 6.5 Update 2 (1.2.1.34-1vmw.650.2.50.8294253). To so do, simply copy nvme.v00 to /bootbank from either an existing ESXi 6.5 Update 2 system or directly from the ISO. Please note, any future updates or patches to the ESXi host will most likely override the updated driver.
For my 1TB HP EX920 I had to downgrade to 6.7 GA driver for NVMe to get 6.7u1 to see my drive. Haven’t checked 67u2, but I recond it will be the same. Maybe 67GA is the same driver as in 65u2?
It was discussed here: https://www.reddit.com/r/vmware/comments/a80r3y/issue_with_hp_nvme_drive_in_esxi/
In my experience, I had the same issue with 6.7U1 and 6.7U2 – I’m assuming its using the same driver.
6.7U1 was when they updated the driver. The 6.7GA driver seems to work fine. I really wish this was resolved, makes homelab stuff kind of a pain.
Thanks for this! I got my 1TB back!
Does somone have the issue with samsung evo nvme?
Turning off “Intel Rapid Storage Technology” worked for me to get a corsair MP510 detected and working
How did you do this? I cannot get this to work.
I did replace nvme.v00 with the the version from 6.7.0 GA.
Now ESXi is no longer booting 🙁
Did you get this working on ESXi 6.7 Update 2? I really don’t need to rebuild ESXi again.
were you able to resolve? I cant discover my datastores
I also wonder if the samsung evo 970 nvme has the same problem. I just received my adata SX8200 pro and after rolling back the driver version it works fine..but I wonder if I should go for the Samsung after all, if it wouldn’t have these driver issues.
Just stumbled across this article. I literally went to best buy after reading this and purchased a Samsung 970 EVO PLUS 500GB to test on a homelab system (ASROCK X470D4U Motherboard w/ AMD 2700x CPU) running a fresh install of ESXi 6.7 U2. ESXi had no issues detecting the drive, creating datastores, and VMs etc.
replacing the nvme.00 in the /bootbank per the steps provided will PSOD. You wont be able to boot anymore with error:
Failed to verify signatures of the following vib(s): [nvme]. All tardisks validated.
I ran into this issue with my own HP Homelab setup. I documented it here:
https://www.ovdamn.com/2019/07/not-all-nvm-drives-are-created-equal.html
It’s actually funny that you mentioned it, I tweeted at William Lam about this. He updated this post to show that the issue is that his Crucial drive didn’t support NVMe 1.3, however the HP EX920 is listed as supporting NVMe 1.3. I also have the 1TB version. My understanding is that the EX920 gets pretty hot, I’ve never seen mine go offline, but I don’t do a whole ton with it.
Running 6.5U3: Samsung 970 EVO = Yes + Intel 660p 1Tb = No. After cloning 6.5U3 image profile using Web Client/Auto deploy, deselect nvme 1.2.2.28 and select nvme 1.2.1.34, then generate install ISO. Shame, 660p officially supports NVMe 1.3 too
Thank you! That helped a lot! The only image available for download was the HPE one:
https://my.vmware.com/group/vmware/details?downloadGroup=OEM-ESXI65U2-HPE&productId=614