mac pro

ESXi Thunderbolt Driver to Fibre Channel Storage from ATTO

09/12/2016 by William Lam 6 Comments

One of the things I always enjoy doing at VMworld, when I am not running around and I have a few minutes to myself, is to check out the VMware Solutions Exchange. This is where you can learn and interact with hundreds of our VMware Certified Partners showcasing their new solutions and innovations that they have built on top of VMware's products.

UPDATE (08/22/17) - ATTO's ESXi Thunderbolt Driver is now officially on the VMware HCL, please see this blog post here for more details.

While walking through the show floor, I had stopped by the ATTO Technology booth who has been a long time partner of VMware in the storage and networking connectivity space. What caught my eye was that they had just released a Beta of an ESXi Thunderbolt Driver in the form of an ESXi VIB that would allow customers to connect their Apple Mac Pro 6,1 using the Thunderbolt 2 interface to an external Fibre Channel storage array. I believe ATTO might be the first vendor ever to produce a Thunderbolt Driver for ESXi. This is really exciting news if you ask me, especially as more and more of our customers are looking to virtualize Mac OS X guests in their Datacenters using vSphere.

Historically, the only option to connect a Mac Pro 6,1 to an external Fibre Channel array was to use something like a Sonnet Chassis. Now, you can potentially connect up to 6 of the built-in Thunderbolt 2 interfaces on the Mac Pro's to your external storage array using this new solution from ATTO. Before I go into some of the details, ATTO did want me to mention that this solution is currently not officially supported by VMware nor is it on VMware's HCL. ATTO will be providing full support on their software as well as VMware's software stack during the duration of the beta program. In terms of official certification on VMware's HCL, I suspect that it will most likely depend on customer demand which would influence whether ATTO applies for an official certification, which again, would be the first of its kind for Thunderbolt.

The way in which this solution works is that you install the ATTO Thunderbolt Driver on your ESXi host and this will allow it to communicate with an ATTO ThunderLink device which provides the Thunderbolt 2 to Fibre Channel connectivity. You have the option of using either the FC2082 which provides 20Gb/s Thunderbolt 2 (2-port) to 8Gb/s FC (2-Port) Device or the FC2182 which provides 20Gb/s Thunderbolt 2 (2-port) to 16Gb/s FC (2-Port) Device. Below is a diagram from the ATTO digital solution brief on Thunderbolt Driver for ESXi which outlines the configuration.

If you are interested in taking part in ATTO's ESXi Thunderbolt Driver Beta program or would like to learn more about the solution, you can reach out directly to Carllene Mowry (cmowry@attotech.com) who is running the program. For more information be sure to check out the ATTO digital brief on Thunderbolt Driver for ESXi.

Lastly, I was also fortunate to have a quick chat with Carllene and team to get a few additional exclusive tidbits on some of the things the ATTO team is working on next. The first of which is support for the Thunderbolt 3 (aka USB-C) interface to Fibre Channel which will be quite nice for newer platforms that include that interface, including home lab setups such as the Intel NUC. Speaking of Intel NUC, this is just one of the many other platforms that include either Thunderbolt 2 or 3 interfaces. Although the solution today is specifically supporting the Mac Pro, I know ATTO folks are interested to hear from customers on other systems with Thunderbolt interface and providing similiar capabilities.

The other really exciting development that is currently being investigated is support for Thunderbolt 2 or 3 to 10GbE connectivity on ESXi. As you can imagine, this is really going to open up some really cool new use cases, especially around things like VSAN which can easily benefit from this. It is still in early development but from my understanding, ATTO is already seeing a lot of interest in this area as well as how this might work with VSAN. I am hoping I will be able to share more details as this further develops. If any of these updates sounds interesting, do leave a comment to let the ATTO folks know and I will make sure they monitor the thread.

Apple Mac Pro 6,1 PCIe SSD issue resolved w/ESXi 6.0 Update 2

03/15/2016 by William Lam 6 Comments

Early last year, the new Apple Mac Pro 6,1 (aka black can design) was certified and fully supported on vSphere 6.0 which I had blogged about here. Several months later, customers discovered that some of the newer Mac Pro 6,1 units were shipping with different model of their PCIe SSD device than what was originally released at GA. This was problematic because ESXi was not aware of this newer device and could not detect during or after installation. Although a work around was identified for customers looking to install either ESXi 5.x or 6.x on the newer Apple Mac Pros, it definitely was not ideal.

It has taken a bit longer than expected, but the issue has now been resolved with the latest release of ESXi 6.0 Update 2. A similar fix will be available for customers running ESXi 5.5 in a future update. You can find the direct download for ESXi 6.0 Update 2 in link below which includes a pointer to the release notes in case you are interested in other fixes included in this release.

vSphere ESXi 6.0u2 - https://my.vmware.com/web/vmware/details?downloadGroup=ESXI60U2&productId=491&rPId=10348

Heads up: ESXi 5.x & 6.0 unable to detect newer Apple Mac Pro 6,1 local SSD Device

07/27/2015 by William Lam 12 Comments

Over the last couple of weeks there have been several reports coming in from customers that the local SSD device found in newer Apple Mac Pro 6,1 were no longer being detected by ESXi. Starting with ESXi 5.5 Patch03 and ESXi 6.0, the Apple Mac Pro 6,1 was officially supported but it looks like the latest versions of the Mac Pro 6,1 that are being shipped contain a slightly different local SSD device which is not recognized by ESXi.

This was not the first time that this has happened, when the 2014 Mac Mini were first released, they too had a similar issue in which a custom VIB was required to get the internal device to get recognized by ESXi. There is an internal bug (PR 1487494) that is currently tracking the issue and if you are also experiencing this problem, please file an SR and have the GSS Engineer attach your case to this bug.

In the meantime, there is an unofficial workaround which was discovered by one of my readers (Mr. Spock) that by installing the community SATA-XACHI VIB over on Andreas Peetz VIB Depot site, both ESXi 5.5 and ESXi 6.0 will then recognize the local SSD Device. You will need to either use VMware Image Builder or Andreas ESXi Customizer tool to create a custom image if you decide to install ESXi directly on the local SSD device. I personally would recommend installing ESXi on USB device, this would allow you to install the VIB as a post-installation and not requiring a custom ESXi image.

ESXi 6.0 works OOTB for Apple Mac Mini & Mac Pro

02/06/2015 by William Lam 145 Comments

Over the years, there have been various Apple issues that have required different hacks/tweaks to get a stock ESXi image to install on an Apple Mac Mini. Even though the only officially supported Apple platform for ESXi is the Apple Mac Pro, VMware Engineering, CPD and Hardware QE have been working hard to back porting as many of these "fixes/tweaks" to provide the best user experience possible for installing ESXi on non-supported Apple hardware. As you can imagine, the amount of work required to manage hardware on the official VMware HCL is no small task and then trying to back port non-supported platforms is even more challenging from a support standpoint.

The release of ESXi 6.0 is a significant release in my opinion as it contains the final few fixes that have plagued earlier version of the Mac Mini platform, especially starting with the Mac Mini 6,2 and greater models. I am please to announce that the stock ESXi 6.0 image now works on all Mac Mini platforms starting from 5,1 and newer as well as Mac Pro 6,1 platform and newer. There are no additional tweaks or custom ISOs that will be required for a fully functional ESXi installation with proper networking enabled. In fact, if you have a Thunderbolt Ethernet Adapter connected to either a Mac Mini or Mac Pro, you will also notice that it is automatically recognized without any additional driver or tweaks to the driver map files.

Disclaimer: Running ESXi on an Apple Mac Mini is not officially supported by VMware, please use at your own risk

One caveat that I would like to highlight is for the recent 2014 Mac Mini 7,1 or newer models where the SATA HDD is not automatically detected due to a change in the disk model made by Apple. Unfortunately, due to the late release of the 2014 Mac Mini, the required PCI ID to recognize the drive could not be added to the stock image of ESXi 6.0. Luckily, the resolution is quite easy and you can download and apply this custom VIB to get the SATA HDD recognized. I have been told that the plan is to get this fix in the next update/patch release so hopefully in the near future, no additional tweaks are required ... unless Apple decides to change something on us again 🙂

Here are screenshots of running ESXi 6.0 on both the Apple Mac Mini and Mac Pro:

Thunderbolt Storage for ESXi

01/21/2015 by William Lam 45 Comments

A question that I frequently receive is whether ESXi supports Thunderbolt-based storage devices. This is especially interesting for folks running ESXi on an Apple Mac Mini due to the limited number of IO connections the Mac Minis' have. If you look on VMware's HCL, you will not find any supported Thunderbolt Storage devices nor are there any that are being actively tested with ESXi, at least as far as I know.

Having said that, generally speaking from an ESXi host point of view, the Thunderbolt interface is just seen as an extended PCIe bus. This means that whatever storage device is connected on the other end can work with ESXi as long as there is a driver in ESXi that can communicate with that devices. This is analogous to having a RAID card and having the proper device driver on ESXi to see its storage.

Even though VMware is not actively testing Thunderbolt-based storage devices, there are a few folks out in the community who have and have been successful. I wanted to share these stories with the community for those that might be interested in this topic and hopefully others who have had similar success can also share more details about their setup.

UPDATE (09/12/16) - ESXi Thunderbolt Driver to Fibre Channel Storage from ATTO

Disclaimer: All solutions listed below are from the community and decisions to purchase based on these solutions will be at your own risk. I hold no responsibility if the listed solutions do not work for whatever reason.

Solution #1 - Pegaus R6 Thunderbolt Storage Enclosure

This was the first Thunderbolt storage device that I had ever seen confirmed publicly to work with ESXi after installing a STEX driver VIB. You can find more details here.

Solution #2 - Sonnet Echo Express III-R Rackmount Thunderbolt 2 Expansion Chassis & RacMac Mini Enclosure

This next solution was recently shared with me from Marc Huppert who has recently expanded his home lab. Marc combined a Thunderbolt expansion chassis with a Mac Mini chassis to exposed Fibre Channel storage to his Mac Minis. You can find more details here.

Solution #3 - xMac Mini Server Enclosure

I came across this solution while searching online which also uses another Mac Mini Thunderbolt expansion chassis connected to Fibre Channel based storage. You can find more details here.

Solution #4 - Sonnet xMac Pro Server Enclosure

Thanks to Joshua for sharing his solution. You can find more details in the comments here.

Solution #5 - LaCie Rugged Thunderbolt drives

Thanks to Philip for sharing his solution. You can find more details in the comments here.

Solution #6 - ARC-8050T2 Thunderbolt 2 RAID

Thanks to Jason for sharing his solution. You can find more details in the comments here.

Solution #7 - Another Sonnet xMac Pro Server Enclosure + EMC VNX

Thanks to Johann for sharing his solution. You can find more details here.

Solution #8 - LaCie Little Big Disk Thunderbolt 2 with 2013 Mac Pro w/ESXi 6.0

Thanks to Thomas for sharing his solution. You can find more details here.

Solution #9 - Sonnet Echo Express III with Mac Pro 6,1 and ATTO ExpressSAS H680 w/ESXi 6.0

Thanks to Grasshopper for sharing details here and here.

If there are other Thunderbolt-based storage devices that you or others have had success with ESXi, feel free to leave a comment with details and I will add it to the post. If there are any Thunderbolt storage device vendors that would like to send me a demo unit, I would be more than happy to give the system a test to see if it works with ESXi 🙂

How to install ESXi 5.5 Patch03 on the new Mac Pro 6,1?

10/31/2014 by William Lam 23 Comments

I have received several questions from customers asking how to go about installing the latest ESXi 5.5 Update 2 Patch03 on the Apple Mac Pro 6,1. Luckily, I was able to borrow one of the Apple Mac Pro 6,1 we had within VMware Engineering for a couple of days to document the process.

Step 1 - A prerequisite that must be met prior to installing ESXi is to have the Apple boot ROM update on the Mac Pro 6,1 running on MP61.0116.B05 or higher. This great piece of tidbit came from Josh who discovered issues while trying to install ESXi and found out this was a requirement after opening a case with Apple Engineering. It turns out that there is not an EFI update and the ONLY way to update the boot ROM was to install Yosemite (OS X 10.10) as it contains an update which can be applied to the Mac Pro. Thanks Josh for sharing this tip with us!

You can check the boot ROM by either following this Apple KB here or by running system info with an OS X image on a bootable USB device which is what I did to verify as seen in the screenshot below.

Step 2 - Once you have met the prerequisite, you will need to download the offline bundle for ESXi 5.5 Update 2 Patch03 which contains the Mac Pro 6,1 enablement along with other bug fixes. You can do so by going to VMware Patche portal and under ESXi, you should find ESXi550-201410001.zip at the very top.

Step 3 - Next you need to convert this offline bundle into an ISO image that you can load onto a USB device, this is the simplest way to install ESXi. To do so, you will need a Windows system as it uses a tool called VMware Image Builder which is only available for Windows. Image Builder is part of the PowerCLI toolkit which can be downloaded here.

Step 4 - Once PowerCLI has been installed go ahead and launch the it and we are ready to start authoring our ISO image

Step 5 - Add ESXi offline bundle that we download by running the following command:

Add-EsxSoftwareDepot ESXi550-201410001.zip

Step 6 - You will need to select the particular ESXi Image Profile to create your ISO image from, you can view the four Image Profiles by running the following command:

Get-EsxImageProfile | format-wide

Step 7 - You will want to select the one that contains the all patches including security and VMware Tools called ESXi-5.5.0-20141004001-standard by running the following command:

New-EsxImageProfile -CloneProfile "ESXi-5.5.0-20141004001-standard" -name "ESXi55u2-p03" -Vendor virtuallyGhetto

Step 8 - We now need to export the Image Profile we have selected to an ISO by running the following command:

Export-EsxImageProfile -ImageProfile "ESXi55u2-p03" -ExportToISO -filepath C:\VMware-ESXi-5.5u2p03-Mac-Pro-6-1.iso

Step 9 - Once the ISO has been created, you can now create a bootable USB containing your ESXi installation. I like to use Unetbootin but there are several other tools you can use, select whichever one you are comfortable with.

Step 10 - Plug the USB device into your Mac Pro and make sure to hold down the "ALT" key so you can select the device to boot from and you can start your ESXi installation as you would normally.

Here is a screenshot of the Mac Pro running the latest ESXi 5.5 Update 2 Patch03 release:

Thunderbolt Ethernet Adapter Support

I know there were a couple of questions from folks asking whether the Thunderbolt Ethernet Adapter would be recognized by ESXi on the new Mac Pro 6,1 and I can confirm, it does as shown in the screenshot below:

The Mac Pro 6,1 has two on-board Ethernet ports and comes with 6 x Thunderbolt connections, so you can connect quite a bit of networks if you need to.

GPU Passthru Support

I know a couple of you have asked whether the two Radeon 7870 GPU's could be passthru into a guest OS such as Mac OS X or Windows and it looks like they can from the ESXi point of view, however this is not officially supported by VMware, so YMMV on whether the guestOS can actually make use of the GPU.

For Windows it looks like it was able to properly detect the GPU (as shown in the screenshot) below and for Mac OS X it does not look like it's properly detecting the GPU. I will see if I can investigate this further but there is a good chance that passthru for Mac OS X guest will not work.

If you have made it this far and realize you rather not go through that long process (which is quite short actually), then I you will be quite happy to see that I have done the hard work for you and have created an ESXi 5.5 Update 2 Patch03 ISO which you can download here. Hope you enjoy your new Mac Pro and running ESXi on top of it!