virtuallyGhetto

smc

Retrieving Apple hardware details (Model, Serial, Board & EFI Boot ROM & SMC Version) from ESXi

by 2 Comments

For customers who run ESXi on Apple Hardware, retrieving the Apple hardware specific information such as the Board ID, EFI Boot Room and SMC Version for troubleshooting or auditing purposes can be challenging. Historically, this information is only available when running the hardware with an Apple MacOS operating system and customers would have to either boot the system into the MacOS Recovery Mode to run the system profiler tool from the command-line or install a full blown MacOS operating system to retrieve the necessary information as outlined in this Apple KB.

For some customers, this may not even be an option as Apple hardware does not provide any remote management capabilities and customers must physically be in front of the system to perform this process and imagine needing to do this across a fleet of Apple servers. 

While working on a recent case with a customer, I was curious if the Apple specific information could be retrieved from within ESXi and remove the need to boot into MacOS? I reached out to one of the Engineers, Darius Davis, who I work with on a frequent basis when it comes to MacOS-related topics to see if this was possible.

[Read more...] about Retrieving Apple hardware details (Model, Serial, Board & EFI Boot ROM & SMC Version) from ESXi

How to build custom ESXi ISO for Apple Mac Mini?

by 43 Comments

For those of you who own an Apple Mac Mini 6,2 may recall some of the, lets call them "challenges" on getting ESXi to run on the Mini. These challenges range from known SMC issues from Apple to missing or updated Broadcom tg3 network drivers. Though there workarounds for these issue, the process was quite complex. I took it upon myself to help simplify it by building custom ESXi ISO's for most of the major ESXi releases so that users could simply install ESXi as they normally would and by-pass all the complexity.

This has worked for the majority of folks but I have received several requests for those that may not be comfortable with just downloading a random ISO on the internet, which I can fully understand. The other reason is that some folks would like to build their own custom ISO and include other drivers/packages and others are just interested in the process. This has been on my to-do list for awhile but it was finding the time to document the process but also I normally like to take it a step further and see how I can make it even more simple 🙂

Disclaimer: Running ESXi on an Apple Mac Mini is not officially supported by VMware, please use at your own risk

With the recent release of vSphere 5.5 Update 2, I thought this would be the perfect opportunity to show how you can build your own custom ESXi ISO to run on the Apple Mac Mini 6,2.

Note: Earlier versions of Mac Mini should work fine for the most part without additional modifications.

Before I get started, I would also like to mention that several of the "challenges" such as having an updated Broadcom tg3 drivers have been fixed in the latest ESXi 5.5 Update 2 release, so out of the box you will be able to see the on-board network device working as expected and Ethernet Thunderbolt will also be functional if you are using that device with no additional drivers being required. I have been able to successfully install the default out of the box ESXi 5.5 Update 2 ISO from VMware on my Apple Mac Mini 5,3 without any additional changes.

Here is the process for building your own custom ESXi ISO for your Mac Mini:

Step 1 - Download the ESXi ISO you wish to work with

Step 2 - You will need access to a Linux system (recommend CentOS) that has mkisofs utlity, which is used to author an ISO

Step 3 - Download my custom.tgz which will automatically handle the SMC issue for Apple Mac Mini 6,2

Step 4 - Download my ghettotize-esxi-iso.sh which is a shell script that will automatically take an ESXi ISO and author a new ISO containing the fixes. The script is pretty straight forward and you can take a look at the script for all the details.

Here is an example of running the script against the latest ESXi 5.5 Update 2 ISO:

build-custom-esxi-iso-for-mac-min-0
As you can see at the end of the script, you should get a new authored ISO with a -NEW in the filename:

build-custom-esxi-iso-for-mac-min-1
Once you have the new ISO, you can then take that and load that onto a USB device. I like using unetbootin which is a handy utility that is supported on all platforms and creates a bootable USB device with the ISO provided. As you can see the process is pretty straight forward and though it took a bit of "experimentation" on my end to make it completely seamless, you can see there is too much to the process in general.

For those of you who rather not go through the process and just wants the goods and trusts me, I have also built out a new updated image for latest ESXi 5.5 Update 2 release. I did not have access to a Mac Mini 6,2 to confirm the build, but I am pretty confident it should work as I tested an earlier 5.5 Update 2 build several months back. You can find the latest ISO along with past ones I have created below.

How to run Nested Mac OS X guest on ESXi VM on top VMware Fusion?

by Leave a Comment

You might be asking, why would anyone want to do this? Well, luckily this is not a "because you can" type of answer but was it was an interesting solution that one of our VMware Engineers (Darius) had shared with me after helping out on this VMTN Community forum thread.

The user was running VMware Fusion on his physical Mac OS X system and wanted to be able to test OS X Mavericks under ESXi. Not having a physical ESXi host to test with, the next best thing was to run a ESXi VM under VMware Fusion and then run the Mavericks guest on top of that.

Here is a quick diagram of the user setup:

nested-mac-osx-vm-on-esxi-on-fusion0
The issue with just simply doing this is that for a Mac OS X guest to properly run on ESXi, the underlying hardware must be Apple Hardware. The reason for this is not a technical challenges, but rather a legal one per Apple's EULA. The way in which ESXi detects that the underlying hardware is Apple is by checking whether Apple's SMC (System Management Controller) is available.

In the scenario above, the Nested ESXi VM is not automatically passing through the SMC from the physical Mac OS X system and hence the Mac OS X VM at the very top of the stack will not properly function. The solution that Darius found was to add the following two Advanced VM Settings (VMX) entries to the ESXi VM:

smc.present = "TRUE"
smbios.reflectHost = "TRUE"

This will allow the passing of the underlying SMC up into the Nested ESXi VM which will then allow Mac OS X guest VMs to properly function. We can also confirm this by check the Nested ESXi MOB by pointing a browser to the following URL: https://[ESXI-IP]/mob/?moid=ha-host&doPath=hardware

nested-mac-osx-vm-on-esxi-on-fusion3
If you did not add the two entries above, then the smcPresent property would show up as false. In our case, we did add the following two entries and we now run our Mac OS X Guest. Here are a couple of screenshots of performing this on my iMac at home running the same exact configuration:

nested-mac-osx-vm-on-esxi-on-fusion1nested-mac-osx-vm-on-esxi-on-fusion2
Thanks Darius for sharing this with me and the community! I am sure this will come in handy for anyone wanting to test Mac OS X guests under ESXi but do not have a physical ESXi host and can easily substitute using VMware Fusion.

Running ESXi 5.0 & 5.1 on 2012 Mac Mini 6,2

by 59 Comments

If you recently purchased the new 2012 Apple Mac Mini 6,2 which was just released not too long ago and tried to install either ESXi 5.0 or 5.1, you probably noticed a PSOD (Pink/Purple Screen of Death) during the installation. This is currently a known issue and there is an extensive VMTN thread (9,300+ views) about this problem which also includes a fix through a collaboration between VMTN community user zer010gic and VMware Engineer dariusd. Even though the Apple Mac Mini is not an officially supported hardware platform for running ESXi, it is great to see VMware engineers going out their way and trying to help the VMware community find a solution as well as providing an "unofficial" fix in this case.

I would also like to point out that this issue only applies to the new 2012 Apple Mac Mini, for previous models such as the Apple Mac Mini 5,1 or 5,3 you can install ESXi 5.0 or 5.1 without any issues. For more details, please refer to the instructions in this blog post.

Disclaimer: The Apple Mac Mini is not officially supported by VMware. The only supported platform for ESXi 5.0 for Apple hardware is the Apple XServe 3,1 and for ESXi 5.1 is the Apple Mac Pro, which you can get more details here.

Before jumping into the solution, if you think VMware should support the Apple Mac Mini for running ESXi, please provide feedback to VMware by submitting a Feature Request. The more feedback that VMware receives from customers along with business justifications, the better our product management team can prioritize features that are most important to our customers.

Here are the current problem/solutions when trying to install on the new Mac Mini:

Problem: PSOD during ESXi 5.0 or 5.1 installation.
Solution: Add iovDisableIR=true to the kernel option before attempting installation. When you are asked to reboot, be prepared to enter iovDisableIR=true again (SHIFT+O) which is required to get ESXi to boot after installation. Once the system has booted up, go ahead and run "esxcli system settings kernel set -s iovDisableIR -v true" in the ESXi Shell to persist the kernel setting. This is a "temp" workaround while PSOD is being investigated.

Problem: Unable to install new OSX Server on a VM or power on existing OSX Server VMs.
Solution: There appears to be a significant change in Apple's SMC (System Management Controller) device in the newer models that prevents the Apple SMC VMkernel driver from properly loading. A tempoary fix was provided to zer010gic to create a custom ISO until the fix is integrated into a future release.

Note: There may be other minor/unconfirmed issues listed on the VMTN thread, but for basic ESXi installation/usage + OSX Server VM creation/installation, the above solutions should be sufficent.

Instead of having everyone walk through the process of creating a custom ESXi ISO which includes the two fixes mentioned above as well as the bundling the updated tg3 Broadcom network drivers for network connectivity, zer010gic has generously created and is hosting ESXi 5.1 ISOs for users to download and use. It contains some work that I have been doing with zer010gic to create an ESXi 5.1 ISO that does not require any manual intervention outside of the normal ESXi installation. I recently completed the rest of this work which is based off of the oriignal ISO that zer010gic has shared on the VMTN community (unfortunately I have not been able to get a hold of him to provide him with the necessary bits and I have decided to post a modified ISO).

Here is a step by step instruction for zer010gic ESXi 5.1 ISO

Step 1 - Download zer010gic ESXi-5.1-MacMini-SMC-6-2.iso.

Step 2 - Transfer ISO to either USB key or CD-ROM

Step 3 - Perform ESXi installation as you would, but when you get to the very last step prior to rebooting, be ready for some typing when the host boots back up (this is important else you will get a PSOD)

Step 4 - When ESXi starts to boot up, hit SHIFT+O which will allow you to add additional kernel boot option. Add the following text the bootUUID (remember to add a space first)

iovDisableIR=true

This step is required to ensure your ESXi boots up properly for the first time so you can permanently enable this kernel option using ESXCLI which will then persist this upon sub-sequent reboots.

Step 5 - Login to ESXi Shell (you may need to enable it first) and run the following ESXCLI command:

esxcli system settings kernel set -s iovDisableIR -v true

Once this is set, you no longer have to do this again. If you prefer not to go through these manual steps, please refer to the section below for a modified ESXi 5.1 ISO which automates all this for you.

Here is my modified ESXi 5.1 ISO which does not require any additional user intervention

Step 1 - Download my ESXi-5.1-MacMini-SMC-BOOT-FIX-6-2.iso

Step 2 - Transfer ISO to either USB key or CD-ROM

Step 3 - Go through normal ESXi install and enjoy

Note: For details on how I automated the kernel setting setting, take a look at the very end.

So if you are looking to refresh your home lab, you just may want to consider using the new Apple Mac Minis, especially with small form factor footprint 🙂

Note: A couple of users mentioned it took a bit of time to boot up, specifically when usbarbitrator module is being loaded. I noticed this too and it took quite a bit of time, probably 5-6 minutes. If you do not plan on any USB pass-through from the Mac Mini to your guestOSes, you can actually disable this service which should help speed the bootup. If you wish to disable usbarbitrator, run the following command:

chkconfig usbarbitrator stop

ESXi ISO Customization Details

If you take a look at the steps required to install the ISO provided by zer010gic, most of the heavy work has already been done for you. The only "manual" part that is required from the user is to enter a kernel option during the first boot and then run an ESXCLI command to persist this kernel setting which will prevent Mac Mini from PSODing. Removing these these manual steps is actually harder than it looks because of when you need to actually perform the changes. After much trial and error, I came up with the following script below (it's not the cleanest, but it works).

Basically the script is loaded from custom.tgz and executed before the installation begins and it generates a script stored in /tmp/customboot.sh which will look for the boot.cfg configuration file stored in the primary bootbank. This is where we insert the iovDisableIR=true parameter so the user is not required to do this after the first boot up. The challenge with this is the boot.cfg does not exists until after the installation has completed, so what I ended up doing was insert a command into /usr/lib/vmware/weasel/process_end.py which is part of the weasel installer for ESXi and is the very last script that is called when a user hits reboot. The command points back to the /tmp/customboot.sh which will perform the insert into boot.cfg right before rebooting. To automatically take care of the ESXCLI configuration, I added the ESXCLI command to /etc/rc.local.d/local.sh which will automatically run after all init scripts have executed. Then finally, I need to clean up local.sh since I only need that that run once which is handled by another script that is also created and stored in /etc/init.d/customcleanup which will just clean up local.sh file as well as delete itself. Simple right? 😉

Note: There is probably a more optimal way of doing this, probably using one of the weasel installer scripts and just set the boot.cfg option and then clean up with an init script, but I decided to leverage some of my earlier work for Disabling LUN Duringn ESXi Installation

Here is the script within the custom.tgz file: