Company: AutoTrader.com
Product: VMware vSphere
Hardware: Apple Mac Mini

[William] - Hi Chris, good afternoon. I know we have chatted a few times on Twitter before but for the folks that do not know you, can you quickly introduce yourself and what you do?

[Chris] - My name is Chris Nakagaki and I work for Autotrader.com as Sr. Systems Engineer. My current role involves day-to-day operations of VMware vSphere products in addition to defining best practices around the virtual infrastructure. Not to mention, help drive automation in my organization. Occasionally, I'll post something useful on my tech.zsoldier.com blog.

[William] - You had sent me an email after I published the first community story around how VMware leverages Mac Mini’s. I hear you are doing something pretty cool with the Mac Mini’s as well for your organization? Could you share some details on how your organization is using VMware and Mac Minis?

[Chris] - A couple of our subsidiary companies, in this case (vAuto and AutoTrader.com) essentially needed OS X VM's for QA testing of iOS applications, and general Mac browser testing. Rather than delivering individual Mac Mini to every developer and/or VMware fusion, etc. vAuto first approached us and we came up with this idea of clustering some Mac Mini's. They wanted to run ESXi on them to host relatively small VM's that could be centrally managed and accessed from any number of developers and with the Apple EULA, this was the only option due to the restriction. Besides that, it was just a really cool idea since we're all Mac/Apple fans anyways

[William] - That’s awesome, never seen a customer come up with both the request and a solution at the same time 😉 Have you had any experiences running vSphere on the Mac Mini’s before? Any challenges you faced while exploring this solution?

[Chris] - Thankfully you (William Lam) had run into a lot of the problems for us already.  So it was really easy to setup using the custom ISO and VIB you created to put our little 'MacCloud' together. The other non-software aspect we ran into though was the fact that the Mac Mini's do not have an out of band management interface. So we are currently looking for some smart power supplies and/or iKVM so that we can actually place these in our 'real' datacenters.

[William] - Hey no worries, I rather be the guinea pig and get all the kinks out so customers like yourself can just enjoy the benefits of running vSphere and ESXi on Apple hardware! How large is the MacCloud right now and what is the current hardware and software configuration?

[Chris] - Our MacCloud is only 3 Mac Mini's right now as we're still kind of 'feeling' it out. But soon after vAuto started using it, the word got out and we setup some test systems for our AutoTrader.com developers. In addition, our client engineering group uses a tool called Casper to manage our Macs. He needed a Distribution Point, preferably a system that had AFP, so we set him up with one and he was able to use it to deliver updates/applications. And the I/O for VSAN is so good with the SSD's, it screams.

Each Mac Mini is the 'Server' version, i7@ 2.6GHz, 1 SSD (128GB), 1HDD (1TB) and 16GB memory using USB to boot into ESXI. For the software, we are currently using vSphere Enterprise Plus and the vCenter Server happens to be the VCSA. The MacCloud is also being monitored by vCOPs

Here is a picture of the front/back of the Mac Mini rack:

autotrader-mac-mini-1

autotrader-mac-mini-2
[William] - That’s amazing, it sounds like the environment is really satisfying your developers and I can see why word has spread. So, did I read that right? You are currently using VSAN on the Mac Mini’s!? How has the performance been and what made you decide to leverage VSAN?

[Chris] - The minute VSAN went into beta, that's all I could personally think about for my own home lab with the Mac Mini's. That just naturally translated when the business had a need and I could satisfy my curiosity in one fell swoop. So being that these workloads aren't heavy I/O, I haven't been all that concerned with it. The VM Storage Policies have been all left at default because I don't see a need to change right now.

My team and I are actively keeping a pulse on all the users of the VM's hosted on here though. vCOPs shows that everything is working efficiently, but we want to make sure that is translating to a good user experience. The Casper DP is one in particular that I'm curious about since the disk I/O profile on that one should be a bit more consistent.

[William] - That is really cool to see customers already leveraging VSAN for their production usage and great to hear the experiences has been solid so far. You mention the use of vCOPs for monitoring the VMs, are you also using vCOPS to monitor the underlying Mac Mini’s and how do you handle hardware issues?

[Chris] - Honestly, right now, we're just relying on the vCenter CIM service to tell us if it finds a hardware problem. The first obvious problem I have with that is I'm not so sure it would notice a hard drive failure. Like VMware, we'd probably just bring it into a local Apple store and have any components still under warranty replaced. 'Normally' we have SNMP traps sent from vCenter to HP BSM. Being that this is such a small environment with lots of questions, we simply use vCOPs to alert us of any unusual behavior or problems. Many of of our vCenter alarms are 'self-correcting' alarms.

[William] - It sounds like your MacCloud is quite mature with so many different capabilities. Any plans in the near term to expand, I can already see more developers asking for similar setup? Will you be increasing your Mac Mini VSAN Cluster or potentially create a new one?

[Chris] - Most likely yes. My hope is that Apple and VMware will see the value in these community initiatives to hopefully make a 'support' Mac Mini with some native 10Gb capabilities. In the meantime, I can only see this growing to some really awesome potential.

[William] - Awesome to hear! Well, I do not want to take up any more of your time but before we conclude. Is there any tips or recommendations you would offer other fellow vSphere Administrators looking to run vSphere on Mac Mini? Any words of wisdom that you can offer?

[Chris] - Download William's ISO, upload and install the VIB from local system. Remote VIB install doesn't work because of static line (might be specific to windows) and last but not least. TRY IT!

[William] - haha. Thanks for the plug!

If you are interested in sharing your story with the community (can be completely anonymous) on how you use VMware and Mac OS X in Production, you can reach out to me here.

 

Thanks for the comment!