virtuallyGhetto

mac pro

Community stories of VMware & Apple OS X in Production: Part 8

by 3 Comments

Company: Mid-Pacific Institute (Private School in Hawaii)
Software: VMware vSphere and Fusion
Hardware: Apple Mac Pro

[William] - Hi Derick, I appreciate you taking some time out of your busy schedule to talk with us all the way from Hawaii 🙂 Before we get started, can you quickly introduce yourself?

[Derick] - Sure William. My name is Derick Okihara, and I work for Mid-Pacific Institute. We are a private K-12 institution with about 1600 students. My role here is general IT and server administration. I've been working with computers since I was in high school. I have been a long time Apple user (since //gs), but really started working with them professionally about 10 years ago. We currently have a 1:1 iPad program for the students, and 2:1 iPad+laptops for faculty, so there's a lot of technology to support.

[William] - That’s awesome Derick. Look forward to hearing more about your environment. Speaking of which, I hear you are currently managing some Apple hardware running on VMware? Could you tell us a little bit about the hardware configuration and the VMware software you are currently using?

[Derick] - We are currently using vCenter with ESXi 5.5, we have 2 Mac Pro (5,1s) in a cluster with a Synology 1813+ shared storage. The network storage is connected via the iSCSI software initiator using round-robin. We also use VMware Fusion for the Mac. The Mac Pros have 24GB of RAM and 4 port Intel Gigabit Ethernet cards for a total of 6 ports each.

[William] - What made you decide on using a Synology for shared storage and what configuration/capacity did you go with? Were there any other options you were looking at?

[Derick] - A lot of the decisions for this setup was made on price. How this all started, was that I was asked to create an ESXi server to host a VM Appliance to run our campus wide Informacast speaker system. I had already been planning an ESXI deployment on the Mac, testing on Mac Mini. Instead of building a PC server just for this appliance, I was given the OK to build on an existing Mac Pro, so it could serve multiple purposes.

Being forward thinking, I knew we needed redundancy, so I opted to go for network storage. With a tight budget, and being able to use CPUs we already had, I decided on the Synology 1813+ for it's 4 gigabit ports. This allowed me to later add the 2nd host in our vcenter cluster when we expanded.

[William] - Can you talk a little bit about the type of workloads and applications you are currently virtualizing on the Mac Pro’s and are these all OSX VMs?

[Derick] - Since this is still version 1.0, we aren't heavily taxing our cluster. Right now it hosts 5 VMs (2 Virtual Appliance, 2 OS X Server, 1 Windows Server). I'd want to add more RAM as well, OS X VMs are very RAM hungry.

The OS X servers are a student file server (AFP/SMB) and an Apple Caching server / Munki repository. The Windows server is mostly a test bed, the Virtual Appliances are the aforementioned Informacast manager and VCenter Appliance.

Here is a picture Derick's two Apple Mac Pros:

derick-mac-pro
[William] - The Mac Pro’s have a maximum amount of memory that it supports, do you plan on expanding the infrastructure to accommodate additional workloads or would you be looking at upgrading to the latest generation of Mac Pro (black)?

[Derick] - Honestly, with our current needs and budget, I think I would be looking at the next generation Mac Mini combined with some sort of PCI-E enclosure. Like the Sonnet XMac server. I know the Mini will likely not be fully supported, but I like what i've seen on virtuallyGhetto with the current generation 🙂 That is, if the next gen Mac Mini supports 32GB of RAM!

[William] - Very cool! So from your point of view, you would rather have more Mac Mini’s than a couple of Mac Pro’s? It sounds like cost plays a huge factor, but what other constraints or requirements that is making you lean more towards Mac Mini’s instead of going to a new Mac Pro which can get up to 64GB of memory and 6-Core CPU?

[Derick] - Footprint - the Mac Pros we currently have take up a large amount of rack space. Even the new mac pros would not be rack mountable without an additional enclosure. For us, having 3 x Mac Mini with 32GB of RAM would be ideal price/performance ratio. (We have a 3 CPU license). Eventually our Mac Pro 5,1's will die, so I'm already thinking about what's next. Having 3 x Mac Mini servers in a cluster, that takes up only 3U would be pretty sweet!

[William] - Speaking of support, did you purchase any type of extended contracts with Apple on the hardware or you going to deal with them on a case by case basis? Have you had any issues with failing hardware on the Mac Pro’s?

[Derick] - We only had the initial Apple Care (now since expired). We have 1 spare Mac Pro currently running other loads but that could be migrated in the event that we have a hardware failure. We have not had many issues on the Mac Pro 5,1s other than internal hard drive failure. They've been rock solid.

[William] - Has there been any interesting issues or challenges you had faced while setting up this infrastructure? Either the hardware, software or managing the VMs?

[Derick] - This whole process was a learning experience for me. At a previous job, I had inherited an ESXI server running multiple CentOS and Ubuntu VMs, but I had never set it up myself, let alone on a Mac. Thanks to the multiple resources on the web (Rich Trouton's blog, VirtuallyGhetto, and a P2V script from Alan Gordon at MacSysAdmin) the process and gotchas were mostly done before me

The biggest challenge for me was configuring the Synology for iSCSI-round robin. In my research I found that one could utilize multiple gigabit connections with Multi-path IO for higher bandwidth. After lots of configuring and back and forth with Synology / VMware support, I finally found the proper settings that allowed me to utilize more than 1 gigabit link.
However, after I updated to ESXi 5.5, it broke.

I was stuck, because I needed to upgrade to 5.5 in order to run an OS X caching server (12-character serial number fix in 5.5). But Synology said the 1813+ was not compatible with 5.5 and would not help me. Long story short, one of my hosts is running 5.5 (with OS X Caching server) and the other host is running 5.1 (file services) because it needs the greater throughput.

[William] - Derick, I want thank you for taking the time and sharing with us your experiences with managing VMware and Apple hardware. Before I let you go, do you have any tips for our readers that may be in a similar environment (academic) and needs to build out an infrastructure to support their end users? Any gotchas or things you would recommend if you had to do this all over again?

[Derick] - Anyone looking to reduce their machine footprint should definitely look into virtualization. VMware has very attractive pricing for the EDU market if you're looking to build a cluster with high availability, or you can run a single host for free. The best piece of advice I can give is just to test thoroughly. Virtualization is very complicated, and combines a multitude of areas of expertise (Storage, Networks, Workloads, and ESXI platform itself). It can be daunting but it's very rewarding. If you get stuck, just ask William on Twitter @lamw, jk

If you are interested in sharing your story with the community (can be completely anonymous) on how you use VMware and Mac OS X in Production, you can reach out to me here.

 

Community stories of VMware & Apple OS X in Production: Part 7

by Leave a Comment

Company: Fortune 150 (Retail)
Software: vSphere + vSphere Replication
Hardware: Apple Mac Pro

[William] - Hi Vitaliy, thank you for reaching out and wanting to share your experiences with the community on managing a VMware and Apple OS X infrastructure. Can you tell us a little bit about yourself and what you currently do?

[Vitaliy] - I am a Senior Systems Analyst for a Fortune 150 company that wishes to remain anonymous (aka I do not have legal clearance to use the company name). I am part of a team that is responsible for providing IT infrastructure for many creative and marketing applications -- think pre-press and advertising.

[William] - Can you provide us some details about the VMware and OS X infrastructure that you’re supporting? Software/Hardware specs that you decided to go with and the workload characteristics?

[Vitaliy] - Prior to virtualization we were running two dozen Xserves with OS X 10.6 running a wide range of applications from Open Directory to custom in-house scripts. We have virtualized the whole environment with just 4 Mac Pro machines, each machine has 12 cores and 64GB of memory giving us a total of about 128GHz and 256GB of memory.

We have exhausted all the PCI-X slots on the Mac Pro's by adding two dual port network cards and a dual port HBA. As a result we have two redundant management, data, and vMotion ports on each machine. Oh, one thing worth mentioning is that VMware officially only supports 32GB of memory per Mac Pro but we have been running 64GB with no issues. For the past year we have been running vSphere 5.1 and just upgraded to 5.5 last week.

We have been using HP 3PAR SAN for our storage back-end and over the last couple of weeks we have migrated to an Oracle SAN. The whole process was completely seamless and transparent to the users thanks to VMware.

Here is a picture of the Mac Pro setup courtesy of Vitaliy:

mac-pro-vitaliy
[William] - Wow, that’s great to hear you’ve been able to really push the Mac Pro’s. You must have been happy to be able to consolidate all those Xserves! What was your approach for virtualizing OS X from the physical Xserve to Mac Pro? Did you rebuild or leverage some type of V2V?

[Vitaliy] - We decided to rebuild from scratch. We were running an outdated version of OS X 10.6 and all the applications running on top of that were just as old.

[William] - Can you talk to how you provision your OS X Virtual Machines and Applications and how it gets to the end users? Do users get their own systems or is this a shared infrastructure?

[Vitaliy] - It's a shared infrastructure, generally a VM is dedicated to a particular application. We created a "base VM" that has basic settings like power/energy saver settings, local accounts, monitoring software, etc. preconfigured and whenever we need a new virtual machine we simply clone it and change the hostname and IP address on the new VM. Perhaps a template would've been a cleaner solution but this is what we do. We are currently looking into automating configuration with either Puppet or Casper.

When we initially rolled out a couple of OS X virtual machines we noticed that CPU usage on the VMware cluster spiked up to almost a 100% while the virtual machines were idle. It turned out that the default OS X screensaver uses GPU power to generate that flare effect and because not enough GPU memory was available it resorted to using up all the CPU. Disabling the screensaver or switching to a text based one quickly fixed that issue ...

[William] - Thanks for the excellent tip on OS X screensaver, this is a handy one to know about! How do you go about monitoring the Mac Pro infrastructure? What’s the process for replacing failed hardware components and have you had any challenges with this?

[Vitaliy] - We treat it the same way as the rest of our environment -- each vSphere node and virtual machine is monitored via Nagios. We have this cluster running for little over a year now and luckily we have not had to deal with any hardware failure.

[William] - For your OS X Virtual Machines, do you have a need for backups or a DR strategy? If so, could you share some details on what you are currently using?

[Vitaliy] - We have a replica of our production environment at a remote disaster recovery site and we use vSphere Replication to copy all the VMs nightly. We also heavily rely on the snapshot feature prior to making any operating system or application changes, it has been a lifesaver so far.

[William] - Vitaliy, I want to say thank you very much for taking some time out of your super busy schedule to have a chat. Before I let you go, do you have any words of wisdom for others looking to manage a similar infrastructure? Anything you would do differently and any resources you have found useful in aiding you to support a VMware / OS X infrastructure?

[Vitaliy] - Speak to your manager, legal department, or whoever is in charge about interpreting Apple EULA. I have heard of at least three different interpretations and all have legal implications. I am very happy with our environment and would not change a thing if I had to build it again. Your blog, virtuallyGhetto, has been a great resource as you are the only one talking about VMware products running on Apple hardware.

If you are interested in sharing your story with the community (can be completely anonymous) on how you use VMware and Mac OS X in Production, you can reach out to me here.

 

Community stories of VMware & Apple OS X in Production: Part 5

by 2 Comments

Company: Artwork Systems Nordic A/S (AWSN)
Software: VMware vSphere
Hardware: Apple Mac Pro

[William] - Hi Mads, thank you for taking some time this morning to share with the community your past experiences managing a VMware and Apple OS X environment. Before we get started, can you introduce yourself and what you currently do?

[Mads] - My name is Mads Fog Albrechtslund, and I currently work as a vSphere Consultant for Businessman A/S Denmark. The reason for my current employment, is primarily a Mac based vSphere project I did at my former employer, Artwork Systems Nordic A/S also in Denmark. Before I became a vSphere Consultant, my primary job function was as a Mac Consultant, in which I have several Apple related certifications.

[William] - Could you describe what your vSphere project was about?

[Mads] - The vSphere Project, was that of virtualizing and consolidating the infrastructure of Artwork Systems Nordic A/S (AWSN). AWSN is a reseller of hardware and software to the graphical industry, thereby running a lot of Apple systems and software that require Mac OS X underneath.

When I started at the company in early 2009, there were around 8-10 servers, and only 9 employees. Every server was just a desktop Mac or PC, running multiple services at once, trying to use the hardware at best. I started by consolidating and somewhat standardizing all these machines, into a Rack cabinet.

But I still wanted to make it better, more flexible and faster to deploy new OS'es when they are needed. I also wanted to move away from running multiple services on a single OS. I started looking into virtualization around late 2010, before VMware even made vSphere compatible with the Mac's. And we started working with a competitor of VMware, which at the time was about to release a bare-metal hypervisor that was compatible with Mac hardware.

We invested time, money and hardware in that initial project, only to around 6 month later to find out that the vendor would drop that bare-metal software again.

[William] - Ouch! I guess that is one of the risks when working with a new company/startup. So what did you end up doing after the company dropped support for bare-metal support?

[Mads] - So when VMware release vSphere 5.0 which was compatible with Apple hardware, I asked my boss to try again. He said "Sure, go ahead…. but we don't have a lot of money to do this with". So I needed to make this project as cheap as possible.

What I ended up with was 3 Mac Pro's (2x 2008 and 1x 2009), which I got almost free from a customer, extra RAM (32GB in each Mac Pro), extra NIC's (4 NIC's in each Mac Pro), a Synology RS812+ NAS and VMware vSphere Essentials bundle.

Here is a picture of the 3 Mac Pros:

awsn-mac-pro
[William] - I too remember when VMware announced support for Apple Hardware with vSphere 5.0, that was a huge deal for many customers. Were there any performance or availability requirements that you had to take into considerations while designing this solution? Did all Virtual Machines run off of the NAS system or was it a mix between local and remote storage?

[Mads] - All VM's ran off the NAS over iSCSI. I did consider the availability of that design, but given the constraints of the money of the project, there was not much of a choice. I did not want to run the VM's on the local disks inside each Mac Pro, considering that if one Mac Pro died, I would not easily have the possibility to power-on that VM on another Mac Pro.

The performance of the NAS was not great, but good enough. After I left, the NAS was upgraded to a Synology DS1813+, and then using the old Synology RS812+ as a backup destination. The load on the VM's was light, as there only was 10 employees in the company, and most of the VM's was only for testing or designing solutions for the customers.

[William] - What type of Virtual Machines and applications were you running on the Mac Pros?

[Mads] - The 3 Mac Pro's are running around 20 VM's, where most of them are either OS X based or Linux Virtual Appliance's. My plan was to do one service per OS, to keep it as simple as possible. Almost all the OS X based VMs are running OS X 10.8 Mountain Lion. Some of them are just plain Client installations, but most of them have the Server app installed, to run Open Directory, DNS or File Server.

The Client installations are running specific software that the company sells, like graphical processing software from Enfocus or FTP software from Rumpus. There is also an older OS X based VM, running Mac OS X Server 10.6, which runs a special graphical procession software called Odystar from a company called Esko. This software only exist on Mac OS X, and it also requires a HASP USB dongle for its license. Most of the VM's are configured as low as possible, which for most is 1 vCPU and 2GB ram.

The Mail server for the company, is based on Kerio Connect software, which is also something that the company is a reseller of for its smaller graphical customers. That software exist either as a virtual appliance, a Windows install or a Mac based install. We ended up with choosing a Mac based installation, because we knew it better.

[William] - How did you go about monitoring the Virtual Machines as well as the underlying hardware? Any particular tools that you found worked well for your organization?

[Mads ] - We did not do much of monitoring, of neither the VMs or the hardware. I was onsite, and sitting almost beside the rack most of the time, so if there was any trouble either physically or virtual, I could fix it fast. I had configured email reporting in all the solutions that gave the option (vCenter Server, Synology NAS and some of the applications).

[William] - I know you had started this project back in 2010 and there was definitely a limited amount of hardware options to run Apple OS X VMs. Today, there are a few more options and if you were to do it again, would you have done anything differently? Would you still consider the Mac Pro (Tower) or look at potentially the newer Mac Pro (Black) or even the Mac Mini’s?

[Mads] - We did start out by looking at the Mac Mini's, but considering that we could only run 3 hosts because of the vSphere Essential license, we needed to get more RAM in each host, than the Mac Mini's could provide. The Tower based Mac Pro is still the best option for this installation, given that it is available for a reasonable price, runs more than 16GB ram and you can get 2x CPU sockets in each host.

The new black version of the Mac Pro, is especially not a good fit, primarily because of the price and because of the dual GPU's and only 1 CPU. I would love a Mac Mini with 32GB ram, that would properly fit perfectly, considering the advances in CPU technology over the 2008/2009 CPU's in the Mac Pro's currently running the environment.

[William] - Mads, thank you very much for spending your morning and sharing with us your experiences with running vSphere on the Mac Pros. You have provided a lot of good information that I know will surely help the VMware and Apple community. One final question before I let you go. Is there any tips/tricks you would recommend for someone looking to start a similar project? Any particular resources you would recommend people check out?

[Mads] - First of a big thanks to yourself, for provide great content on http://www.virtuallyghetto.com. I have also provided my own experiences both on my personal blog www.hazenet.dk and on businessman's company blog bmspeak.businessmann.dk

On my own blog, I have written about issues with screensavers in Mac OS X VM's and I have also written a long blog post about how make a never booted Mac OS X template VM, which don't have any UUID's set.

If you are interested in sharing your story with the community (can be completely anonymous) on how you use VMware and Mac OS X in Production, you can reach out to me here.

 

Community stories of VMware & Apple OS X in Production: Part 3

by 2 Comments

In this third post of the community stories, I had the pleasure to interview with Blake Garner who shares with us how he leverages VMware and Apple Technologies in his production environment.

Company: Adobe
Product: VMware Fusion
Hardware: Apple Mac Mini & Mac Pro

[William] - Hi Blake, thanks for your comment on Twitter today, it sounds like this blog series might be useful for your organization. I understand that you have some experiences working with VMware and Apple OS X that you can share with the community? Before we get started, can you quickly introduce yourself?

[Blake] - Sure, I have been working at Adobe for 11 years in the Seattle location. Started off as a Dev/Test lab administrator focused on Macs, Printers & color management. Been working with VMware seriously starting with VMware Fusion 1.0 and Lab Manager 2.x.

[William] - Wow, a VMware old timer 🙂 So, I take it from your Tweet that you must be doing some really cool stuff with VMware and Apple OS X? Can you describe the environment and how you leverage these technologies together?

[Blake] - We provide "bare metal" Mac Mini's and Mac Pro's to our users. They are mostly developers and testers of Adobe desktop software. The process involves approvals and users requesting a Mac from our web portal.

Once users have the Mac they load VMware Fusion which is the majority of the time. We do not have a centralized command and control of VMware Fusion. A lot of the teams then hook up VMware Fusion to our testing harnesses. Things like Jenkins and a lot of custom code to manage the VM's. We looked into using ESXi/vSphere/vCloud on the Mac but due to the Apple EULA restrictions it just wasn't a good fit.

[William] - Very cool! This sounds like you’re offering Mac as a Service for your internal customers? How do you go about managing the requests of which Mac Mini or Mac Pro are leased out? Is all this custom software Adobe has built?

[Blake] - Yes we do both Mac and x86 systems (Windows & Linux) as part of our bare metal offering on our internal IT Cloud. The bare metal part is custom code. We have rolled our own deployment systems to go along with it as well. Users access one portal that currently spans vCloud, AWS & bare metal along with a few other services.

[William] - You mention that the majority of users install VMware Fusion on the “bare metal” once they get their assigned Mac Mini or Mac Pro. What is VMware Fusion being used for?

[Blake] - Lot's of automation. vmrun gets a good workout here. One team has a "Test as a Service" that can control VMware Fusion and rollback snapshots to provide a clean testing state. The consumers of the Mac's enlist them into their own existing automation systems. Often if you look at the VM's you will see Creative Cloud applications running through test cases super fast. Builds of products for Mac OS and iOS also happen.

[William] - Ah, so they are leveraging VMware Fusion as a platform to be able to run sort of a “Continuous Integration” build environment for your internal Mac OS X and iOS builds? This sounds like it could be quite challenging to manage, have the end users had any issues or have they automated everything all already?

[Blake] - In a larger software company you end up with a number of approaches. I did not get involved at the build level enough to speak on that. Adobe IT tends to focus on the common services that all the teams can use. In general there is a lot of automation herding going on..

[William] - Gotcha. So, going back to physical Mac Mini and Mac Pro, roughly how many are you managing and what type of configurations did you spec out? Any particular reasoning for choosing these configurations?

[Blake] - Our initial launch of the bare metal mac service has 50 Mac Pro's and 50 Mac Minis split equally between two sites. This service is just a couple months old and we expect it to grow fast as the engineers figure out they can get Mac's if their manager approves a monthly fee to their cost center.

We have one config for Mac Pro and Mac Mini. The Mac Mini 6,2 has 250GB SSD and i7 2.6GHz with 16GB RAM. The Mac Pro 6,1 6-Core E5 3.5 GHz with 500GB SSD, 16GB RAM and lower end GPU's as those are not used that much. We really just looked for a sweet spot that matches what Creative Cloud needs

Here is a picture of one of the Mac Pro racks courtesy of Blake:

adobemacpro
[William] - How do you monitor for hardware issues and what is the most common issues have you seen for both the Mac Mini and Mac Pro?

[Blake] - That is a real challenge with Apple hardware. With no out of band management we rely on our staff to troubleshoot via KVM or in person if needed. Pre-release software can really crash a system and that often needs a finger on the power button. I'm looking into using our remote controlled PDU to power cycle systems via the portal. If a system is truly FUBAR we just give people a new one.

In a lot of cases we can simply re-image the system via netboot and add it back to the available pool. VMware Fusion comes really in handy for developing our custom netboot environments as well. Netbooting an OS X VM's is one of my favorite features of VMware Fusion.

[William] - I can definitely see it being easier to re-image than to troubleshoot unless you are seeing a consistent issue. Curious, what version of Mac OS X and VMware Fusion are you using today? In addition to snapshot & Netbooting, any other useful features VMware Fusion provides?

[Blake] - VMware Fusion 6.0.3 is what we support today. On the hardware we encourage people to use the latest release version of OS X. In the VM's it depends on what kind of testing. Validation of pre-release OS's is a big chunk of the testing work. For OS X that happens in VMware Fusion and other x86 it's done in vCloud and some VMware Workstation. The automation capabilities of VMware Fusion are a key component. That's what gives us the upper hand in managing these virtual systems.

[William] - Hey Blake, I really appreciate you taking the time and sharing with us on how Adobe leverages VMware and Mac OS X. Before I let you go, do you have any words of advice for other customers looking to provide a similar type of environment? Any gotchas or things you would change if you could start fresh?

[Blake] - Go talk to those people who have offices crammed full of systems and find out what they are doing. You can often find the common requirements and start building against that. Don't try and dictate the whole solution to engineers. Once they are happy customers it's much easier to get them onboard with centralized services. Provide a set of functional services and engineers will pick it up quickly. Keep adding services and they will grow along with you.

Things that I would change all end up on our features roadmap. I have my eyes on providing an API to access to re-image or reserve bare metal systems and providing vagrant along with VMware Fusion for automation.

If you are interested in sharing your story with the community (can be completely anonymous) on how you use VMware and Mac OS X in Production, you can reach out to me here.

 

Quick Update – ESXi support for Apple Mac Pro 6,1

by 3 Comments

I know many of you have been asking about ESXi support for the latest Mac Pro 6,1 that was released from Apple late last year and I just wanted to give a quick update. VMware Engineering has been hard at work on getting this new platform certified and supported with ESXi, however, there were some unforeseen challenges that is currently preventing the current version of ESXi to run on the new Mac Pro.

VMware is working closely with Apple's hardware team to resolve these issues and we expect to have a Mac Pro 6,1 supported with ESXi 5.5 in the future. In the meantime, if you wish to evaluate ESXi on the new Mac Pro (though not officially supported), you can sign up for the new vSphere Beta and run a Beta version of ESXi on the new Mac Pro.

Here is a screenshot of Mac Pro 6,1 running the Beta version of ESXi:

esxi-mac-pro-6.1-0
There are a couple of workarounds that is required for right now, which will all be resolved by GA. For more details, please refer to this VMTN thread.

Apple Mac Pro 5,1 now on VMware HCL for ESXi 5.5

by 5 Comments

I was just informed yesterday that VMware HCL (Hardware Compatibility List) has been updated to  include the current Apple Mac Pro 5,1 as a supported platform for ESXi 5.5. The Apple Mac Pro was not originally listed and may have caused some confusion. This was a mistake on our part and luckily it has now been resolved.

The Apple Mac Pro 5,1 now supports the following versions of ESXi:

  • ESXi 5.1
  • ESXi 5.1 Update 1
  • ESXi 5.5

I know a few of you have asked about the upcoming Mac Pro (black circular one) and whether it would be certified for ESXi.

All I can say, it is something that we will look at once the new Mac Pro becomes generally available (VMware does not get early access to Apple hardware).