UEFI PXE boot is possible in ESXi 6.0

A couple of days ago I received an interesting question from fellow colleague Paudie O'Riordan, who works over in our Storage and Availability Business Unit at VMware. He was helping a customer who was interested in PXE booting/installing ESXi using UEFI which is short for Unified Extensible Firmware Interface. Historically, we only had support for PXE booting/installing ESXi using the BIOS firmware. You also could boot an ESXi ISO using UEFI, but we did not have support for UEFI when it came to booting/installing ESXi over the network using PXE and other variants such as iPXE/gPXE.

For those of you who may not know, UEFI is meant to eventually replace the legacy BIOS firmware. There are many benefits with using UEFI over BIOS, a recent article that does a good job of explaining the differences can be found here. In doing some research and pinging a few of our ESXi experts internally, I found that UEFI PXE boot support is actually possible with ESXi 6.0. Not only is it possible to PXE boot/install ESXi 6.x using UEFI, but the changes in the EFI boot image are also backwards compatible, which means you could potentially PXE boot/install an older release of ESXi.

Note: Auto Deploy still requires legacy BIOS firmware, UEFI is not currently supported today. This is something we will be addressing in the future, so stay tuned.

Not having worked with ESXi and UEFI before, I thought this would be a great opportunity for me to give this a try in my homelab which would also allow me to document the process in case others were interested. For my PXE server, I am using CentOS 6.7 Minimal (64-Bit) which runs both the DHCP and TFTP services but you can use any distro that you are comfortable with.

Step 1 - Download and install CentOS 6.7 Minimal (64-Bit)

Step 2 - Login to the CentOS system via terminal and perform the following commands which will update the system and install the DHCP and TFTP services:

yum -y update
yum -y install dhcp tftp-server

Step 3 - Download and upload an ESXi 6.x ISO to the CentOS system. In example here, I am using latest ESXi 6.0 Update 1 image (VMware-VMvisor-Installer-6.0.0.update01-3029758.x86_64.iso).

Step 4 - Extract the contents of the ESXi ISO to the TFTP directory by running the following commands:

mount -o loop VMware-VMvisor-Installer-6.0.0.update01-3029758.x86_64.iso /mnt/
cp -rf /mnt/ /var/lib/tftpboot/esxi60u1
umount /mnt/
rm VMware-VMvisor-Installer-6.0.0.update01-3029758.x86_64.iso

Step 5 - Copy the custom ESXi bootx64.efi bootloader image to the root of the extracted ESXi directory by running the following command:

cp /var/lib/tftpboot/esxi60u1/efi/boot/bootx64.efi /var/lib/tftpboot/esxi60u1/mboot.efi

Step 6 - Next, we need to edit our DHCP configuration file /etc/dhcp/dhcpd.conf to point our hosts to the mboot.efi image. Below is an example configuration and you will need to replace it with the network configuration of your environment. If you are running the TFTP server on another system, you will need to change the next-server property to the address of that system else you will just specify the same IP Address as the DHCP server.

Step 7 - Next, we will need to edit our TFTP configuration file /etc/xinetd.d/tftp to enable the TFTP service by modifying the following line from yes to no:

disable = no

Step 8 - By default, the ESXi's boot.cfg configuration file refers to all packages under / path. We will need to remove that reference and can easily do so by running the following command:

sed -i 's/\///g' /var/lib/tftpboot/esxi60u1/boot.cfg

Step 9 - Finally, we need to restart both the TFTP (under xinetd) and DHCP services. For testing purposes, I have also disabled firewall for ipv4/ipv6, of course in a real production environment you will probably want to only open the ports required for TFTP/DHCP.

/etc/init.d/xinetd restart
/etc/init.d/dhcpd restart
/etc/init.d/iptables stop
/etc/init.d/ip6tables stop

We can now boot up either a physical host that is configured to use UEFI firmware OR we can also easily test using Nested ESXi. The only change we need to make to our ESXi VM is by setting the firmware mode from BIOS to EFI which can be done using the vSphere Web/C# Client as shown in the two screenshots below:

uefi-pxe-boot-esxi-6.0-0 uefi-pxe-boot-esxi-6.0-1
If everything was successfully configured, we should now see our system PXE boot into ESXi installer using UEFI as seen in the screenshot below.

If you run into any issues, I would recommend checking system logs on your PXE server (/var/log/messages) to see if there are any errors. You can also troubleshoot by manually using tftp client and connecting to your TFTP Server to ensure you are able to pull down the files such as the boot.cfg by running the following command:

get esxi60u1/boot.cfg

For additional resources on scripted installation of ESXi also referred to as Kickstart, be sure to take a look here. I also would like to give a big shoutout and thanks to Tim Mann, one of the Engineers responsible for adding UEFI support into ESXi and for answering some of my questions while I was setting up my environment.

How to configure shared folders in VMware AppCatalyst?

A widely used feature in VMware's hosted products (Fusion & Workstation) is the shared folders capabilities which allows you to easily share files between the host system and your Virtual Machines. With the latest release of VMware's AppCatalyst TP2, shared folders is now officially supported and can be configured using the AppCatalyst's REST API. Below are the instructions on setting up shared folders for your VMs running in AppCatalyst. Thanks to Fabio for sharing the details.

I will assume that you already have a VM that is running under AppCatalyst. If you do not, you can run the following commands to quickly deploy a new VM called "photon" from the default VMware Photon OS template and retrieve the IP Address assigned to the VM (which we will make use later).

/opt/vmware/appcatalyst/bin/appcatalyst vm create photon
/opt/vmware/appcatalyst/bin/appcatalyst vmpower on photon
/opt/vmware/appcatalyst/bin/appcatalyst guest getip photon

If you have not used AppCatalyst before, be sure to check out this getting started guide here.

Step 1 - Open a terminal and start the AppCatalyst Daemon by running the following command:


Step 2 - Open a browser to connect to the following URL https://localhost:8080 to access the REST API explorer provided by Swagger.

Step 3 - Expand the POST /vms/{id}/folders which is the API to configure shared folders for a particular VM. You will need to fill in the id property which specifies the name of the VM and the state property defines the shared folder configuration.

Here is an example for creating a shared folder in the guest called "shared-folder" and mapping that to the host folder under /Users/lamw/Development. The flags property specifies how the folder will be accessed by the VM. Currently, there is only one flag implemented which is "4" and means read/write access. In the future there maybe more flags implemented for different types of access.


Once you have entered the information into the UI, to execute the API request you just need to click on the "Try it out" button at the bottom. If the operation was successful, you should get back a 200 response from the UI.

Screen Shot 2015-10-02 at 9.33.31 AM
Note: The guestPath property is not actually a path in the VM but rather the name of the directory which will map to the host shared folder.

Step 4 - We can now login to our VM to confirm that the shared folder has been configured. If you are using the default Photon template, you can login by specifying the default SSH keys included with AppCatalyst and the IP Address of the Photon VM we deployed earlier.

Here is the command to login (be sure to replace with the IP Address from your enviornment):

ssh -i /opt/vmware/appcatalyst/etc/appcatalyst_insecure_ssh_key photon@

Step 5 - Once logged in, if we now look under /mnt/hgfs directory, we should now see the shared directory automatically created and mapped to the host shared folder we created earlier.

Screen Shot 2015-10-02 at 9.35.13 AM
If you wish to remove shared folders from a VM, you can do so by executing the DELETE /vms/{id}/folders/{folderId} API and specifying the id for both the VM and folder. Hopefully in the future, shared folders can also be easily consumed through the simple AppCatalyst CLI, but for now you can do so using the REST API.

Erasing existing disk partitions now available in the vSphere Web Client (vSphere 6.0 Update 1)

One of the primary challenges when trying re-purpose existing storage devices is ensuring that all data and existing partitions have been completely removed. Often times, customers end up resorting to third-party tools like GParted which requires you to boot your server into the LiveCD before you can remove the existing partitions. This is less than ideal, especially if you need to perform this operation across multiple systems.

For customers who wish to re-purpose their existing storage devices for other use, including VSAN, there is now a new UI option in the vSphere Web Client introduced in vSphere 6.0 Update 1 to help assist with this procedure. I had not seen anyone talk about this feature yet and figure I would share some details as this is something I have heard customers ask for in the past. You can find this new option (icon with disk and eraser) by clicking onto a specific ESXi host and then selecting the Manage->Storage Adapters and then highlighting the specific storage device you wish to erase as seen in the screenshot below.

Once the erase partition icon or action is selected, you will then be presented with a summary of the existing partitions on the disk and then prompted to confirm that you wish to delete ALL partitions on the disk.

After the operation has successfully completed, you can now re-purpose the storage device for other use like VSAN!

For those of you who are interested from an Automation standpoint, this UI operation actually makes use of an existing vSphere API that has been for quite some time called updateDiskPartitions() under the StorageSystem manager of an ESXi host. To erase all partitions, you simply pass in an empty spec to the API method.

In addition, I also want to quickly mention that you will also have the ability to edit and erase existing disk partitions using the ESXi Embedded Host Client Fling which will be available in a future update. Below is a quick screenshot on what that would look like. 


Handy tidbits & workarounds for the VCS to VCSA Migration Fling

The VCS (Windows VC) to VCSA Migration Fling has been out for a little over 6 months and the response from customers thus far has simply been phenomenal. We have also received some great feedback (200+ comments) from customers who have tried out the Fling in either a Dev/Test environment and some even in their production environment for those that are a bit more on the adventurous side. I have also had the pleasure in talking to some of these customers who have been successful in migrating off of their Windows vCenter Server (both large and small) and onto the vCenter Server Appliance (VCSA) and sharing additional feedback they may have about the Fling and how we can further improve.

Given the popularity of this topic, I thought it would be useful to aggregate some of the learning's, tidbits and workarounds that have been discovered in the past 6months to help any new or even existing users who might be interested in trying out the Fling. We really do appreciate all the feedback that everyone has given in the various forms and in fact, several of the workarounds were ones provided by our customers. As you know, the Fling today is not currently officially supported, however the feedback has really helped our PM/Engineering team. In fact, you can even get a sneak peak at an early Tech Preview we did at VMworld here to give you an idea on how some of your feedback has influenced a feature that may or may not be out in the near future 😉

Tidbit 1 Microsoft Windows 2012 is currently not supported.
Additional Info There is a known winexe bug which is affects migrating from this specific OS platform.
Workaround Engineering has a fix for this and is currently in the process of testing the fix along with legal review. There is not an ETA due to the review but we hope to release an update to Fling that includes this fix very soon. Stay tuned!
Tidbit 2 Use of non-default (custom) ports on Microsoft SQL Server Database is not supported
Additional Info The Fling currently assumes the SQL Server Database is running on port 1433
Workaround Engineering has a fix for this and is currently in the process of testing the fix along with legal review. There is not an ETA due to the review but we hope to release an update to Fling that includes this fix very soon. Stay tuned!
Tidbit 3 Use of an Embedded Microsoft SQL Server or Microsoft SQL Express Database on the vCenter Server is not supported
Additional Info Since the source Windows vCenter Server must be powered off during the database migration; running the database on the same source vCenter Server is not possible.
Workaround One option is to re-ip the source Windows vCenter Server and ensuring the vCenter Server service is completely disable which would allow the Migration Appliance to communicate with the database. This is not ideal as you are modifying the source Windows vCenter Server but has worked in our testing. Second option that several other customers have recommended instead is to export the vCenter Server Database to a single instance of a Microsoft SQL Server or Microsoft SQL Express and that has worked really well.
Tidbit 4 Clustered database such as Microsoft Clustering Services (MSCS) is not supported
Additional Info There have been issues from some customers when trying to connect to an instance of the vCenter Server Database behind an MSCS Cluster.
Workaround Exporting the vCenter Server Database to a single instance of a Microsoft SQL Server or Microsoft SQL Express and then using the Fling has worked for several customers.
Tidbit 5 Issues connecting to a non-default named instance (e.g. SERVERNAME\VCENTER) of the vCenter Server Database.
Additional Info Some customers have had issues with the connection string to a non-default named instance of the vCenter Server Database during the database migration portion of the Fling.
Workaround A solution that was identified by a customer used the following: http://stackoverflow.com/a/11921896/2668394
Tidbit 6 Upgrade to VCSA 6.0 after migrating from Windows vCenter Server 5.5 to VCSA 5.5 fails
Additional Info You see the following error "Extra sequences: vpx_host_cnx_seq;" in /var/log/vmware/upgrade/vcdb_req.err during the upgrade to VCSA 6.0. These sequences are only found and valid in a Microsoft SQL Server Database and are not relevant in an vPostgres Database and just simply need to be dropped as they are not used at all.
Workaround Login to the VCSA 6.0 appliance as root and run the following command: /opt/vmware/vpostgres/current/bin/psql -U postgres -d VCDB -c "drop sequence if exists vpx_host_cnx_seq cascade"

If you are running into issues while through the the migration, one thing you can do is login to the Migration Appliance and go to another virtual console (ALT+F2) and view the Migration logs  under /var/log/migrate.log SSH is currently not installed by default. If you wish to pull out the logs for additional support, you can install which will require internet access and you can do so by running the following commands:

sudo apt-get -y update
sudo apt-get -y install openssh-server
sudo /etc/init.d/ssh start

The credentials to the Migration Appliance is vmware/vmware

Lastly, if there are other tidbits or workarounds that you would like to share, feel free to leave a comment and I will get it added to the list.

Help support an awesome cause: College of Adaptive Arts

Working at VMware, I get the opportunity to collaborate with some of the most passionate Engineers, Technologists and Customer Advocates around. In addition to solving hard and technical problems for our customers, my colleagues are also very passionate philanthropists who are always looking for ways to give back not only financially but by also donating their time, talent and creativity to help others. They all truly embrace the VMware EPIC^2 values.

About a month ago, I was having a conversation with a good friend of mine, Minh Dao who works as a Program Manager within the Storage and Availability Business Unit at VMware. He was telling me a story about an organization he was recently involved with called the College of Adaptive Arts. This organization provides young adults with special needs with the same opportunity that many of us take for granted after graduating from high school, which is a higher education in college.

Screen Shot 2015-09-21 at 3.27.57 PM
Minh was actually introduced to the program by Suchitha Chetia, a Director of QE who has been involved with the organization for some time. She had invited Minh out one weekend to their San Jose campus to see first hand the work of this organization. He told me he was really moved by what he had seen and experienced. These young adults just wanted to make a difference in the world just like any other young adult would. They dream big but often lack the resources to achieve those dreams. As with any higher education, it is not cheap. CAA offers classes in different disciplines such as communication, arts and music, TV and film, dance, etc. Funding for each discipline for an entire year costs about $15,000 USD. This year, the CAA will be hosting its 4th Annual Golf Classic on Monday, October 5th at the Santa Teresa Golf Club. It’s a joyful and invigorating day of golf whereby a group of three plays a round of golf with a golfer from the College of Adaptive Arts golf team. All funds raised will go to support various programs offered by CAA for the enrolled students.

Suchitha and Minh have created a VMware team to help raise funds for the CAA Organization. In addition to donating myself and supporting this awesome cause, I wanted to also share their story with my readers as I know many of you are also passionate philanthropists yourselves. Currently, Minh has set a goal of $1,500 but I am hopeful that we can easily blow that out of the water with your help. Any amount will help and will go a long way and the last day to donate is Oct 5th. Thank you in advance.

Note: For any VMware Employees who donate, please remember to submit a gift match through the VMware Foundation. This is a quick and easy way to double the donation!

How to Donate: