esxi 6.0

Automatically retrieve CVE CVSS score for all ESXi security bulletins

07/20/2018 by William Lam 9 Comments

I always enjoying learning new things, especially when it is outside of my immediate domain expertise and if I can thrown in some Automation to help solve a solution, it is a win for everyone. I bring this up because, yesterday I had noticed an interesting question from one of our field folks where their customer is looking to implement a process for applying ESXi security patches to help determine compliance timeline (e.g. when a specific security update will be applied to infrastructure).

To do this, the customer would like to use the Common Vulnerability Scoring System (CVSS) score which ranges from 0-10, 0 being low and 10 being high. The CVSS score is part of the Common Vulnerabilities and Exposures (CVE) which is also referenced for every ESXi security patch (bulletin) that is published by VMware. The question that came up was how easily it would be to determine the CVSS score for a given ESXi security patch. First, I will outline the "manual" process and once that is understood, I will demonstrate an automated solution which customers can take advantage of to easily retrieve this information for all ESXi security patches.

Auditing & Automating Disabled Protocols (TLS/SSLv3) for ESXi 6.0u3 & 6.5 using PowerCLI

05/09/2017 by William Lam 27 Comments

A couple of weeks back, I had received a question from one of our TAMs in regards to automating the disablement of specific TLS/SSL protocols for their ESXi 6.0 Update 3 hosts. As of vSphere 6.0 Update 3 and vSphere 6.5, customers now have the ability to completely disable TLS 1.0, TLS 1.1 and SSLv3 using the new TLS Reconfiguration Tool. Mike Foley did a nice write up here if you are interested in more details.

The TLS Reconfiguration Tool works well if you have the same version of vSphere for both your vCenter Server and ESXi host, but has challenges when you are in a mixed environment like this particular customer. In their environment, they are running vCenter Server 6.5 and ESXi 6.5 Update 3 which prevented them from using the TLS Reconfiguration Tool as this is a limitation with the tool today.

UPDATE (05/11/17) - Added support for ESXi 6.5 hosts as well

Given the TLS Reconfiguration Tool was written in Python, I was able to take a closer look at its implementation and I found that the settings that controlled the disabled protocols were just merely a few ESXi Advanced Settings which meant that this could be automated using standard vSphere Automation Tools that our customers were already familiar with. As part of this exercise, I also discovered the tool currently does NOT support disabling TLS/SSLv3 protocols for the Small Footprint CIM Broker (SFCB) service which is also required if you want to be in full compliance for a particular TLS protocol. Although there is not a direct SFCB API that allows you to manage the sfcb.cfg configuration file, there is still a way we can automate this without requiring SSH to the ESXi host which would technically be the alternative. Lastly, I was a bit surprised to see the TLS Reconfiguration Tool did not have a "query" option for listing the current disabled protocols for all ESXi hosts, but they do have it for vCenter Server itself.

To help this particular customer and others who may have specific TLS compliance requirements, I have created the following PowerCLI script called ESXiDisableProtocolConfiguration.ps1 which includes the following two functions:

Get-ESXiDPC - Retrieve the current disabled protocols for all ESXi hosts within a vSphere Cluster
Set-ESXiDPC - Configure the specific disabled protocols for all ESXi hosts within a vSphere Cluster

Copying files from a USB (FAT32 or NTFS) device to ESXi

01/24/2017 by William Lam 11 Comments

It is not uncommon, especially in troubleshooting scenarios where you might find yourself needing to transfer files to or from an ESXi host using a USB device as it may not be reachable on the network. Another common case for directly attaching a USB device to an ESXi host is to transfer a large amount of Virtual Machines that were exported from another system and rather than streaming the content from your desktop, you may want to connect it directly to ESXi host. In fact, I had this very use case when I was a customer after we had acquired a company and needed to transfer their assets to our infrastructure. The IT admins just copied everything onto a USB device and then shipped us the drive for processing.

Historically, it was understood that ESXi could only access a USB device (requires disabling the USB arbitrator service) if it contains a FAT16 partition which are then automatically mounted under the /vmfs/volumes/ path. The biggest issue with FAT16 is that the size of the partition has to be <=2GB which severely limits its use for larger files. Another alternative that came up in recent years years is that you could run VMFS on a USB device, but that obviously would require you to format the USB device with VMFS and it would only be readable between ESXi hosts. If you were looking for something more generic like FAT32 which supports a larger partition size, it was assumed this was not possible, at least I was under that impression.

It was only recently as part of a project I had been working on where I was re-visiting this topic that I had discovered that other partition types such as FAT32 and even NTFS from a USB device could actually be accessed by ESXi 6.x. The assumption that I and probably others had made was that just because the partitions were not visible or mounted by ESXi, it does not mean the underlying USB device would also not be accessible. To access a FAT32 partition from a USB device in ESXi, you can use the mcopy utility from the ESXi Shell and for accessing an NTFS partition from a USB device in ESXi, you can use ntfscat utility. It actually took me some trial/error to get the correct syntax, but you can see how to use the utilities below.

Functional USB-C Ethernet Adapter for ESXi 5.5, 6.0 & 6.5

01/22/2017 by William Lam 22 Comments

While attending an offsite this week, there were some discussions amongst my colleagues about their new Apple Mac Pro and its USB-C only ports. The discussion was completely unrelated to work, however that did get me thinking about the USB-C peripheral market and specifically their ethernet adapters. While searching online, I came across several new USB-C to gigabit ethernet adapters that were now available and one in particular that was very interesting, was the Plugable USB-C to 10/100/1000 Gigabit Ethernet LAN Network Adapter. What caught my eye about this specific network adapter was that it uses the exact same ASIX AX88179 driver as my USB 3.0 to Ethernet Adapter ESXi VIBs were built off of! There was a good chance this might just work.

As you can probably guess, I was pretty excited and quickly ordered one of the Plugable USB-C Ethernet Adapters. The next challenge was getting access to a system that has a USB-C port. After asking around, I finally got my hands on a Dell XPS 13 which has a USB-C port that I could use for a few days. Funny enough, the Dell laptop only has USB 3.0 and USB-C ports, so the first challenge was to disable Secure Boot since I had built a custom ESXi 6.5 image that included my USB 3.0 Ethernet Adapter VIB. Below are the ESXi VIBs or offline bundles that will be required for this solution.

UPDATE (02/12/19) - A new VMware Native Driver for USB-based NICs has just been released, please use this driver going forward.

Please see this blog post for more detailed instructions on installing the VIB as well as accessing the vusbX pNIC.

Disclaimer: This is not officially supported by VMware. Use at your own risk.

Once I got ESXi up and running, I was disappointed to see that the USB-C device was not being detected. I had tried a few more things but nothing worked and I decided to sleep on it. The next morning, I realize maybe there was some additional settings that needed to be tweaked in the BIOS. With a bit of trial/error, I found out that you needed to enable the "Thunderbolt Boot Support" which apparently is disabled by default, at least on this Dell system. Below is a screenshot of the BIOS USB/Thunderbolt Settings and this was the only change required from the system defaults.

Once I rebooted, I immediately saw the link up on the USB-C device while ESXi was starting up 😀

Using vSphere Auto Deploy to Netboot ESXi onto Apple Mac Hardware

01/17/2017 by William Lam 2 Comments

Last week I published an article that demonstrated for the first time on how to netboot an ESXi installation onto Apple Mac Hardware. As you can imagine, this was very exciting news for our VMware/Apple customers, who historically have not had this capability before. Customers can now automate and install ESXi over the network onto their Apple Mac Hardware just like you would for other non-Apple hardware.

With the ability to boot ESXi over the network for Apple Mac Hardware, it is now also possible for customers to take advantage of the vSphere Auto Deploy feature. Auto Deploy allows customers to easily and quickly provision ESXi hosts at scale and integrates directly with vCenter Server to automatically join and apply specific defined host configuration policies. This is a great time to check out Auto Deploy, especially with all the new enhancements that were introduced in vSphere 6.5 like custom script bundles for example.

Below are the instructions on how to setup Auto Deploy to work with Apple Mac Hardware.

How to Netboot install ESXi onto Apple Mac Hardware?

01/13/2017 by William Lam 10 Comments

The ability to perform an ESXi Scripted Installation over the network has been a basic capability for non-Apple hardware customers since the initial release of classic ESX. However, for customers who run ESXi on Apple Mac Hardware (first introduced in vSphere 5.0), being able to remotely boot and install ESXi over the network has not been possible and customers could only dream of this capability which many of us have probably taken for granted.

Unlike traditional scripted network installations which commonly uses Preboot eXecution Environment (PXE), Apple Mac Hardware actually uses its own developed Boot Service Discover Protocol (BSDP) which ESXi and other OSses do not support. In addition, there are very few DHCP servers that even support BSDP (at least this may have been true 4 years ago when I had initially inquired about this topic). It was expected that if you were going to Netboot (equivalent of PXE/Kickstart in the Apple world) a server that you would be running a Mac OS X system. Even if you had set this up, a Netboot installation was wildly different from a traditional PXE installation and it would be pretty difficult to near impossible to get it working with an ESXi image. With no real viable solution over the years, it was believed that a Netboot installation of ESXi onto Mac Hardware just may not be possible.

tl;dr - If you are interested in the background to the eventual solution, continue reading. If not and you just want the goods, jump down a bit further. Though, I do think it is pretty interesting and worth getting the full context 🙂