vSphere 6.5

Enhancements to Hybrid Linked Mode (HLM) in VMC using the new vCenter Cloud Gateway

10/04/2018 by William Lam Leave a Comment

It has been almost a year since VMware introduced the Hybrid Linked Mode (HLM) capability, which provides customers with a consistent operating experience for managing and consuming resources from both their on-premises and VMware Cloud on AWS (VMC) environments. Feedback from customers on HLM has been fantastic, especially when new or prospective VMC customers learn about HLM for the very first time. Customers were pleasantly surprised at how seamless the experience was when consuming VMC resources, using a familiar interface, the vSphere UI.

Here is a quick recap of what HLM provides today:

HLM allows customers to link a single VMC instance to a single on-prem SSO Domain which can contain one or more vCenter Servers (Enhanced Linked Mode) while maintaining separate administrative domains (e.g. on-prem user is Administrator while VMC user is CloudAdmin only)
SSO Domains will be different between on-prem and VMC, however it is a 1:1 relationship
A trust is established where the on-prem vCenter Server trusts the incoming connections from VMC as they share the same Active Directory identity source. Data is sync'ed uni-directionally from on-prem to VMC
Can be configured at any point in the on-prem vCenter Server lifecycle, no restrictions to initial install and can easily be un-linked unlike ELM
Both Embedded & External vCenter Server deployments are supported
HLM supports different versions of vCenter Server between on-prem (6.5d+) and VMC, especially as VMC will almost always run a newer version of vSphere
Users MUST login to VMC vCenter Server for single-pane of glass management (H5 Client supported only), logging into on-prem vCenter Server will NOT show VMC vCenter Server
Roles are NOT replicated due to the restrictive access model in VMC

Automatically retrieve CVE CVSS score for all ESXi security bulletins

07/20/2018 by William Lam 9 Comments

I always enjoying learning new things, especially when it is outside of my immediate domain expertise and if I can thrown in some Automation to help solve a solution, it is a win for everyone. I bring this up because, yesterday I had noticed an interesting question from one of our field folks where their customer is looking to implement a process for applying ESXi security patches to help determine compliance timeline (e.g. when a specific security update will be applied to infrastructure).

To do this, the customer would like to use the Common Vulnerability Scoring System (CVSS) score which ranges from 0-10, 0 being low and 10 being high. The CVSS score is part of the Common Vulnerabilities and Exposures (CVE) which is also referenced for every ESXi security patch (bulletin) that is published by VMware. The question that came up was how easily it would be to determine the CVSS score for a given ESXi security patch. First, I will outline the "manual" process and once that is understood, I will demonstrate an automated solution which customers can take advantage of to easily retrieve this information for all ESXi security patches.

Using ESXi Kickstart %firstboot with Secure Boot

06/26/2018 by William Lam 5 Comments

If you install ESXi via a Kickstart script and make use of the %firstboot option to execute commands on the first boot of the ESXi host after installation, you should be aware of its incompatibility with the Secure Boot feature. If you install ESXi where Secure Boot is enabled, the Kickstart will install ESXi normally only execute up to the %post section. However, it will not execute the %firstboot scripts and if you look at the /var/log/kickstart.log after the host boots, you should see the following message:

INFO UEFI Secure Boot Enabled, skipping execution of /var/lib/vmware/firstboot/001.firstboot_001

If you have Secure Boot enabled, %firstboot is not supported. The reason for this is Secure Boot mandates only known tardisks which can hold executable scripts, and a kickstart script is an unknown source so it can not run when Secure Boot is enabled. If you wish to continue using %firstboot scripts, the only option is to disable Secure Boot and then re-enable it after the installation. A preferred alternative is to convert your %firstboot logic into an external script which can then be applied using the vSphere API (recommended method) and this way you can still customize your ESXi host after the initial installations. I have already filed an internal documentation bug to add a note regarding Secure Boot and %firstboot, hopefully that will roll out with the net documentation refresh.

Quick Tip – What hashing algorithm is supported for ESXi Kickstart password?

05/21/2018 by William Lam 1 Comment

I had a question the other day asking whether the encrypted password which can be specified within an ESXi Kickstart file (denoted by the --isencrypted flag) can use a different hashing algorithm other than MD5? The answer is absolutely yes. In fact, MD5 as a default hashing algorithm has NOT been used for a number of releases, probably dating back to classic ESX (you know, the version that had the Service Console).

For all recent releases of ESXi including 5.5 to 6.7, the default hashing algorithm has been SHA512 for quite some time now. Below are two ways in which you can check which default hashing algorithm is currently being used:

Option 1 - SSH to ESXi host and take a look at /etc/pam.d/passwd

Option 2 - SSH to ESXi host and take a look at /etc/shadow and look at the field prior to the salt.

As a reference:

$1$ - MD5
$5$ - SHA256
$6$ - SHA512

Bulk VM Migration using new Cross vCenter vMotion Utility Fling

12/20/2017 by William Lam 28 Comments

Over the last few years, I have spoken to a number of customers who have greatly benefited from the ability to live migrate Virtual Machines across different vCenter Servers that are NOT part of the same vCenter Single Sign-On (SSO) Domain, which I had first shared back in 2015 here and here. This extended capability of the Cross vCenter vMotion feature enabled customers to solve new use cases that were challenging, especially for scenarios such as Datacenter migration, consolidation or even migrating existing workloads from their current environment into new SDDC deployments such as VMware Cloud Foundation (VCF) as an example.

Although customers could initiate Cross vCenter vMotions using the vSphere API which included PowerCLI (Move-VM cmdlet was enhanced in 6.5, more details here), the overall experience was still not as friendly. This was especially true for customers who may only have a small number of VMs to migrate and prefer a UI-based interface rather than an API/CLI only option. In addition, for large number of VM migrations, there was not an easy way to perform "batch" VM migrations that was easily consumable for folks who may not have a strong background in Automation or the vSphere APIs.

Today, I am pleased to share a new VMware Fling called the Cross vCenter Migration Utility that will help simplify the consumption of initiating VM migration(s) across different vCenter Servers, especially between dispart SSO Domains where a graphical interface was not available. This solution was developed out of our VMware Cloud Foundation (VCF) Engineering group which is part of the Integrated Systems Business Unit at VMware. I had spoken to a number of folks within the group about the extended Cross vCenter vMotion capability and I was super excited when I heard they were planning to release this tool as a Fling and make it available to all customers. I was fortunately to have been involved in the project alongside the Engineering lead Vishal Gupta and we are excited that we can finally talk about this project and see how customers will be using this new tool.

UPDATE (05/07/18) - The Fling has just been updated to 2.0 with the following new features:

Added support to select individual host as the placement target
Added support for migrating VMs with shared datastore
Added clone functionality in addition to relocate
Added resource summary details for placement targets
Added a prompt to verify site thumbprint during SSL verification
Added a link to refresh vm list in the inventory view
Updated REST APIs to add operation type parameter

Cross vCenter Migration Utility Fling

Cross vCenter vMotion Requirements: KB 2106952

Download Fling here

Features

Completely UI-driven workflow for VM migration
Provides REST API for managing migration operations
Works with vCenter not part of the same SSO domain
Supports both live/cold migration of VMs
Batch migration of multiple VMs in parallel
Flexible network mappings b/w source and destination sites

Deployment models for vSphere Content Library

12/01/2017 by William Lam 4 Comments

When talking to customers about vSphere Content Library deployments, one question I normally get is how best deploy Content Library for optimal workload deployment, especially in scenarios where remote or branch offices are involved? There are two main deployment models for vSphere Content Library as the title has alluded to. The main difference between the two is whether you have a single vCenter Server or if you have multiple vCenter Servers, with each managing its own vSphere infrastructure?

Lets refer to the single vCenter Server case as Scenario 1 and the multi-vCenter Server case as Scenario 2 and below are the two scenarios outlined with additional details.

Scenario 1 (Single vCenter Server):

In this scenario, which is a fairly common deployment for many smaller to mid-size organizations, where you only have a single or very few vCenter Server(s). They are used to manage several remote locations which only consists of ESXi hosts running at each of the locations and storage local to the site is available. In addition, there are several expected behaviors of the content itself which I have formulated into the following table below:

Content Management	Content Distribution	Content Deployment
Centrally Managed	Sync across the WAN	Workloads stored and deployed locally

For this type of an environment, you would first setup a published library which stores all the content that you wish to distribute across your remote sites. Next, you would create subscriber library(s) consuming the published library, but instead of storing the replicated content locally, it is actually stored at each of the remote locations and their respective vSphere Datastore(s). This ensures that content is synchronized from our published library out to each of the remote locations, but when content is requested for deployment, the traffic is local to the site rather than going across the WAN.

In the above scenario, since there is only a single vCenter Server, if it ever becomes unavailable then provisioning and management to the remote location will also be unavailable. This is the expected behavior regardless if Content Library is configured.