fling

New SDDC Certificate Replacement Fling

07/11/2018 by William Lam 8 Comments

Certificate lifecycle management is not something anyone looks forward to, it is time consuming and usually not automated. However, it is a necessity for many of our customers. The process gets even more challenging when needing replace certificates across multiple VMware products, not only careful orchestration but also properly reestablishing trust between product just adds another layer of operational complexity. Within the Integrated System Business Unit (ISBU) at VMware, which produces both the VMware Validated Design (VVD) and VMware Cloud Foundation (VCF), the team has been working on a way to simplify certificate management, not only for individual products (working with product teams) but also holistically at the VMware SDDC level.

This initially started with the development of a tool called Certificate Generation Utility (CertGen), which helps customers generate new certificates for various products within the VMware SDDC. Although it was developed for the VVD, any VMware customer who consumed products within the VVD, could also leverage this tool. We all know certificate generation can be a pain, but it is not as challenging or as complex as the actual certificate replacement process itself which is also fully documented by the VVD team here.

This is where the new Fling comes in, the SDDC Certificate Tool, which automates the manual steps outlined by the VVD and helps customers easily replace certificates that they have created (CertGen or another process) and automatically orchestrates this across the different products within the SDDC. The tool is command-line driven and uses a JSON configuration file which can contain all or a subset of the VMware SDDC products, which is great for supporting different environments and allows for easy source control. Extensive pre-checks are also built into the tool to validate the certificates themselves (e.g. expiry, chain validation, etc) also also preventing miss-match of information (e.g. SAN entries, number of nodes, etc) which then get compared against your actual environment before any changes are applied. The JSON also contains a section referred to as Service Accounts, which is merely other VMware product accounts that the tool supports to reestablish trust after replacing the certificate for given product.

Bulk VM Migration using new Cross vCenter vMotion Utility Fling

12/20/2017 by William Lam 28 Comments

Over the last few years, I have spoken to a number of customers who have greatly benefited from the ability to live migrate Virtual Machines across different vCenter Servers that are NOT part of the same vCenter Single Sign-On (SSO) Domain, which I had first shared back in 2015 here and here. This extended capability of the Cross vCenter vMotion feature enabled customers to solve new use cases that were challenging, especially for scenarios such as Datacenter migration, consolidation or even migrating existing workloads from their current environment into new SDDC deployments such as VMware Cloud Foundation (VCF) as an example.

Although customers could initiate Cross vCenter vMotions using the vSphere API which included PowerCLI (Move-VM cmdlet was enhanced in 6.5, more details here), the overall experience was still not as friendly. This was especially true for customers who may only have a small number of VMs to migrate and prefer a UI-based interface rather than an API/CLI only option. In addition, for large number of VM migrations, there was not an easy way to perform "batch" VM migrations that was easily consumable for folks who may not have a strong background in Automation or the vSphere APIs.

Today, I am pleased to share a new VMware Fling called the Cross vCenter Migration Utility that will help simplify the consumption of initiating VM migration(s) across different vCenter Servers, especially between dispart SSO Domains where a graphical interface was not available. This solution was developed out of our VMware Cloud Foundation (VCF) Engineering group which is part of the Integrated Systems Business Unit at VMware. I had spoken to a number of folks within the group about the extended Cross vCenter vMotion capability and I was super excited when I heard they were planning to release this tool as a Fling and make it available to all customers. I was fortunately to have been involved in the project alongside the Engineering lead Vishal Gupta and we are excited that we can finally talk about this project and see how customers will be using this new tool.

UPDATE (05/07/18) - The Fling has just been updated to 2.0 with the following new features:

Added support to select individual host as the placement target
Added support for migrating VMs with shared datastore
Added clone functionality in addition to relocate
Added resource summary details for placement targets
Added a prompt to verify site thumbprint during SSL verification
Added a link to refresh vm list in the inventory view
Updated REST APIs to add operation type parameter

Cross vCenter Migration Utility Fling

Cross vCenter vMotion Requirements: KB 2106952

Download Fling here

Features

Completely UI-driven workflow for VM migration
Provides REST API for managing migration operations
Works with vCenter not part of the same SSO domain
Supports both live/cold migration of VMs
Batch migration of multiple VMs in parallel
Flexible network mappings b/w source and destination sites

vSphere SDK for JavaScript Fling released

02/03/2016 by William Lam 1 Comment

The VMware Fling team has just released another cool new Fling, the vSphere SDK for JavaScript. This Fling is especially interesting as it provides the underlying SDK framework used by the popular ESXi Embedded Host Client Fling which was released back in August of last year. I came to learn about this project during last years internal R&D Innovation Offsite (RADIO) conference which is held annually and can be thought of as the VMworld conference for VMware employees.

One of the biggest highlight of the conference for me personally is checking out the expo floor where you get to see what other VMware Engineering teams have been working on whether it is the next big feature, product or new ideas that they might be thinking about. It was during my walk through that I met Rostislav Hristov, one of the Engineers who worked on the vSphere SDK for JavaScript. I was really impressed at what Rostislav built and luckily he was already in touch with the Embedded Host Client Engineers to see how they could leverage his JavaScript SDK as the initial prototype had made calls directly using the vSphere MOB which was not very friendly to develop against.

There has been a number of improvements to the vSphere SDK for JavaScript since I last saw it and although the name contains "vSphere", it definitely supports more than just the vSphere API endpoint. In fact, with this single SDK you can interact with vCenter Server Single Sign-On (SSO) API, vCloud Suite API which covers capabilities like Tagging and Content Library as well as the Site Recovery Manager (SRM) APIs! For customers and partners that are looking to develop their own web portals or interfaces that can integrate with these APIs, this will be a handy tool to have.

To get started, the vSphere SDK for JavaScript contains a README file that contains additional instructions on setting up the SDK as well as a couple of samples that demonstrates each of the supported API endpoints:

vimService.js - Sample using the vSphere API
stsService.js - Sample using the SSO API
cisServices.js - Sample using the vCloud Suite API
srmService.js - Sample using the SRM API

Here is the command to run the vimService.js sample which will also require you to set the environmental variable NODE_TLS_REJECT_UNAUTHORIZED=0 if you are using the default VMware self-signed SSL Certificate.

NODE_TLS_REJECT_UNAUTHORIZED=0 babel-node samples/vimService.js

Once the sample has started up, you will be prompted with a URL to open in your browser. In the vimService.js example, you will be able to login to either a vCenter Server or ESXi host as seen in the screenshot below.

Once logged in, you should see a simple listing of the different inventory objects in your vSphere enviornment.

In the stsService.js sample, once logged in by specifying the address to your PSC/SSO instance, you should see that a SAML token was issued.

The cisService.js sample exercises several operations using a mixture of both the vSphere API as well as the new vCloud Suite API. It does require connecting to a vCenter Server 6.0 environment as it will be performing operations using the new vSphere Content Library feature as well as some VM operations. Do not worry, once the operations have been completed, the script will automatically clean itself up. This is a great sample if you want to see how you could make use of the different APIs all through this single SDK.

I did not have an SRM environment up and running to test the srmService.js sample, but you can see from the code that it will list all of the recovery plans and their current state. For more details on how the individual APIs work, you can refer to the documentation included in the vSphere SDK for JavaScript or the official API documentation for the individual products. If you have any feedback or comments about this Fling, please leave a comment on the Fling site here as I am sure the Engineers would love to hear what you think!

Windows Server 2012 fix is out for the VCS to VCSA Migration Fling!

01/05/2016 by William Lam

Happy New Years everyone! Here is the first post of 2016 🙂

I know many of you have been anxiously waiting for a fix to the Winexe bug that causes the VCS to VCSA Migration Fling to fail when migrating from a vCenter Server that runs on Microsoft Windows Server 2012 (includes R2) OS to the VCSA. It has taken a bit longer than we all had hoped due to legal reviews and such, but today I am please to announce that we now have an updated release to the VCS to VCSA Migration appliance (v0.9.1). For customers using Windows Server 2012, WinRM will be used instead of Winexe due to the Windows bug that has been identified and will require a couple of additional configurations on the source Windows vCenter Server prior to starting the migration process. For more details, please refer to the documentation on the Fling site. Customers using Windows Server 2008, Winexe will continue to work as it does today.

In addition to this fix, we also now support specifying a custom port for the Microsoft SQL Server Database in case you had changed it from the default of 1433, which a few customers have asked for.

I would like to give a big thanks to both Mike and Ravi for working tirelessly on getting this fix out as soon as they could along with all the legal hurdles. Let us know if you have any feedback by either leaving a comment below or better yet, interact with the Engineers by leaving a comment on the Fling site!

Upgrading ESXi itself is now possible with the new Embedded Host Client v4

12/21/2015 by William Lam 13 Comments

It looks like we just got an early Christmas gift from Etienne, George and the VMware Flings team with the release of v4 of the ESXi Embedded Host Client (EHC). One of the new features that I am most excited about is the ability to upgrade ESXi itself using the "Update" feature which was first introduced in EHC v3. In the previous release of EHC, you could install or patch any ESXi VIB using this interface which I had blogged about here. With the latest release, Etienne has expanded its functionality to not only allow for a VIB URL but also a metadata.zip URL which ESXi offline bundles contain.

Here is an example of using EHC v4 to upgrade an ESXi 6.0 host to ESXi 6.0 Update 1:

Step 1 - Download ESXi offline bundle of choice, in this example we are downloading the offline bundle for ESXi 6.0 Update 1:

Step 2 - You will need to extract the contents of the offline bundle and then upload the content to either a vSphere Datastore or an HTTP accessible URL as shown in the screenshots below.

Currently it is not possible to upload an entire directory using the Datastore Browser with EHC. This is something I have reported internally and hopefully will be fixed in a future update.

Step 3 - Click on the "Help" menu on the upper right hand corner of the Embedded Host Client and select the "Update" option which should prompt you to specify a URL. You can either specify a local datastore path to the metadata.zip such as the following:

/vmfs/volumes/datastore1/ESXi600-201510001/metadata.zip

or you can specify an HTTP(s) URL to the metadata.zip like the following:

http://192.168.1.180/ESXi600-201510001/metadata.zip

Step 4 - Once you click the "Update" button, you will be asked to confirm and then you should see task kicked off called "Install Host Patch V2". Once the task has completed, you will need to reboot the ESXi host for the changes to take effect. If you are interested in viewing the update logs without having to log in via SSH, you can click on the "Monitor" tab of the Host option and under "Logs", click on /var/log/esxupdate.log to see the progress.

If you have more than one or two hosts, I definitely recommend uploading the extracted offline bundle onto a webserver or a shared vSphere Datastore which you can the access from multiple ESXi hosts. This is a very nice way to easily upgrade your ESXi hosts without having to go into the ESXi Shell or enable SSH. Pretty neat feature if you ask me! There are many other new enhancements included in this release, be sure to give them a try and let us know if you have any feedback by leaving a comment on my blog or better yet, directly on the Fling site.

What's new in Embedded Host Client v4

Host
- Ability to change host acceptance level
- Ability to edit lock down exception users
- Ability to edit system swap settings
VM
- VM list has been optimized for performance, reducing data download by a factor of 5.
- Ability to edit VM advance options
- Ability to edit VM video adapter settings
- Add a PCI pass through device (unable to remove device though)
- SRIOV support for Network card devices
- Ability to change browser console keyboard layout (Japanese and German are the currently supported layouts)
- Cmd+a or Ctrl+a to select all VMs in list
- Soft-power off and reset if Tools is installed is now supported
General
- New Tools and links menu under Help
- Update mechanism can now take a URL or data store path to an metadata zipfile, allowing to update ESXi itself
- Localization and internationalization (French, Spanish, Japanese, German, Chinese (traditional and simplified) and Korean
- Ability to disable session timeout
- A huge number of bugfixes and minor improvements

Neat way of installing or updating any VIB using just the ESXi Embedded Host Client

11/10/2015 by William Lam 3 Comments

A couple of months back I had tossed out an idea on Twitter asking if others would like to see an automatic update mechanism built into the ESXi Embedded Host Client which would allow users to easily update to newer releases of the Fling versus the current method which requires copying the VIB and then running command in the ESXi Shell.

Wonder if its just me,but would others like to see an automatic update mechanism in the ESXi Embedded Host Client UI? pic.twitter.com/R9KFMOE4zu

— William Lam (@lamw) August 26, 2015

To no surprise, the feedback was an astounding yes! Literally within a couple of hours, Etienne Le Sueur, one of the two VMware Engineers working on the Fling shared a screenshot that demonstrated that this would possible. The first release of this feature would simply ask for the URL to the updated ESXi Embedded Host Client VIB and this was included in the v3 release of the Fling.

One additional tidbit that Etienne had shared was that the way this feature was implemented, it was not only limited to Embedded Host Client VIB but you could do this for any ESXi VIB. This is done by using the vSphere API and calling into the InstallHostPatchV2_Task() method which allows you to install or update an ESXi VIB from a URL source. Most recently, there a twitter conversation between myself, Etienne and Christian Mohn on how this capability could be further extended to include updating ESXi itself which can either be from an Image Profile or offline bundle. For those with a detailed eye, you may have noticed that the same API method can also support an offline bundle URL which would make this possible. As of right now, the feature is actually included in an internal build of the Embedded Host Client, but perhaps we will see this in a future update of the Embedded Host Client? 😉

Going back to the original topic of this blog post, to use the VIB install/update mechanism, you would need to first upload the ESXi VIB to an HTTP Server and then specify the URL. This is fine if you have an existing HTTP Server but if you do not, it is sort of a pain and though there are other methods like uploading directly to the ESXi's python based HTTP Server as mentioned by Christian, it would still require using something like SCP which is an additional step. My initial goal and hope was to be able to install or update an ESXi VIB or ESXi itself using purely the Embedded Host Client. This would keep things simple and not require things like SSH to be enabled on the ESXi host.

After a bit of brainstorming with Etienne, he actually found a super clever way of accomplishing this after our conversation. The idea I had was to make use of the ESXi Datastore to store the VIB which can be uploaded through the Embedded Host Client. By default, there is also an HTTP based interface to the datastore, however it requires authentication which would be a problem. The neat idea that was suggested was why not try to specify the local VMFS path to the ESXi VIB (e.g./vmfs/volumes/datastore1/my.vib)? It turns out that this actually works as well!

With just two easy steps, you can now upload an ESXi VIB and then install/update all using just the Embedded Host Client with no additional dependencies

Step 1 - Navigate to the Datastore section in the Embedded Host Client and then upload the ESXi VIB that you wish to install or update.

Step 2 - To install/update the VIB, click on Help in the upper right hand corner of the Embedded Host Client and select the "Update" option. Specify the local VMFS path to ESXi VIB and then click on Update to apply.

Note: A reboot may be required after applying a new VIB. It will be your responsibility to shutdown the VMs and reboot the ESXi host for changes to go into effect if required.

At this point, you should also see a task kicked off applying the VIB. If there are any errors thrown, they will be displayed else you should see a successful task completion. For educational purposes, here is a quick screenshot of /var/log/esxupdate.log showing the VIB being applied, this can be used for further troubleshooting if required.

Hope you enjoyed this neat little trick and with just two easy steps you can install or update any ESXi VIB using the Embedded Host Client without additional dependencies or enabling SSH on the ESXi host.