Auditing/Logging vCenter Server authentication & authorization activities

Recently, I have seen an increase in the number of requests from our field and customers inquiring about logging various vCenter Server authentication and authorization activities. The topics vary from identifying which log files contain which activities to to why some of this information is not available in the vCenter Server Events UI or why they are available else where. In most of these cases, customers were also looking for a way to forward these activities to their remote syslog infrastructure for auditing and tracking purposes whether that is using vRealize Log Insight (which all vSphere customers get 25 free OSI licenses!) or some other logging solution.

Having explored this topic lightly in the past and given the amount of interests, I thought I would dive a bit deeper and look at some of the common authentication and authorization workflows and provide examples of what the log entries look like and where you can find them. However, before jumping right in, I think is is worth spending a few minutes looking at the history of authentication (commonly referred to as AuthN) and authorization (commonly referred to as AuthZ) for vCenter Server and where we had started from and where we are at today to give you the full context.

History of vCenter Server AuthN/AuthZ

Prior to vSphere 5.1, vCenter Server handled both Authentication (AuthN) and Authorization (AuthZ). As a Client, you would connect directly to vCenter Server and the AuthN service will verify who you are whether that is a local account on the OS or an Active Directory user which required vCenter Server to be joined to your AD Domain. Once you have been authenticated, the AuthZ service will then take over and verify the privileges you have been assigned to perform specific operations within vCenter Server.

In vSphere 5.1, a new service was introduced called Single Sign-On (SSO) which now takes over for AuthN services from vCenter Server. Once authenticated, it will then allow you to connect to the vCenter Server which then handles AuthZ activities

Although it may not be apparent, one major implication is where are successful and failed authentications being logged? In the past, these would reside within vCenter Server since it handled both AuthN/Authz activities, vCenter Server even included specific authentication Events that can then be seen using the UI and/or API. However, with SSO in the picture, authentication is no longer in vCenter Server but with SSO. This is why when you have a failed login using the vSphere Web Client (Flex/H5) UI it does not show up in vCenter Server and it because the logging is done but within the SSO service (which now resides in the Platform Services Controller for more recent vCenter releases).

Continue reading

Workaround to deploy vSphere Integrated Containers 1.1 OVA using PowerCLI (SHA256 not supported)

Last week I had noticed several folks were having issues deploying the latest vSphere Integrated Containers (vIC) 1.1 OVA using PowerCLI. The following error message was observed when using the Get-OvfConfiguration cmdlet which is needed before importing an OVF/OVA:

PowerCLI doesn't support SHA256 hash codes in OVF manifest

As you probably have guessed, the issue is that PowerCLI currently does not support the SHA256 hashing algorithm, which the latest vIC OVA was generated with. I suspect this is probably related to the change with OVFTool 4.2 which now defaults to SHA256 which also has some implications on which vSphere UI you can use to import OVF/OVAs which I had written about here. As of today, PowerCLI currently only supports SHA1 and anything greater will not work. I have already reported this to Jake Robinson who is the PM for PowerCLI and hopefully this will get addressed in a future update.

In the meantime, you can deploy vIC using either the vSphere Web Client and/or ESXi Embedded Host Client, both support SHA256. If you wish to Automate the deployment of vIC, the only option right now is to convert the OVA from SHA256 to SHA1. You can easily do this by using OVFTool which is available on all OS platforms. If you already have downloaded the vCenter Server Appliance (VCSA) ISO, you can even make use of its bundled OVFTool in case you did not want to install OVFTool (You can find it under vcsa/ovftool in extracted ISO).

To convert the hashing algorithm, we just need to pass in our desire hash to the --shaAlgorithm parameter.

ovftool.exe --shaAlgorithm=SHA1 C:\Users\primp\Desktop\vic-v1.1.1_56a309fb.ova C:\Users\primp\Desktop\vic-v1.1.1_56a309fb-SHA1.ova

Once the conversion is done, you can delete the original vIC OVA and then use PowerCLI to import the new OVA just like you would with any other OVF/OVA!

How to move vSAN Datastore into a Folder?

A question came up the other day from a customer that wanted to move a vSAN-based datastore into a vSphere Folder, but was having some trouble with this operation. vSphere Folders are commonly used by administrators for organizational purposes and/or permission delegation. When the customer tried to move their vSAN datastore into a folder using the vSphere Web Client (applies to HTML5 Web Client as well), what they found was that nothing happens even though the UI indicates the operation should be possible with the (+) symbol.

I also saw the same behavior described by the customer and was curious if this was a UI only issue or a general limitation. To quickly verify, I decided to perform the operation using the vSphere API instead of the UI. Behind the scenes, the UI simply calls the MoveIntoFolder_Task() vSphere API which allows you to move various vSphere Inventory objects into a vSphere Folder. As many of you know by now, the vSphere APIs can be consumed in variety of "SDKs" or programming/scripting languages which includes PowerCLI. For PowerCLI users, this functionality is further simplified and abstracted away using the Move-Datastore cmdlet which I will be using in our demonstration.

In my setup, I have two vSAN Datastores, one from a vSphere 6.0u3 environment and another from vSphere 6.5. Lets say I want to move the 60u3 datastore to HR folder and 66 datastore to Engineering folder. The following PowerCLI snippet below does exactly that:

Move-Datastore -Datastore (Get-Datastore "vsanDatastore-60u3") -Destination (Get-Folder "HR")
Move-Datastore -Datastore (Get-Datastore "vsanDatastore-66") -Destination (Get-Folder "Engineering")

Using the vSphere API/PowerCLI, the operation looks to have been successful. Lets now going back to our vSphere Web Client and see if the operation actually went through?

Look at that, both our vSAN Datastores is now part of a vSphere Folder! This looks like a UI (Flex/H5) only issue and I have also confirmed that this will be fixed in a future update of vSphere. For now, if you need to move vSAN-based datastores into a vSphere Folder, simply use the vSphere API as a workaround.

Note: I also found that if you need to move the vSAN Datastore back to the Datacenter level, you will also need to invoke that operation using the vSphere API as the UI also prevents this operation.

Introducing Alexa to a few more VMware APIs

Over the weekend, while taking a break from putting together some furniture as it was my time for my daughters nap, I got that the chance to explore and create a new Alexa Skill which integrates with a few of VMware's APIs. This has been something I wanted to try out for some time but have not had any spare time. I had even purchased an Amazon Echo Dot but its just currently being used as a music player for the family. A couple of weeks back I saw an awesome blog post from Cody De Arkland where he demonstrates how to easily integrate the new vCenter Server 6.5 REST APIs into an Alexa Skill which can then be consumed using an Amazon Echo device.

Cody's write-up was fantastic and I was able to get everything up and running in about 20-25minutes with a few minor trial/error. It was great to see how easy it was for a non-developer like Cody to easily consume the new vCenter Server REST APIs which includes basic VM Management as well access to the VMware Management Appliance Interface or VAMI for short. Given Cody already did the hard work to create the initial Alexa integration, I figure it might be cool to extend his work and introduce Alexa to a few more VMware's APIs including the traditional vSphere API (SOAP) and the new vSAN Management API.

UPDATE (06/15/17) - Just added support for PowerCLI, it was a little tricky as Flask app is written in Python and so poor man workaround was to call Powershell/PowerCLI using subprocess.

Since Cody's integration module was written using Python, it was pretty simple to add support for both pyvmomi (vSphere SDK for Python) and vSAN Management SDK. To install pyvmomi, you can simply run

pip3 install pyvmomi

and for installing vSAN Management SDK, have a look at this blog post here.

Here is a quick video that I had recorded which demonstrates the use of both the vSphere API and vSAN Management API using my Amazon Echo.

You can find all my changes in this forked repo lamw/alexavsphereskill and make sure to follow Cody's blog post here for instructions on how to get setup. For those wondering if Cody will be publishing an Alexa Skill for general consumption, I know he is working on some awesome updates to make it even easier to consume. Here is a sneak peak at just some of the recent updates that Cody is working on ...

Stay tuned on this blog and Github repo for future updates!

One thing to note which I was not aware of until Cody mentioned it, is that once your Alexa Skill is built, you can directly access it from your own personal Amazon Echo without needing to publish it. You need to activate the Alexa Skill by saying "Alexa Start [APP-NAME]" where name is the name used in the "Invocation Name" field as shown in the screenshot below when setting up your Alexa Skill. I should also mention that if you decide to change the Alexa Skill name itself, which I had initially done and called it "vGhetto Control", make sure you update the Flask App name in to the same name (spaces are converted to underscores) or you will run into issues.

Managing ESXi Embedded Host Client settings

There was a question the other day about managing ESXi Embedded Host Client (EHC) settings which you can find by click on the logged-in username and navigating to the "Settings" section as shown in the screenshot below. Customers can manage things like the default VM Console used whether that is the HTML5 VMRC or the Standalone VMRC to auto-refresh and even sharing usage information to help improve the product back to VMware.

In addition to configuring the EHC settings within UI, you can also manage them via automation using the vSphere API and any one of your favorite vSphere SDK/CLIs. The EHC settings are exposed as a set of ESXi Advanced Settings as shown in the screenshot below. These settings are applied on a per-ESXi basis and NOT on a per-user basis.

Below is a table that summarizes the 7 different EHC settings which you can programmatically query to retrieve their current value, default value and whether they have been overridden simliar to what the UI provides today.

Key Description Default Value
UserVars.HostClientCEIPOptIn  Whether or not to opt-in for CEIP in Host Client: 0 for ask 1 for yes 2 for no 0
UserVars.HostClientDefaultConsole  Default console type in Host Client webmks
UserVars.HostClientEnableMOTDNotification  Whether or not to enable MOTD notification on login for Host Client 1
UserVars.HostClientEnableVisualEffects  Whether or not to enable visual effects for Host Client 1
UserVars.HostClientSessionTimeout  Default timeout for Host Client sessions in seconds 900
UserVars.HostClientShowOnlyRecentObjects  Whether or not to show only recent objects in Host Client 1
UserVars.HostClientWelcomeMessage  Welcome message displayed on login in Host Client  Welcome to {{hostname}}

Note: The language settings is based on your browser settings which you can overridden, but it looks like we may not have exposed that as a configurable setting via automation.