virtuallyGhetto

vSphere SDK

An update on how to retrieve useful information from a vSphere login?

by 4 Comments

There was an internal Socialcast question today in which the answer could be found in my how to identify the origin of a vSphere login article. After responding to the question, I had realized that I wrote that article almost 6 years ago and what is even more crazy is that it is still very applicable today. The article explains how you can identify a vSphere login by enabling the "trivial" logging option in vCenter Server (extremely verbose, so please use with caution). Once enabled, you can go through the vpxd.log file and find things about a user login such as the the IP Address of the client as well as the type of vSphere interface they had used to login to whether that is using the vSphere C# Client or PowerCLI for example. Although this extracted information can be very useful, the process to retrieve this is not very ideal, especially having to increase your vCenter Server logging verbosity to the extreme which can force other more critical log events to roll over.

Given that this article written back when vSphere 4.1 was still the current release, I figure I should give the process another look to see if there was a better method in retrieving this information. While quickly browsing around the SessionManager object and specifically the UserSession property, I noticed there have been quite a few enhancements that were introduced in vSphere 5.1. It looks like you can now easily retrieve things like the User Agent, IP Address of the client as well as the number of API invocations for anyone who is currently logged into a given vSphere environment. Perhaps someone internally saw my blog post and thought it would be useful to add these properties directly into the vSphere API rather than poking around in the verbose logs 😀

To exercise these new vSphere APIs, I have create a quick PowerCLI function called Get-vSphereLogins The script will iterate through all currently logged in vSphere sessions and provide the following output: Username, IP Address, API Count & Login Time. It also excludes the current session initiating the query as well as any of the VC Extension logins. Here is a screenshot of my environment using several different vSphere API interfaces to login to my vSphere environment:

retreiving-useful-information-about-vsphere-login-0
With the information above, not only can you tell who is logging in but also where (IP Address) and most importantly how (User Agent) they are logging in. One thing to be aware of is that the User Agent is not always populated and even if it is, it may not provide you with enough information on the specific interface a given user is logging in from. For example, it looks like a script written using the vSphere SDK for Python does not actually set the User Agent, so it is empty.

Here is an updated table using some of the latest vSphere interfaces to log into a vSphere 6.0 Update 2 environment and their respective observed User Agents:

Interface User Agent
vSphere C# Client VMware vSphere Client/6.0.0
vSphere Web Client VMware vim-java 1.0
vSphere MOB Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_1) AppleWebKit/537.36 (KHTML like Gecko) Chrome/54.0.2840.71 Safari/537.36
PowerCLI PowerCLI/6.5.0
vSphere SDK for Perl VI Perl
vSphere SDK for Ruby (rbvmomi) Ruby
vSphere SDK for Python (pyvmomi) None

Note: In vSphere 6.5, the User Agent that is returned for the vSphere Web Client session looks to be using web-client/6.5.0

Finally, saving the best for last. The VMware Engineer(s) not only added these new properties into the vSphere API, but they have also made them readily available using the vSphere Web Client. To view all the session information, navigate to your vCenter Server instance and under Manage->Sessions you can get the exact same view as using the vSphere API. By default, the IP Address, User Agent & API Invocations are hidden by default. You just need to right click on the table header and add those additional field as shown in the screenshot below.

retreiving-useful-information-about-vsphere-login-1
Longer term, it would be great to see that each of the "official" VMware CLI/SDKs as well as other interfaces can uniquely identify themselves with a well defined string. This not only helps with understanding the types of tools customers are using but also helps with any types of internal audits customers may require. If you think this would be useful to have, please feel free to leave a comment or any other things you feel would be useful to include.

New vSphere 6.5 APIs worth checking out

by 11 Comments

With the upcoming new release of vSphere, there are quite a few new vSphere APIs to look forward to and consume from an Automation standpoint. Similiar to what I had done in the past with previous major releases of vSphere, here is a list of of some of the new vSphere APIs (SOAP based) that I think are worth checking out whether they are new features vSphere 6.5 will introduced or new enhancements to existing functionality which will benefit our vSphere Administrators and/or Developers.

If you would like to see the complete list of new vSphere 6.5 (SOAP based) APIs, be sure to check out the vSphere 6.5 API Reference Guide which will include a "What's New" section on all the new Managed Objects, Methods, Properties, etc. when vSphere 6.5 is generally available.

CryptoManager / CryptoManagerKmip - VM Encryption is one of the new features in vSphere 6.5 and with these APIs, you will be able to manage and configure the VM Encryption settings including associating with KMIP server. For enabling/disabling VM Encryption at the VM and disk level, have a look at VirtualMachine->crypto and VirtualMachine->deviceChange->backing property.

  • GenerateClientCsr
  • GenerateKey
  • GenerateSelfSignedClientCert
  • ListKmipServers
  • MarkDefault
  • RegisterKmipServer
  • RemoveKmipServer
  • RetrieveClientCert
  • RetrieveClientCsr
  • RetrieveKmipServerCert
  • RetrieveKmipServersStatus_Task
  • RetrieveSelfSignedClientCert
  • UpdateKmipServer
  • UpdateKmsSignedCsrClientCert
  • UpdateSelfSignedClientCert
  • UploadClientCert
  • UploadKmipServerCert

FailoverClusterConfigurator - To setup the new vCenter Server High Availability (VCHA) feature which is only available in the VCSA, use these APIs which include deploying and configuring the passive and witness nodes.

  • configureVcha_Task
  • createPassiveNode_Task
  • createWitnessNode_Task
  • deployVcha_Task
  • destroyVcha_Task
  • getVchaConfig
  • prepareVcha_Task

FailoverClusterManager -  Have a look at these APIs to initiate a failover or view the current VCHA configuration.

  • getClusterMode
  • GetVchaClusterHealth
  • initiateFailover_Task
  • setClusterMode_Task

HostVStorageObjectManager - An API only feature in vSphere 6.5 which will allow you to create and manage Virtual Disks as a "First Class" citizen. This particular API is for managing First Class Disks (FCD) when talking directly to an ESXi host.

  • HostCloneVStorageObject_Task
  • HostCreateDisk_Task
  • HostDeleteVStorageObject_Task
  • HostExtendDisk_Task
  • HostInflateDisk_Task
  • HostListVStorageObject
  • HostReconcileDatastoreInventory_Task
  • HostRegisterDisk
  • HostRelocateVStorageObject_Task
  • HostRenameVStorageObject
  • HostRetrieveVStorageObject
  • HostRetrieveVStorageObjectState
  • HostScheduleReconcileDatastoreInventory

VcenterVStorageObjectManager - An API only feature in vSphere 6.5 which will allow you to create and manage Virtual Disks as a "First Class" citizen. This particular API is for managing First Class Disks (FCD) when talking directly to a vCenter Server.

  • AttachTagToVStorageObject
  • CloneVStorageObject_Task
  • CreateDisk_Task
  • DeleteVStorageObject_Task
  • DetachTagFromVStorageObject
  • ExtendDisk_Task
  • InflateDisk_Task
  • ListTagsAttachedToVStorageObject
  • ListVStorageObject
  • ListVStorageObjectsAttachedToTag
  • ReconcileDatastoreInventory_Task
  • RegisterDisk
  • RelocateVStorageObject_Task
  • RenameVStorageObject
  • RetrieveVStorageObject
  • RetrieveVStorageObjectState
  • ScheduleReconcileDatastoreInventory

DatastoreNamespaceManager->ConvertNamespacePathToUuidPath() - From a troubleshooting standpoint, do you ever wish you can easily translate the human readable VM path (e.g. /vmfs/volumes/vsanDatastore/myVM/myVM.vmx to the VSAN/VVOL equivalent identifier which is UUID based? Well, this is now possible with this new API!

AuthorizationManager->FetchUserPrivilegeOnEntities() - This is a pretty neat API as it allows you to easily query an existing user to see the current privileges has been assigned. This could could come in handy to quickly audit a particular privilege for a user.

HostImageConfigManager->installDate() - Have a look at this blog post Super easy way of getting ESXi installation date in vSphere 6.5 for more details.

HostImageConfigManager->fetchSoftwarePackages() - This is another nice API to easily retrieve all the VIBs installed on an ESXi host. This is the equilvenet of running "esxcli software vib list" and you will now have all the additional metadata info that was historically only available via ESXCLI. Here is an example PowerCLI function called Get-ESXInstalledVib which exercises this new API.

HostStorageSystem->UpdateVmfsUnmapPriority() - Have a look at the blog post Configure new automatic Space Reclamation (VMFS UNMAP) using vSphere 6.5 APIs for more details.

VirtualMachine->{AttachDisk_Task(),DetachDisk_Task()} - This API allows you to attach and detach First Class Disks that you may have created earlier using the FCD APIs as shown above.

VirtualMachine->config->bootOptions->EfiSecureBootEnabled - To take advantage of the new VM Secure Boot feature in vSphere 6.5, you simply just toggle this property. Here are two PowerCLI functions called Get-SecureBoot/Set-SecureBoot which exercises this new API.

In addition, vSphere 6.5 also introduces a new REST-based API that covers several areas such as basic VM Lifecycle Management (simliar to that of the existing vSphere SOAP-based API), vSphere Content Library, vSphere Tagging and Virtual Appliance Management for the vCenter Server Appliance (VCSA). You can interact with these new APIs by using any of the vSphere Automation SDKs (.Net, Java, Python, Ruby or Perl) or even just simply using cURL from the command-line. It is really that easy!

Lastly, to make exploring these new REST-based APIs easier for both administrators as well as developers, there is now a new API Explorer that is included specifically with the VCSA in vSphere 6.5. You can think of it like a vSphere MOB 2.0 but way easier to use. Some of you may recognize the interface as shown in the screenshot below which uses the Swagger UI. This interface allows you to quickly browse through all the APIs, no need to refer to the documentation as the APIs are self-documented and made available in this interface. Best of all, you can even try out the APIs by simply logging into your vCenter Server and then selecting an API and clicking on the "Try it out now" button!

To access the API Explorer, you simply open a web browser and enter the following URL: https://[VC-HOSTNAME-OR-IP]/apiexplorer/

vsphere-6-5-apis-apiexplorer
There will also be native PowerCLI cmdlets (Get-CisService) to these new REST API and below is a quick example of retrieving the version (GET /system/version) of the VCSA:

$vcsaVersion = Get-CisService -Name  'com.vmware.appliance.system.version'
$vcsaVersion.get()

vsphere-6-5-apis-powercli

Remotely query an ESXi host without adding it to vCenter Server

by Leave a Comment

I was browsing through the vSphere API Reference the other day and I came across an interesting vSphere API called QueryConnectionInfoViaSpec() which was introduced in vSphere 6.0. This new API is similiar to the existing QueryConnectionInfo() method which also allows you to remotely query an ESXi host without requiring you to add it to your vCenter Server Inventory, which I did not know about before. Some of the information that you can retrieve is whether it is currently being managed by an existing vCenter Server and if so, what is its IP Address, any registered Virtual Machines, networks and datastores that may be configured along with underlying host capabilities.

The main difference between this new API is that instead of accepting a list of arguments to the method, it accepts a HostConnectSpec. It is a minor difference, but what this allows you to do is to be able to re-use that exact same spec in the AddHost_Task() API if you determine that you would like to add the ESXi host to your vCenter Server Inventory after inspecting the remote ESXi host. I think this is a useful API if you want to validate that you are not stealing an ESXi host from another managed vCenter Server and if you are, at least you know which vCenter Server it was being managed by. To demonstrate this vSphere API, I have created a simple PowerCLI script called Get-RemoteESXi.ps1 which requires a vCenter Server login as well as the Hostname/IP Address and credentials of the remote ESXi host you wish to query.

Here is an example output of running the script which stores the returned output to the $results variable.

query-remote-esxi-without-adding-to-vcenter-server
If you want to check whether the ESXi host is being managed by an existing vCenter Server, you can check the following property which will hold the IP Address of the vCenter Server if it is currently being managed:

$result.Host.ManagementServerIp

If you want to check if there are running VMs already on the ESXi host, you can check the following property:

$result.vm

As mentioned earlier, there is a ton of other information returned from the remote ESXi host and I will leave it as an exercise for the reader to explore further.

Quick Tip – How to quickly find the VM Display Name of your vCenter Server?

by 1 Comment

It is a pretty common practice for customers to use their vCenter Server to manage the underlying ESXi host which is also running the vCenter Server VM, sometimes referred to as a self managed vSphere Cluster. For organizations that have a strict naming convention which includes the VM display name as well as the OS hostname (FQDN), it is generally not too difficult to identify the actual VM that is running your vCenter Server.

However, for customers who may not have a VM display name that can easily be associated to their vCenter Server, it can be pretty difficult and time consuming to locate the vCenter Server VM for troubleshooting and/or updating purposes. Historically, this mapping and association has been left to our customers to document whether it is using a sticky, CMDB, vSphere Tags/Folders, etc. to be able to identify some of these more important VMs.

Luckily, there are some built-in mechanisms within vCenter Server itself that we can leverage to help us quickly identify the VM display name of our vCenter Server. Just this week while working on something, I actually came across another method which I think is even easier than the one I was familiar with.

Option #1 (harder) - When vCenter Server detects that it is in a self-managed configuration, it will automatically add a "System" metadata tag (not to be confused with vSphere Tags) to the actual vCenter Server VM. For vCenter Server, the "Tag" will have the value of SYSTEM/COM.VMWARE.VIM.VC which you can query on a VM using the vSphere API. Below is a PowerCLI example exercising the vSphere API:

$vm = Get-VM -name VCSA-60u2
(Get-View $vm).ExtensionData.Tag

quickly_find_vcenter_server_vm_display_name-0
The main issue here with this option is that because this information is at the VM level, you must first find the VM. This means, you would have to iterate through all of your VMs and seeing which has this particular property set with this value.

Option #2 (easier) - As mentioned earlier, I came across this new option by accident while browsing through the vCenter Server Advanced Settings. I noticed an interesting value for the following key: config.registry.key_VCVmId After a quick investigation, I found that vCenter Server also sets this Advanced Setting to the Managed Object Reference (MoRef) ID of the actual vCenter Server VM. This means, in one step, I know exactly which VM is my vCenter Server and I just need to convert the MoRef ID to a VM Object to then query the VM display name. How cool!?

The following PowerCLI example demonstrates how to extract the vCenter Server Advanced Setting, convert the MoRef ID to a VM Object which we can then query for its VM Display Name.

As you can see from the screenshot below, with just a few lines of code, we can quickly figure out our vCenter Server VM regardless if our inventory is 10 or 10,000 VMs. In fact, you can even output the exact ESXi host that is currently running the VM as well, but I will leave that as an exercise for the reader 🙂

quickly_find_vcenter_server_vm_display_name-1

vSphere SDK for JavaScript Fling released

by 2 Comments

The VMware Fling team has just released another cool new Fling, the vSphere SDK for JavaScript. This Fling is especially interesting as it provides the underlying SDK framework used by the popular ESXi Embedded Host Client Fling which was released back in August of last year. I came to learn about this project during last years internal R&D Innovation Offsite (RADIO) conference which is held annually and can be thought of as the VMworld conference for VMware employees.

One of the biggest highlight of the conference for me personally is checking out the expo floor where you get to see what other VMware Engineering teams have been working on whether it is the next big feature, product or new ideas that they might be thinking about. It was during my walk through that I met Rostislav Hristov, one of the Engineers who worked on the vSphere SDK for JavaScript. I was really impressed at what Rostislav built and luckily he was already in touch with the Embedded Host Client Engineers to see how they could leverage his JavaScript SDK as the initial prototype had made calls directly using the vSphere MOB which was not very friendly to develop against.

There has been a number of improvements to the vSphere SDK for JavaScript since I last saw it and although the name contains "vSphere", it definitely supports more than just the vSphere API endpoint. In fact, with this single SDK you can interact with vCenter Server Single Sign-On (SSO) API, vCloud Suite API which covers capabilities like Tagging and Content Library as well as the Site Recovery Manager (SRM) APIs! For customers and partners that are looking to develop their own web portals or interfaces that can integrate with these APIs, this will be a handy tool to have.

To get started, the vSphere SDK for JavaScript contains a README file that contains additional instructions on setting up the SDK as well as a couple of samples that demonstrates each of the supported API endpoints:

  • vimService.js - Sample using the vSphere API
  • stsService.js - Sample using the SSO API
  • cisServices.js - Sample using the vCloud Suite API
  • srmService.js - Sample using the SRM API

Here is the command to run the vimService.js sample which will also require you to set the environmental variable NODE_TLS_REJECT_UNAUTHORIZED=0 if you are using the default VMware self-signed SSL Certificate.

NODE_TLS_REJECT_UNAUTHORIZED=0 babel-node samples/vimService.js

vsphere-sdk-for-javascript-0
Once the sample has started up, you will be prompted with a URL to open in your browser. In the vimService.js example, you will be able to login to either a vCenter Server or ESXi host as seen in the screenshot below.

vsphere-sdk-for-javascript-1
Once logged in, you should see a simple listing of the different inventory objects in your vSphere enviornment.

vsphere-sdk-for-javascript-2
In the stsService.js sample, once logged in by specifying the address to your PSC/SSO instance, you should see that a SAML token was issued.

vsphere-sdk-for-javascript-3
The cisService.js sample exercises several operations using a mixture of both the vSphere API as well as the new vCloud Suite API. It does require connecting to a vCenter Server 6.0 environment as it will be performing operations using the new vSphere Content Library feature as well as some VM operations. Do not worry, once the operations have been completed, the script will automatically clean itself up. This is a great sample if you want to see how you could make use of the different APIs all through this single SDK.

vsphere-sdk-for-javascript-4-new
I did not have an SRM environment up and running to test the srmService.js sample, but you can see from the code that it will list all of the recovery plans and their current state. For more details on how the individual APIs work, you can refer to the documentation included in the vSphere SDK for JavaScript or the official API documentation for the individual products. If you have any feedback or comments about this Fling, please leave a comment on the Fling site here as I am sure the Engineers would love to hear what you think!

pyvmomi (vSphere SDK for Python) 5.5.0-2014.1 released!

by 1 Comment

I just saw an awesome update from Shawn Hartsock, a fellow VMware colleague. For those of you who do not know him, Shawn works in our Ecosystems and Solutions Engineering (EASE) organization and is the primary maintainer of VMware's pyvmomi (vSphere SDK for Python) open-source project. The pyvmomi project was open sourced since last December which I had written about here, it has received over 3K+ downloads and has a very active community. Much of this success has been due to the hard from Shawn fostering an active community around pyvmomi.

The announcement today from Shawn is a new release of pyvmomi at version 5.5.0-2014.1:

As mentioned earlier, the pyvmomi project is a very active project and Shawn is constantly engaging with users looking for feedback, suggestions or requests for new samples to build. If you are interested in vSphere Automation and would like to leverage Python, be sure to check out the pyvmomi Github repository! Lastly, if you have written some cool scripts/applications or would like to request specific sample scripts, be sure to send a pull request to Shawn as we would love to see more contributions and collaborations from the community!