virtuallyGhetto

vsphere sdk for perl

Creating SE Sparse Linked Clones Using the vSphere 5.1 API

by 9 Comments

In my previous article, I showed you how you can easily create your own SE Sparse (Space-Efficient) disks for you virtual machines running on vSphere 5.1. In addition to just creating new virtual disks using the new disk format, you can also create SE Sparse Linked Clones by leveraging the vSphere 5.1 API. I had previously shared a vSphere SDK for Perl sample script called vGhettoLinkedClone.pl that allows you to create your own Linked Clones and I have updated the script to now support the new SE Sparse disk format.

The script now includes new parameter called --convert which allows you to specify whether or not the destination Linked Clone should follow the same source virtual disk format or the new SE Sparse disk format. Remember to upgrade your VMs to the latest ESXi 5.1 compatibility and later (virtual hardware version 9) if you wish to leverage the new SE Sparse disk format. The way this is accomplished is by specifying a new disk backing  VirtualDiskFlatVer2BackingInfo in the VirtualMachineRelocateSpec and passing in the deltaDiskFormat to be of type seSparseFormat.

Disclaimer: This is for educational purposes only, this is not officially supported by VMware. Please test this in a development environment before using it on actual systems.

Note: Linked Clones is a feature of vCenter Server, make sure you are connecting to a vCenter Server 5.1

If you have a VM that has a VMDK that is NOT an SE Sparse disk format but wish to create Linked Clones with the new SE Sparse disk format, then you can use the --convert sesparse option, here is an example command:

./vGhettoLinkedClone.pl --server vcenter51-1.primp-industries.com --username root --vmname Regular-VM --vmname_destination Linkedclone-SeSparse-Destination --snapname pristine --vmhost vesxi51-1.primp-industries.com --datastore datastore1 --convert sesparse

If you have a VM that has a VMDK that IS an SE Sparse disk format and you wish to create Linked Clones with the same disk format, then you can use the --convert source option, here is an example command:

./vGhettoLinkedClone.pl --server vcenter51-1.primp-industries.com --username root --vmname SeSparse-VM --vmname_destination Linkedclone-SeSparse-Destination --snapname pristine --vmhost vesxi51-1.primp-industries.com --datastore datastore1 --convert source

If we head over to our vSphere Web Client and take a look at our new Linked Clones, we can confirm their virtual disks is using the new SE Sparse disk format.

How to Create an SE Sparse (Space-Efficient) Disk in vSphere 5.1

by 8 Comments

You probably may have heard, that with the upcoming release of vSphere 5.1, a new virtual machine disk format will be introduced called called SE Sparse (Space-Efficient). One of it's features is to provide the ability to reclaim unused blocks from within the guestOS. I would highly recommend you check out a recent blog post vSphere 5.1 Storage Enhancements – Part 2: SE Sparse Disks by Cormac Hogan for more details about the new SE Sparse disk format as well as other storage improvements in vSphere 5.1.

As Cormac points out, this new disk format will initially be leveraged by VMware View (in a future release from my understanding), as there are additional integrations required to use this feature than just using the new SE Sparse disk format. Having said that, the SE Sparse disk format is a feature of the vSphere 5.1 platform and with that, you do have the ability to create an SE Sparse disk.

Disclaimer: This is for educational purposes only, this is not officially supported by VMware. Please test this in a development environment before using it on actual systems.

There are two methods in which you can create an SE Sparse disk, directly on the ESXi Shell of an ESXi 5.1 host or remotely connecting to an ESXi 5.1 host.

Option 1 - Using vmkfstools on ESXi Shell 

Though it may not be documented, you can easily create a new VMDK with the new SE Sparse disk format by running the following command (10GB disk in this example):

vmkfstools -c 10g -d sesparse WindowsXP.vmdk

Here is a screenshot of new SE Sparse disk descriptor file to prove we have successfully created a new VMDK using the new format:

Option 2 - Using vSphere 5.1 API w/modified remote version of vmkfstools

As mentioned, the SE Sparse disk format is a feature of the vSphere 5.1 platform and as so, you can also leverage the vSphere 5.1 API to create a new VMDK using the virtualDiskManager and specifying the new SeSparseVirtualDiskSpec.

Note: Even though the vSphere API reference mentions the ability to set grain size via grainSizeKb property, I have found that it is not possible and just leaving it blank will automatically default to 1024K (1MB) which might be a system default for now.

You can download the modified version of the remote vmkfstools called vmkfstools-lamw which requires the the installation of vCLI 5.1 or vMA 5.1.

Here is an example of creating the same 10GB VMDK using the new SE Sparse disk format:

./vmkfstools-lamw --server 172.30.0.187 --username root -c 10G -d sesparse "[datastore1] WindowsXP.vmdk"

After you have created your new SE Sparse disk, the next logical step is assign it to a virtual machine. Since this is a new feature in vSphere 5.1, you will need to use the new vSphere Web Client to perform the operation as the legacy vSphere C# Client is not aware of this new disk type. You will also need to ensure that the virtual machine is running the latest ESXi 5.1 compatibility and later (virtual hardware version 9).

Once you have added our newly created disk from the datastore, it should now show up in the vSphere Web Client as Flex-SE for the disk type.

Additional Resources:

 

Auditing vMotion Migrations

by 3 Comments

I saw an interesting question this week about auditing vMotion events and the number of times a VM has migrated to a particular ESX(i) host for license compliance. You can view this information using the Task/Events in your vCenter Server but you can also extract out the various types of events using the EventManager in the vSphere API. You will be able to go as far back in time as your vCenter Server's database retention policy allows you to. We will be searching for the VmMigratedEvent Event which will include variety of information including the source and destination host for the VM. The destination host will only be populated upon a successful vMotion.

Of course I had to write a script to help automate this, so here is a vSphere SDK for Perl script called getNumberOfvMotions.pl that accepts the name of an existing VM and will return the number of vMotions that has been performed on the VM as well as the list of destination hosts and the number of times it has migrated to those hosts. You will need a system that has the vCLI installed or you can you use vMA.

Note:  If you want to look at past vMotion for a VM that no longer exists, this is still possible, but you will need to parse the "message" within the Event as you can no longer look up that VM object in vCenter.

Here is an example of the script running:

You can easily modify the script to audit all VM's in your environment or just use a simple "for" loop to go through a set of VM's you are interested in, but I will leave that as an exercise for you.

VM Provisioning on Datastore Clusters in vSphere 5

by 23 Comments

Last year I wrote an article called Automating Storage DRS & Datastore Cluster Management in vSphere 5 and I provided a pretty comprehensive vSphere SDK for Perl script to help administrators automate Storage DRS configurations. These past few months I have noticed an increase in interest on the VMTN developer forums relating to Storage DRS. Majority of the questions has been related to which vSphere API methods to use and how to use these methods for cloning VMs to datastore clusters.

If you have cloned a VM before, the underlying vSphere API method being used is the CloneVM_Task(), but when cloning a VM to a datastore cluster, a different API must be used.

In vSphere 5, VMware introduced several new API methods in the StorageResourceManager and the two specific ones relating to provisioning VMs are RecommendDatastores() and ApplyStorageDrsRecommendation_Task(). The process to clone a VM to a datastore cluster is a two step process:

  1. Call RecommendDatastores() which accepts very similar input as the CloneVM_Task(). In addition, you need to specify the datastore cluster also known as a Storage Pod in the vSphere API as well as the "type" (create, clone, relocate, reconfigure), in this example we will be performing a clone operation. This method will then generate a recommendation on where to place the VM which is based on the SDRS placement engine. No provisioning will occur at this point, just a placement recommendation.
  2. To perform the actual provision of the VM, you will need to call ApplyStorageDrsRecommendation_Task() which accepts a recommendation ID that was generated from the first step. Once the recommendation is applied, the provisioning of the VM will start just like it did when you called the CloneVM_Task().

Note: The RecommendDatastores() will return multiple recommendations, the best one will be first entry in the array. This is the same algorithm used when performing this same operation in the vSphere Client, it also selects the first recommendation.

Now that we understand how the APIs work, let's take a look at how we can leverage this in a script for some automation! Here is a simple vSphere SDK for Perl script called datastoreClusterVMProvisioning.pl which allows you to clone an existing VM onto a datastore cluster. You will need a system that has vCLI 5.0 installed or you can use VMware vMA 5 to run the script. You will also need to connect to a vCenter Server 5 for all SDRS operations.

The script requires 4 input parameters:

  • vmname - Name of the VM you wish to clone from
  • clonename - Name of the cloned VM
  • vmfolder - Name of a vCenter folder
  • datastorecluster - Name of a datastore cluster

Here is a screenshot of cloning an existing VM onto a datastore cluster:

The script is pretty straight forward and it can easily be adapted to include other configurations as required in your own environment.

Hopefully this gives you a better idea on how to leverage the new provisioning APIs for Storage DRS and start automating your VM deployments onto datastore clusters and get the benefits Storage DRS in your vSphere environment.

Automating VMware Tools Upgrade Policy

by 16 Comments

I received a question this week from a reader who was looking to change the VMware Tools upgrade policy for a few hundred virtual machines and wanted to know if it was possible to automate this. The answer is absolutely yes!

By default the VMware Tools upgrade policy is disabled and set to "manual" mode. If you want the vSphere platform to automatically check and upgrade VMware Tools upon a system power cycle, then you can enable it by going to Edit VM->Options->Tools->Check and upgrade Tools during power cycling.

To update this configuration, there is a property in the vSphere API called toolsUpgradePolicy which can accept two values: manual or upgradeAtPowerCycle.

Here is a vSphere SDK for Perl script updateVMToolsPolicy.pl that supports two types of operations: "list" and "update". The list operation will show you all VMs and their currently configured upgrade policy, by default they should be all manual unless you have changed it manually. The update operation will allow you to configure a list of VMs and policy you designate. This change can be done while the VM is running, you do not need to make any changes to the guestOS that is running.

Here is an example of the "list" operation:

If you want to take all the VMs that have "manual" policy and change them over to "upgradeAtPowerCycle", you can copy the output to a file and then use a find or UNIX/Linux grep command to search for entries that have the word "manual".

Here is the command you can use if you are on a UNIX/Linux system:

cat output | grep "manual" | awk -F '["|"]' '{print $2}'

Here is the command to get the first column which contains the VM display name:

cat output | grep "manual" | awk -F '["|"]' '{print $2}' > VMLIST

Lastly, you just need to take the previous command and redirect that to a file which will then be used in the "update" operation. You can also take the output and using an editor to get to the final output, use whatever you are comfortable with.

Here is an example of the commands listed above:

Now that we have the list of VMs we are interested in updating, we just need to select the policy and perform the "update" command. Here is an example:

So there you have it, you can now easily automate the the VMware Tools upgrade policy for any or all your VMs without having to edit each one manually.

Automating vCenter Server Advanced Settings

by 1 Comment

This weekend I needed to make a change to one of the advanced settings in my vCenter Server. This can be done using the vSphere Client, but it is also possible to use the vSphere API, (although it is not well-known). This can come in handy especially when you need to make a change across several dozen vCenter Servers and not have to manually login to each one. In the vSphere API, there is a property under the service content called setting which contains an array of advanced settings that can be modified.

I wrote a quick vSphere SDK for Perl script called vCenterAdvSettingMgmt.pl which allows for two type of operations "list" and "update". When you use the update operation, you will be required to pass in both the key and value that you would like to change.

Here is an example of the "list" operation:

Here is an example of the "update" operation where I am updating the "VirtualCenter.ManagedIP" setting:

You can verify the change by re-running the "list" operation:

This script came in very handy when deploying the new vCenter Infrastructure Navigator which requires "VirtualCenter.ManagedIP" to be configured prior to deploying.