spbm

Feedback on default behavior for VM Storage Policy

06/05/2018 by William Lam 1 Comment

Today, the vCenter REST (vSphere Automation) APIs currently does not support the use of VM Storage Policies when relocating (vMotion, Cross Datacenter vMotion & Storage vMotion) or cloning an existing Virtual Machine. Customers have provided feedback that this is something that they would like to see get added to the current REST APIs and while this is being looked at, there were a couple of open questions from Engineering.

The following 2-question survey below is to help us understand what the "default" behavior should be when a Virtual Machine is being relocated or cloned within a vCenter Server and a VM Storage Policy is NOT specified when using the APIs. The reason for this is that our existing APIs for relocate and clone today are very flexible and not everything needs to be specified as part of the relocate or clone API specification. However, due to this flexibility, you may observe different behaviors and we would like to understand what the default behavior should be when some of these paraemters are not specified. In the case where you want to be explicit, you can always specify the VM Storage Policy, but the survey is to understand when it is not specified.

Survey: https://goo.gl/forms/aQjVTly2MmVHeRsp1

Thank you for taking the time to provide your feedback, this will help us build an easy and robust API when dealing with relocate and clone operations using the vCenter REST APIs.

Minimum permissions to view VM Storage Policies

01/19/2017 by William Lam Leave a Comment

Any way to allow vCenter users to see #SPBM policies without using global permissions?
No pol visible w/cluster level permissions #vmware

— Aaron Patten (@Jedimt) January 18, 2017

I saw this question from Aaron yesterday while scrolling through my Twitter timeline and after answering it, I figure I write a quick blog post about it in case this comes up in the future.

There are two specific privileges around managing VM Storage Policies: Update and View as shown in the screenshot below. If you only want to allow users to be able to see all the available VM Storage Policies that have been defined, then you just need to create a new Role with only the "View" privilege.

Secondly, it is important to note that VM Storage Policies are defined and managed at a vCenter Server level. This means that when you assign the permission, it needs to be applied at the root vCenter Server level (you do not have to propagate it down wards if you do not wish to show the rest of the vSphere Inventory). Global permissions are not required, but if you have multiple vCenter Servers which are all part of the same SSO Domain, you may want to consider this if users are allowed to login to any one of the vCenter Servers.

Once you have assigned the permission to either the user or group, then you can have them login using either the vSphere Web Client or using the SBPM APIs and you will now be able to view all defined VM Storage Policies.

SPBM APIs are now included in pyvmomi (vSphere SDK for Python)

01/19/2017 by William Lam 4 Comments

I have been spending quite a bit of time lately with PowerCLI Core, especially with one of my pet projects. One of the limitations that PowerCLI Core has today is that the Storage cmdlets which includes vSAN and VVol functionality has not been ported over yet. This means that if you need to do something with VM Storage Policies for example, it would not be possible with PowerCLI Core and you would have to use the Windows PowerCLI version instead.

While investigating for an alternative solution to PowerCLI Core to get access to the Storage Policy Based Management (SPBM) APIs, I was pleasantly surprised to learn that pyvmomi (vSphere SDK for Python) had recently added support for the SPBM APIs in their 6.0.0.2016.4 release last year. I had accidentally stumbled onto this news while looking through the pyvmomi Github issues, specifically this one here. I was surprise to see there was no mention of this enhancement in the pyvmomi release notes.

This is great news for pyvmomi consumers and given this was news to me, I am guessing it might be news for others so figure I would also share the info. While looking into using the SPBM APIs from pyvmomi, I did not see any sample scripts showing how to use the SPBM API. Given I needed to write a script for my project, I figure I would also create a couple of examples to help others get started.

Exploring VSAN APIs Part 6 – Modifying Virtual Machine VM Storage Policy

03/20/2014 by William Lam 6 Comments

One of the biggest benefit of VSAN is the ability to specify granular storage policies on a per Virtual Machine basis. These storage policies is managed through VMware's Storage Policy Based Management system and is automatically enforced by VSAN to ensure compliance. A VM Storage Policy can be assigned during the initial deployment of a Virtual Machine or it can be modified afterwards, for example if the Virtual Machine's SLA's has changed because the workload has changed. From the vSphere Web Client, modifying a Virtual Machine's VM Storage Policy is simply selecting the VM Storage Policy and re-applying which is also available programmatically through the vSphere API.

Using the vSphere API method ReconfigVM_Task(), you will be able to modify the VM Storage Policy for the VM Home Namespace and/or individual Virtual Disks. To modify the VM Home Namespace, there is a property defined at the root of the Virtual Machine config spec called vmProfile which accepts the VM Storage Policy ID extracted from the SPBM API. To modify the VM Storage Policy for an individual Virtual Disk, you will need to set the profile property which is exposed on a Virtual Device with the VM Storage Policy ID. To demonstrate this functionality, I have created a sample vSphere SDK for Perl script called changeVMStoragePolicy.pl

Disclaimer: These scripts are provided for informational and educational purposes only. It should be thoroughly tested before attempting to use in a production environment.

In my environment, I have a Virtual Machine called VM1 which has been defined with a VM Storage Policy called "Copper" as seen in screenshot below:

Let's say I want to change the Virtual Machine's VM Storage Policy to another policy called "Aluminum", I first need to extract the VM Policy ID from SPBM API and then pass it into the script like the following:

./changeVMStoragePolicy.pl --server vcenter55-1.primp-industries.com --username root --vmname VM1 --profileid cd6908b2-0704-4733-ad9b-a9a8f200ab0a

Once the Virtual Machine has been reconfigured, we can then take a look in our vSphere Web Client and we can see the VM Storage Policy has now been changed and VSAN will automatically enforce these new requirements.

If you wish to assign a VM Storage Policy as part of a new Virtual Machine creation, you just need to set the vmProfile and profile properties which is similar to a reconfiguration operation.

Extending RVC to support renaming VM Storage Policies

03/18/2014 by William Lam 1 Comment

I was recently using RVC (Ruby vSphere Console) to setup one of my VSAN lab environments and I had noticed that in the SPBM namespace, that you could create and delete a VM Storage Policy, but you could not rename an existing one. The great thing about RVC is that it is very extensible and I thought it would be useful to have a spbm.profile_rename command, so I decided to build it!

The management of VM Storage Policies is performed through the SPBM API and there is a method called PbmUpdate() which allows you to rename an existing VM Storage Policy. In my environment, I exclusively use the VCSA (vCenter Server Appliance) and in the /root directory, you should see a .rvc directory. To extend the SPBM namepace, you just need to create a new file called spbm.rb which should contain the following snippet of code:

opts :profile_rename do
  summary "Rename a VM Storage Profile"
  arg :profile, nil, :lookup => RbVmomi::PBM::PbmCapabilityProfile
  arg :name, "New name", :type => :string
end

def profile_rename profile, name
  _catch_spbm_resets(nil) do
    pbm = profile.instance_variable_get(:@connection)
    pm = pbm.serviceContent.profileManager
    spec = PBM::PbmCapabilityProfileUpdateSpec(
      :name => name,
    )
    pm.PbmUpdate(:profileId => profile.profileId, :updateSpec => spec)
  end
end

opts :profile_rename do

summary "Rename a VM Storage Profile"

arg :profile, nil, :lookup => RbVmomi::PBM::PbmCapabilityProfile

arg :name, "New name", :type => :string

end

def profile_rename profile, name

_catch_spbm_resets(nil) do

pbm = profile.instance_variable_get(:@connection)

pm = pbm.serviceContent.profileManager

spec = PBM::PbmCapabilityProfileUpdateSpec(

:name => name,

)

pm.PbmUpdate(:profileId => profile.profileId, :updateSpec => spec)

end

Once you have saved the file, you can now connect to RVC and you should see a new command called spbm.profile_rename which takes an existing VM Storage Policy and the new name of the policy.

Here is an example of what that would look like where I have a VM Storage Policy called "Platinum" and I want to rename it to "Adamantium":

spbm.profile_rename localhost/Datacenter/storage/vmprofiles/Platinum/ Adamantium

VMware VSAN APIs

02/11/2014 by William Lam 15 Comments

For customers who are currently evaluating VMware VSAN, the vSphere Web Client is the primary interface for managing and configuring VSAN. However, these exact same VSAN operations can also be accessed programmatically from an Automation perspective. Given that I had spent quite a bit of time playing with VSAN in the lab and exploring some of its APIs, I thought it might be useful to share some of the common VSAN operations mapped to their respective APIs.

There are two main categories of operations that you should be aware of when working with VSAN:

VSAN specific operations
VM Storage Policy operations

VSAN Operations

The VSAN specific operations are available as part of the vSphere 5.5 API and the list below are the supported operations:

Enable/Disable VSAN on a vSphere Cluster (automatic claiming):

Use the ReconfigureComputeResource_Task() and set spec->vsanConfig->enabled to either true or false & spec->vsanConfig->defaultConfig->autoClaimStorage to true

Manual Disk Claiming for an ESXi host:

Each ESXi host provides a vsanSystem manager at configManager->vsanSystem which provides the following methods for disk management
- AddDisks_Task()
- InitializeDisks_Task()
- QueryDisksForVsan()
- QueryHostStatus()
- RemoveDisk_Task()
- RemoveDiskMapping_Task()
- UpdateVsan_Task()

Enable VSAN traffic type for a VMkernel interface

Use the ReconfigureComputeResource_Task() and set spec->vsanHostConfigSpec->HostSystem to the ESXi host you wish to configure and spec->vsanHostConfigSpec->networkInfo->port->device to the specific VMkernel interface you wish to enable VSAN traffic type on

Internal VSAN manager for accessing low level CMMDS (Clustering Monitoring, Membership and Directory Services), object & disk management APIs:

Each ESXi host provides a vsanInternalSystem manager at configManager->vsanInternalSystem which provides the following methods for working with the lower level system of VSAN
- QueryCmmds()
- QueryObjectsOnPhysicalVsanDisk()
- QueryPhysicalVsanDisks()
- QueryVsanObjects()

Change the VM Storage Policy for a Virtual Machine Home OR Virtual Machine Home and Virtual Disk:

Use the ReconfigVM_Task() and set spec->vmProfile to the VM Storage Policy ID

Change the VM Storage Policy for Virtual Machine Virtual Disk:

Use the ReconfigVM_Task() and set spec->deviceChange->device to the specif Virtual Disk to be reconfigured and set spec->deviceChange->vmProfile to the VM Storage Policy ID

Enter Maintenance Mode for an ESXi host in a VSAN Cluster:

Use the EnterMaintenanceMode_Task() and set spec->
maintenanceSpec->vsanMode->objectAction to the specific data accessibility mode defined here

Create and Delete directories on a VSAN Datastore:

Use the DatastoreNamespaceManager which provides the following two methods
- CreateDirectory()
- DeleteDirectory()

SPBM

For VM Storage Policies, VSAN leverages the SPBM (Storage Policy Based Management) framework which allows administrators to create policies that define specific storage capabilities such as performance and reliability that can then be applied to a Virtual Machine. The SPBM API is exposed as a separate API endpoint in vCenter Server and will be required if you wish to create additional VM Storage Policy leveraging VSAN storage capabilities.

Query list of available VM Storage Profiles defined:

Use PbmQueryProfile()

Create VM Storage Policy:

Use PbmCreate()

Delete VM Storage Policy:

Use PbmDelete()

Check VM Storage Policy Compliance:

Use PbmCheckCompliance()

Given a Virtual Machine Home or Virtual Disk, provide associated VM Storage Profile:

Use PbmQueryAssociatedEntity()

Given a VM Storage Profile, provide the associated Virtual Machine Home or Virtual Disk:

PbmQueryAssociatedEntity

Use PbmQueryAssociatedProfiles()

PbmQueryAssociatedProfiles

Hopefully this provides a good overview of some of the common VSAN / SPBM operations and their associated API methods. If I have some spare cycles, I may create some sample scripts leveraging some of these API methods. In the meantime, you can check out these RVC (vSphere SDK for Ruby) modules leveraging both VSAN and SPBM APIs here and here.

Additional Resources:

If you wish to evaluate VSAN and its associated APIs, please sign up for the public beta here.
Sign up for a special VSAN event here
VM Storage Policy APIs aka Storage Profile APIs will be available in vSphere 5.5
SPBM (Storage Policy Based Management) MOB in vSphere 5.5