vm storage profile

Feedback on default behavior for VM Storage Policy

06/05/2018 by William Lam 1 Comment

Today, the vCenter REST (vSphere Automation) APIs currently does not support the use of VM Storage Policies when relocating (vMotion, Cross Datacenter vMotion & Storage vMotion) or cloning an existing Virtual Machine. Customers have provided feedback that this is something that they would like to see get added to the current REST APIs and while this is being looked at, there were a couple of open questions from Engineering.

The following 2-question survey below is to help us understand what the "default" behavior should be when a Virtual Machine is being relocated or cloned within a vCenter Server and a VM Storage Policy is NOT specified when using the APIs. The reason for this is that our existing APIs for relocate and clone today are very flexible and not everything needs to be specified as part of the relocate or clone API specification. However, due to this flexibility, you may observe different behaviors and we would like to understand what the default behavior should be when some of these paraemters are not specified. In the case where you want to be explicit, you can always specify the VM Storage Policy, but the survey is to understand when it is not specified.

Survey: https://goo.gl/forms/aQjVTly2MmVHeRsp1

Thank you for taking the time to provide your feedback, this will help us build an easy and robust API when dealing with relocate and clone operations using the vCenter REST APIs.

Minimum permissions to view VM Storage Policies

01/19/2017 by William Lam Leave a Comment

Any way to allow vCenter users to see #SPBM policies without using global permissions?
No pol visible w/cluster level permissions #vmware

— Aaron Patten (@Jedimt) January 18, 2017

I saw this question from Aaron yesterday while scrolling through my Twitter timeline and after answering it, I figure I write a quick blog post about it in case this comes up in the future.

There are two specific privileges around managing VM Storage Policies: Update and View as shown in the screenshot below. If you only want to allow users to be able to see all the available VM Storage Policies that have been defined, then you just need to create a new Role with only the "View" privilege.

Secondly, it is important to note that VM Storage Policies are defined and managed at a vCenter Server level. This means that when you assign the permission, it needs to be applied at the root vCenter Server level (you do not have to propagate it down wards if you do not wish to show the rest of the vSphere Inventory). Global permissions are not required, but if you have multiple vCenter Servers which are all part of the same SSO Domain, you may want to consider this if users are allowed to login to any one of the vCenter Servers.

Once you have assigned the permission to either the user or group, then you can have them login using either the vSphere Web Client or using the SBPM APIs and you will now be able to view all defined VM Storage Policies.

SPBM APIs are now included in pyvmomi (vSphere SDK for Python)

01/19/2017 by William Lam 3 Comments

I have been spending quite a bit of time lately with PowerCLI Core, especially with one of my pet projects. One of the limitations that PowerCLI Core has today is that the Storage cmdlets which includes vSAN and VVol functionality has not been ported over yet. This means that if you need to do something with VM Storage Policies for example, it would not be possible with PowerCLI Core and you would have to use the Windows PowerCLI version instead.

While investigating for an alternative solution to PowerCLI Core to get access to the Storage Policy Based Management (SPBM) APIs, I was pleasantly surprised to learn that pyvmomi (vSphere SDK for Python) had recently added support for the SPBM APIs in their 6.0.0.2016.4 release last year. I had accidentally stumbled onto this news while looking through the pyvmomi Github issues, specifically this one here. I was surprise to see there was no mention of this enhancement in the pyvmomi release notes.

This is great news for pyvmomi consumers and given this was news to me, I am guessing it might be news for others so figure I would also share the info. While looking into using the SPBM APIs from pyvmomi, I did not see any sample scripts showing how to use the SPBM API. Given I needed to write a script for my project, I figure I would also create a couple of examples to help others get started.

New “raw” VM Storage Policy support in OVFTool 4.2 simplifies bootstrapping VMs onto VSAN

01/11/2017 by William Lam 1 Comment

Several weeks back I came across a handy little tidbit on an enhancement that was added to the latest version of OVFTool (4.2) that greatly simplifies the bootstrapping of a vCenter Server or any other VM for that matter onto a vSAN Datastore. This is generally used when setting up a pure greenfield environment where your vCenter Server may not be setup yet and you want to run it on top of vSAN which is fully supported configuration. The process of "bootstrapping" a VCSA onto a single node vSAN datastore is documented on my blog here. The high level steps are as follows:

Temporarily change default ESXi VM Storage Policy to allow forced provisioning and FTT=0 (e.g. no protection)
Claim disks for creating vSAN Diskgroup(s)
Create vSAN Cluster
Deploy VCSA
Apply the default vSAN VM Storage Policy to VCSA VM
Revert the temporarily change of default ESXi VM Storage Policy

With this new OVFTool enhancement, Step 1 and 6 is no longer needed from the standpoint of needing to change the default VM Storage Policy on the ESXi host using the ESXi Shell or using the vSphere API. Instead, you can now pass in a "raw" VM Storage Policy (SPBM) to apply to the specific OVF/OVA that is being deployed rather having to make a global change on the ESXi host. This also helps reduce the post-deployment steps as you only need to re-apply the default vSAN VM Storage Policy to the vCenter Server VM and not have to touch ESXi host settings once the vCenter Server is up and running.

To use this new raw VM Storage Policy feature in OVFTool, there is a new command-line option called --defaultStorageRawProfile which accepts the "raw" VM Storage Policy as you would normally provide to SPBM APIs such as "((\"hostFailuresToTolerate\" i1) (\"forceProvisioning\" i1))" for example.

The really cool thing about this feature is that you can take advantage of this directly with the new OVFTool argument passthrough feature that was introduced in the VCSA 6.5 CLI deployment utility. Combining these solutions together, you can easily simplify the "bootstrapping" of a VCSA onto a vSAN Datastore. Below is the snippet that you would include in the VCSA JSON configuration file used for deployment.

"ovftool.arguments" : {
"defaultStorageRawProfile" : "((\"hostFailuresToTolerate\" i1) (\"forceProvisioning\" i1))"
}

Although a tiny enhancement, I think this is a pretty neat capability, especially being able to make use of it natively within the VCSA configuration file. It is definitely great to see us continue to simplify how VMware management infrastructure is deployed and definitely stay tuned on what else we have cooking in the future for this particular area 🙂

Unable to apply VSAN VM Storage Policy on NSX Controller/Edge VMs?

01/07/2015 by William Lam 2 Comments

This post was inspired by a recent Twitter conversation with Joep Piscaer who ran into an interesting challenge with VSAN and NSX.

I want to apply a VSAN VM Storage Policy to a NSX Controller, but I’m getting an "The method is disabled by ‘vShield_SVM’” error. Any ideas?

— Joep Piscaer (@jpiscaer) December 31, 2014

The issue that Joep encountered was not being able to apply a VSAN VM Storage Policy onto an NSX Controller VM which resided on a VSAN Datastore. Below is a screenshot of the error message "The method is disabled by vShield_SVM" if you tried to apply the VM Storage Policy.

The reason Joep is seeing this error is because the NSX Controller VM is a special "Service VM" that is being managed by a specific solution, in this case it is VMware NSX. To ensure that users do not accidentally modify these "Service VMs", certain set of functionality has been disabled on these VMs from regular users. Any configuration changes that are required are initiated through the solution itself which has full administrative access to these VMs. This issue is actually not specific to the NSX Controller VMs but also applies to the NSX Edge VMs: ESR (Edge Service Router) and DLR (Distributed Logical Router).

In fact, this applies to any "Service VMs" which are being managed by a VMware Solution or 3rd party Solution. You will notice that you will not be able to edit these VMs like you would normally on other VMs. One suggestion from a community member was to check out the VMware KB 2008957 which has users manually tweaking the VCDB, which I am not a big fan of if I can help it. If you want to know why I do not recommend this, check out this post here where kittens might get harmed. Now, getting back to Joep's request, is there a solution for him? After all, his request is a valid one where he has deployed an NSX Controller VM on a VSAN Datastore and wishes to apply a specific VSAN VM Storage Policy.

Luckily, there is cleaner work around that does not involve messing around with the VCDB and crafting ugly SQL queries. The way these "methods" or operations are disabled on a particular set of VMs is through the use of a private vSphere API available through vCenter Server called disableMethods. You can actually view the list of disableMethods by viewing a particular VM using the vSphere MOB under config->disableMethod property as seen in the screenshot below.

The list of disableMethods map to the specific vSphere API calls for a VM and in the case of modifying a VM which includes applying a VM Storage Policy, the method that is used is is called the ReconfigVM_Task which we can see in the screenshot mapping to vim.VirtualMachine.reconfigure. If we want to be able to apply a VM Storage Policy, we simply just need to temporarily remove this particular operation from the disabbleMethods list.

Here are the instructions for enabling ReconfigVM_Task method:

Step 1 - You will need to find the MoRef (Managed Object Reference) ID of the VM that you wish to enable the method on. You can do this by either browsing through the vSphere MOB, using this vSphere SDK for Perl script or this PowerCLI snippet:

Get-VM -Name [VM-NAME] | ft -Property Id

Step 2 - Open a web browser to the following URL:

https://[VC-IP]/mob/?moid=AuthorizationManager&method=enableMethods

Step 3 - You will need to replace the following two parameters (make sure to replace the VM MoRef ID with the one you found in Step 1):

parameter	value
entity	<entity type="ManagedEntity" xsi:type="ManagedObjectReference">vm-35</entity>
method	<method>ReconfigVM_Task</method>

Step 4 - Once you have updated fields as shown in the screenshot below, to execute the API call you just need to click on the "Invoke Method" link on the bottom right.

If everything was successful, you should see some output from the operation listing the methods that are still currently disabled. You can also confirm that everything is working by refreshing the vSphere Web Client or if you are using the vSphere C# Client, the "Edit Settings" option should now be available. Lastly, if I now apply a VSAN VM Storage Policy, I will no longer get the error and as you can see from the screenshot below, I now have successfully applied my "VSAN-Platinum-VM-Storage-Policy" for my NSX Controller VM. I would strongly recommend that you re-enable the original disable method by following the instructions below.

Here are the instructions for disabling ReconfigVM_Task method:

Step 1 - You will need to find the MoRef (Managed Object Reference) ID of the VM that you wish to enable the method on. You can do this by following Step 1 from the enable instructions

Step 2 - Open a web browser to the following URL:

https://[VC-IP]/mob/?moid=AuthorizationManager&method=disableMethods

Step 3 - You will need to replace the following three parameters (make sure to replace the VM MoRef ID with the one you found in Step 1 and any random number works for sourceId property):

parameter	value
entity	<entity type="ManagedEntity" xsi:type="ManagedObjectReference">vm-35</entity>
method	<DisabledMethodRequest><method>ReconfigVM_Task</method><DisabledMethodRequest>
sourceId	1234

Step 4 - Once you have updated fields as shown in the screenshot below, to execute the API call you just need to click on the "Invoke Method" link on the bottom right.

If everything was successful, you should see a void output and if you go to your vSphere Web/C# Client, you should see that the "Edit Settings" option is now disabled again for this VM. Though applying a VSAN VM Storage Policy is pretty trivial, there are some additional things to be aware of when working with special "Service VMs" like the NSX Controller. Hopefully we can improve this workflow in the future and provide for a better user experience but in the mean time, you can use this workaround.

Exploring VSAN APIs Part 6 – Modifying Virtual Machine VM Storage Policy

03/20/2014 by William Lam 6 Comments

One of the biggest benefit of VSAN is the ability to specify granular storage policies on a per Virtual Machine basis. These storage policies is managed through VMware's Storage Policy Based Management system and is automatically enforced by VSAN to ensure compliance. A VM Storage Policy can be assigned during the initial deployment of a Virtual Machine or it can be modified afterwards, for example if the Virtual Machine's SLA's has changed because the workload has changed. From the vSphere Web Client, modifying a Virtual Machine's VM Storage Policy is simply selecting the VM Storage Policy and re-applying which is also available programmatically through the vSphere API.

Using the vSphere API method ReconfigVM_Task(), you will be able to modify the VM Storage Policy for the VM Home Namespace and/or individual Virtual Disks. To modify the VM Home Namespace, there is a property defined at the root of the Virtual Machine config spec called vmProfile which accepts the VM Storage Policy ID extracted from the SPBM API. To modify the VM Storage Policy for an individual Virtual Disk, you will need to set the profile property which is exposed on a Virtual Device with the VM Storage Policy ID. To demonstrate this functionality, I have created a sample vSphere SDK for Perl script called changeVMStoragePolicy.pl

Disclaimer: These scripts are provided for informational and educational purposes only. It should be thoroughly tested before attempting to use in a production environment.

In my environment, I have a Virtual Machine called VM1 which has been defined with a VM Storage Policy called "Copper" as seen in screenshot below:

Let's say I want to change the Virtual Machine's VM Storage Policy to another policy called "Aluminum", I first need to extract the VM Policy ID from SPBM API and then pass it into the script like the following:

./changeVMStoragePolicy.pl --server vcenter55-1.primp-industries.com --username root --vmname VM1 --profileid cd6908b2-0704-4733-ad9b-a9a8f200ab0a

Once the Virtual Machine has been reconfigured, we can then take a look in our vSphere Web Client and we can see the VM Storage Policy has now been changed and VSAN will automatically enforce these new requirements.

If you wish to assign a VM Storage Policy as part of a new Virtual Machine creation, you just need to set the vmProfile and profile properties which is similar to a reconfiguration operation.