virtuallyGhetto

VSAN 6.6

New Nested ESXi 6.x Content Library 

by 14 Comments

A few years back I had showed how you could create and host your own 3rd Party vSphere Content Library which allows customers to decouple their content from the underlying vSphere environment and centralizing their content and making it available to number of vCenter Servers by simply just having an HTTP(s) endpoint. The other huge benefit is being able to take advantage of the existing web content tools for optimizing delivery or retrieval whether that is replication, caching, etc. and not relying a single vCenter Server for providing Content Library publication. In addition to showing how to create your own content libraries, I also had built my own 3rd Party vSphere Content Library which contains a variety of my Nested ESXi Templates (empty VM shells) running on Amazon S3 which can be consumed by anyone as long as you are running vCenter Server 6.0 or newer.

Although the empty Nested ESXi Templates were quite useful for myself and customers, it would have also been nice to include my pre-built Nested ESXi Virtual Appliances which I had recently updated to support vSphere 6.0 Update 3 and vSphere 6.5d (vSAN 6.6). Thanks to Dana Nourie, who runs our wildly popular VMware Flings Program, was kind enough to help me with the content hosting and now anyone can also subscribe to my Nested ESXi VA's and automatically have the content sync down using the vSphere Content Library feature.

UPDATE 1 (07/31/17) - The Nested ESXi Content Library has been updated to include the latest ESXi 6.5 Update 1 VA. If you are already subscribing to the library, it should have already pulled down the content (or at least the metadata which you can then force synchronization) or you can simply subscribe to the library and have access to the latest ESXi VA.

UPDATE 2 (05/07/18) - The Nested ESXi Content Library has been updated to include the latest ESXi 6.5 Update 2 VA. If you are already subscribing to the library, it should have already pulled down the content (or at least the metadata which you can then force synchronization) or you can simply subscribe to the library and have access to the latest ESXi VA.

To get started, just create a new vSphere Content Library and enter the following subscription URL: https://download3.vmware.com/software/vmw-tools/lib.json 


You can either download the content immediately or only when you need to use it. I recommend the former since its only two images which totals up to a whopping 1GB 😉

Once the creation of the Content Library has been completed, you should see the following two Nested ESXi VAs in the library which are now ready for deployment!


For more information about the Nested ESXi 6.0u3/6.5d VA's and how they work, please have a look at this blog post here. For more information about the Nested ESXi Templates and how to subscribe to the 3rd Party vSphere Content Library, please have a look at this blog post here.

Automating the new native VCSA bootstrap “Easy Install” in vSAN 6.6

by 10 Comments

In case you missed the previous article, have a read here which goes into greater detail behind the new VCSA bootstrap installer (also known as vSAN Easy Install) which is part of the new vSAN 6.6 release. As I hinted at the end of the previous post, customers not only have a simplified way of bootstrapping the VCSA on vSAN from a UI standpoint but they can also completely automate this leveraging some of the new vSAN Management 6.6 APIs, which are the same APIs that the UI uses.

A new Managed Object called VsanVcsaDeployerSystem is now available when connecting to either a standalone ESXi host as well as a vCenter Server. It contains the following three methods:

Here is the workflow for automating the VCSA bootstrap installer:

Step 1 - Connect directly to the ESXi host which you wish to bootstrap vSAN. You will use the VsanPrepareVsanForVcsa() API which accepts a list of disks for either a Hybrid or All-Flash vSAN datastore.

Step 2 - Deploy the VCSA like you normally would using the CLI Installer. You will specify the ESXi host that you had just prepared in Step 1 which includes the vSAN Datastore that was setup as part of that process.

Step 3 - Once the VCSA has been successfully deployed, you will connect to the vCenter Server and use the VsanPostConfigForVcsa() API which will create a vSphere Datacenter, vSphere Cluster and enable it with vSAN (which can also include Dedupe/Compression if you are using an All-Flash setup) and then automatically add the ESXi host that you had just bootstrapped. If you have provisioned other ESXi hosts that have not been configured with vSAN, you can also include that into the API request. The really nice thing about this "post" API is that rather than having to call into several existing vSphere APIs to setup vCenter Server, you can do all of that just using this single API!

To help demonstrate the use of the these new vSAN Management APIs, I have created a simple Python script which exercises these new APIs called vsan-vcsa-deployer-sample.py The script supports three operations: listdisk, prepare and post.

Here is an example of running the listdisk operation which will list all available disks that are currently not in use and can be used by vSAN:

python vsan-vcsa-deployer-sample.py -s 192.168.1.100 -u root -p VMware1! --operation listdisk

Once you have the disks information, you can then use the prepare operation as shown below to bootstrap your ESXi host:

python vsan-vcsa-deployer-sample.py -s 192.168.1.100 -u root -p VMware1! --operation prepare --cache "SAMSUNG MZVPV128" --capacity "Samsung SSD 850"


At this point, you are now ready to deploy the VCSA using the CLI Installer. Once that has completed, you can complete the process by using the post operation and provide the required parameters to setup vCenter Server including the ESXi host that you had just bootstrapped so it can be added to the vCenter Server inventory as shown below:

python vsan-vcsa-deployer-sample.py -s 192.168.1.200 -u '*protected email*' -p VMware1! --operation post --datacenterName "VSAN-Datacenter" --clusterName "VSAN-Cluster" --esxName 192.168.1.100 --esxUsername root --esxPassword VMware1!


Once the post operation has completed, you will have a fully configured vCenter Server which you can check by logging into the vSphere Web Client. Pretty slick, if you ask me!

Correlating vSAN perf metrics from vSphere Web Client to both PowerCLI & vSAN Mgmt API

by 4 Comments

While going through the PowerCLI 6.5.1 release notes last week, in addition to the new Get-VsanView cmdlet which exposes the complete vSAN Management API through PowerCLI, I had learned that we had also released a new Get-VsanStat cmdlet. This new cmdlet allows customers to easily retrieve the various vSAN Performance Metrics provided by the vSAN Performance Service.


The really nice thing about vSAN Performance Service is that all vSAN stats are now available directly in the vSphere Web Client, where as before, this information was only available as part of the vSAN Observer which was a completely different interface. The other huge benefit to using the vSAN Performance Service is that it also stores the historical stats compared to the vSAN Observer which was primarily used for real time troubleshooting. As of vSAN 6.6, the vSAN Observer has been deprecated and will only be used by GSS in limited scenarios, everything you were able to do with the vSAN Observer is now possible with the vSAN Performance Service.

While trying out the new Get-VsanStat cmdlet, I had found it to be quite difficult to easily map the vSAN metrics I saw in the UI to the specific PowerCLI query required to extract that information. The documentation was also quite light and only included a single sample and although some of the metrics could easily be deduced, there were many others that I was just unsure of. I had also tried using the vSAN Management API directly, thinking that I might have more luck but it was also challenging to use for other reasons and I still ran into the same problem which was how do I easily map what I saw from the UI down to the API or even associating that back to PowerCLI.

After spending a few days with BOTH PowerCLI and the vSAN Management API and with a bit of frustration, I think I have finally figured out how to map what I saw from the UI back to the both CLI and API. This was not an easy task as and I had to cross-reference multiple datasources to build up this mapping and I thought I would put together a reference which outlines this mapping so that others would have to go through the same pain. IMHO, this should be a pretty straight forward task. In addition, I have also provided a PowerCLI sample for each of the metric types as well as the associated vSAN Management API mapping as those differ in name as well. This hopefully should make it easy for anyone to start using either of these interfaces for collecting vSAN metrics from an Automation standpoint. As part of this exercise, I also ran into a variety of bugs which I have already filed internally and all this information has been feedback to Engineering teams to hopefully improve both our CLI and API in future updates.

[Read more...] about Correlating vSAN perf metrics from vSphere Web Client to both PowerCLI & vSAN Mgmt API

Managing & silencing vSAN Health Checks using PowerCLI

by 3 Comments

One neat capability that was introduced with vSAN 6.6 is the ability to "silence" and disable specific vSAN Health Checks. A recent use for this came up on Duncan's blog where the vMotion health check would fail if you are using a vMotion network stack. As you can imagine, this feature can also come in handy for vSphere Home Labs where your hardware may not be on the official VMware HCL and wish to disable those specific vSAN Health Checks.


[Read more...] about Managing & silencing vSAN Health Checks using PowerCLI

Getting started w/the new PowerCLI 6.5.1 Get-VsanView cmdlet

by 1 Comment

One of the things that I am most excited about from an Automation standpoint with the vSAN 6.6 release is that customers using PowerCLI will now have complete access to the vSAN Management API which we had initially introduced back in vSphere 6.0 Update 2. In PowerCLI 6.5R1, customers only had access to high level vSAN cmdlets which did a pretty good job covering the broad set of vSAN functionality. However, it did not expose the complete vSAN Management API and this has been something many customers have been asking about.

With the new PowerCLI 6.5.1 release, a new Get-VsanView cmdlet is now available that will exposes the complete vSAN Management API using PowerCLI. Since the vSAN Management API has been around since vSphere 6.0 Update 2, you will also be able to use this new cmdlet against a vSAN 6.2, vSAN 6.5 and vSAN 6.6 environment! You simply just need to update your PowerCLI installation which you should always do to get the latest fixes and enhancements.

When you connect to either a vCenter Server and/or ESXi host, you will be able to view all available vSAN Managed Objects for the system by simply running the cmdlet without any arguments as shown in the screenshot below.


If you wish to access a specific vSAN Managed Object type, then you will need to pass in the vSAN MoRef ID from the given list. Here is an example of accessing the "VsanVcClusterHealthSystem-vsan-cluster-health-system" which will give us access to the VsanVcClusterHealthSystem.

[Read more...] about Getting started w/the new PowerCLI 6.5.1 Get-VsanView cmdlet

SMART drive data now available using vSAN Management 6.6 API

by 1 Comment

One of the major storage enhancements that was introduced in vSphere 5.1 as part of the new I/O Device Management (IODM) framework was the addition of SMART (Self Monitoring, Analysis And Reporting Technology) data for monitoring FC, FCoE, iSCSI, SAS protocol statistics, this is especially useful for monitoring the health of an SSD device. Historically, there was not a public vSphere API to consume this information and customers had to rely on ESXCLI which is not very friendly from a programmatic standpoint.


One of the nice enhancements that was introduced in vSAN 6.6 from an API standpoint is that you can now access SMART data using the vSAN Management 6.6 API. The other really cool thing about this enhancement is that although this API was added under the vSAN Management API, you do not actually have to be using vSAN to be able to use this new API!

There are two methods in which you can access the SMART data:

  • vCenter Server - When connecting to a vCenter Server, you can access the VsanQueryVcClusterSmartStatsSummary() method which is available as part of the VsanVcClusterHealthSystem and you simply just provide it the name of a vSphere Cluster.
  • ESXi Host - When connecting directly to an ESXi host, you can access the VsanHostQuerySmartStats() method which is available as part of the HostVsanHealthSystem.

To demonstrate how these two new APIs work, I have create two sample scripts: vsan-smarts-data-sample.py using vSAN Management SDK for Python and VSANSmartsData.ps1 using the new PowerCLI Get-VsanView cmdlet.

Here is an example of running the python sample:

python vsan-smarts-data-sample.py -s 192.168.1.200 -u '*protected email*' -p 'VMware1!' -c VSAN-Cluster


Here is an example of running the PowerCLI sample:

Get-VSANSmartsData -Cluster VSAN-Cluster