I got a question from my buddy Paudie O'Riordan this morning where he was noticing a strange issue while trying to upgrade his ESXi hosts from 6.0 to 6.5 (all on the VMware HCL). Like many of our customers, he runs ESXi on USB device and when he attempted to upgrade using ESXi Scripted Install (Go Automation!), he was surprised to find that his USB device was not being detected.
Interestingly, I had literally just finished answering a similar question on our internal Socialcast forum and I had wondered if Paudie was also seeing the same problem. The issue looks to be related to the new USB Native Driver (vmkusb) that was introduced in ESXi 6.5 where is it is unable to claim the specific USB device.
Although you can disable the USB Native Driver and fall back to the legacy driver as mentioned in this VMware KB 2147650, but because this is happening during the installation/upgrade process, it can get a bit tricky.
The native remote syslog functionality in the vCenter Server Appliance (VCSA) for vSphere 6.5 introduces several new changes from vSphere 6.0. With some of the questions that I have been receiving on this topic, I figure it would be useful to take a closer look at some of the different behaviors and configuration differences. Hopefully by the end of this article, you will have a better understanding of the syslog capabilities in VCSA 6.5
Remote Syslog Configuration
In VCSA 6.0, to configure the remote syslog configuration, you needed to use the vSphere Web Client. Although this may have felt like a convenience, it also added an unnecessary dependency on both vCenter Single-Sign On (SSO) and the vSphere Web Client UI. In VCSA 6.5, the remote syslog configurations is now part of the VAMI UI (https://[VCSA]:5480) which is an out-of-band interface that can still function if either SSO or vCenter Server is down. Once you have saved your changes, the syslog client will automatically be restarted for the changes to go into effect. If you wish to disable the remote syslog functionality, simply click on the reset button.
Note: If you decide to use port 1514, I have found that you must use the TLS protocol rather than TCP or else it will not work.
In the previous article (Part 6), we demonstrated how you can easily associate the list of VMDKs to their respective OS partitions within a VCSA or PSC node, which is useful when you need to increase the disk capacity for a specific OS partition. In Part 7, we are now going to drill down a bit further into the underlying filesystem and retrieve both the storage capacity as well as utilization of each partition.
VAMI UI Area of Focus
There is not a granular view of the individual OS partitions within the VAMI UI (https://[VCSA]:5480). However, this information is available as part of the VAMI information when logged into the vSphere Web Client by navigating to System Configuration->Nodes->Monitor->Storage as shown in the screenshot below.
The workflow for retrieving the storage statistics using the VAMI APIs is also applicable for retrieving other statistics such as compute, memory and networking. You first need to retrieve the list of VAMI Statistic IDs that are available from the monitoring API endpoint which is demonstrated with the Get-VAMIStatsList function. From each VAMI Stat ID, you can also drill down further to get more details such as description and the unit of measurement. Once you have the Stat IDs that you are interested in, you can then perform a query based on a specific set of search criteria which can be time range (start and stop time), interval, etc. To demonstrate how the query API works, I have created the Get-VAMIStorageUsed function to demonstrate its usage.
VAMI APIs Used
- GET /appliance/monitoring
- GET /appliance/monitoring/query
The first function Get-VAMIStatsList does exactly as it sounds, it simply lists all VAMI Stat IDs that are available. Most of the Stat ID names are pretty descriptive but you can always retrieve more information by performing a GET on /appliance/monitoring/[VAMI-STAT-ID].
As stated in the beginning, we are interested in the grabbing the storage stats. The Get-VAMIStorageUsed function takes a subset of the VAMI Stats ID, specifically the storage ones that map to the OS partitions and retrieves both the total and used capacity (MB) as shown in the screenshot below.
In Part 6, we will take a look at how we can use the new VAMI APIs to easily associate the underlying VMDKs to their respective OS disk partitions for a VCSA or PSC node. In addition, the workflow of increasing the disk capacity for a specific partition has also been simplified further with the new VAMI APIs. After increasing the specific VMDK size, we can now also trigger the partition resize operation using the VAMI APIs, where as before this used to be a manual task that required SSH access. In vSphere 6.5, there have been a few minor changes to the VCSA's VMDK layout and sizes, for more details, please have a look at this blog post here.
VAMI UI Area of Focus
Unfortunately, there is not a page within the VAMI UI (https://[VCSA]:5480) that either lists or provides the actual mapping of the underlying VMDKs to their respective partition types. You can see the different VMDKs using the vSphere Web/C# Client, but historically the mapping of VMDK to partition type was done manually or you would refer to the table found in the blog post referenced above. Lets see if we can pull this information without needing to go to a UI 🙂
VAMI APIs Used
- GET /appliance/system/storage
- POST /appliance/system/storage/resize
I had just deployed a new vRealize Log Insight (vRLI) 4.0 instance in my home lab environment to investigate a behavior that I was seeing with another product, non-vRLI related. Due to the nature of the work, I needed to have a pristine vRLI environment each time to study the results. I had already forwarded some logs into vRLI and rather than deploying another instance or re-deploy the current instance, what I really wanted to be able to do is to just wipe all the logs in vRLI but did not see an option within the UI. I also could have used VM snapshots, but was hoping there was a cleaner solution that vRLI provided out of the box.
The next place I looked immediately after was Mr. Log Insight's site aka Steve Flanders blog but there was nothing there about this other than archiving. After a few Google searches, I came across this exact same question on the vRLI Ideas site but sadly there was no solution and it was dated back in 2014. Though Steve makes a good point about just letting the logs rotate out automatically, in my case, this was not an option and I needed a pristine environment.
Being the curious one, I figured there has to be a way, even if it is not officially recommended nor supported. As you probably have guessed, I did find a way but I would caution that you read the disclaimer below before proceeding further. This was something I needed to do in my lab to test a few scenarios that was non-vRLI related, but I needed syslog target, so this is why I am using vRLI 🙂
Disclaimer: This is probably not officially supported nor recommended by VMware. Please use at your own risk. YOU WILL LOSE ALL YOUR LOGS