VIN 2.0 Supports New Export to CSV & Maps Feature

VMware just released the new vCenter Operations Management Suite 5.6 this evening which includes all new updates to the following products:

• vCenter Operations Manager 5.6 
• vCenter Configuration Manager 5.6
• vFabric Hyperic 5.0
• vCenter Infrastructure Navigator 2.0
• vCenter Charge Back Manager 2.5

There are too many cool new features in each of these products to list them all out (I recommend you check out the release notes for each product). Though, two new features that I would like to point out is from latest vSphere Infrastructure Navigator 2.0 release (also known as VIN) which allows you to export the VIN data to CSV output as well as exporting the maps to PNG.

You have the option of exporting all virtual machines from a given vCenter Server or filter out a subset of the virtual machines by navigating to a specific object using the object navigator in the vSphere Web Client. In the screenshot below, I would like to export all virtual machines that VIN is collecting for a particular vSphere Cluster, so I select the Virtual Machines in that Cluster on the left and then select the Manage tab and then click on Application Services.

The export to CSV button is located right next to the filter box and this will export the entire table view that is seen to a CSV file which you can then save onto your local desktop. The filename will automatically be saved as vin-inventory.csv and here is a screenshot of the output file:

Note: If you filter the list of virtual machines using the Filter box, the export will still capture all virtual machines under this view. If you wish to capture a specific set of VMs, you will need to use the object navigator to filter out the specific objects before exporting.

To export the application dependency maps from VIN, you will need to be in the context of an individual virtual machine and again under Manage tab and then click on Application Services. The map export button is located right next to the zoom in/out option in the upper right corner as shown in the screenshot below.

I really like map export feature as you can sit back and let VIN do all the hard work of mapping out all the applications and VM dependencies for us in a graphical manner and then just export the picture which can then be used with documentation, CMDB diagrams, auditing, etc. These are just two out of the many new features found in the latest release of VIN 2.0. I highly recommend you give VIN a try if you are not already using it in your vSphere environment!

One additional note that I would like to point out, the data being exported to CSV does not capture all the application details such as the services, ports, processes, etc. that you might see from the VIN UI. If you wish to extract all this information from VIN for your own use, you can do so by taking a look at the following articles:

• Extracting Information from VIN (vSphere Infrastructure Navigator) Part 1
• Extracting Information from VIN (vSphere Infrastructure Navigator) Part 2
• Alternative Way of Extracting VIN (vSphere Infrastructure Navigator) Information

Getting Rid of the Inventory Tree in the New vSphere Web Client

I don't know about you, but I really like using the new inventory list compared to the old inventory tree when I need to find something in new vSphere Web Client. The inventory list does not rely on the static and limited hierarchical tree view to display your vSphere objects. Instead, it groups common vSphere objects together (works across multiple vCenter Servers) along with links to other related objects. This allows you to quickly navigate to a particular vSphere object and with just a click away to other related objects for further inspection. Finally, you will no longer have to worry about the "white screen of death" which was a common problem when trying to display huge inventories and sometimes even smaller ones while using the tree view. 

To be honest, I was not a fan of the inventory list at first, but after spending some time with it, I quickly realized the benefits of moving away from the old hierarchical tree view. I actually like the new inventory list so much, that I personally wanted like to get rid of the inventory tree view as it is an extra mouse movement to get to the inventory list. I sometimes even accidentally click on the inventory tree when browsing too quickly through the vSphere Web Client.

I thought it might be a long shot to see if it was possible to remove the inventory tree since I assumed it might be part of the compiled code. Surprisingly, I found out from one of the developers, there was actually a pretty simple way (aka "hack") of removing the inventory tree. 

Disclaimer: This is probably not officially supported by VMware, please use at your own risk.

In the example below, I am using the VCSA (vCenter Server Appliance) which has the vSphere Web Client installed by default but this should also work for a Windows vCenter Server that has the vSphere Web Client Server installed.

Step 1 - We need to make a backup of the following file /usr/lib/vmware-vsphere-client/plugin-packages/vsphere-client/plugins/inventory-viewer-war-5.1.0.war which contains the file that we need to edit. The command below will just make a backup copy called inventory-viewer-war-5.1.0.war.BAK

cp /usr/lib/vmware-vsphere-client/plugin-packages/vsphere-client/plugins/inventory-viewer-war-5.1.0.war /usr/lib/vmware-vsphere-client/plugin-packages/vsphere-client/plugins/inventory-viewer-war-5.1.0.war.BAK

Step 2 - Next, we will go ahead and extract the contents of the WAR file which is basically a zip archive in our home directory so that we can edit a file. Run the following command which will extract the contents into a directory called TEMP under /root.

unzip /usr/lib/vmware-vsphere-client/plugin-packages/vsphere-client/plugins/inventory-viewer-war-5.1.0.war -d ~/TEMP

Step 3 - Change into the ~/TEMP directory and you should see a file called plugin.xml which we will be editing. Use an editor such as vi and locate the following section and comment it all out using the <!-- <text> --> notation as shown below
Step 4 - Once you have finished editing the plugin.xml file, go ahead and save the file. Now we will need to re-create the inventory-viewer-war-5.1.0.war file and to do so, inside the TEMP directory, run the following command:

zip -r inventory-viewer-war-5.1.0.war *

Step 5 - We now need to copy the modified inventory-viewer-war-5.1.0.war back into the vSphere Client Plugins directory. Run the following command to copy the WAR file into plugins directory:

cp inventory-viewer-war-5.1.0.war /usr/lib/vmware-vsphere-client/plugin-packages/vsphere-client/plugins/inventory-viewer-war-5.1.0.war

Step 6- Finally, for the changes to go into effect, we just need to restart the vSphere Web Client service by running the following command:

/etc/init.d/vsphere-client restart

If everything was successful, then you should be able to login to the vSphere Web Client and when you click on the main vCenter home on the left, you should no longer see the inventory tree view, just the inventory lists.

Even though we removed the inventory tree from the object navigator, you can still access the four tree views using the shortcuts found on the home page:

If you really want to disable those as well, you can comment out the following four sections:
OR better yet, re-link them to the main vCenter home view by adjusting the targetViewUid to point to vsphere.core.viHome.domainView

Alternative Way of Extracting VIN (vSphere Infrastructure Navigator) Information

Here is an alternative method of extracting the super useful information from VIN (vSphere Infrastructure Navigator) which I stumbled across while troubleshooting an earlier VIN 1.2 deployment. This method is much simpler from a data extraction point of view and has less prerequisites compared to the first method documented here and here, but it does require a bit more data processing.

Once VIN is up running, there are a set of zip files that are created in /var/log/vadm/results which are name with the virtual machines MoRef ID. We can quickly view the contents of one of these files (without needing to extract the zip file) by using zcat, here is the command to view one of the zip files: zcat /var/logs/vadm/vm-238.zip

From the screenshots above, we can see it contains the same set of information as displayed in the VIN UI. The information is broken down into several section including high level summary of the VM, individual processes running in the virtual machine, application/services known by VIN as well as a netstat table.

Instead of operating on these files directly, you should make a separate copy of the directory for further processing. Since the file name contains the VM's MoRef ID instead of the human readable VM display name, I wrote a quick script called extractVINData.sh which creates a copy of data and rename the files to VM display name.

To use the script, you just need to download it and upload it to your VIN appliance (make sure you set the execute permission on the script). Then just run the script and it will automatically copy the directory to /root/vin-results and translate the VM MoRef ID by correlating with the /var/log/vadm/engine.log file.

Here is a screenshot of the script output:

So if you do not mind some parsing, you can easily extract the data VIN is collecting by simply SSHing to the VIN appliance and periodically copying out /var/log/vadm/results directory. From my quick test, this directory is automatically updated as new VMs are brought online and inventoried by VIN.

VCSA (vCenter Server Appliance) 5.1 VCDB & SSODB Password

I recently helped answer a question internally about the default credentials to the VCSA 5.1 (vCenter Server Appliance) vCenter Server and SSO (Single-Sign On) Database for troubleshooting purposes. I thought I share the details in case this might help others.

With the release of the vSphere 5.1, the VCSA now runs VMware's vPostgres database for both the VCDB and SSODB. You should also know there is no default credentials for the database as the passwords are automatically generated during the initial application install and the database password will be unique on every VCSA.

However, you can still retrieve the password for both the VCDB as well as SSODB (this took a bit of digging in the appliance).

Disclaimer: You should not have any reason to go into the actual DB of either vCenter Server or SSO other than potential troubleshooting with VMware Support. Please use caution if you do choose to connect to the DB, as you can potentially impact your system.

VCDB Credentials:
You can view the credentials for the VCDB in the following file: /etc/vmware-vpx/vcdb.properties

Here is a screenshot of the file content as well as using psql client located on the VCSA (/opt/vmware/vpostgres/1.0/bin/psql) to connect to the VCDB:

SSODB Credentials:
You can view the credentials for the SSODB in the following file: /usr/lib/vmware-sso/webapps/lookupservice/WEB-INF/classes/config.properties

Here is a screenshot of the file content as well as using psql client located on the VCSA to connect to the SSODB:

Extracting Information from VIN (vSphere Infrastructure Navigator) Part 2

In my previous article Extracting Information from VIN (vSphere Infrastructure Navigator) Part 1, we took a look at the data VIN was collecting through an interface called Jolokia. Utilizing a tool called j4psh, we were able to easily view and explore the data in VIN remotely. In this article, we will take a look at how easily you can extract the data we explored in the previous article using a very simple script.

While going through the Jolokia website and walking through the tutorial, I found there were several programmatic clients that could be used to connect to the Jolokia service which includes Java, Javascript and Perl. Since I am most familiar with Perl, I wrote a very simple Perl script called getVINData.pl leveraging the information from my previous article.

Disclaimer: This is not officially supported by VMware, use at your own risk.

Before using the script, you will need to run through the two per-requisites outlined in the previous blog article: VIN appliance setup and installation of Jmx4Perl. Once you have completed these two steps, you are now ready to execute the script (make sure the script has the executable permission set). The script is pretty straight forward it accepts two input parameter: VIN hostname/IP Address and the name of the virtual machine you wish to query.

In the example below, I am connecting to my VIN host which has an IP Address of 172.30.0.150 and I am querying a virtual machine with the name Analytics VM (one of the vC Ops VMs).

From the screenshot above we can see the following:

• The vCenter Server that VIN is currently registered to
• VM Summary information
• Applications/Services currently running on the VM
• VM Dependencies

If we take a look at the vSphere Web Client and the VIN data for this particular VM, we should see the same set information:

Though the script already contains quite a bit of information, it is just a sample of what can be done. With further exploration you can easily extend the script to extract other pieces of information and possibly even use other scripting/programming languages to connect to this interface. As I mentioned before, VIN is a very powerful tool for your vSphere infrastructure and now you can gain additional benefits by leveraging it's valuable data externally!

Extracting Information from VIN (vSphere Infrastructure Navigator) Part 1

vSphere Infrastructure Navigator or VIN for short is a really useful tool that helps you map out and visualize application services and dependencies between your virtual machines. This is extremely helpful when trying to identify what applications or ports a given virtual machine is running and how it is related or dependent on other virtual machines within your virtual infrastructure.

One thing that I found lacking in VIN today is the ability to export or extract some of this useful information for external use such as a CMDB (Configuration Management Database) as one potential use case. This would allow users to always have an up-to-date CMDB that does not rely on a manual process of adding CI (Configuration Items) and services to your CMDB which can potentially lead to incorrect or more importantly, out of date information (I have seen this happen in the past as an System Admin).

While deploying the latest release of VIN 1.2 in my home lab, I decided to poke around in the VIN appliance and I noticed a web application called Jolokia (not the hot pepper) that was running. After a quick Google, I came across this Jolokia tutorial and with some trial and error, I was able to validate the steps against VIN and was able to access the data VIN was discovering about the virtual machines.

In part 1 of this article, I will show how to get setup and start exploring the data VIN is collecting and in part 2, I will provide a sample script which can be used to automate the extraction of some of this data found in VIN. In the example below, I will be leveraging the vMA appliance, you could use another system that can install the Jolokia client packages, please refer to tutorial for more details.

Disclaimer: This is not officially supported by VMware, use at your own risk. 

VIN Appliance Setup:

Before we begin, we need to configure a few things on the VIN appliance which will allow us to connect remotely which is disabled by default.

Step 1 - Edit the following configuration file /opt/vadm-engine/webapps/jolokia/WEB-INF/classes/jolokia-access.xml and specify the remote IP Address you wish to connect to VIN from. By default, VIN is configured to only allow connections from localhost (127.0.0.1).

Step 2 - Run the following script called /opt/vadm-engine/bin/disable_security.sh which disables some of the security configurations such as firewalls, JMX security, etc. which is required to connect to Jolokia interface remotely.

Step 3 - Finally, for the changes to go into effect, you will need to restart the VIN discovery engine by running the following command: /etc/init.d/vadm-engine restart

Step 4 - To verify the configuration above, go ahead and login to the remote host of the IP Address that you added to the jolokia-access.xml file and perform the following curl command: curl http://[VIN_IP_ADDRESS]:8080/jolokia and you should get back a reply which confirms you can communicate with the Jolokia service on your VIN host remotely.

Note: You can also take the URL and paste that into a web browser and it should also respond with the same information. If you do use a browser, make sure you add the IP Address of the desktop that is running the browser to jolokia-access.xml file.

Installing Jmx4Perl on vMA:

 
Step 5 - To explore the data in VIN, we will be installing Jmx4Perl as noted in the Jolokia tutorial which includes an interactive utility called j4psh which will make it easier to view the information in VIN. You will need to install Jmx4Perl via CPAN or manually install the following Perl packages in this order:

• JSON-2.53
• Term-Clui-1.66
• Config-General-2.51
• Term-ShellUI-0.92
• Module-Find-0.11
• jmx4perl-1.06

Luckily, you do not need to manually do this as I wrote a quick shell script called setupJMX4PERL.sh which will download the packages above and install Jmx4Perl for you. All you need to do is just execute the script and then select "yes" when prompted.

Step 6 - Once Jmx4Perl has been installed, you should now have a utility called j4psh which we will use to connect to the Jolokia service running on your VIN appliance remotely from vMA. Run the following command: j4psh http://[VIN_IP_ADDRESS]:8080/jolokia 

Note: This can take awhile, in my lab environment it took up to 10 minutes before it connected. Please be patient as it could take longer, I can only assume some amount of data is being cached on the client. 

Once you have successfully connected, you should now have a prompt displaying the VIN address. There are multiple namespaces or MBeans and we will be looking at the com.vmware.vadm which is the VIN application. To change into this specific Mbean, run the following command: cd com.vmware.vadm as shown in the screenshot above. Once you have switched to VIN's MBean, you can then run the ls command, similar to UNIX/Linux ls (list) command. You can also just type help for all available commands.

What you see listed above in the screenshot are the different sub-MBeans that are available in the com.vmware.vadm Mbean and we will take a look at a few of these to show you how to navigate your way around.

Exploring VIN MBeans:

Note: Please be very careful when executing operations, I would highly recommend you do this in a lab environment to ensure you do not impact your VIN system. 

The first MBean we will take a look at is name=inceptionConfigurationMBean,type=ConfigurationMBean and as you might have guessed from the name it is related to some type of configuration. Go ahead and cd into that Mbean and then type ls.

We can see from this Mbean, there are a bunch of attributes that we can access. To view an attribute, we just run the cat command and the name of the attribute. In this example, I am checking the value of vc.credentials.host which should contain the address of the vCenter Server that the VIN Server is currently registered with. Before proceeding to the next Mbean, you will need to cd back out to main MBean, to do so, just type cd .. as you would on a UNIX/Linux machine to change to previous directory.

The next Mean we will take a look at is name=vcInventory,type=VcConnector and as you might have guess this is dealing with the vCenter inventory. Go ahead and cd to this Mbean and type ls.

For this Mbean, when we perform the list command, we are only shown operations or methods for this Mbean. The operations shown on the left will display the required parameters and we can see they all require a string input except for the getAllVms() so let's go ahead and execute that and see what happens (of course we can probably guess, a list of all VMs will be displayed). To execute an operation, you will need to run the exec command, go ahead and run the following command: exec getAllVms as shown in the screenshot above. As you can see from the screenshot, we are listing all VMs that have been discovered by VIN including a nice summary of each virtual machine.

Let's go ahead and run through two additional operations that accept parameters such as getVmByName and getVmByIp. You will use the exec command but now you will need specify an additional parameter based on the operation. The screenshot below demonstrates calling both of these operations and specifying the appropriate input parameter:

The next two operations will be located in the same Mbean which deals with the application services found on each VM as well as any external VM dependencies. The first operation is name=ApplicationService,type=ApplicationService and we will be looking specifically at the findApplicationComponentsByInfrastructureElemenBusinessKeys operation which accepts the MoRef Id of the VM in question which must be in JSON format (this took me awhile to figure out). From the previous operations above, you will need a MoRef Id of a VM you are interested in seeing what application services are running and run the following command: exec findInfrastructureElementsByBusinessKeys '[{"moid":"vm-232"}]' and be sure to replace the MoRef Id with your own.

As you can see from the output, it contains quite a bit of information including the three services this VM is running and one of these services is vCenter Operations. If you hop over to the vSphere Web Client and find this VM, you should see the same set of services listed.

The second operation that we will take a look at is called findOutgoingDependentInfrastructureElements and it also accepts MoRef Id as input and noticed the help option shows it requiring a single entry as well as an array entry of the MoRef Id, again in JSON format. To view the dependency of this VM to other VMs in your virtual infrastructure, run the following command:  exec findOutgoingDependentInfrastructureElements '{"moid":"vm-385"}' '[{"moid":"vm-385"}]' and be sure to replace the MoRef Id with your own.

From the output, you can see the VM's that are depended on by this VM as well as other information.

I have not explored all the Mbeans and operations, but I highly encourage you to browse around and see what you can find. Just be careful when executing operations that you are performing READ operations and not WRITE operations as you can impact your VIN system. In part 2 of this article, I will show you how to easily export some of this useful information found in VIN using a simple script.