virtuallyGhetto

ruby vsphere console

How to convert vSAN RVC commands into PowerCLI and/or other vSphere SDKs?

by Leave a Comment

A common request that I see come up from our field and customers is getting specific vSAN Ruby vSphere Console (RVC) commands to be made more generally available in other vSphere CLI/SDKs like PowerCLI for example. Funny enough, many folks do not realize that this functionality has been there since vSAN 6.2 and specifically with the release of the vSAN Management APIs which exposes all vSAN functionality programmatically whether you are consuming it from the vSphere Web Client, Embedded Host Client or from RVC. All of these tools have been built using the vSAN Management APIs.

Although we have supported a variety of vSAN Management SDKs (language bindings) since its first release, I will say that PowerCLI consumption of the vSAN Management API has only been made available recently with PowerCLI 6.5.1 and it supports the latest release of vSAN 6.6 and can go all the way back to vSAN 6.2. Even with PowerCLI support, I still continue to see vSAN RVC requests come up time after time and it seems like folks still have not made the connection that RVC is just simply using the vSAN Management API just like UI does.

What is even more interesting is that the source code of RVC can be viewed by anyone to see how each command is implemented and which APIs are being used. RVC is built using rbvmomi (vSphere SDK for Ruby) which provides access to both the vSphere and vSAN Management APIs. Given the number of requests that I have seen, I am going to assume that this is not common knowledge and I figured the best way to show how this work is with a real world example. I decided to take the vsan.check_limits RVC command and create an equilvenet PowerCLI script that uses the vSAN Management API to provide the exact same information.

Note: You will need to know how to use the vSphere/vSAN Management APIs and knowing a little of Ruby can also help. If you are new to vSAN Management APIs, have a look at this blog post on how to get started.

Here is a screenshot of running the vsan.check_limits RVC command:


Here is a screenshot of running the PowerCLI script that I have created:


As you would expect, the data is exactly the same since they both consume the same underlying vSAN Management API.

So, how do we get started?

[Read more...] about How to convert vSAN RVC commands into PowerCLI and/or other vSphere SDKs?

Docker Container for the Ruby vSphere Console (RVC)

by 2 Comments

The Ruby vSphere Console (RVC) is an extremely useful tool for vSphere Administrators and has been bundled as part of vCenter Server (Windows and the vCenter Server Appliance) since vSphere 6.0. One feature that is only available in the VCSA's version of RVC is the VSAN Observer which is used to capture and analyze performance statistics for a VSAN environment for troubleshooting purposes.

For customers who are still using the Windows version of vCenter Server and wish to leverage this tool, it is generally recommended that you deploy a standalone VCSA just for the VSAN Observer capability which does not require any additional licensing. Although it only takes 10 minutes or so to setup, having to download and deploy a full blown VCSA to just use the VSAN Observer is definitely not ideal, especially if you are resource constrained in your environment. You also may only need the VSAN Observer for a short amount of time, but it could take you longer to deploy and in a troubleshooting situation, time is of the essence.

I recently came across an internal Socialcast thread and one of the suggestion was why not build a tiny Photon OS VM that already contained RVC? Instead of building a specific Photon OS that was specific to RVC, why not just create a Docker Container for RVC? This also means you could pull down the Docker Container from Photon OS or any other system that has Docker installed. In fact, I had already built a Docker Container for some handy VMware Utilities, it would be simple enough to just have an RVC Docker Container.

The one challenge that I had was that the current RVC github repo does not contain the latest vSphere 6.x changes. The fix was simple, I just copied the latest RVC files from a vSphere 6.0 Update 1 deployment of the VCSA (/opt/vmware/rvc and /usr/bin/rvc) and used that to build my RVC Docker Container which is now hosted on Docker Hub here and includes the Dockerfile in case someone was interested in how I built it.

To use the RVC Docker Container, you just need access to a Linux Container Host, for example VMware Photon OS which can be deployed using an ISO or OVA. For instructions on setting that up, please take a look here which should only take a minute or so. Once logged in, you just need to run the following commands to pull down the RVC Docker Container and to star the container:

docker pull lamw/rvc
docker run --rm -it lamw/rvc

ruby-vsphere-console-docker-container-1
As seen in the screenshot above, once the Docker Container has started, you can then access RVC like you normally would. Below is an quick example of logging into one of my VSAN environments and using RVC to run the VSAN Health Check command.

ruby-vsphere-console-docker-container-0
If you wish to run the VSAN Observer with the live web server, you will need to map the port from the Linux Container Host to the VSAN Observer port which runs on 8010 by default when starting the RVC Docker Container. To keep things simple, I would recommend mapping 80->8010 and you would run the following command:

docker run --rm -it -p 80:8010 lamw/rvc

Once the RVC Docker Container has started, you can then start the VSAN Observer with --run-webserver option and if you connect to the IP Address of your Linux Container Host using a browser, you should see the VSAN Observer Stats UI.

Hopefully this will come in handy for anyone who needs to quickly access RVC.

Automating full configuration of a VSAN Stretched Cluster using RVC

by Leave a Comment

A couple of weeks back I had spent some time setting up several VSAN Stretched Clusters in my lab for some testing and although it was extremely easy to setup using the vSphere Web Client, I still prefer to stand up the environment completely automated 🙂

In looking to automate the VSAN Stretched Cluster configuration, I was interested in something that would pretty much work out of the box and not require any additional download or setup. The obvious answer would be to use the Ruby vSphere Console (RVC) is a really awesome tool that is available as part of vCenter Server included in both Windows vCenter Server and the VCSA.

For those of you who have not used RVC before, I highly recommend you give it a try and you can take a look at this article to see some of the cool features and benefits. I am making use of the RVC script option which I have written about in the past here to perform the VSAN Stretched Configuration. One of the new RVC namespaces that have been introduced in vSphere 6.0 Update 1 is the vsan.stretchedcluster.* commands and the one we are specifically interested in is the vsan.stretchedcluster.config_witness command.

There are a couple of things the script expects from an environment setup, so I will just spend a few minutes covering the pre-reqs and the assumptions before diving into the script. I will assume you already have a vCenter Server deployed and configured with an empty inventory. I also assume you have already deployed at least two ESXi hosts and a VSAN Witness VM that meets all the VSAN pre-reqs like at least one VSAN enabled VMkernel interface and associated disk requirements. Below is a screenshot of the vSphere Web Client of the initial environment.

automate-the-full-configuration-of-vsan-stretched-cluster-using-rvc-0
Next, we will need to download the RVC script deploy_stretch_cluster.rb and upload that to your vCenter Server. Before you can execute the script, you will need to edit the script and adjust the variable names based on your environment. Once you have saved the changes, you can then run the RVC script by running the following command:

rvc -s deploy_stretch_cluster.rb [VC-USERNAME]@localhost

Here is a screenshot of running the script on the VCSA using Nested ESXi VMs + VSAN Witness VM for the Stretched Clustering configuration:

aautomate-the-full-configuration-of-vsan-stretched-cluster-using-rvc-1
If everything executed successfully, you should see a "Task result: success" which signifies that the VSAN Witness VM was successfully added to the VSAN Stretched Cluster. If we now refresh the vSphere Web Client and under the Fault Domains configurations in the VSAN Cluster, we now see both our 2-Node VSAN Cluster and the VSAN Witness VM.

automate-the-full-configuration-of-vsan-stretched-cluster-using-rvc-2

Hopefully this script can also benefit others who are interested in quickly standing up a VSAN Stretched Cluster, especially for evaluation or testing purposes. Enjoy getting your VSAN on!

How to run the VSAN Observer in “collection” mode in the background?

by 1 Comment

The VSAN Observer is a very powerful tool that allows you to get in-depth performance analysis of your VSAN environment. One of the really useful feature is the ability to run the VSAN Observer in "collection" mode by using the --generate-html-bundle option. Something that I have noticed when running the VSAN Observer in collection mode is that you not close the current SSH session, else the collection will stop. I have even tried running the VSAN Observer using RVC's not very well known "script" feature and then back-grounding the process, but after a minute or so the collection also just stops.

The only workaround that I have found is by using Screen, a full-screen windows session manager usually found on most Linux/UNIX and Mac OS X systems. Having used Screen in the past life as a Systems Administrator, I can say it is an extremely useful tool when needing to perform long running tasks and not have to worry about your SSH session being disconnected. You can start a session, disconnect and then re-connect at a later time to monitor the progress.

If you are on a Mac, then Screen should have already been installed. Below are the steps to run the VSAN Observer in the VCSA using Screen:

Step 1 - Start screen and give the session a name such as "VSAN-Observer" for example:

screen -S VSAN-Observer

Step 2 - SSH to your VCSA and login to RVC and start the VSAN Observer using the collection mode as you normally would. For step by step instructions, check out Rawlinson Rivera's article here on setting up the VSAN Observer.

Step 3 - Once the VSAN Observer is running, enter the following key combinations which will detach your Screen session:

Crtl+a d

Step 4 - To list the available Screen sessions, you can run the following command:

screen -list

vsan-observer-rvc-script-1
Step 5 - To re-attach to your Screen session, you will need to specify the session name. In our example, it was called VSAN-Observer:

screen -r VSAN-Observer

An alternative to Step 2, instead of running the VSAN Observer interactively, I actually prefer to run the VSAN Observer using RVC's script option. It is just less typing for me and makes it easy to collect stats across multiple VSAN environments

To do so, you will need to create a script file that contains the following:

The RVC script option actually accepts a Ruby script to execute and if we take a look at the script, we are just passing some arguments to the vsan.observer command.

To use the RVC script instead of interactively logging in, you can run the following command:

rvc -s [SCRIPT-NAME] [USERNAME:PASSWORD]@localhost

vsan-observer-rvc-script-0
I think a nice feature enhancement to the VSAN Observer is the ability to automatically background the collection process without having to rely on the existing SSH connection, perhaps this is something Christian may consider for a future update to RVC 🙂 In the meantime, this is a pretty decent work around

How to upgrade to the latest VSAN Beta Refresh of RVC on Windows?

by 4 Comments

A variety of new updates can be found in the latest VSAN Beta Refresh including a new RVC (Ruby vSphere Console) namespace called SPBM (Storage Policy Based Management) that allows users manage and configure VM Storage Policies. However, this update is currently only available for the VCSA (vCenter Server Appliance) and there is no update package for the Windows based vCenter Server (I hear this is being looked into).

UPDATE 12/04/13 - I just received an update from Cormac that we have just released a VSAN Beta Refresh for vCenter Server for Windows which you can download here.

For those of you who are running vCenter Server on Windows for the VSAN Beta and wish to try out the latest release of RVC which includes additional fixes as well as the new SPBM namespace can do so by just updating to the latest RVC package.

Step 1 - Download the rvc_1.3.3-1_x86_64.rpm package from the VSAN Beta website and upload that to your Windows vCenter Server.

Step 2 - Download and install 7zip on the Windows vCenter Server or another Windows server which can be used to extract the contents of the RPM package.

Step 3 - Right click on the RPM package and select the "Extract to rvc_1.3.3-1_x86_64/" option which should create a new folder that contains a new file called rvc_1.3.3-1_x86_64.cpio.

Step 4 - Right click on the CPIO package and select the "Extract to rvc_1.3.3-1_x86_64/" option which will create another folder containing the actual RVC bits.

Step 5 - Copy the contents from inside of opt\vmware\rvc to the following path on your vCenter Server C:\Program Files\VMware\Infrastructure\VirtualCenter Server\support\rvc Once you have replaced the updated RVC bits, you can now change into that directory and launch RVC by just typing "rvc" and hitting enter. To confirm that RVC has been successfully updated, you can then type "spbm." and hit tab and you should see eight new SPBM commands.

To learn more about RVC for Windows, I would highly recommend you check out Eric Bussink's blog article here.

Automating VCSA Network Configurations For Greenfield Deployments

by 6 Comments

If you deploy the VCSA (vCenter Server Appliance) or other virtual appliances directly onto an ESXi host,
you will notice the network configuration wizard for the virtual appliance is not available as you would expect when deploying to a vCenter Server.

The reason for this is that ESXi does not support some of the advanced OVF/OVA properties such as the Networking section and you will need to deploy the OVF/OVA to a vCenter Server to be able to configure these advanced options. This poses a problem if you need to deploy the VCSA in a greenfield environment where you will not have an existing vCenter Server running and you will be deploying directly to the ESXi host. Unless you have a DHCP enabled network, you will most likely need to manually go into the vSphere C# Client to change the network configuration as it was unable to obtain an IP Address.

Though this is a one time configuration, it is still not ideal and would require the use of a Windows system to access the vSphere C# Client. You can actually get around this by leveraging the GuestOperations API (previously known as VIX API) which allows you to perform operations within the guestOS that is running VMware Tools. The other nice thing about the GuestOperations API is that it does not require any network connectivity from the virtual machine.

Note: The GuestOperations API can be accessed in variety of ways and in this article I am demonstrating just two methods and does not require a Windows system. You can also access the GuestOperations API using PowerCLI if you are more comfortable with Windows and do not wish to use the vSphere C# Client to manually configure the network settings for the VCSA. I would also like to stress that though this article is about the VCSA, you can easily apply this to any VMware based virtual appliance or virtual appliance running VMware Tools.

The most important thing to identity before using the GuestOperations API is the specific command or program you wish to invoke and the argument it accepts. To configure the network configuration for the VCSA or any other VMware based virtual appliance, you would use /opt/vmware/share/vami/vami_set_network If you just run this command by itself, there are variety of options from IPv4 to IPv6, static or dhcp configuration. In our example, we will be configuring a Static IPv4 address for our VCSA and the command we would run is the following:

/opt/vmware/share/vami/vami_set_network "eth0 STATICV4 192.168.1.150 255.255.255.0 192.168.1.1"

Method 1 - Using RVC (Ruby vSphere Console)

 

RVC is a nice open-source tool for interactively managing and configuring your vSphere infrastructure. RVC can be installed on any platform, in this example, I am running RVC on my Apple OS X laptop.

Step 1 - We first need to deploy the VCSA OVA and we can do so by using the ovftool via the command-line which can also be installed on Mac OS X system.

Step 2 - We then login to our ESXi host using RVC.

Step 3 - Next we will need to "change directory" to the location of our VM, in this example my VCSA is called VCSA-5.1. We can then run the "info ." command to view the summary of our VM. We can see that our VM is powered on from our initial deployment and we are ready to apply our network configurations in the next step.

Step 4 - To be able to run the above command, we will need to first authenticate into the guestOS. To do so, we will run the "vm_guest.authenticate ." and we will be prompted for the VCSA password. By default, the command assumes the username is root but that can also be specified on the command-line. If you are successful, you should not see any errors and then we can run the "vm_guest.start_program" command. Run the following to set a IPv4 static IP Address:

vm_guest.start_program . --program-path /opt/vmware/share/vami/vami_set_network --arguments "eth0 STATICV4 192.168.1.150 255.255.255.0 192.168.1.1"

Note: All commands in RVC can be tabbed out with auto-completion.

If the command was successful, you can quit RVC and you should be able to ping the IP Address that you have just configured.

Method 2 - Using vSphere SDK for Perl Script

 

Awhile back I wrote a script called guestOperations.pl which is a vSphere SDK for Perl script that implements the new GuestOperations API. This is a generic script which can be used to remotely connect to either a vCenter Server or ESXi host and perform operations within a guestOS as long as VMware Tools is installed and running. In this example, I also have the vSphere SDK for Perl installed on my Mac OS X laptop, but you can also install this SDK on any platform as well.

Step 1 - We will first use the "validate" operation to ensure our credentials to the guestOS is correct, but more importantly ensure that VMware Tools is up and running.

If the operation was successful, we should see our guest credentials validated. If not, you may need to wait a minute or two while VMware Tools is still loading up.

Step 2 - To invoke the command to configure the network configuration, we will use the "startprog" operation and run the following:

./guestOpsManagement.pl --server mini --username root --guestusername root --vm VCSA-5.1 --operation startprog --program_path /opt/vmware/share/vami/vami_set_network --program_args "eth0 STATICV4 192.168.1.150 255.255.255.0 192.168.1.1" --working_dir /

If the command was successful, then you should now be able to ping the IP Address that you have just configured.

As you can see, with the use of the GuestOperations API, you can do more than just setup the network configuration for a VM, you can run pretty much any command within the guestOS as you normally would if you were to RDP or SSH in. This is a very powerful interface that you can leverage to help you automate your virtual machine deployment and configurations!