virtuallyGhetto

VAMI

Introducing Alexa to a few more VMware APIs

by 3 Comments

Over the weekend, while taking a break from putting together some furniture as it was my time for my daughters nap, I got that the chance to explore and create a new Alexa Skill which integrates with a few of VMware's APIs. This has been something I wanted to try out for some time but have not had any spare time. I had even purchased an Amazon Echo Dot but its just currently being used as a music player for the family. A couple of weeks back I saw an awesome blog post from Cody De Arkland where he demonstrates how to easily integrate the new vCenter Server 6.5 REST APIs into an Alexa Skill which can then be consumed using an Amazon Echo device.

Cody's write-up was fantastic and I was able to get everything up and running in about 20-25minutes with a few minor trial/error. It was great to see how easy it was for a non-developer like Cody to easily consume the new vCenter Server REST APIs which includes basic VM Management as well access to the VMware Management Appliance Interface or VAMI for short. Given Cody already did the hard work to create the initial Alexa integration, I figure it might be cool to extend his work and introduce Alexa to a few more VMware's APIs including the traditional vSphere API (SOAP) and the new vSAN Management API.

UPDATE (06/15/17) - Just added support for PowerCLI, it was a little tricky as Flask app is written in Python and so poor man workaround was to call Powershell/PowerCLI using subprocess.

Since Cody's integration module was written using Python, it was pretty simple to add support for both pyvmomi (vSphere SDK for Python) and vSAN Management SDK. To install pyvmomi, you can simply run

pip3 install pyvmomi

and for installing vSAN Management SDK, have a look at this blog post here.

Here is a quick video that I had recorded which demonstrates the use of both the vSphere API and vSAN Management API using my Amazon Echo.

You can find all my changes in this forked repo lamw/alexavsphereskill and make sure to follow Cody's blog post here for instructions on how to get setup. For those wondering if Cody will be publishing an Alexa Skill for general consumption, I know he is working on some awesome updates to make it even easier to consume. Here is a sneak peak at just some of the recent updates that Cody is working on ...

Stay tuned on this blog and Github repo for future updates!

One thing to note which I was not aware of until Cody mentioned it, is that once your Alexa Skill is built, you can directly access it from your own personal Amazon Echo without needing to publish it. You need to activate the Alexa Skill by saying "Alexa Start [APP-NAME]" where name is the name used in the "Invocation Name" field as shown in the screenshot below when setting up your Alexa Skill. I should also mention that if you decide to change the Alexa Skill name itself, which I had initially done and called it "vGhetto Control", make sure you update the Flask App name in __init__.py to the same name (spaces are converted to underscores) or you will run into issues.

Automating post-configurations for both PSC & VCSA 6.0u1 using appliancesh

by 4 Comments

In vSphere 6.0, we introduced a new command-line option to allow you to automate both the deployment and upgrade of a vCenter Server Appliance (VCSA) and Platform Services Controller (PSC) using a simple JSON configuration file. This has been a very popular request from customers and one that I have been asking for some time now and was glad to see it was finally made available with the VCSA. One thing that was still missing from an Automation standpoint was being able to some basic post-configurations after the initial deployment. Common operations such as adding additional user accounts, configuring SNMP for monitoring or adding proxy server were available but had to be done interactively and manually.

In vSphere 6.0 Update 1, an enhancement was made to the appliancesh interface which will now allow customers to automate the post-configurations of either a VCSA or PSC by simply re-directing a series of appliancesh commands within a file using SSH. Although SSH may not be ideal for all customers and having a programmatic interface via an API is ultimately where we want to get to; This at least allows customers to automate the end-to-end deployment of both the VCSA and PSC as well as covering any additional post-configurations that might be required to stand up a vSphere environment.

To make use of this feature, you simply create a file that contains the list of appliancesh commands that you wish to run on either the VCSA and/or PSC. Here is an example configuration called psc.config (you can name it anything you want):

Once you have saved the configuration file, you simply SSH to either your VCSA or PSC and re-direct the configuration file by running the following command:

ssh *protected email* < psc.config

Once authenticated, the series of appliancesh commands will be executed and then you will be automatically logged off as seen in the screenshot below.
automating-post-configurations-for-psc-and-vcsa-using-appliancesh-0
If you have any feedback in this particular area, please leave a comment as I know both PM/Engineering are interested in hearing your thoughts and what you might want to see in the future in terms of post-configuration of the VCSA and PSC.

How to upgrade from VCSA 5.x & 6.x to VCSA 6.0 Update 1?

by 100 Comments

I have seen quite a few questions come in on how to properly upgrade from an existing vCenter Server Appliance (VCSA) 5.x and/or 6.x environment to the latest vSphere 6.0 Update 1 which was just released today. Before I jump straight into the process, I think its worth covering on how updates (patches) and upgrades have traditionally been handled for the VCSA. In an update or patch scenario, you are staying within a major release of vSphere (e.g. vSphere 5.0) and moving to something like vSphere 5.0 p01 and in this case, an in-place update or patch is performed. In an upgrade scenario, where you are moving from one major release (e.g. vSphere 5.0) to another major release (e.g. vSphere 6.0), a "migration based" approach is taken. This means that you would need to deploy the new VCSA that you wish to upgrade to and then migrate the data from your old VCSA appliance to the new one which is part of the upgrade workflow. This "migration based" approach was also true for any "U" (Update) releases (e.g. vSphere 5.5 to vSphere 5.5 Update 1). 

For major releases, this makes perfect sense and provides customers a nice way to easily rollback if something goes wrong. You simply power off the new VCSA and then power on your original VCSA and you are back in business. For update releases, we have heard from our customers that this process was not ideal and though there is always a risk when updating software (which is why I always recommend customers test thoroughly in a Dev/Test environment before moving to production), the amount of changes in the code is significantly less when compared to a major upgrade. One of the new features that we have introduced in vSphere 6.0 Update 1 is an in-place upgrade for "U" (Update) releases which I have already blogged about here among other new features.

This means that if you are coming from a VCSA 6.x environment and you wish to upgrade to vSphere 6.0 Update 1, you simply just mount the vSphere 6.0 Update 1 Patch ISO to your VCSA 6.x environment and perform the update from the command-line via the appliancesh interface. This is quite nice as it reduces the need to copy data between your old and new appliance and helps reduce the overall downtime. In fact, you can upgrade to vSphere 6.0 Update 1 in about 10min or so using this new method. If you are coming from a VCSA 5.x (5.0, 5.1 or 5.5) environment, this would be consider a major to major upgrade and you would need to follow the "migration based" approach to upgrade to vSphere 6.0 Update 1. One other thing to note after you have upgraded to vSphere 6.0 Update 1, we have now re-introduced URL based patching via the VAMI interface. This means in the future, you no longer need to update or patch from an ISO but can do so directly from VMware's online repository.

Below are the instructions on upgrading from VCSA 6.x to VCSA 6.0 Update 1:

Note (09/14/15):

  • If you have an External PSC with your VCSA 6.x and wish to upgrade, the process shown below is the same for both the PSC and the VCSA. You will want to first upgrade your PSC first as that provides authentication to your vCenter Server. Once the PSC has been upgraded and accessible on the network again, you will then want to move to your VCSA. If you are interested in the proper sequence and ordering of VMware Products to update, you can also check out this handy VMware KB 2109760 which provides all the details
  • Thanks to fellow reader Idan for reporting this but it looks like after an upgrade of the VCSA, the default VMware URL for the VAMI is not working. You will need to update it to point to the following URL https://vapp-updates.vmware.com/vai-catalog/valm/vmw/647ee3fc-e6c6-4b06-9dc2-f295d12d135c/6.0.0.10000.latest/ instead of the default one as shown in the screenshot below. This is only applicable for upgrade scenarios. If you deploy a new VCSA 6.0 Update 1, it will automatically be using the correct URL

incorrect-vami-repo-url
Step 0 - Ensure you have a proper backup and take a snapshot of your VCSA 6.x appliance before beginning.

Step 1 - Download the VCSA 6.0 Update 1 Full Patch (VMware-vCenter-Server-Appliance-6.0.0.10000-3018521-patch-FP.iso) by visiting the VMware Patch Download site.

upgrade-from-vcsa-6.0-to-vcsa-6.0-update-1-0
Step 2 - Mount the VCSA 6.0 Update 1 Patch ISO to your VCSA 6.x appliance using either the vSphere Web/C# Client

Step 3 - Login to your VCSA 6.x appliance via SSH to the appliancesh interface. If you have disabled that, simply type "appliancesh" and login with the root credentials.

Step 4 - Run the following command to stage and install the patches from the VCSA 6.0 Update 1 Patch ISO:

software-packages install --iso --acceptEulas

upgrade-from-vcsa-6.0-to-vcsa-6.0-update-1-1
Note: If you run into any errors while either staging or installing the patches, you should drop into the bash shell and take a look at /var/log/vmware/applmgmt/software-packages.log file for additional information. One common issue that I have seen in the past is if your /storage/log partition is full and you may need to perform a clean up before continuing.

Step 5 - Once the upgrade has completed, you just need to reboot your VCSA by running the following command:

shutdown reboot -r "Updated to vSphere 6.0u1"

upgrade-from-vcsa-6.0-to-vcsa-6.0-update-1-2
Step 6 - A quick way to confirm that you have successfully upgraded your VCSA to vSphere 6.0 Update 1, simply open a browser to the following URL: https://[VCSA-IP]:5480 and it should take you to the new HTML5 VAMI interface.

upgrade-from-vcsa-6.0-to-vcsa-6.0-update-1-3
If you would like additional information, take a look at this VMware KB 2119924.

How to finally inject OVF properties into VCSA when deploying directly onto ESXi?

by 40 Comments

One of my biggest pet peeve when it comes to deploying the VCSA (vCenter Server Appliance) and other OVF/OVA directly onto an ESXi host is the lack of OVF property support. If you have deployed the VCSA before, you are probably aware of the different user experience when deploying to a vCenter Server versus deploying directly to an ESXi host. For those of you who are not familiar, the difference is when you deploy an OVF/OVA that contains custom OVF properties such as the VCSA, you have the ability to provide input to these parameters when deploying to a vCenter Server as seen in the screenshot below.

ovftool-inject-ovf-0
However, when you deploy to an ESXi host (which is a common use case for greenfield deployments), you will notice that these OVF properties are not available. So, why are the OVF properties missing when deploying to ESXi? The simple answer is that the vCenter Server has a database.

OVF properties work by being injected into the Virtual Appliance through VMware Tools and into the OVF run time environment when a Virtual Appliance is powered on. If a user decides not to power on the Virtual Appliance immediately after deployment, the OVF properties and their values must be persisted. For vCenter Server, we have a database in which we can store the OVF properties, but for ESXi a database does not exists which can properly store the OVF properties and this is why they are not been supported by ESXi.

UPDATE (10/22) - ovftool 4.0 has been officially released. You can download it here.

The good news is that a solution has been in the works and you can actually get a sneak peak and try it out if you have either the recent VMware Fusion 2014 or Workstation 2014 Tech Preview installed. The solution was to add a new ovftool parameter called injectOvfEnv. As the name suggest, it allows the injection of OVF property values into a Virtual Appliance during power on. I have known about this solution for some time but it was only available in an unreleased version of ovftool. Since VMware Fusion and Workstation automatically bundles ovftool, I was curious if they would include the upcoming version and it looks like they did! 🙂

So now, instead of having to use the VAMI commands to setup the networking for a greenfield deployment with VCSA, you can simply add the following two parameters: --X:injectOvfEnv --powerOn along with the list of OVF property values.

I have created a simple shell script called deploy_vcsa.sh to demonstrate this functionality and you can easily create an equivalent BAT or PowerShell script for a Windows environment. You will need to modify the variables in the script to match your environment and they should all be pretty self-explanatory.

Here is an example output of running the script and deploying the VCSA directly onto my Mac Mini running ESXi 5.5u1:

ovftool-inject-ovf-1
Once the VCSA is fully booted up, we can take a look using the vSphere C# Client that it contains the IP Network information we provided through the OVF properties:

ovftool-inject-ovf-2
As you can see this new version of ovftool works without requiring anything special on the ESXi host and should technically work for the last several releases. I think this is a pretty good reason to check out the latest Fusion and Workstation Tech Previews 🙂

Quick Tip – Configuring HTTP proxy for VAMI via CLI

by 4 Comments

An HTTP Proxy is commonly used by customers who do not have direct internet access from within their datacenter to either upload logs or download patches from a particular website. I recently had to configure this within our environment to ensure we could patch against the external repository as we also have the same restriction as majority of our customers. To configure a proxy server for a VMware Virtual Appliance, you can do so using the VAMI interface under the Network section as seen in the screenshot below.

vami-proxy-configuration

I was looking to configure this from the command-line and through some quick test, here is how you do it. There is a command called /opt/vmware/share/vami/vami_set_proxy which accepts two parameters: proxy server and the proxy port.

Here is an example of how the command works:

/opt/vmware/share/vami/vami_set_proxy proxy.vghetto.com 3128

You can view the current proxy settings by running the following command:

/opt/vmware/share/vami/vami_proxy

There are two additional commands that show only the proxy server and proxy port respectively:

/opt/vmware/share/vami/vami_proxy_port
/opt/vmware/share/vami/vami_proxy_server

One thing you may have noticed is that these commands do not support configuring a proxy username or password as the VAMI UI does. After looking at the script does, I found that it is just writing it out into /etc/environment configuration file. If you require a proxy username and password, you could just directly edit the file and append the following as an example:

http_proxy=http://username:*protected email*:3128

How To Create Offline Update Repository For VMware Virtual Appliances

by 8 Comments

Virtual appliances built from VMware Studio provides a very easy mechanism of updating or upgrading the software on the appliance by using the VAMI (Virtual Appliance Management Interface) web interface. The VAMI web interface provides three methods of updating or upgrading an appliance: online update repository hosted by the author of the appliance, CD-ROM or alternate update repository can be specified.

UPDATE 02/23/16 - It looks like there were two tiny changes with the latest VAMI Update Repos starting with vCenter Server Appliance (VCSA) 6.0 Update 1. The first being a new signature file called manifest-latest.xml.sign and the .sha256 and .sig files are no longer used or available for download. The second is an additional patch-metadata-scripts.zip file that is under the pool-package directory which maybe required depending on the virtual appliance in question. I have updated my script to take care of these files in case they are needed for newer versions of the VAMI interface.

UPDATE 07/10/15 - VMware has just released a new Fling called VAMI Update Repository Appliance (VURA) which provides an easy way for customers to create offline VAMI repositories for their Virtual Appliances.

For VMware virtual appliances, a VMware hosted update repository is configured by default and internet connectivity to the configured URL will be required (proxy configurations are supported). However, there are environments where network connectivity to VMware's online repository is just not possible or the update repository must be hosted internally due to security requirements and this is where the third option can be used.

The process to setup your own update repository is not really documented and I have been noticing more requests from customers looking for a way to update or upgrade their VMware virtual appliances without requiring access to VMware's online repository. There are also other virtual appliances such as VCSA (vCenter Server Appliance) which ships both the update contents for an ISO and zip file which can then be used with the two other update/upgrade methods.

Though these files can be generated from VMware Studio as part of the appliance build process, the majority of the VMware virtual appliances do not provide these files for download.

I decided to research this topic a bit and look into building my own offline update repository based on the online update repository from VMware. I figured it should be fairly easy to replicate what is being hosted to a local web server that runs within your own datacenter. After some investigation, I found the  process to be pretty straight forward and only requirement is a web server that can be used to host the contents for update repository. In this example, I will show you how to build an update repository to upgrade VIN (vSphere Infrastructure) 1.2 to 2.0.

Step 1 - Login to the VAMI interface of VIN (https://[VIN-IP]:5480) and under the Update tab and make a note of the the default repository URL.

Step 2 - Download buildVARepo.sh shell script and upload that to a Linux based web server which will automatically build out our update repository.

Step 3 - The script accepts two arguments: default repository URL for a particular virtual appliance (from the previous step) and the name of the directory in which the repository will be created. In this example, I will be using the VIN repository URL and I will name the repository vin:

./buildVARepo.sh http://vapp-updates.vmware.com/vai-catalog/valm/vmw/302ce45f-64cc-4b34-b470-e9408dbbc60d/1.2.0.290.latest vin

The system that the script runs will need to have access to the URL above as it needs to download the required manifest files. Using the manifest files, parses out the package name and downloads the RPM packages to the web server using wget.

Step 4 - The result is a directory structure that will look like the following:

vin/manifest = List of XML manifest and signature files that describes the update and the path to the appliance packages
vin/package-pool = The RPM packages for the appliances for a given update

Depending on the location of where the script was executed, you may need to move it to the proper path in which your web server is configured to serve up content. You should be able to open a browser and point that to the /vin directory and view the contents.

Step 5 - We now log back into the VAMI interface and specify our update repository URL which will be http://[IP-OR-HOSTNMAE]/[REPO-NAME] and save the settings.

Step 6 - Now we head over to the Status sub-tab under Update and click on the "Check Updates" and we should see a new update for our virtual appliance. To update the appliance, we then select "Install Updates" and shortly after we should see our VIN appliance upgrade to 2.0

Note: Not all virtual appliances provide upgrades to the latest versions of the appliance, be sure to check the documentation of each individual appliances to see what is supported.