• Skip to primary navigation
  • Skip to main content
  • Skip to primary sidebar

virtuallyGhetto

  • About
  • Privacy
  • VMware Cloud
  • Home Lab
  • Nested Virtualization
  • Automation
    • VMware Kickstart
    • VMware API/SDK/CLI
    • VMware vMA/VIMA
    • VMware OVF / OVFTOOL
  • Apple Mac
  • VCSA
  • VSAN

vma

Cool Docker Container for VMware Utilities

02/17/2015 by William Lam 11 Comments

Last week during lunch I learned about a cool little project that my colleague Alan Renouf was working on in his spare time at night. He was interested in learning more about Docker and thought the best way to learn about something new was by using it, which is normally how I learn as well. He came up with a nifty idea to create a Docker Image that would house a bunch of useful VMware tools which included several VMware open source projects as well.

UPDATE (11/23/16) - The Docker Container has now been updated with all the latest vSphere 6.5 SDK/CLI/Tools. We also plan to make this new version of the Docker Image available on Docker Hub, so stay tuned for those details shortly.

Some customers in the past have built similar offerings by using a free VMware Appliance called vMA (vSphere Management Assistance). vMA is nothing more than a stripped down version of SLES that has the vSphere CLI (vCLI) pre-installed. In my opinion, vMA is pretty limited and you can not install additional packages without voiding official support. Even if you decide to ignore support and install custom packages, I have often seen this break existing dependencies. When I talk to customers about their use of vMA, most used it because it was just there, but the majority prefer to use their own harden distribution of Linux and install their own admin utilities and packages which may also include non-VMware tools.

I personally have no problem building my own VM appliance that contains the various VMware packages, utilities and scripts that I use on a daily basis. However, not everyone is comfortable with this idea. Wow could this be further simplified and automated? Well, enter vmware-utils a Docker Image that allows you to automatically build a new image that contains some of the most popular and widely used VMware Utilities.

I wanted to enhance the awesome work that Alan had done with couple more VMware open source tools that I thought might be useful to VMware Administrators, which I actually wrote about here in my List of VMware CLIs, SDKS and DevOps Tools article. I have already submitted a pull request for my changes here. If there are other tools or packages you think that are useful and wish to contribute back, feel free to clone the repository and submit a pull request!

The latest vmware-utils now contains the following:

  • vSphere CLI 6.5
  • PowerCLI Core 1.0
  • vSphere Management SDK 6.5
  • vSphere SDK for Perl 6.5
  • vSphere SDK for Ruby (rbvmomi)
  • vSphere SDK for Python (pyvmomi)
  • vSphere Automation SDK for Ruby 6.5
  • vSphere Automation SDK for Python 6.5
  • vSphere Automation SDK for Perl 6.5
  • vSphere Automation SDK for Java 6.5
  • VSAN Management SDK for Ruby 6.5
  • VSAN Management SDK for Python 6.5
  • VSAN Management SDK for Java 6.5
  • VSAN Management SDK for Perl 6.5
  • Virtual Disk Development Kit (VDDK) 6.5
  • OVFTool 4.2
  • PowerCLI Community Repository
  • PowerCLI Core Docker Container Samples
  • William Lam's vGhetto Script Repository
  • Pyvmomi Community Samples
  • Docker Client v1.12.3
  • Docker Compose v1.8.1

For those of you who are new to Docker, a great way to quickly get started is by using an awesome tool called boot2docker which allows you to run Docker Containers on either a Windows or Mac OS X system. This also helps remove any barriers if you do not want to setup a Linux machine to get Docker of if you are like me, running on Mac OS X and rather not have to spin up a VM just to use Docker. Below are the steps on getting boot2docker working and building your own vmware-utils Docker Image.

Step 1 - Download the Docker Client for your specific OS (Windows, Linux or Mac OS X)

Step 2 - Take a look at the vmware-utils README, I spent some time updating it to make it more consumable for new users of Docker and follow the "How" section which will have you download the 4 VMware utilities as well as the vmware-utils DockerFile which we will need to build the Docker Container.

Step 3 - Create a directory and place all files into that directory. In this example, I have called the directory "vmware-utils".

vmware-utils-docker-container-0
Step 3 - We are now ready to build our vmware-utils Docker Image. Change into the "vmware-utils" directory that contains the files you downloaded earlier we will need to specify a "tag" for our image as part of the build command. In this example, I have called my image "lamw/vmware-utils" and to start the build process run the following command:

docker build -t lamw/vmware-utils .

Step 4 - The build itself may take some time depending on the speed of your internet connection. You will know when it has successfully completed when it states "Successfully built X" where X will be some unique ID as seen in the screenshot below.

vmware-utils-docker-container-4
Step 5 - Once the Docker Image has finished building, you can then run and connect to the Container by running the following command:

docker run --rm -it lamw/vmware-utils

vmware-utils-docker-container-5
At this point, you are now logged into the vmware-utils Docker Container that you have just built! It contains all the VMware Utilities that I have listed earlier and for more details on what has been installed and the location of the utilities, take a look at the vmware-utils Github documentation. If there are other tools you would like to see, feel free to contribute back by cloning the repository and submitting a pull request. I am definitely looking forward to seeing how this project evolves and providing a more dynamic way of creating a vMA-like experience without the current limitations. Keep up the awesome work Alan!

Share this...
  • Twitter
  • Facebook
  • Linkedin
  • Reddit
  • Pinterest

Filed Under: Automation, Docker, vRealize Suite, vSphere Tagged With: api, boot2docker, container, DevOps, Docker, dockerfile, vcloud air, vma, vSphere API

Free Linux & Windows Syslog Alternatives to depercated vi-logger in vMA 5

07/25/2011 by William Lam 12 Comments

Those of you who currently use vi-logger in vMA 4.x as a free syslog server for your ESX(i) hosts may notice this functionality has been removed in the latest vMA 5 release. VMware decided to remove the syslog functionality in vMA in favor of combining it with the vCenter Server. If you decide to run vCenter 5 on Windows, you have the option of installing an additional syslog collector on the same or separate Windows system and registering it as a vCenter plugin. If you are using the new VCVA (vCenter Server Virtual Appliance), there is also a syslog collector that is installed by default.

Using vMA's vi-logger was an easy and free solution, but you still have some alternatives without having to use vCenter or install/build a new syslog server. The following will document a free syslog solution for both a Linux or Windows platform.

Linux Syslog server alternative using vMA 5.0
You can actually leverage the existing syslog server on the latest vMA 5 release and with a few customization, get it setup to start collecting logs from your ESX(i) hosts as before with vi-logger.

Step 1 - It is recommend that you configure an additional disk on vMA for your syslogs as the size of vMA is quite tiny for additional use. I will assume that you know how to add and configure an additional disk, if not you can do a simple search on Google. In this example, I have a second disk that is 10GB and it is mounted up under /var/log/remote which is where the ESX(i) logs will be stored in.

Step 2 - You will need to edit the syslog configuration under /etc/syslog-ng/syslog-ng.conf and you will need to add three entries. The first addition is to configure the source for log messages from the network and enabling both udp/tcp on port 514, you may add the following under the default "src" entry.

source network {
udp6( port(514) );
tcp6( port(514) );
};

The next two entries will define the destination and how it'll log. You will add the following at the end of the syslog-ng.conf configuration file.

destination log_remote {
file("/var/log/remote/$HOST_FROM/$YEAR-$MONTH/messages-$YEAR-$MONTH-$DAY"
create_dirs(yes) frac-digits(3)
template("$ISODATE $PROGRAM $MSGONLY\n")
template_escape(no)
);
};
log {
source(network);
destination(log_remote);
};

The "log_remote" destination will send all logs from your ESX(i) hosts into /var/log/remote and will have the following format: $HOST_FROM/$YEAR-$MONTH/messages-$YEAR-$MONTH-$DAY

Step 3 -  Now you will need to restart the syslog server for the changes to take effect. You will need to run the following command: sudo /etc/init.d/syslog restart

If everything went successful, you should now be able to configure your ESX(i) hosts to point to your vMA 5 system and you should see logs appearing under /var/log/remote

Note: You will need to use sudo to view the directory under /var/log/remote and to view the logs

Windows Syslog server alternative using vCenter Syslog Collector
The vCenter Syslog Collector can be installed and used without the use of vCenter, you can easily turn any existing or new Windows system into a syslog server for your ESX(i) hosts for free.

Step 1 -  It is recommend that you configure a seperate disk on the Windows system that you are going to be using for your syslog server. I will assume that you know how to add and configure an additional disk, if not you can do a simple search on Google. In this example, I have a second disk that is 10GB and listed as Syslog (E: drive)

Step 2 - You will need access to the vCenter Server 5.0 installation ISO or executable to install the Syslog Collector utility. Start the installer and select and install VMware Syslog Collector

Step 3 - You have the option of using the local C:\ drive, but I would recommend setting up a separate drive if you can. If you decide to change the default log location, you need to ensure that you specify the following directory structure VMware\VMware Syslog Collector\Data else you will run into issues with the installation. In this example, I have moved my logs into E:\ drive and the path looks like the following: E:\VMware\VMware Syslog Collector\Data. You also have the ability to change the size of the log files before rotation and the number of logs before rotating.

Step 4 - If you are installing the Syslog Collector on the same host as vCenter Server, you should select the integrated installation else you should select a standalone installation.

Step 5 - The next screen will be the default ports to enable for both TCP/UDP and SSL which can be configured or left as the default as recommend.

Step 6 - The screen is how the Syslog Collector will be identified on the network and it should just be the IP Address of the host.

If everything went successful, you should now be able to configure your ESX(i) hosts to point to your Windows Syslog Collector system and you should see logs appearing under E:\VMware\VMware Syslog Collector\Data

As you can see even with vi-logger being removed in the latest version of vMA 5, you can easily still configure a free syslog server with your ESX(i) hosts on either a Linux or Windows platform.

Share this...
  • Twitter
  • Facebook
  • Linkedin
  • Reddit
  • Pinterest

Filed Under: Uncategorized Tagged With: esxi5, syslog, vilogger, vma, vMA5, vSphere 5

How to install Ruby vSphere Console on vMA

04/06/2011 by William Lam 5 Comments

If you are a Ruby and VMware fan (heck, even if you are not a Ruby fan), you should check out the "unofficial" VMware Lab Fling - Ruby vSphere Console (RVC) created by Rich Lane of VMware. RVC is described as a Linux console UI for vSphere, built on the RbVmomi (another VMware Fling) bindings to the vSphere API. RVC is platform agnostic, as long as you can get Ruby/Gem installed, you can run RVC on Windows (I believe), UNIX/Linux and even OSX!

Rich just released another update today supporting Linked Clones functionality for virtual machines and maintenance mode for ESX(i) host. I wanted to check out the latest update and I thought give it a try on vMA, here are the instructions for installing Ruby, Gem (Ruby package manager) and RVC on vMA 4.1. If you do not want to manually go through this manual process, jump towards the bottom for an automated script that does all this for you 🙂

Step 1 - You will need to download a few dependencies for both Ruby and Gems, the easiest way to do so is to setup a YUM repo pointing to CentOS mirrors. You will need to create a .repo file under /etc/yum.repos.d and in this example, we'll just call it /etc/yum.repos.d/CentOS-Base.repo

Copy the following into this file, make sure you create the file using sudo:

Step 2 - Install the following packages:

yum -y install gcc gcc-c++ zlib-devel openssl-devel readline-devel libxml2 libxml2-devel libxslt libxslt-devel

Step 3 - Download Ruby and Gems using wget:

wget ftp://ftp.ruby-lang.org/pub/ruby/1.8/ruby-1.8.7.tar.gz
wget http://rubyforge.org/frs/download.php/74619/rubygems-1.7.2.tgz

Step 4 - Install Ruby

tar -zxvf ruby-1.8.7.tar.gz
cd ruby-1.8.7
./configure
make
sudo make install
cd ..

You can verify that Ruby was installed correctly by running the following:

[[email protected] rubygems-1.7.2]$ ruby --version
ruby 1.8.7 (2008-05-31 patchlevel 0) [x86_64-linux]

Step 5 - Install Gem

tar -zxvf rubygems-1.7.2.tgz
cd rubygems-1.7.2
sudo ruby setup.rb

You can verify that Gem was installed correctly by running the following:

[[email protected] rubygems-1.7.2]$ gem --version
1.7.2

Step 6 - Update Gem (not necessary if you downloaded the latest version which is 1.7.2)

sudo gem update --system

Step 7 - Install ffi gem, this is used for faster tab competition

sudo gem install ffi

Step 8 - Install Ruby vSphere Console (RVC) gem

sudo gem install rvc

You can verify that RVC was installed correctly by running the following command:

[[email protected] ~]$ rvc -h
Ruby vSphere Console.

I also wrote a quick RVC installation script which does exactly the above without the manual labor.

Step 1 - Download installRVC.sh script to vMA:

wget http://vghetto.svn.sourceforge.net/viewvc/vghetto/other/installRVC.sh

Step 2 - Set the script to be executable:

chmod +x installRVC.sh

Step 3 - Run the installation script

sudo ./installRVC.sh

Once completed, you should get a successful message that RVC is installed on your vMA host.

Now you are ready to start playing with RVC!

Here is an example of the two latest features added by Rich

Linked Clone:

Maintenance Mode:

As you can see it is very easy to use and intuitive, what would be even cooler is to have the various objects be colorized so you can easily distinguish between them, perhaps feature enhancement Rich?

If there are other vSphere operations that you would like to see get implemented, drop your feedback on this VMTN thread which is being monitored by Rich. You can also follow Rich on twitter for the latest updates about RVC.

Hopefully VMware will make this into an official fling and put it on their VMware Lab's website to get the word out!

Share this...
  • Twitter
  • Facebook
  • Linkedin
  • Reddit
  • Pinterest

Filed Under: Uncategorized Tagged With: ruby vsphere console, rvc, vma

ESXi Lockdown mode does not play nice with vMA

02/16/2011 by William Lam 1 Comment

Today on the VMTN community forums, a user identified an interesting side effect when using vMA's vilogger and enabling lockdown mode on an ESXi host. What the user found was the vilogger daemon stopped collecting logs when lockdown mode was enabled for an ESXi host. At first, I thought lockdown mode should have no affect on vilogger, as it only disables the "root" account from accessing ESXi host other than from the DCUI (Directo Console User Interface).

I replicated the setup in my lab using an ESXi host that was being managed by vMA via vi-fastpass (fpauth) and enabled vilogger for this host. I verified log collection was functional before enabling lockdown mode on the ESXi host, right away vilogger stopped collecting logs when lockdown mode was enabled. When using the "vilogger list" command, the status of the ESXi host goes from "collecting" to "No Permission". I found this to be quite odd and verified what the user was describing in his environment.

Next was to take a look at the vilogger logs which is stored in /var/log/vmware/vma/vilogd.log and I found the same "No Permission" error.

I decided to login to the ESXi host and tailed the hostd logs to see what was going on when lockdown mode was being enabled. What I found was pretty surprising to me, there was a task that removed permissions from the vi-adminXX user account, I was pretty sure at this point, the culprit was related to lockdown mode.

I decided to take a look at VMware's documentation to see what the behavior of Lockdown Mode was and the following snippet taken from vSphere's online documentation explains it all:

The text highlighted in red is the key to the issue the user is facing and specifically the very last section where it states:

you cannot run vCLI commands from an administration server, from a script, or from vMA against the host

This meant that not only the root account was locked out, but all other accounts found on the ESXi host whether they are custom from your environment or from auxiliary systems such as VMware vMA, would be completely disabled. What is even more interesting, even read-only accounts would no longer function, they too had to go to vCenter to be re-proxied to specific ESXi host.

This has a few implications for users considering Lockdown Mode:

  1. All scripts including resxtop and user authentication must go through vCenter. If vCenter went down, you have no remote way to access your ESXi host. This also meant that you could not remotely start up vCenter if it was hosted in a virtual machine but rather from DCUI after enabling Local Tech Support Mode
  2. The use of vMA's vilogger is completely useless when Lockdown Mode is enabled for ESXi host. Users may want to consider setting up a traditional syslog server and have the logs forwarded from the ESXi host

IMHO, I don't think Lockdown Mode should crippled the vilogger functionality, the logging is a "Read" operation and I think re-configuring it to "read-only" role should have suffice. I also think that VMware could have done a better job working with the vMA engineers to support this functionality and have some documentation regarding this issue. For now, if you rely on any type of automation that goes directly to an ESXi host and you are thinking about Lockdown mode, you may want to think twice.

Share this...
  • Twitter
  • Facebook
  • Linkedin
  • Reddit
  • Pinterest

Filed Under: Uncategorized Tagged With: esxi4.1, lockdown mode, vilogger, vma

How to backup/restore vMA’s config + vi-fastpass DB

02/04/2011 by William Lam 1 Comment

I recently wrote an article on the process of cloning vMA which provides a way of backing up vMA. Due to the ease of deploying vMA as a virtual appliance (using OVF), there aren't too many reasons you would need to backup this virtual machine. If you lost the system for whatever reason, you can easily re-deploy with just a few clicks.

However, having said that, if you extensively make use of vi-fastpass fpauth and manage a lot of targets whether they are ESX(i) or vCenter hosts, you need to understand it is not simply just re-deploying another vMA host. When you add a target to vMA's vi-fastpass, two accounts are provisioned on the host "vi-adminXX" and "vi-userXX", these accounts are associated with an encrypted cipher located on vMA which allows for "fastpass" access to the host without having to re-type the password to the host each time. If you were to re-deploy a new vMA host and add the targets again, your host will not only contain the old entries but now a new set of accounts for your new vMA host. This can be an issue as you start to have stale accounts on your ESX(i) or vCenter host.

To prevent this issue, you can actually backup both vMA's configurations which is primarily stored in a sqlite database and the vi-fastpass credential store. In the following example, I have two ESXi hosts being managed by my primary vMA and I also have a standby vMA DR host in which I will backup the files to.

First, you want to make sure you do not have any active vifptarget sessions, this is not a requirement but it can ensure you do not copy over any vMA "session cache" files to your DR site. You can check by doing a long listing in /home/vi-admin/.vmware and looking for the directory vmasessioncache which will contain any active cached sessions if you recently initialized a fastpass target.

Note: Again this is not really necessary and you can exclude the vmasessioncache directory as part of your backup

You will first need to "dump" the existing vMA database into a file and provide a name of your choice, you will need to run the following command:

sqlite3 /var/opt/vmware/vMA/vMA.db .dump > vMA.db.backup

Next, you will need to "scp" the following files to your vMA DR:

  •  vi-fastpass encrypted credential store file
    • /home/vi-admin/.vmware
  •  vMA's configuration dealing with vi-fastpass targets + vi-logger
    • /home/vi-admin/vMA.db.backup
  •  vMA's default logging configs + paths
    • /etc/vmware/vMA/vMA.conf 

You now should login to your vMA DR host and you should see only two files in the home directory of the vi-admin user: vMA.conf and vMA.db.backup (.vmware directory is a hidden directory in /home/vi-admin)

From here, you will restore vMA.conf and you will need to run the following command:

sudo mv vMA.conf /etc/vmware/vMA/vMA.conf

Next, you will restore vMA.db and you will need to run the following command:

sudo sqlite3 /var/opt/vmware/vMA/vMA.db < vMA.db.backup

At this point, we can verify the database contents by just running the ".dump" command:

sqlite3 /var/opt/vmware/vMA/vMA.db .dump

Now, we're not done yet, we need to run one additional tool that will perform some VMware "black magic" which will allow this new vMA to access all your ESX(i) and/or vCenter targets just as you had it before on your primary vMA.

You will need to create a file that provides some dynamic shared libraries for the tool we are going to execute. Create a file under /etc/ld.so.conf.d/vmware-vma using "sudo" and paste the following two lines:

  • /opt/vmware/vma/lib
  • /opt/vmware/vma/lib64

Now you will need to run the following to read in the configuration:

sudo ldconfig -f /etc/ld.so.conf.d/vmware-vma

Now you are ready to run the "migratecredstore" utility which is located under /opt/vmware/vma/bin which will perform the "black magic" and make sure you use sudo.

Once you see the successful message on completing the migration of your credential store, you have now fully restored your original vMA configuration. Here we perform a list of the active servers that was once accessible on primary vMA and initializing a target and verifying that it does in fact work.

One thing to note, if you still have access to your primary vMA, both your vMA's are now in an active/active state, with caveat that your primary vMA is the only one allowed to make any changes. What I mean by this, is when you initially add a host to vMA's fastpass, it not only creates two accounts, but it also associates it's system's UUID as part of the unique identifier which is stored on the host with the key VMAID

This means that if you deleted the target off your vMA DR host, it does not actually remove this entry on your ESX(i) and/or vCenter host. Only the primary vMA which has the matching UUID is able to remove the entry all together when you perform a "vifp removeserver" operation.You can see the system UUID by using the dmidecode utility.

We can also see this within the vMA database, when viewing this on the vMA DR host, you will actually see both entries of the primary and the DR vMA UUIDs because we restored the database with the original vMA's config.

If you need your vMA DR host to be able to modify entries or rotate passwords, you will need to shutdown vMA and update it's bios.uuid within the .vmx entry. You must use the "original" vMA's UUID which you should see from the database by running the following command:

sqlite3 /var/opt/vmware/vMA/vMA.db "select * from management_info;"

You will also need to delete the "new" UUID to ensure that only one exists which should be the "original" UUID, you can so by running the following command substituting your UUID:

sudo sqlite3 /var/opt/vmware/vMA/vMA.db "delete from management_info where myUUID='422E4042-63EE-86D1-D22A-79B6ABCA8D68';"

At this point, your vMA DR is now your primary and your old vMA is no longer needed.

Share this...
  • Twitter
  • Facebook
  • Linkedin
  • Reddit
  • Pinterest

Filed Under: Uncategorized Tagged With: credentialstore, vi-fastpass, vma, vMA.db

Cloning vMA still not supported?

02/01/2011 by William Lam 5 Comments

Cloning and/or moving vMA is not supported by VMware, this has been the case since VIMA 1.0. If you take look at the "Known Issues" in the vMA release notes, you will see there is no still workaround. Users may find it useful to be able to clone vMA for backup purposes or to deploy another vMA without having to re-download or re-uploading .OVF from your local desktop/VMware's website.

If you ever tried to clone VIMA 1.0 and boot the system up, you will notice a warning message during the init process when vMA verifies the system's UUID. If it detects that it has been cloned, it will actually disable itself and makes the vMA components unusable until you reinitialize vMA using vimaclean. This is still the case with vMA 4.1, except the vMA components are no longer disabled when you perform a clone.

You will just see the warning message during the init process, but all functionality continues to work. Before the vMA specific components starts up, it makes a call to /opt/vmware/vma/bin/verifyuuid. It verifies the system UUID which you can extract using dmidecode, this is the same UUID found within the .vmx entry called uuid.bios. When you clone a VM, a new UUID will be generated which causes an issue when it tries to verify.

Prior to vMA 4.x, VIMA 1.0 stored all it's configurations within a set of XML files located in /etc/vmware/viconfig, these files held the config for both the vi-fastpass targets and vilogger config and policies. With the release of vMA 4.0, these configurations were migrated to a sqlite3 database which is stored in /var/opt/vmware/vMA/vMA.db. Within this database, it also stored as the old flat file configuration, the system UUID of vMA. This is what verifyuuid was checking against to see if it was cloned or moved, you can actually query the vMA DB to extract this information using a simple SQL query.

To resolve the warning message, you just need to update the new UUID in the vMA DB using an update statement.

Instead of manually typing all that out, I actually created a very simple script that queries for the current system UUID and then updates the vMA DB with the correct UUID. Here is a sample execution:

Download: updatevMAUUID.sh

I suspect that VMware may have allowed cloning of vMA, but just did not update their release notes? The only thing you have to do is manually fix the UUID if you do not want to see the warning message. If you would like to fix this for VIMA 1.0, you will need to do the same thing except you need to modify the UUID found in /etc/vmware/viconfig/viconfig.xml and restart vMA for the changes to effect.

Share this...
  • Twitter
  • Facebook
  • Linkedin
  • Reddit
  • Pinterest

Filed Under: Uncategorized Tagged With: sqlite3, vma, vMA.db

  • Go to page 1
  • Go to page 2
  • Go to page 3
  • Go to page 4
  • Go to Next Page »

Primary Sidebar

Author

William Lam is a Senior Staff Solution Architect working in the VMware Cloud team within the Cloud Services Business Unit (CSBU) at VMware. He focuses on Automation, Integration and Operation for the VMware Cloud Software Defined Datacenters (SDDC)

  • Email
  • GitHub
  • LinkedIn
  • RSS
  • Twitter
  • Vimeo

Sponsors

Privacy & Cookies: This site uses cookies. By continuing to use this website, you agree to their use.
To find out more, including how to control cookies, see here: Cookie Policy