How to configure Likewise "Open" AD intergration on vMA

I recently received a question about whether it was possible to configure Active Directory integration with vMA. Out of the box, this is not a feature that is available by default but can be set up. There are many articles online that provide instructions on configuring AD integration on UNIX/Linux host but they may not always be as straight forward to implement. 

While pondering about this question, I remember reading an article about the OEM partnership between Likewise and VMware, in which Likewise's authentication software will be integrated into future releases of the vSphere platform. There has also been rumors that the Likewise software will be appearing in the next release of vSphere which may provide AD integration out of the box. 

Likewise has an open source product called "Open" which integrates with UNIX, Linux and Mac systems to Microsoft Active Directory, allowing users to authenticate with their Windows domain credentials. I thought it would be interesting to see if I could get "Open" running on VMware vMA and surely it was pretty straight forward. 

1. You will need to register to download the latest version of the Likewise software which can be found here:

Note: Make sure you select the 64bit version and the non-GUI version of "Open".

2. You will now upload the installer LikewiseIdentityServiceOpen-5.3.0.7798-linux-x86_64-rpm-installer to your vMA host using either UNIX/Linux scp or WinSCP if you are on a Windows systems.

3. Set the installer to be an executable by running the following command:

[vi-admin@kate ~]$ chmod +x LikewiseIdentityServiceOpen-5.3.0.7798-linux-x86_64-rpm-installer
4. Now we will begin the installation, you will need to use sudo (accept all defaults):

[vi-admin@kate ~]$ sudo ./LikewiseIdentityServiceOpen-5.3.0.7798-linux-x86_64-rpm-installer

We trust you have received the usual lecture from the local System
Administrator. It usually boils down to these three things:

#1) Respect the privacy of others.
#2) Think before you type.
#3) With great power comes great responsibility.

Password:
----------------------------------------------------------------------------
Welcome to the Likewise Identity Service [Open] Setup Wizard.

----------------------------------------------------------------------------
Please read the following License Agreement. You must accept the terms of this
agreement before continuing with the installation.

Press [Enter] to continue :
Likewise Open is provided under the terms of the GNU General
Public License (GPL version 2) and the GNU Library General
Public License (LGPL version 2.1). The additional components
listed below are covered under separate license agreements:

Samba 3.0 Client libraries and tools - GPLv2
MIT Kerberos - MIT Kerberos 5 and other licenses
OpenLDAP - OpenLDAP Public License
Novell DCE-RPC - BSD
LibXML2 - BSD
libuuid from e2fsprogs - BSD
libiconv - LGPLv2
OpenSSL - BSD

For more details and for the full text for each of these
licenses, read the LICENSES and COPYING files included with
this software.

Press [Enter] to continue :

Do you accept this license? [y/n]: y

----------------------------------------------------------------------------
32-bit Compatbility Libraries

Should the 32-bit compatibility libraries be installed? These are only needed if 32-bit programs will be accessing the Likewise authentication code. If you do not know the answer, just leave it as "Auto".

[1] Auto
[2] Yes
[3] No
Please choose an option [1] :

----------------------------------------------------------------------------
Setup is now ready to begin installing Likewise Identity Service [Open] on your computer.

Do you want to continue? [Y/n]: y

----------------------------------------------------------------------------
Please wait while Setup installs Likewise Identity Service [Open] on your computer.

Installing
0% ______________ 50% ______________ 100%
######################################Info: Likewise
--------

To join an Active Directory domain using a command-line interface, run:

/opt/likewise/bin/domainjoin-cli

Press [Enter] to continue :
###
5. Once the setup has finished, you will want to edit the lsassd.conf configuration file. The two changes that you will be making are:

• Allow a user to login to vMA without having to specify the username and the full domain (e.g. username@domain@vmahost)
• Changing the default login shell from /bin/sh to /bin/bash

Start by editing /etc/likewise/lsassd.conf

• uncomment "assume-default-domain = yes"
• change "login-shell-template = /bin/sh" to "login-shell-template = /bin/bash"

[vi-admin@kate ~]$ sudo vi /etc/likewise/lsassd.conf
Note: If you are using a newer version of "Open" where lsassd.conf no longer exists, please take a look at the "Open" documentation on updating the configurations listed above - http://www.likewise.com/resources/documentation_library/manuals/open/likewise-open-guide.html

6. Before we join the vMA host to the Active Directory server, ensure that DNS is properly configured and that both forward and reserve lookups are correct on the vMA host.

[vi-admin@kate ~]$ host kate
kate.primp-industries.com has address 172.30.0.189

[vi-admin@kate ~]$ host 172.30.0.189
189.0.30.172.in-addr.arpa domain name pointer kate.primp-industries.com.
7. We will now join the vMA host to an AD server. The syntax will be "domainjoin-cli join [domain] [username]"

[vi-admin@kate ~]$ sudo domainjoin-cli join primp-industries.com Administrator

Joining to AD Domain: primp-industries.com
With Computer DNS Name: kate.primp-industries.com

Administrator@PRIMP-INDUSTRIES.COM's password:
Warning: System restart required
Your system has been configured to authenticate to Active Directory for the first time. It is recommended that you restart your system to ensure that all
applications recognize the new settings.

Warning: Unknown pam module
The likewise PAM module cannot be configured for the wbem service. This services uses the '$ISA/pam_unix.so' module, which is not in this program's list of
known modules. Please email Likewise technical support and include a copy of /etc/pam.conf or /etc/pam.d.

Warning: A resumable error occurred while processing a module
Even though the configuration of 'pam' was executed, the configuration did not fully complete. Please contact Likewise support.

SUCCESS
Note: Do not worry about the warning message, it is normal and you do not need to restart the system for the changes to take effect.

If you have any issues trying to join a domain, you can enable logging which can be helpful for troubleshooting. To do so, you will specify two additional parameters which will denote the log level and where to output the log, whether that is to the console or to a file

[vi-admin@kate ~]$ sudo domainjoin-cli --loglevel verbose --logfile joindomain.log join primp-industries.com administrator
Joining to AD Domain: primp-industries.com
With Computer DNS Name: kate.primp-industries.com

administrator@PRIMP-INDUSTRIES.COM's password:
Warning: Unknown pam module
The likewise PAM module cannot be configured for the wbem service. This services uses the '$ISA/pam_unix.so' module, which is not in this program's list of
known modules. Please email Likewise technical support and include a copy of /etc/pam.conf or /etc/pam.d.

Warning: A resumable error occurred while processing a module
Even though the configuration of 'pam' was executed, the configuration did not fully complete. Please contact Likewise support.

SUCCESS
From the above example, you should have a new log file created called joindomain.log

8. To verify that you have successfully joined the domain, you can run the following command to query:

[vi-admin@kate ~]$ sudo domainjoin-cli query

Name = kate
Domain = PRIMP-INDUSTRIES.COM
Distinguished Name = CN=KATE,CN=Computers,DC=primp-industries,DC=com
9. Before you try to login with a user in the domain, you need to reload the configuration changes that were made earlier. To do so, you will execute the following:

[vi-admin@kate ~]$ sudo /opt/likewise/bin/lw-refresh-configuration

Configuration successfully loaded from disk.
10. Now, we will test a login using an account on the AD server:

[vi-admin@kate ~]$ ssh primp@localhost
Password:
Your password will expire today

Welcome to vMA
run 'vma-help' or see http://www.vmware.com/go/vma4 for more details.

[primp@kate ~]$ pwd
/home/local/PRIMP-IND/primp
We can also verify this user on the AD Server by running the following query:

Default level 0 info

[vi-admin@kate ~]$ /opt/likewise/bin/lw-find-user-by-name primp

User info (Level-0):
====================
Name: primp
SID: S-1-5-21-503341760-968948550-2164105906-1105
Uid: 1058014289
Gid: 1058013696
Gecos: primp primp
Shell: /bin/bash
Home dir: /home/local/PRIMP-IND/primp
Logon restriction: NO
level 2 info

[vi-admin@kate ~]$ /opt/likewise/bin/lw-find-user-by-name primp --level 2

User info (Level-2):
====================
Name: primp
SID: S-1-5-21-503341760-968948550-2164105906-1105
UPN: primp@PRIMP-INDUSTRIES.COM
Generated UPN: NO
DN: CN=primp primp,CN=Users,DC=primp-industries,DC=com
Uid: 1058014289
Gid: 1058013696
Gecos: primp primp
Shell: /bin/bash
Home dir: /home/local/PRIMP-IND/primp
LMHash length: 0
NTHash length: 0
Local User: NO
Account disabled (or locked): FALSE
Account expired: FALSE
Password never expires: TRUE
Password expired: FALSE
Prompt for password change: YES
User can change password: YES
Days till password expires: 0
Logon restriction: NO
To unjoin and leave the domain, you will use the following:

To preview the files that will require changes for leaving a domain use

[vi-admin@kate ~]$ sudo domainjoin-cli leave --advanced --preview

Leaving AD Domain: PRIMP-INDUSTRIES.COM
[F] DDNS - Configure Dynamic DNS Entry for this host
[X] [S] ssh - configure ssh and sshd
[X] [N] pam - configure pam.d/pam.conf
[F] nsswitch - enable/disable Likewise nsswitch module
[X] [N] krb5 - configure krb5.conf
[X] [N] stop - stop daemons
[X] [N] leave - disable machine account
[F] keytab - initialize kerberos keytab

Key to flags
[F]ully configured - the system is already configured for this step
[S]ufficiently configured - the system meets the minimum configuration
requirements for this step
[N]ecessary - this step must be run or manually performed.

[X] - this step is enabled and will make changes
[ ] - this step is disabled and will not make changes
To confirm and leave the domain, use

[vi-admin@kate ~]$ sudo domainjoin-cli leave

Leaving AD Domain: PRIMP-INDUSTRIES.COM
SUCCESS


All Likewise utilities are installed under /opt/likewise/bin and for more information on these utilities and how to use them, check out the Likewise documentation here.

UPDATE:
The instructions above can also be used to setup "open" on classic ESX w/Service Console, ESXi will not work however.

Is vSphere 4.1 release really imminent?

There have been a few articles floating around web regarding the potential features and speculations on when the next version of vSphere will be released. While doing some research on a new article that I am working on, I stumbled onto an interesting VMware website, called VMLive. VMLive is VMware's interactive webinar series designed specifically for their partner community. 

It seems that Australia-New Zealand VMLive channel is getting a head start on next release, which looks to be vSphere 4.1! I'm curious if other regions are getting similar offerings?

Here is a screenshot of the 4 upcoming webinars:

Here is the link, assuming it does not get pulled.

• http://app.connect.vmware.com/e/es.aspx?s=524&e=14780133&elq=acf5047eb86d490cb660ce78db98d2e7&OPENID=Browser

How to install vCLI 4.0 Update 2 on vMA

There was a question today on the VMTN forums about obtaining the latest version of resxtop for vMA to utilize the new NFS datastore counters. Unfortunately, there is no automatic method of updating vMA to get the new version of resxtop, which is part of the vCLI 4.0 Update 2 package. The current release of vMA 4.0 contains the GA release of vCLI 4.0 (May 2009). VMware has since released both and Update 1 and Update 2 of vCLI.

Even though there is no automatic way of upgrading the vCLI on vMA, it is actually pretty easy to download the latest version and upgrade it yourself. Before starting, you will want to download vCLI 4.0 Update 2 and ensure that it is the 64bit version.

Download: vCLI 4.0 Update 2

You will need to copy the tarball to your vMA host using either UNIX/Linux scp command or WinSCP if you're on a Windows system. Once the package has been uploaded, you will login to your vMA host and you should see the package in the current working directory:
[vi-admin@kate ~]$ ls
VMware-vSphere-CLI-4.0.0-253290.x86_64.tar.gz
Now you will extract the contents of the tarball using the following command:

[vi-admin@kate ~]$ tar -zxvf VMware-vSphere-CLI-4.0.0-253290.x86_64.tar.gz
After extracting the contents, you should now have a new folder called vmware-vsphere-cli-distrib:

[vi-admin@kate ~]$ ls -l
total 18304
-rw-r--r-- 1 vi-admin root 18714362 Jun 28 10:35 VMware-vSphere-CLI-4.0.0-253290.x86_64.tar.gz
drwxr-xr-x 10 vi-admin root 4096 Apr 23 01:01 vmware-vsphere-cli-distrib
You will now cd into the vmware-vsphere-cli-distrib directory and run the installer. The first time you run this, you will get an error and you need to remove the installer db at this time to proceed with the installation:

[vi-admin@kate vmware-vsphere-cli-distrib]$ sudo ./vmware-install.pl
A previous installation of vSphere CLI has been detected.

Uninstallation of previous install failed. Would you like to remove the install
DB? [no] yes

Removing installer DB, please re-run the installer.
Note: (This is necessary since the installer script does not support a clean upgrade from what I can tell)

Once you have successfully removed the installer db, you will need re-run the previous command which will start the installation (accept all the defaults and ensure you do overwrite the utilities):

[vi-admin@kate vmware-vsphere-cli-distrib]$ sudo ./vmware-install.pl

......

The installation of vSphere CLI 4.0.0 build-253290 for Linux completed
successfully. You can decide to remove this software from your system at any
time by invoking the following command:
"/usr/bin/vmware-uninstall-vSphere-CLI.pl".

This installer has successfully installed both vSphere CLI and the vSphere SDK
for Perl.

Enjoy,

--the VMware team
After this, you now have the latest version of vCLI 4.0 Update 2 installed on your vMA host.

The biggest feature with this new release of the vCLI is the NFS datastore metrics which has been a sought after for awhile. One other feature that has not gotten too much attention in the new version of esxtop/resxtop is the power management metrics, denoted by the new "y" option.

ESXi syslog caveat

There are two easy methods of capturing logs for both an ESX or ESXi server:

1. Use VMware vMA and vilogger to setup a syslog server. Simon Long wrote a very detailed article on how to configured and set it all up, check it out here.
2. Configure ESX or ESXi server to remotely log to an existing syslog server.

In ESXi, there are potentially three sets of logs to be captured: messages, hostd, and vpxa (if your host is being managed by vCenter). If you are using option #1 and you have enabled logging using vilogger, you will see the following:

If you are using option #2, there is a caveat to be aware of if you are using vCenter to manage your ESXi host, the vpxa logs are not actually being captured.

Here is an ESXi 4.0 Update 1 host that is configured to remotely log to a syslog server, tailing the logs you will notice only the messages and hostd logs:

I was recently made aware of this problem from VMTN community user aenagy who ran into this in his environment. After filing a support request with VMware, he found out there is an entry that is not added to vpxa.cfg configuration file when the host is joined to a vCenter. Per VMware, it was not considered a bug, but I would disagree.

To enable vpxa log capture, you will need to login to the unsupported Busybox console, also known as Tech Support Mode, for instructions please take a look at this VMware KB. You need to edit the following /etc/opt/vmware/vpxa/vpxa.cfg this is the configuration file for the vCenter agent running on the ESXi host:

<outputToSyslog>true<outputToSyslog>
<syslog>
    <ident>vpxa</ident>
    <facility>local4</facility>
</syslog>

 Append the above entries between the <log>...</log> tags. Once you have updated the vpxa.cfg file, you will want to run the follow command on the Busybox console to ensure the changes are saved and backed up to the local bootbank for ESXi. There is an automated cron job that runs every hour which calls /sbin/auto-backup.sh

You can just run that command manually which will backup the changes and you will see the diff at the bottom with the changes that were made:

~ # /sbin/auto-backup.sh

config implicitly loaded
local.tgz
etc/dropbear/dropbear_dss_host_key
etc/dropbear/dropbear_rsa_host_key
etc/opt/vmware/vpxa/vpxa.cfg
etc/opt/vmware/vpxa/dasConfig.xml
etc/opt/vmware/aam/configBackup.tar.gz
etc/opt/init.d/vmware-aam
etc/sysconfig/network
etc/vmware/hostd/authorization.xml
etc/vmware/hostd/hostsvc.xml
etc/vmware/hostd/pools.xml
etc/vmware/hostd/vmAutoStart.xml
etc/vmware/hostd/proxy.xml
etc/vmware/ssl/rui.crt
etc/vmware/ssl/rui.key
etc/vmware/vmkiscsid/initiatorname.iscsi
etc/vmware/vmkiscsid/iscsid.conf
etc/vmware/config
etc/vmware/dvsdata.db
etc/vmware/esx.conf
etc/vmware/license.cfg
etc/vmware/locker.conf
etc/vmware/snmp.xml
etc/vmware/vmware.lic
etc/dhclient-vmk0.leases
etc/group
etc/hosts
etc/inetd.conf
etc/chkconfig.db
etc/passwd
etc/random-seed
etc/resolv.conf
etc/shadow
etc/syslog.conf
etc/sfcb/repository/root/interop/cim_indicationfilter.idx
etc/sfcb/repository/root/interop/cim_indicationhandlercimxml.idx
etc/sfcb/repository/root/interop/cim_listenerdestinationcimxml.idx
etc/sfcb/repository/root/interop/cim_indicationsubscription.idx
--- /etc/opt/vmware/vpxa/vpxa.cfg Wed Jun 23 18:48:34 2010
+++ /tmp/auto-backup.26548.dir/etc/opt/vmware/vpxa/vpxa.cfg Wed Jun 23 18:47:00 2010
@@ -8,11 +8,6 @@
<maxfilenum>10</maxfilenum>
<maxfilesize>1048576</maxfilesize>
<outputtoconsole>false</outputtoconsole>
- <outputToSyslog>false</outputToSyslog>
- <syslog>
- <ident>vpxa</ident>
- <facility>local4</facility>
- <syslog>
</log>
<nfc>
<loglevel>error</loglevel>
config implicitly loaded
Saving current state in /bootbank
Clock updated.
Time: 18:48:38 Date: 06/23/2010 UTC


Once the backup has completed, you will need to reboot the host before the configuration can take effect. Once your host is back online, if you take a look at the logs on your syslog server, you will notice now that vpxa logs are also being captured:

Hopefully VMware resolves this, because this is definitely a bug and can be a painful one to remediate, especially if you are managing a few hundred ESXi hosts.

Thanks to aenagy for the information.

UPDATE:

After the blog post, I received a few questions:

1) The first question was if the modification would persist from an upgrade and I can confirm it does. My test was originally from an ESXi 4.0u1 and after the change, I upgraded the host using VUM to 4.0u2 and the changes were preserved. The reason for this is because the /sbin/auto-backup.sh was executed which backs up the ESXi configuration and is reloaded upon ever reboot including upgrades. 

2) The other question was around automating this change and the answer is yes! Assuming you have SSH access enabled on your ESXi host, you can use the following "ash" script (ESXi does not have a bash) to push the changes out to your host.

You can download the script here called esxiVPXASyslogFix.sh and make sure you have the right permissions on the script to execute.

 The script will first check to see if the entry exists, if it does not, it will then update the vxpa.cfg and automatically /sbin/auto-backup.sh

 Once the backup has completed, you will see a message that asks you to reboot the host for the changes to take effect.

 

My VCAP4-DCA beta experience

Over the weekend I participated in the VMware VCAP4-DCA beta exam, which is set to release later this year. My exam consisted of 41 live labs and I completed the exam in a little under 4.5 hours. Due to the NDA restriction, I will not be able to go into the specifics of the exam but I can share with you my experience while taking the exam.

The biggest issue I had while taking the exam was the user experience, which for me personally sucked! After 30 minutes or so into the exam, I noticed the display started to overlay multiple images on top of each other and not refreshing properly. Not only did I have interlaced images, certain parts of the screen were blank white and required multiple minimizations to force a redraw of the screen, which did not always work. This was very frustrating to deal with throughout the exam and at one point I had to call the proctor in since I was not able to figure out what I was clicking on as it actually was selecting something else. I ended up moving forward with the exam since I was already half way into the exam and there was nothing more the testing facility could have done. I did have the staff create an incident report which will be submitted to VMware for feedback on the display and connectivity issue.

After the exam ended, I spoke to the proctors and they had mentioned they had a similar issue earlier in the week with another candidate taking a VMware certification. They did not say if it was for VCAP-DCA but did say it was not an issue with Pearson but with the actual labs at VMware where these environments were hosted. If a user paid $400 for this exam and you received the type of experience I had, you would not be a happy camper. VMware still has some work to do in this area before the exam is ready to roll out later this year.

Other than a few minor issues/typos with some of the questions and pristineness of the environment, I was very impressed overall with the exam. I thought that the questions and tasks were very fair, some definitely took much longer to complete than others, so be cautious of your time. I thought I would be able to go back and check on some of the questions I had marked to come back to but ended up running out of time at the end. Luckily, I was able to provide comments on every question, though not in the detail as I would have liked. Looking back at the VCAP-DCA blueprint, I think the certification team did a pretty good job of providing an outline of topics to study for. Though it can be daunting at first glimpse, it definitely will be your best source of reference material. Unfortunately most of the beta invitees had less than 2 weeks if not less to prepare for this exam.

I also concur with Chris Dearden's observations with the blueprint, another beta participant of the VCAP-DCA exam. I felt that the exam was geared more towards consultants and/or outsourcing backgrounds than just VMware administrators. The topics on the blueprint ranged from the core vSphere components such as vCenter, ESX and ESXi but also additional auxiliary components that may not be used in all vSphere environments (e.g. Orchestrator, vSheild Zones, Heartbeat, etc). I have to agree with Chris' comment, it is probably rare that you will find all of VMware's enterprise products implemented in a single production vSphere environment other than maybe a test or development lab. For me, this was the first time I  have implemented some of these systems in my lab while studying for the exam.This was definitely a challenge with unfamiliar technology and a limited amount of time to to prepare for the exam.

Speaking of a lab, I would highty recommend that you setup a test lab as mentioned by Chris. You will probably need a minimum of two hosts to test and setup the various configuration laid out in the blueprint. I was fortunate enough to have access to two physical hosts and deployed 4 vESX(i) (virtual ESX & ESXi) hosts while studying for the exam.

In terms of advice, I would say to make sure you go through the blueprint thoroughly and make sure you can perform all the skills and abilities listed in each of the objectives. It is also very important to be able to perform the tasks using both the CLI and the GUI, making sure you are comfortable with both.

All in all, I was pretty happy with the exam minus the user experience. I will probably be a wreck until the results are announced (4-6 weeks maybe?) and hopefully I will pass. In the meantime, I will be able to get back to some of the scripts that I have put on hold while studying for this exam!

Good luck to anyone that may be taking the exam on the last day of the beta program which ends on Monday June 21st, 2010. I think the VMware community will be very happy with the new VCAP-DCA certification when it is released!

Update 10/14 - I just received an email that I passed VCAP4-DCA beta exam! I'm glad the wait finally over

Update 12/15 - I just received an email on my VCAP4-DCA number, I am VCAP4-DCA #6

esxcli Part3 - Automating esxcli using PowerShell

In this final three part esxcli series, I will show you how to automate esxcli operations in a Windows environment using PowerShell. There was a recent question on the VMTN forums regarding this very topic. As a simple solution to the problem, a community member provided a snippet of PowerShell code. Of course the snippet of PowerShell code originated from the great PowerCLI Master - Luc Dekens.

I have slightly modified the original version of the script as I am not a fan of hard coding passwords within a script. This version of the script requires plink which is a command-line interface to the free Windows SSH PuTTY back end. The script performs an SSH connection to either an ESX or ESXi host to run esxcli command locally, with the specified parameters as you would if you were on the Service Console, or the unsupported Busybox console.

Copy the snippet of code below and create a new script called esxcli-automation1.ps1 and replace all the variables matching your environment including the $cmd which specifies the exact esxcli command you are running on the remote host.

esxcli-automation1.ps1

$User = "root"
$Computer = "esxi4-1.primp-industries.com"

#prompt user for password
$passwordInput = Read-Host -AsSecureString:$true "Please enter password for $Computer"

#convert secure password to normal string
$Ptr = [System.Runtime.InteropServices.Marshal]::SecureStringToCoTaskMemUnicode($passwordInput)
$Password = [System.Runtime.InteropServices.Marshal]::PtrToStringUni($Ptr)
[System.Runtime.InteropServices.Marshal]::ZeroFreeCoTaskMemUnicode($Ptr)

$plink = "C:\plink.exe"
$esxcli = "esxcli"
$plinkoptions = " -v -batch -pw '$Password'"
$cmd = 'nmp device list'
$remoteCommand = $esxcli + " " + '"' + $cmd + '"'
$command = $plink + " " + $plinkoptions + " " + $User + "@" + $computer + " " + $remoteCommand
Invoke-Expression -command $command

Here is a sample execution listing devices:

There are a few issues with this script:

• Assumes SSH connectivity is open to the host, which on classic ESX w/Service Console is probably a valid reason but what about ESXi? We know that we should not be using the unsupported Busybox console on ESXi and we can not make the assumption that it will be opened

          Here is the same example when executing against an ESXi host:

• The script can only execute against one host at a time.

In a perfect world, the esxcli API would be properly exposed and integrated into the vSphere API. PowerCLI and other vSphere SDKs would be able to automate these configurations without this hack, but unfortunately this is not the case. However, we can leverage the esxcli utility that is bundled up with the vCLI which can be installed on either a Linux or Windows platform.

With this information, I decided to write my own PowerShell script (yes you heard right, it was slightly difficult without knowing the syntax and format) matching the capabilities similar to that of the vSphere SDK for Perl implementation running on vMA. So here is the second revision of the script called esxcli-automation2.ps1:

esxcli-automation2.ps1

$Username = "root"
$esxcli = "C:\Program Files\VMware\VMware vSphere CLI\bin\esxcli.exe"
#$Params = "swiscsi nic list -d vmhba33"

#list of ESX and ESXi host to perform operation against
$vihosts = @("esxi4-1.primp-industries.com","esxi4-3.primp-industries.com")

#prompt user for password
$passwordInput = Read-Host -AsSecureString:$true "Please enter your common password for hosts"

#convert secure password to normal string
$Ptr = [System.Runtime.InteropServices.Marshal]::SecureStringToCoTaskMemUnicode($passwordInput)
$Password = [System.Runtime.InteropServices.Marshal]::PtrToStringUni($Ptr)
[System.Runtime.InteropServices.Marshal]::ZeroFreeCoTaskMemUnicode($Ptr)

#loop through array of hosts
for($i = 0; $i -le $vihosts.length -1; $i++) {
 write-Host Executing operation on $vihosts[$i]
 #could not find a way to store the params in a variable you must
 #insert esxcli parameters at the very end of the statement below
 & $esxcli --server $vihosts[$i] --username $Username --password $Password swiscsi nic list -d vmhba33
}

Note: PowerCLI is not required, just PowerShell.

This new version of the script allows you specify a list of ESX or ESXi host to perform a common esxcli operation. The script also prompts for the password as the previous script, so you do not have to hard code the password to the host. You will need to update the script with the path to the plink.exe and the list of hosts you want to perform the operation against.

Here is an example of verifying iSCSI multipathing configuration against two hosts (1 ESX and 1 ESXi) without the use of SSH:

As you can see, you can easily integrate this with other existing PowerCLI scripts that perform configuration changes across a given set of host(s). Until VMware provides access to the esxcli APIs, these are some of the methods/hacks that need to be implemented when automating configuration changes using esxcli.

Here are some PowerShell references that were used in creating the two scripts:

• http://technet.microsoft.com/en-us/library/ee176935.aspx

• http://www.rickcable.com/blog/default.asp?Display=33

• http://bsonposh.com/archives/247

esxcli Part2 - Automating esxcli using vMA

In the previous article, esxcli Part1, we discussed what esxcli is and what it can be used for. In this article, we will show you how you can automate and perform a common esxcli operation against a set of ESX and/or ESXi hosts using vMA. As mentioned in the previous article, esxcli needs to be executed against a specific host, it is not vCenter aware and cannot connect to vCenter to perform an operation against a host.

One of the pre-requisites before starting is that all ESX and ESXi hosts need to be managed by vMA and accessible using vi-fastpass authentication. Please refer to the vMA documentation on configuring this on vMA.

Download: esxcli-automation.pl

The following script is based off of useVIFastpassOnvMAToRunPerlScriptWithoutClearTextPassword.pl. The modified script utilizes the vi-fastpass library to properly access a set of target(s) and performs the specified esxcli operation without having to provide credentials to each and every host.

The following example will demonstrate iSCSI multipath configuration on two hosts using the method described by Duncan Epping in this article.

1) Create a file containing the list host to perform the operations against:

[vi-admin@scofield ]$ cat hosts
esxi4-1.primp-industries.com
esxi4-3.primp-industries.com
2. Run the script with the specific esxcli parameters as you would normally by passing it into the --cmd_option argument (double quote the arguments):

Binding iSCSI interface to vmk0

[vi-admin@scofield ]$ ./esxcli-automation.pl --hostlist hosts --cmd_option "swiscsi nic add -n vmk0 -d vmhba33"

Executing script on "esxi4-1.primp-industries.com" ...

Executing script on "esxi4-3.primp-industries.com" ...
Binding iSCSI interface to vmk1

[vi-admin@scofield ]$ ./esxcli-automation.pl --hostlist hosts --cmd_option "swiscsi nic add -n vmk1 -d vmhba33"

Executing script on "esxi4-1.primp-industries.com" ...

Executing script on "esxi4-3.primp-industries.com" ...
3. Verify iSCSI multipathing has been properly configured:

[vi-admin@scofield ]$ ./esxcli-automation.pl --hostlist hosts --cmd_option "swiscsi nic list -d vmhba33"

Executing script on "esxi4-1.primp-industries.com" ...
vmk0
pNic name: vmnic0
ipv4 address: 172.30.0.183
ipv4 net mask: 255.255.255.0
ipv6 addresses:
mac address: 00:50:56:92:69:7a
mtu: 1500
toe: false
tso: true
tcp checksum: false
vlan: true
link connected: true
ethernet speed: 1000
packets received: 167051261
packets sent: 232132
NIC driver: e1000
driver version: 8.0.3.1-NAPI
firmware version: N/A

vmk1
pNic name: vmnic1
ipv4 address: 172.30.0.184
ipv4 net mask: 255.255.255.0
ipv6 addresses:
mac address: 00:50:56:92:68:6c
mtu: 1500
toe: false
tso: true
tcp checksum: false
vlan: true
link connected: true
ethernet speed: 1000
packets received: 145924
packets sent: 54502
NIC driver: e1000
driver version: 8.0.3.1-NAPI
firmware version: N/A

Executing script on "esxi4-3.primp-industries.com" ...
vmk0
pNic name: vmnic0
ipv4 address: 172.30.0.233
ipv4 net mask: 255.255.255.0
ipv6 addresses:
mac address: 00:50:56:92:33:95
mtu: 1500
toe: false
tso: true
tcp checksum: false
vlan: true
link connected: true
ethernet speed: 1000
packets received: 138535
packets sent: 8026
NIC driver: e1000
driver version: 8.0.3.1-NAPI
firmware version: N/A

vmk1
pNic name: vmnic1
ipv4 address: 172.30.0.234
ipv4 net mask: 255.255.255.0
ipv6 addresses:
mac address: 00:50:56:92:27:65
mtu: 1500
toe: false
tso: true
tcp checksum: false
vlan: true
link connected: true
ethernet speed: 1000
packets received: 145924
packets sent: 112
NIC driver: e1000
driver version: 8.0.3.1-NAPI
firmware version: N/A

Now you will be able to automate any of the supported esxcli operations against multiple hosts!

In Part3, we will take a look at automating esxcli operations in a Windows environment using Powershell.

esxcli Part1 - What is esxcli?

esxcli is a new CLI (commandline interface) framework in vSphere that provides a modular architecture for various components called namespaces running in the VMkernel. Some of these namespaces are nmp (Native Multipathing Plugin) for the new VMware Pluggable Storage Architecture, corestorage for claim rules used for masking certain devices to a host, and swiscsi for managing iSCSI interface.

esxcli can be executed within the classic ESX Service Console, the unsupported Busybox console in ESXi or using the vCLI's remote version of esxcli. There are currently 3 namespaces (nmp,corestorage and swiscsi) with the current release of vSphere and we may see others introduced in future releases of vSphere. One important thing to note is that because these modules run within the host, using the vCLI's version of esxcli, you will need to authenticate to the host first to see what modules will be available for access. Currently, esxcli is not vCenter aware, meaning you must connect to a specific ESX or ESXi host when performing an operation.

Here is an example of esxcli executed without connecting to the host first:

Here is an example of esxcli being executed after connecting to the host:

When invoking the esxcli command, you may also notice an esxcli.log is generated. If the command is successfully executed, the log will generally be empty, but if there was an error you may want to take a look at esxcli.log if the command does not provide any output to the screen.

Here is an example of using an auth configuration file and because of the case sensitivity of esxcli, the VI_PROTOCOL entry is failing with HTTPS vs https:


[vi-admin@scofield ~]$ cat esxcli.log

[root CRITICAL] Exception:Unsupported protocol
[root CRITICAL] Traceback (most recent call last):
File "esxcli.py", line 387, in _GetStub
File "/vmware/esx40-dev/esx40/bora/vim/py/esxcli/Session.py", line 239, in stub
File "/vmware/esx40-dev/esx40/bora/vim/py/esxcli/Session.py", line 299, in Login
Exception: Unsupported protocol

There is not a whole lot of information available to the public about esxcli. From what I understand after talking to a few VMware engineers, esxcli has an API, but it is currently not exposed to the public for consumption. Not only is there an API, but 3rd party providers or users can potentially create their own modules and install it using the VIB format also known as vSphere Installation Bundle.

Some well known packages that are currently being distributed in the VIB format today are: Cisco Nexus 1000V VEM, HP Insight Manager Agent, EMC PowerPathV/E, Xsigo ESX IB drivers and VMware ESX/ESXi/vMA updates, to name a few. Hopefully in the future, VMware will expose the esxcli API functionality to the developer ecosystem.

Here are a few blog posts with detail examples on using esxcli with the various namespaces:

• http://www.yellow-bricks.com/2009/03/18/iscsi-multipathing-with-esxcliexploring-the-next-version-of-esx/
• http://www.yellow-bricks.com/2009/03/19/pluggable-storage-architecture-exploring-the-next-version-of-esxvcenter/
• http://virtualgeek.typepad.com/virtual_geek/2009/09/a-multivendor-post-on-using-iscsi-with-vmware-vsphere.html
• http://www.punchingclouds.com/?p=965

Stay tuned for Part2 and Part3 where we will look at automating esxcli operations using both the vSphere SDK for Perl and Windows PowerShell!

Is the release vSphere 4.0 Update 2 imminent?

There's already been some chatter in the community regarding vSphere 4.0 Update 2, including a VMware KB article:

http://kb.vmware.com/kb/1021023
http://www.yellow-bricks.com/2010/03/30/whats-the-point-of-setting-iops1/
http://www.vladan.fr/vsphere-4-update-2-is-next/
http://virtualization.info/en/news/2010/03/vsphere-40-update-2-to-add-new-ha.html
http://www.yellow-bricks.com/2010/03/29/cool-new-ha-feature-coming-up-to-prevent-a-split-brain-situation/

While recently downloading a copy of ESX 4.0u1 for testing, I noticed the download URL was the following:

http://www.vmware.com/downloads/download.do?downloadGroup=ESX40U1

The link will generally take you to landing page to login if you have not ready logged in:

I thought I try to plug-in "2" and see what happens, to my surprise it actually displayed some interesting text:

http://www.vmware.com/downloads/download.do?downloadGroup=ESX40U2

When you login, it will throw an error that the download group does not exists.

However, if you tried another invalid entry say "ESX40U3", you'll get an error all together such as the following even before logging in:

http://www.vmware.com/downloads/download.do?downloadGroup=ESX40U3

So does this mean vSphere 4.0 Update 2 about to be released? Your guess is as good as mine, we will just have to wait and see.

How to remove stale targets from vMA

If you have used vMA's vi-fastpass authentication, you will know how easy it is to setup using vifp utility which supports both ESX/ESXi and vCenter targets. 

Here's an example of adding ESXi target:

[vi-admin@scofield ~]$ sudo vifp addserver esxi3-1.primp-industries.com
root@esxi3-1.primp-industries.com's password:

Here's an example of the listing of the available fastpass targets:

[vi-admin@scofield ~]$ sudo vifp listservers
esxi3-1.primp-industries.com ESXi

During this process, two accounts (vi-userXX & vi-adminXX) are created on the target host with a password that vMA management creates and caches it locally in an obfuscated but not encrypted form. This will allow you to initialize a fastpass target using vifpinit utility and execute commands against the target host without having to manually type in the credentials.

The fastpass targets are stored in 2 configuration files on vMA:

1) The obfuscated cached credentials is stored in /home/vi-admin/.vmware/credstore/vicredentials.xml

If you cat out the contents, it will look something like this:

<passwordentry>
   <host>esxi3-1.primp-industries.com</host>
   <username>vi-admin00</username>
   <password>XXXXXXXXXXXXXXXXXXXXXXX</password>
</passwordentry>

2) A More detailed configuration for each of the targets along is stored in /etc/vmware/viconfig/viconfig.xml

If you cat out the contents, it will look something like this:

<host>
   <hostname>esxi3-1.primp-industries.com</hostname>
   <portNumber>443</portNumber>
   <esxID>524d18f6-8bbb-2c5f-a366-6d191813fbe3</esxID>
   <protocol>https</protocol>
   <servicePath>/sdk</servicePath>
   <vi-admin>vi-admin00</vi-admin>
   <vi-user>vi-user00</vi-user>
   <isVIUsersCreatedByFP>true</isVIUsersCreatedByFP>
   <targetType>ESX</targetType>
   <lastPasswordUpdateTime>1276121961</lastPasswordUpdateTime>
</host>

What happens when you rebuild your host, or the system is no longer available because it has been decommissioned or being used for another purpose? vMA will still think it's managing the host and the fastpass credentials will no longer function as the account is no longer valid the host. If you try to remove the old target, you will see the following error:


[vi-admin@scofield ~]$ sudo vifp removeserver esxi3-1.primp-industries.com
root@esxi3-1.primp-industries.com's password:

Error: Failed to connect. Please make sure the server is up and is of supported version.

The reason this occurs is that vMA is unable to login to the host and remove the two accounts that were initially created and fails to remove the target. What you will need to do is actually pass in an additional parameter to vifp command "--force" which will forcefully remove the target from vMA management. This command actually does not require the user to enter the correct password to the host even if it is still reachable by vMA. By specifying this flag and providing some input when prompted for the password, vMA will purge the target from it is system.


[vi-admin@scofield ~]$ sudo vifp removeserver esxi3-1.primp-industries.com --force
root@esxi3-1.primp-industries.com's password:

After a target is removed from vMA, it is also removed from the two above files. You do not manually tweak either of these configuration files or it may lead to issues on your vMA host.

Best practice for decommissioning a host that has been added to vMA's management is the following:

1. Disable vilogger if you've enabled it for the host
2. Remove target from vMA management
3. Verify the host is no longer being managed by vMA
4. Decomission host

Increase Syslog count in ESXi using Busybox

There was an interesting question on the ESXi forum today using syslog and logging to a local VMFS volume and potentially filling up the datastore. By default, the log files are 2MB each before they are rotated out with a total of 9 copies all together. This ensures that the logs will not grow beyond 18MB and potentially fill up your datastore.

Within the unsupported console of ESXi, there is a little tool called "Busybox" which implements a set of familiar command line utilities called applets. Some of these applets are uptime, crond, chroot, md5sum, etc... If you login to the unsupported console and just type "busybox" on the console, you will see the following:

If you look carefully, you will see that one of the applets is "syslogd", which is the syslog daemon running in ESXi. To access a specific applet, you will just type busybox and then the name of the applet along with -h which forces the applet to provide a set of available options. If we do this for syslogd, you will see the following:

The default configurations should ensure that your logs don't go crazy and fill up the volume, but the downside is that your log history will not be kept forever. That is why it is a best practice to setup a remote syslog server to send all your logs for further processing and auditing if necessary.

However, if you wanted to change this defaut, you can. As you can see from the options above, you can configure not only the size (max of 2MB) before rotating out, but also the number of logs to keep (max of 99). While you cannot increase the size of the individual logs, you can specify a larger number of logs to keep. If you want to make the changes live while the system is running, you'll need to perform the following:

1. Find the current pid of syslogd

~ # ps | grep syslog
4313 4313 busybox syslogd


2. Kill the syslog process

~ # kill $(cat /var/run/syslogd.pid)


3. Let's say you want to keep 50 copies instead of 9

~ # busybox syslogd -b 50


Note:

• The input to -s is in bytes with range (0-2097151)
• The input to -b has range (1-99)

It's important to note that this change will not persist through a reboot. I have not been able to figure out where this is set; it could just be hardcoded in the binary by default. A way around this is to re-define the configuration upon boot up by adding an entry to /etc/rc.local which will kill the current running syslogd and then start up with the new parameters as shown above.

Add the following lines to the end of /etc/rc.local:

kill $(cat /var/run/syslogd.pid)
busybox syslogd -b 50

Now for this change to fully persist, you need to do one more thing, you'll need to run /sbin/auto-backup.sh which will force a local backup of the ESXi configuration files which includes /etc/rc.local so that it'll survive through the next reboot. Now you'll be able to store additional ESXi logs for a longer period of time if you choose to log locally.

The vExpert 2010 Crew

The wait is finally over! The highly anticipated e-mail from John Troyer announcing the second group of individuals for the vExpert 2010 award went out today. I'm very honored to be accepted into the vExpert 2010 program and look forward to sharing it with a great group of individuals! I want to thank John for organizing this program and for all the hard work he has put in to recognizing everyone. Congratulations to the vExpert 2010 crew! Let's make 2010 a great year!

Here are some links/blog posts to the other vExpert's for 2010:

Arnim van Lieshout
Tom Howarth & Wil van Antwerpen
Brian Knudtson
Jase McCarty
Maish Saidel-Keesing
Barry Coombs
Joe Kelly
Mike Laverick
Paul Davey
Aaron Delp
Erik Scholten
Justin Emerson
Didier Pironet
Carlo Costanzo
David Marshall
Scott Lowe
Mark Vaughn
Ed Saipetch

Additional Links:
http://www.vmware.com/communities/vexpert/
http://www.yellow-bricks.com/2009/10/01/vexperts-2010/

Here is a vExpert 2010 twitter list put together by @maishsk that you can follow:
http://twitter.com/maishsk/vmware-vexpert-2010

Adventures with HP SAM and Win2008 R2 domains

For the past two years or so we have been testing out VDI in our residence hall student labs at UCSB which consist of about twenty or so thin clients split between two locations. Now what made all of this feasible was:

1) Advancements in remote display protocols
and
2) our ghetto-link-clones script (http://communities.vmware.com/docs/DOC-9020)

We picked HP's RGS as our remote display protocol because it was the only protocol we found at the time that ensured a "rich desktop user experience". I still prefer it today over the software implementation of PCoIP but that is a topic best left for another day.

Naturally, the only broker that pooled RGS-enabled machines at the time was HP's Session Allocation Manager (today Leostream and others can broker it). Despite bad experiences with some HP software in the past, SAM actually worked out very well for us, both on the server side as well as the client side. I still have yet to see a broker package that allows you to customize the client like HP SAM (this feature is definitely stamped as RAD in my book).

SAM was not perfect though and had the occasional hiccup (e.g. machines being listed as unavailable, annoying but only occurred once in a while and was remedied with a resync operation) until the day came: management decided to upgrade the domain from Win2k3R-something to Win2k8R2. SAM completely stopped working; it was an interesting situation to say the least.

After some testing (with profanity and throwing of objects included), we of course determined that the fault was in SAM (3.0, the latest release at the time in early January did not work). HP's response? No official timeframe to release a fix. This equated to a week of scrambling around looking for competing and "compatible" products (ala Leostream) which just ended up complicating things. The solution? An aggravating night biking back and fourth between the two labs switching the entire VDI environment over to View. Whatever it takes right?

So long story short, if you have an existing HP SAM installation that is running on a pre-Win2k8R2 domain, make sure you update to the latest revision of SAM (3.01) that was just released a couple weeks ago before migrating your domain to Win2k8R2.

For a short description of our VDI environment, please refer to:
http://communities.vmware.com/docs/DOC-9201

Script - hostops-lamw.pl

I recently noticed a question on the ESXi forum about trying to add a host to a vCenter server that had the "SSL host certificate verification" enabled while using the vSphere SDK for Perl Utility hostops.pl on vMA. The user encountered the following error when trying to add the host:

Error:
SOAP Fault:
-----------
Fault string: Authenticity of the host's SSL certificate is not verified.
Fault detail: SSLVerifyFault

The SSL host verification is a feature that came with the release of vSphere that provides a security measure to verify the validity of a host before adding it to your VMware infrastructure. This feature is disabled by default, but when it is enabled, a user will need to accept a dialog box to confirm the SHA1 thumbprint of the host in question.

This particular use case was not handled properly by hostops.pl which caused the error message to be thrown. With a small tweak to VMware's canned script, the new and improved hostops-lamw.pl now supports adding an ESX or ESXi host into vCenter with SSL host verification enabled. You'll still be expected to verify the SHA1 thumbprint, but now you can pass this as an additional parameter which will tell vCenter that you have verified the host and add to vCenter management.

Scott Lowe originally wrote an article on how to verify the SHA1 thumbprint for both an ESX and ESXi host.

On ESX you can run the following:
openssl x509 -sha1 -in /etc/vmware/ssl/rui.crt -noout
-fingerprint


On ESXi, the only real way to verify is by looking at the DCUI's "View Support Information":

However, if you truly trust the ESX or ESXi host that you're going to add to vCenter, there is an alternative way of retrieving the SHA1 thumbprint using the vCLI's vifs and the modified hostops-lamw.pl.

By default, you'll be able to point your web browser to https://[hostname]/host/ssl_cert to see actual SSL certificate on your host, assuming this functionality is not disabled. What you can do is download the ssl_cert to vMA or system with vCLI installed and query for the SHA1 hash and provide that as input to hostops-lamw.pl.

Download: hostops-lamw.pl

Step 1. Download hostops-lamw.pl to either vMA or system running vCLI copy it to the following path:

vMA or Linux host	/usr/lib/vmware-cli/apps/host
Windows	C:\Program Files\VMware\VMware vSphere CLI\Perl\apps\host

Step 2. Download the ssl_cert to vMA:

[vi-admin@scofield ~]$ vifs --server esxi4-1.primp-industries.com --username root --get "/host/ssl_cert" esxi4-1.primp-industries.com-ssl_cert
Enter password:

Downloaded file to esxi4-1.primp-industries.com-ssl_cert successfully.


Step 3. Get the SHA1 thumbprint from the ssl_cert you downloaded:

[vi-admin@scofield ~]$ openssl x509 -sha1 -in esxi4-1.primp-industries.com-ssl_cert -noout -fingerprint

SHA1 Fingerprint=79:BB:39:09:F6:E5:91:BD:B0:C3:F3:09:B4:38:50:FB:ED:9C:53:A5


Step 4. Use the modified hostops-lamw.pl and the new --sslthumbprint providing the SHA1 thumbprint (remember to double quote it) along with the other required input to add the host to vCenter:

[vi-admin@scofield ~]$ ./hostops-lamw.pl --server reflex.primp-industries.com --username primp --operation addhost --target_host esxi4-1.primp-industries.com --target_username root --target_password 'password' --sslthumbprint "79:BB:39:09:F6:E5:91:BD:B0:C3:F3:09:B4:38:50:FB:ED:9C:53:A5" --cluster virtual-cluster

Host 'esxi4-1.primp-industries.com' added successfully

Script - Updated vSphere Health Check 4.0.8

Check the latest update to popular vSphere Health Check Script which has now gone to v4.0.8!

One new change that you'll notice is the two additional menu options at the top of the report when you running the report against a vCenter server.

VPX Settings - All configured vCenter settings

VMware/3rd Party Applications - Display any registered VMware/3rd party applications running within a VM

For a list of other changes, please take a look at the vSphere Health Check Script change log.

Give the new script a try and let me know if have any problems or would like to see other features included in a future release.

Script - ipmiConfig.pl

There was a question on the VMTN community asking about automating the configuration of IPMI/iLO Settings for Power Management via kickstart. Unfortunately, this cannot be configured directly on an ESX/ESXi host, it requires vCenter server. This configuration is a prerequisite to VMware's DPM (Distributed Power Management) which is only available with vCenter and DRS enabled cluster.

Although this configuration can't be added to kickstart, you can create a script that is executed as part of the "post configurations". Before starting, make sure that you have IPMI enabled over LAN which is generally found within your iDRAC/iLO or other out-of-band management console.

Here are two posts on popular hardware (Dell/HP) for configuring IPMI/iLO settings:

Dell configuration
HP configuration

In general, the actual configuration for the host is the same but will differ based on the server vendor and out-of-band management console.

Here is a vSphere SDK for Perl script that allows you to configure IPMI/iLO settings for an ESX or ESXi host:

Download: ipmiConfig.pl

You'll need to provide the following variables:

• vihost = ESX or ESXi host
• ipaddress = IP Address of your iDRAC/iLO/etc.
• macaddress = MAC Address of your iDRAC/iLO/etc. interface
• bmcusername = Username to login to your iDRAC/iLO/etc.
• bmcpassword = You'll be prompted to provide the password

Here is an example run:

[vi-admin@scofield ~]$ ./ipmiSettings.pl --server reflex.primp-industries.com --username primp --vihost dev-esxi1.primp-industries.com --ipaddress 172.50.0.45 --macaddress 00:22:E8:71:C5:47 --bmcusername admin

Please enter your BMC Password:

Trying to configure IPMI/iLO Settings for Power Management on dev-esxi1.primp-industries.com ...


If you've entered everything correctly, the prompt should return right away. If it sits there for more than 20 seconds, then something went wrong and you may have entered something incorrectly. Due to the limitation of this API method, no tasks are returned back to properly handle these errors. You may want to manually configure this via the vSphere Client to double check your settings before re-using the script.


Here is an example of a bad run (entered wrong username + password):

[vi-admin@scofield ~]$ ./ipmiSettings.pl --server reflex.primp-industries.com --username primp --vihost dev-esxi1.primp-industries.com --ipaddress 172.50.0.45 --macaddress 00:22:E8:71:C5:47 --bmcusername root

Please enter your BMC Password:

Trying to configure IPMI/iLO Settings for Power Management on dev-esxi1.primp-industries.com ...

Error: Unable to configure IPMI/iLO Settings: SOAP request error - possibly a protocol issue: 500 SSL read timeout:

The vStorage API, do you really know what it is?

I’ve seen this question posed quite a few times both in the VMware developer and VMTN forum asking what exactly is the vStorage API?

So what is it?

If your answer was VMware marketing term, then you win 50 Schrute Bucks!

The VMware vStorage API is actually a blanket umbrella term that encompasses 4 separate individual APIs, all with different functionalities:

1. vStorage API for Data Protection (VADP)
2. vStorage API for Site Recovery Manager (VASRM)
3. vStorage API for Multi-pathing (VAMP)
4. vStorage API for Array Integration (VAAI)

I’m actually going to quote one of Chad Sakac’s reply on a blog about these 4 APIs as he has done a great job of explaining the differences:

1) vStorage API for Data Protection – a set of APIs focused on local backup/recovery use cases.

2) vStorage APIs for Site Recovery Manager – a set of APIs focused on array vendor remote replication such that they can be orchestrated by Site Recovery Manager.

3) vStorage APIs for Multipathing (otherwise known as the Pluggable Storage Architecture). A set of APIs for 3rd parties to extend vSphere’s core multipathing architectures.

4) vStorage APIs for Array Integration (VAAI). Technically not in vSphere 4, but have been discussed in VMworld events in the past. Will be available in future vSphere-generation releases. This allow array vendors to “offload” various tasks from the ESX host’s vmkernel stack – things like writing blocks that make up VMs, copying/snapshotting blocks, doing thin-provisioning out of space handling, and also a much more advanced global locking mechanism than VMFS uses today. These each will make common actions 5x-10x faster (clone, deploy from template, create a FT VM), and improve VMFS scaling by an order of magnitude. More on that here, for folks that are interested (note that when I wrote this, vStorage API for Data Protection was called the VCB Backup Framework)

Generally when the vStorage APIs are brought up, most people think about backups and the new Change Block Tracking feature in vSphere. That is because VMware and other 3rd party backup vendors has done a good job of marketing this "must have" feature. Leveraging Change Block Tracking helps decrease the duration of a backup by only copying the blocks that have changed and some users have seen up to 5-10x increase in speed.

This is great! But now you might ask the question, how do I use vStorage API for Data Protection and how does it tie into VMware's VCB product? Both the vStorage API for Data Protection and VCB are backup API frameworks, they are similar in functionality but different in features (for a break down of the two, take a look at this VMware blog post). VADP will be pretty much hidden from the end user's perspective, you just need to ensure that backup vendors are implementing it and utilizing Change Block Tracking to efficiently backup your VMs. VADP is only available in vSphere and not in VI 3.5.

If you want to write your own backup application, then you will need to know how to hook into VADP. As far as I understand today, and correct me if I'm wrong, the vStorage API for Data Protection is actually the combination of the vSphere 4.0 API + VMware Virtual Disk Development Kit (VDDK) which are both available to users to develop against. There's actually a guide within the VDDK page on Designing Backup Solutions for VMware vSphere which goes into great detail on creating a backup solution using the Change Block Tracking feature.

The three other APIs (VASRM, VAMP and VAAI) are targeted at third party hardware/software vendors and storage array providers to hook in their special sauces with the various VMware solutions. This includes hooking into Site Recovery Manager, PSA (Pluggable Storage Architecture) plugins and offloading VMware operations such as VM cloning and storage vMotions, etc. onto the actual storage array. These APIs are only exposed to partners who provide solutions to one of these features and are not available to the public for general use.

I personally think vStorage API is going to be a game changer and Change Block Tracking is just one of the many cool features to come!

Hopefully this all made sense and if you're interested to learn more about the vStorage API and some of the upcoming features, take a look at these additional resources:

http://www.ntpro.nl/blog/archives/1461-Storage-Protocol-Choices-Storage-Best-Practices-for-vSphere.html
http://virtualgeek.typepad.com/virtual_geek/2008/09/so-what-does-vs.html
http://www.vmware.com/products/vstorage-apis-for-data-protection/
http://www.yellow-bricks.com/2009/03/19/pluggable-storage-architecture-exploring-the-next-version-of-esxvcenter/