virtuallyGhetto

Is the release vSphere 4.0 Update 2 imminent?

by Leave a Comment

There’s already been some chatter in the community regarding vSphere 4.0 Update 2, including a VMware KB article:

http://kb.vmware.com/kb/1021023
http://www.yellow-bricks.com/2010/03/30/whats-the-point-of-setting-iops1/
http://www.vladan.fr/vsphere-4-update-2-is-next/
http://virtualization.info/en/news/2010/03/vsphere-40-update-2-to-add-new-ha.html
http://www.yellow-bricks.com/2010/03/29/cool-new-ha-feature-coming-up-to-prevent-a-split-brain-situation/

While recently downloading a copy of ESX 4.0u1 for testing, I noticed the download URL was the following:

http://www.vmware.com/downloads/download.do?downloadGroup=ESX40U1

The link will generally take you to landing page to login if you have not ready logged in:

I thought I try to plug-in “2” and see what happens, to my surprise it actually displayed some interesting text:

http://www.vmware.com/downloads/download.do?downloadGroup=ESX40U2

When you login, it will throw an error that the download group does not exists.

However, if you tried another invalid entry say “ESX40U3”, you’ll get an error all together such as the following even before logging in:

http://www.vmware.com/downloads/download.do?downloadGroup=ESX40U3

So does this mean vSphere 4.0 Update 2 about to be released? Your guess is as good as mine, we will just have to wait and see.

Filed Under: Uncategorized Tagged With:

How to remove stale targets from vMA

by Leave a Comment

If you have used vMA’s vi-fastpass authentication, you will know how easy it is to setup using vifp utility which supports both ESX/ESXi and vCenter targets.

Here’s an example of adding ESXi target:

[[email protected] ~]$ sudo vifp addserver esxi3-1.primp-industries.com
*protected email*’s password:

Here’s an example of the listing of the available fastpass targets:

[[email protected] ~]$ sudo vifp listservers
esxi3-1.primp-industries.com ESXi

During this process, two accounts (vi-userXX & vi-adminXX) are created on the target host with a password that vMA management creates and caches it locally in an obfuscated but not encrypted form. This will allow you to initialize a fastpass target using vifpinit utility and execute commands against the target host without having to manually type in the credentials.

The fastpass targets are stored in 2 configuration files on vMA:

1) The obfuscated cached credentials is stored in /home/vi-admin/.vmware/credstore/vicredentials.xml

If you cat out the contents, it will look something like this:

2) A More detailed configuration for each of the targets along is stored in /etc/vmware/viconfig/viconfig.xml

If you cat out the contents, it will look something like this:

What happens when you rebuild your host, or the system is no longer available because it has been decommissioned or being used for another purpose? vMA will still think it’s managing the host and the fastpass credentials will no longer function as the account is no longer valid the host. If you try to remove the old target, you will see the following error:

[[email protected] ~]$ sudo vifp removeserver esxi3-1.primp-industries.com
*protected email*’s password:

Error: Failed to connect. Please make sure the server is up and is of supported version.

The reason this occurs is that vMA is unable to login to the host and remove the two accounts that were initially created and fails to remove the target. What you will need to do is actually pass in an additional parameter to vifp command “–force” which will forcefully remove the target from vMA management. This command actually does not require the user to enter the correct password to the host even if it is still reachable by vMA. By specifying this flag and providing some input when prompted for the password, vMA will purge the target from it is system.

[[email protected] ~]$ sudo vifp removeserver esxi3-1.primp-industries.com –force
*protected email*’s password:

After a target is removed from vMA, it is also removed from the two above files. You do not manually tweak either of these configuration files or it may lead to issues on your vMA host.

Best practice for decommissioning a host that has been added to vMA’s management is the following:

  1. Disable vilogger if you’ve enabled it for the host
  2. Remove target from vMA management
  3. Verify the host is no longer being managed by vMA
  4. Decomission host

Increase Syslog count in ESXi using Busybox

by 2 Comments

There was an interesting question on the ESXi forum today using syslog and logging to a local VMFS volume and potentially filling up the datastore. By default, the log files are 2MB each before they are rotated out with a total of 9 copies all together. This ensures that the logs will not grow beyond 18MB and potentially fill up your datastore.

Within the unsupported console of ESXi, there is a little tool called “Busybox” which implements a set of familiar command line utilities called applets. Some of these applets are uptime, crond, chroot, md5sum, etc… If you login to the unsupported console and just type “busybox” on the console, you will see the following:

If you look carefully, you will see that one of the applets is “syslogd“, which is the syslog daemon running in ESXi. To access a specific applet, you will just type busybox and then the name of the applet along with -h which forces the applet to provide a set of available options. If we do this for syslogd, you will see the following:

The default configurations should ensure that your logs don’t go crazy and fill up the volume, but the downside is that your log history will not be kept forever. That is why it is a best practice to setup a remote syslog server to send all your logs for further processing and auditing if necessary.

However, if you wanted to change this defaut, you can. As you can see from the options above, you can configure not only the size (max of 2MB) before rotating out, but also the number of logs to keep (max of 99). While you cannot increase the size of the individual logs, you can specify a larger number of logs to keep. If you want to make the changes live while the system is running, you’ll need to perform the following:

1. Find the current pid of syslogd

~ # ps | grep syslog
4313 4313 busybox syslogd

2. Kill the syslog process

~ # kill $(cat /var/run/syslogd.pid)

3. Let’s say you want to keep 50 copies instead of 9

~ # busybox syslogd -b 50

Note:

  • The input to -s is in bytes with range (0-2097151)
  • The input to -b has range (1-99)

It’s important to note that this change will not persist through a reboot. I have not been able to figure out where this is set; it could just be hardcoded in the binary by default. A way around this is to re-define the configuration upon boot up by adding an entry to /etc/rc.local which will kill the current running syslogd and then start up with the new parameters as shown above.

Add the following lines to the end of /etc/rc.local:

kill $(cat /var/run/syslogd.pid)
busybox syslogd -b 50

Now for this change to fully persist, you need to do one more thing, you’ll need to run /sbin/auto-backup.sh which will force a local backup of the ESXi configuration files which includes /etc/rc.local so that it’ll survive through the next reboot. Now you’ll be able to store additional ESXi logs for a longer period of time if you choose to log locally.

The vExpert 2010 Crew

by Leave a Comment

The wait is finally over! The highly anticipated e-mail from John Troyer announcing the second group of individuals for the vExpert 2010 award went out today. I’m very honored to be accepted into the vExpert 2010 program and look forward to sharing it with a great group of individuals! I want to thank John for organizing this program and for all the hard work he has put in to recognizing everyone. Congratulations to the vExpert 2010 crew! Let’s make 2010 a great year!

Here are some links/blog posts to the other vExpert’s for 2010:

Arnim van Lieshout
Tom Howarth & Wil van Antwerpen
Brian Knudtson
Jase McCarty
Maish Saidel-Keesing
Barry Coombs
Joe Kelly
Mike Laverick
Paul Davey
Aaron Delp
Erik Scholten
Justin Emerson
Didier Pironet
Carlo Costanzo
David Marshall
Scott Lowe
Mark Vaughn
Ed Saipetch

Additional Links:
http://www.vmware.com/communities/vexpert/
http://www.yellow-bricks.com/2009/10/01/vexperts-2010/

Here is a vExpert 2010 twitter list put together by @maishsk that you can follow:
http://twitter.com/maishsk/vmware-vexpert-2010

Adventures with HP SAM and Win2008 R2 domains

by Leave a Comment

For the past two years or so we have been testing out VDI in our residence hall student labs at UCSB which consist of about twenty or so thin clients split between two locations. Now what made all of this feasible was:

1) Advancements in remote display protocols
and
2) our ghetto-link-clones script (http://communities.vmware.com/docs/DOC-9020)

We picked HP’s RGS as our remote display protocol because it was the only protocol we found at the time that ensured a “rich desktop user experience”. I still prefer it today over the software implementation of PCoIP but that is a topic best left for another day.

Naturally, the only broker that pooled RGS-enabled machines at the time was HP’s Session Allocation Manager (today Leostream and others can broker it). Despite bad experiences with some HP software in the past, SAM actually worked out very well for us, both on the server side as well as the client side. I still have yet to see a broker package that allows you to customize the client like HP SAM (this feature is definitely stamped as RAD in my book).

SAM was not perfect though and had the occasional hiccup (e.g. machines being listed as unavailable, annoying but only occurred once in a while and was remedied with a resync operation) until the day came: management decided to upgrade the domain from Win2k3R-something to Win2k8R2. SAM completely stopped working; it was an interesting situation to say the least.

After some testing (with profanity and throwing of objects included), we of course determined that the fault was in SAM (3.0, the latest release at the time in early January did not work). HP’s response? No official timeframe to release a fix. This equated to a week of scrambling around looking for competing and “compatible” products (ala Leostream) which just ended up complicating things. The solution? An aggravating night biking back and fourth between the two labs switching the entire VDI environment over to View. Whatever it takes right?

So long story short, if you have an existing HP SAM installation that is running on a pre-Win2k8R2 domain, make sure you update to the latest revision of SAM (3.01) that was just released a couple weeks ago before migrating your domain to Win2k8R2.

For a short description of our VDI environment, please refer to:
http://communities.vmware.com/docs/DOC-9201

Script – hostops-lamw.pl

by 1 Comment

I recently noticed a question on the ESXi forum about trying to add a host to a vCenter server that had the “SSL host certificate verification” enabled while using the vSphere SDK for Perl Utility hostops.pl on vMA. The user encountered the following error when trying to add the host:

Error:
SOAP Fault:
———–
Fault string: Authenticity of the host’s SSL certificate is not verified.
Fault detail: SSLVerifyFault

The SSL host verification is a feature that came with the release of vSphere that provides a security measure to verify the validity of a host before adding it to your VMware infrastructure. This feature is disabled by default, but when it is enabled, a user will need to accept a dialog box to confirm the SHA1 thumbprint of the host in question.

This particular use case was not handled properly by hostops.pl which caused the error message to be thrown. With a small tweak to VMware’s canned script, the new and improved hostops-lamw.pl now supports adding an ESX or ESXi host into vCenter with SSL host verification enabled. You’ll still be expected to verify the SHA1 thumbprint, but now you can pass this as an additional parameter which will tell vCenter that you have verified the host and add to vCenter management.

Scott Lowe originally wrote an article on how to verify the SHA1 thumbprint for both an ESX and ESXi host.

On ESX you can run the following:

openssl x509 -sha1 -in /etc/vmware/ssl/rui.crt -noout
-fingerprint

On ESXi, the only real way to verify is by looking at the DCUI’s “View Support Information”:

However, if you truly trust the ESX or ESXi host that you’re going to add to vCenter, there is an alternative way of retrieving the SHA1 thumbprint using the vCLI’s vifs and the modified hostops-lamw.pl.

By default, you’ll be able to point your web browser to https://[hostname]/host/ssl_cert to see actual SSL certificate on your host, assuming this functionality is not disabled. What you can do is download the ssl_cert to vMA or system with vCLI installed and query for the SHA1 hash and provide that as input to hostops-lamw.pl.

Download: hostops-lamw.pl

Step 1. Download hostops-lamw.pl to either vMA or system running vCLI copy it to the following path:

vMA or Linux host /usr/lib/vmware-cli/apps/host
Windows C:\Program Files\VMware\VMware vSphere CLI\Perl\apps\host

Step 2. Download the ssl_cert to vMA:

[[email protected] ~]$ vifs –server esxi4-1.primp-industries.com –username root –get “/host/ssl_cert” esxi4-1.primp-industries.com-ssl_cert
Enter password:

Downloaded file to esxi4-1.primp-industries.com-ssl_cert successfully.

Step 3. Get the SHA1 thumbprint from the ssl_cert you downloaded:

[[email protected] ~]$ openssl x509 -sha1 -in esxi4-1.primp-industries.com-ssl_cert -noout -fingerprint
SHA1 Fingerprint=79:BB:39:09:F6:E5:91:BD:B0:C3:F3:09:B4:38:50:FB:ED:9C:53:A5

Step 4. Use the modified hostops-lamw.pl and the new –sslthumbprint providing the SHA1 thumbprint (remember to double quote it) along with the other required input to add the host to vCenter:

[[email protected] ~]$ ./hostops-lamw.pl –server reflex.primp-industries.com –username primp –operation addhost –target_host esxi4-1.primp-industries.com –target_username root –target_password ‘password’ –sslthumbprint “79:BB:39:09:F6:E5:91:BD:B0:C3:F3:09:B4:38:50:FB:ED:9C:53:A5” –cluster virtual-cluster
Host ‘esxi4-1.primp-industries.com’ added successfully