Monday, May 21, 2012

How to Deploy an OVF/OVA in the ESXi Shell

I recently answered, what I thought was pretty straight forward question on the VMTN forums about whether it was possible to to deploy an OVA directly onto an ESXi host without leveraging remote tools such as the vSphere Client or the ovftool. The response that I provided was, no it was not possible to deploy an OVF/OVA within the ESXi Shell and recommended the user to take a look at the vSphere Client or the ovftool.

For whatever reason, my brain decided to ponder about this specific question over the weekend (even though I had answered dozen or so questions earlier in the week) and came up an idea that could make this work. As many of you know, I am a big fan of the ovftool and I have written several articles about the tool such as here and here. I wanted to see if I could get the ovftool to run in the ESXi Shell as all the necessary libraries and required packages are all self contained within /usr/lib/vmware-ovftool directory. If this works, it would allow a user to deploy a VM from an OVF or OVA format within the ESXi Shell and would not require a remote system which is great for kickstart deployments or ISO installations. As you probably have guessed, I was able to get this to work :)

Disclaimer: This is not officially supported by VMware, please test this in a lab before deploying on production systems. 

Before you begin, you will need to get the ovftool installed on an existing Linux system, you can use vMA for your convenience. Next, you will need to use the scp command to copy the entire /usr/lib/vmware-ovftool directory onto an ESXi host. Ensure you place the contents on either a shared or local datastore as the size of the ovftool content is quite large (~119 MB).

In this example, I am scp'ing the ovftool directory to a local VMFS datastore (/vmfs/volumes/datastore1)
scp -r /usr/lib/vmware-ovftool/ root@vesxi50-7:/vmfs/volumes/datastore1
Once you have successfully copied the ovftool directory over to your ESXi host, you will need to make a small tweak to the file located in /vmfs/volumes/datastore1/vmware-ovftool/ovftool (shell script that calls the ovftool binary). You will need to modify the the first line using the vi editor from #!/bin/bash to #!/bin/sh as ESXi does not recognize the bash shell. You are now ready to copy an OVF or OVA to your ESXi host which should also reside within a shared or local datastore.

In this example, I uploaded a SLES OVF to the same datastore which contains the ovftool as seen below from the datastore browser:
Let's go ahead and perform a simple probe operation on the OVF we just uploaded to ensure that ovftool is working as expected. To do so, you just need to specify the full path to the ovftool as well as the full path to either your OVF or OVA file.
Note: The ovftool does take slightly longer to run in the ESXi Shell compared to a regular system with the ovftool installed.

Now that we have confirmed the ovftool is working, let's go ahead and deploy from the OVF image. Even though we are running the ovftool locally in the ESXi Shell, you will still need to specify the credentials to your ESXi host during deployment as ovftool was not designed for this use case.
Note: You must specify both the username and password in the ovftool command line, as the password prompt does not function properly in the ESXi Shell and you will see a looping of  "*" characters on the screen.

If you are familiar with the ovftool, you know you can specify an OVF/OVA from both a local resource as well as remote location such as a web server. Here is another example of deploying an OVF from a remote web server:
We can see that is pretty easy to deploy an OVF or OVA from within the ESXi Shell, but what about unattended installations such as ESXi kickstart? Yep, we can do that too! The easiest way is to compress the vmware-ovftool directory using tar command and then download it remotely using the wget command during the %firstboot stanza. I would also recommend placing your OVF/OVA images on a remote web server as well for centralize management and deployment.

Here is the sample code snippet that can be used in your kickstart:
# download ovftool tar to local storage
wget http://air.primp-industries.com/vmware-ovftool.tar.gz -O /vmfs/volumes/datastore1/vmware-ovftool.tar.gz

# extract ovftool content to /vmfs/volumes/datastore1
tar -xzvf /vmfs/volumes/datastore1/vmware-ovftool.tar.gz -C /vmfs/volumes/datastore1/

# deploy OVF from remote HTTP source
/vmfs/volumes/datastore1/vmware-ovftool/ovftool -dm=thin -ds=datastore1 "--net:access333=VM Network" "http://air.primp-industries.com/SLES-VM/SLES-VM.ovf" "vi://root:vmware123@172.30.0.192"

# power on VM
vim-cmd vmsvc/power.on $(vim-cmd vmsvc/getallvms | grep "SLES-VM" | awk '{print $1}')
As you can see, virtually anything is possible ... even if you thought it was not earlier :)

16 comments:

  1. Even though I have not tested this myself yet (will do so soon), is it possible to create an ovf/ova file from within the ESXi shell ?

    ReplyDelete
    Replies
    1. @Anonymous,

      Yes, if you know how the ovftool works, it can be used to import/export OVF.

      You would still need to connect to the ESXi host and discover your VMs, but you can easily export an "offline".

      Here is an example:

      /vmfs/volumes/datastore1/vmware-ovftool/ovftool "vi://root:vmware123@172.30.0.192/VM1" /vmfs/volumes/datastore1/VM1.ovf

      Delete
    2. Nice! Should we expect the next version of backup to include an option for ovf/ova backup and restore :)

      Delete
    3. I don't have any plans on exposing this with ghettoVCB.

      As mentioned in the article, ovftool runs a little slower on ESXi Shell due to limited amount of resources and the biggest problem is that to export to an OVF, the VM must be offline. Though this may be okay for some users, I suspect the majority of users would like to perform online backups.

      Delete
    4. Just to keep the idea going ...

      The backups that are produced by ghettoVCB are offline VM's. Would it not be feasible to do a post process where we create an ova/ovf from the backup itself, and use that for storage, backup etc. The advantage would be smaller footprint, and better portability (no issues with moving the file around).

      Just thinking out loud ..

      Delete
  2. In kickstart file, can't we just do an NFS mount and run ovftool from the mounted file system ? That way we don't need to run scp, tar and changing of 'bash' to 'sh' manually (which could be done on the NFS server side itself).

    ReplyDelete
    Replies
    1. Yes that will probably work as well. The article is just to give you an idea on how you can accomplish the task, there's more than one way to do anything :) I would recommend you give it a try and post your findings

      Delete
  3. Which version of ESX are you using? I am trying with ESXi 4.1. I am good up until it comes to deployment. Then i get an error, "Error: Unexpected option vi://root:"...
    Any ideas?

    ReplyDelete
    Replies
    1. This was tested on ESXi 5.0

      Delete
    2. Hi William,

      Do you know what is the Linux OS which is most similar to that of esxi 4.1? I have been trying to auto deploy ovf file on 64 bit ESXi 4.x. Since it is ESXi 4.x, I cannot use ESXi 5.0 and vmware auto deploy. The method described in your post looks promising.

      I first tried to install 64 bit ovftool on the latest vma4, and then scp the vmware-ovftool directory to ESXi 4.1 shell. I got a "segment fault" when I was trying to probe the ovf file.

      And then I tried copying the vmware-ovftool from 32 bit ubuntu 10.04. Probing the ova file is successful (/vmfs/volumes/datastore1/vmware-ovftool/ovftool "http://10.10.1.1/kickstart/myos.ova") . But when I was trying to deploy it , I got the following PANIC error.


      /vmfs/volumes/datastore1/vmware-ovftool/ovftool "http://10.10.1.1/kickstart/myos.ova" "vi://10.10.1.100"
      Opening OVA source: http://10.10.1.1/kickstart/myos.ova
      The manifest validates
      PANIC bora/lib/unicode/unicodeCommon.c:330

      VMware Workstation Error:
      VMware Workstation unrecoverable error: (vthread-3)
      PANIC bora/lib/unicode/unicodeCommon.c:330
      You can request support.
      To collect data to submit to VMware support, select Help > About and click "Collect Support Data". You can also run the "vm-support" script in the Workstation folder directly.
      We will respond on the basis of your support entitlement.

      I have verified copying vmware-ovf directory from a host to another host with the same OS version works. I think if I install the ovftool to a OS which is very similar to that of ESXi 4.1, it will a have good chance to succeed.

      Any suggestions that I can try to make ovftool work for ESXi 4.1, besides finding the OS similar to ESXi 4.1?

      Really appreciate your help!

      Delete
    3. ESXi is it's own OS, specifically the VMkernel and it's not related to any Linux OS. The ESXi shell runs Busybox, you could try to get it working on Busybox and copying the files over, but YMMV. This was only tested on ESXi 5 and it's not officially supported, so it may or may not work even on ESXi 5.0

      You might find these two articles useful:
      http://www.virtuallyghetto.com/2011/02/how-to-compile-busybox-for-esxi-kind-of.html
      http://www.virtuallyghetto.com/2011/02/how-to-compile-busybox-for-esxi-kind-of_15.html

      Good Luck

      Delete
    4. Hi William,
      Thanks for your explanation. I tried the Busybox solution without success. But I did figure out one workable solution. Here is what I did:
      1. install 64 bit CentOS 5.5 x86_64.
      2. on CentOS, get 32 bit ovftool installation script VMware-ovftool-2.0.1-260188-lin.i386.sh from vmware. (newer version or 64 bit version don't work on esxi 4.1 shell in my testing)
      3. install ovftool on CentOS: ./VMware-ovftool-2.0.1-260188-lin.i386.sh
      4. modify first line of /opt/vmware/ovftool/ovftool from "#!/bin/bash" to "#!/bin/sh".
      5. cd /opt/vmware
      6. tar -czvf ovftool.tar.gz ovftool/
      7. copy the tar file to esxi4.1 and untar it.
      8. using ovftool to deploy ova file on esxi4.1: /vmfs/volumes/datastore1/ovftool/ovftool --acceptAllEulas --name="newname" "http://10.10.1.1/kickstart/my.ova" "vi://root@127.0.0.1".

      Hope it might be helpful for someone.

      Thanks.

      Delete
  4. Hi William

    Priceless....I was trying to do this a couple of months back for an audo-deployment project from a DVD disk sent out to a customer...I had to gz the entire VM and extract it into the newly deployed ESXi system (MD5 checks all the way!). Now I can chill...send out new OVF's on DVD and do a nice clean import.

    My only thorn in my side is ESXi5's 2GB USB limit which prevents me sending out loads of OVF's updates on a 32GB USB stick?....I can dream I suppose?


    Oustanding!

    Luca (London)

    ReplyDelete
  5. It looks like VMware changed ovftool binary and ver 3.0.1 is no longer static. I'm having troubles getting it work in ESXi 5.1 shell. Does anyone have a link to an older ovftool bundle ?

    The error is :

    SSLLoadSharedLibraries: Failed to load OpenSSL libraries. libdir is /usr/lib/vmware

    VMware Workstation Error:
    VMware Workstation unrecoverable error: (vthread-3)
    SSLLoadSharedLibraries: Failed to load OpenSSL libraries. libdir is /usr/lib/vmware

    ReplyDelete
    Replies
    1. Same error on ESXi, can an older version work?

      Delete
    2. When I run the ovftool (v. 3.0.1) from the local esxi 5.1 shell I receive the following error:

      ovftool :not found

      Has anyone been successful at running the ovftool from the esxi 5.1 shell?

      Delete