In case you missed the awesome news last Friday, George Kobar who works over in the vCloud Air team shared a really cool solution in which he demonstrates how to efficiently setup Nested ESXi running in vCloud Air which includes support for inner-vm guest communication without requiring Promiscuous Mode. Nested ESXi has been possible on vCloud Air for quite some time, in fact when I was first granted access I had to try it out myself and had written about it here. The great thing about vCloud Air is that it runs directly on vSphere which means you will get all the added benefits of the underlying vSphere platform including things like VHV (Virtual Hardware Assisted-Virtualization) to ensure that your Nested ESXi VM and its virutal workloads runs as efficiently and as performant as possible. If you are new to vCloud Air, I would recommend checking out this tutorial here which goes into some of the basic operations.

Given the updated news regarding Nested ESXi on vCloud Air, I am sure many of you are excited to try out this new trick for those requiring inner-vm guest communication. I figured most of you will be interested in trying out vSphere 6.0, especially with some of the new capabilities like SMP-FT and VSAN 6.0 which runs perfectly fine in a Nested ESXi environment for demo and learning purposes as shown here and here. I thought I would put together a quick guide on how to setup both Nested ESXi 6.0 as well as the new VCSA 6.0 (which does have a few minor caveats but can definitely run in vCloud Director and vCloud Air environment).

nested-esxi-6.0-vcloud-air
vcsa--6.0-vcloud-air
Disclaimer: The usual caveat ... Nested ESXi is not officially supported by VMware

ESXi 6.0

There is no version of vCloud Director for the Enterprise that supports vSphere 6.0 which means there is no direct support for the latest virtual hardware release which is 11 or support for ESXi 6.x guestOS type. This is also true for vCloud Air which is currently running on vSphere 5.5 and because of this reason, you will need to upload a VM that has been configured with ESXi 5.x as the guestOS type when looking to install ESXi 6.0. Once vCloud Air supports vSphere 6.0, then you can upload a VM that has been created with the ESXi 6.x guestOS type.

The easiest way to create Nested ESXi VM in a vCloud Director or vCloud Air environment is to simply import a VM that has already been configured with ESXi guestOS type (this does not need to be an already installed image). To help expedite the deployment of Nested ESXi in vCloud Air, I have built several Nested ESXi OVF Templates that that you can use. You will also need to upload an ESXi 6.0 ISO or whichever version of ESXi you plan on running since both ESX(i) 4.x and 5.x is possible.

VCSA 6.0

One of the challenges I came across when testing the new VCSA 6.0 in a vCloud Director based environment which also affect vCloud Air is that they do not support a few capabilities within the OVF specification, namely Deployment Options. Due to this limitation and few others, we can not directly import the VCSA 6.0 OVA into vCloud Director. Luckily, there is a workaround which I had looked into a few months before the GA of vSphere 6.0 and below are the steps to import a VCSA 6.0 OVA into a vCloud Director environment. If you are looking to run VCSA 5.5, then you can directly import the OVA without going through these steps.

Step 1 - Download and extract the contents of the VCSA 6.0 ISO (Build 2656757 was  used)

Step 2 - Convert VCSA 6.0 OVA located in vcsa/vmware-vcsa into an OVF by either using ovftool, tar or a tool like 7zip.

ovftool --sourceType=OVA vmware-vcsa vmware-vcsa.ovf

Next, you will need to make several modifications to the OVF file. I do have to warn you, there are a few tweaks and I highly recommend that you use the OVF templates that I have already created for you. Make sure to also delete the .mf (manifest file) since you are making changes to the OVF else the OVF validation will throw an error because the files have been modified.

To save you some time, pain and troubles, I have pre-created the following 3 OVFs (based on vSphere 6.0 GA release of VCSA 6.0) which contains all the modifications mentioned in Step 3 which you can download and then jump to Step 4:

Step 3 - The first is to locate the "References" tag located at the top of the OVF file and remove the line containing the RPM reference. At the end it should look something like the following:

In addition, depending on the method you took to convert the OVA to an OVF, you may also need to rename the json and disk file names located in this section to match the extracted contents.

The second is to delete the following section from the OVF that starts with MigrationUpgradeRequisitesSection:

The fourth step is to specify the deployment option type that you wish to use. VCSA 6.0 supports the following: embedded, infrastructure (PSC) and management (VC). You will need to locate the following line containing guestinfo.cis.deployment.node.type and set the value property to one of the three options.

The fifth and final step is to specify the deployment size that you wish use for your VCSA, here are nine different supported options:

  • Embedded
    • tiny
    • small
    • medium
    • large
  • vCenter Server Management Node (only)
    • management-tiny
    • management-small
    • management-medium
    • management-large
  • Platform Services Controller Node (only)
    • infrastructure

Since both vCloud Director and vCloud Air does not support the Deployment Option OVF capability, you will need to specify the deployment you wish to use. Locate the DeploymentOptionSection and the first entry where it shows "default=true", you will need to change the id to match one of the entries show above. For example, if you wanted an Embedded VCSA deployment using the tiny size, you would specify "tiny" in the id field.

Once you have selected the type of deployment, you will also need to remove ALL entries referencing the other deployment types else it will always deploy an Embedded deployment.

Note: I would like to give a big shout-out to Doug Baer who works over in the VMware HOL team, he actually discovered the initial issue with the Deployment Options and found the workaround by removing the other disk references. If not, you would end up needing ~2TB of storage as VCD tries to aggregate all nine deployments into one! When I had initially worked out the steps to deploy a VCSA 6.0, I had only used the Embedded deployment option.

Step 4 - Lastly, you will need to change the "capacity" property as seen below from 1303 to 1306 due to a known vCloud Air issue documented in KB2094271

Step 5 - You are now ready to upload your VCSA 6.0 OVF to your vCloud Director or vCloud Air environment.

Note: For vCloud Air, you will need to use the "Manage in vCloud Director" link to upload the OVF as the vCloud Air interface does not support direct OVA/OVF uploads.

Step 6 - When you are are ready to deploy your VCSA, one very important step that you will need to do is to edit a few of the OVF properties in the VM before powering it on. If you power on the VCSA before performing this step, the system will need to be deleted and re-deployed as the OVF properties are only read in on the initial first boot which is required for proper configuration.

  • Make sure to disable guest customization, to do so right click on the VM and select Guest OS Customization and uncheck "Enable guest customization"
  • To edit the OVF properties, right click on the VM and select Properties. Click on Guest Properties and you will ONLY be editing the following three sections

Networking Configuration

vcsa-6.0-networking-configurations
System Configuration

vcsa-6.0-system-configurations
SSO Configuration

vcsa-6.0-sso-configuration
For an Embedded Configuration, you will need to edit the following (below is an example of the data input):

Host Network IP Address: 192.168.110.100
Host Network IP Address Family: ipv4
Host Network DNS Servers: 192.168.110.10
Host Network Default Gateway: 192.168.110.1
Host Network Mode: static
Host Network Identity: vc-01a.corp.local
Host Network Prefix: 24
Tools-based Time Synchronization Enable: check OR NTP Servers
Root Password: VMware1!
SSH Enabled: check/uncheck
Directory Domain Name: vghetto.local
New Identity Domain: check
Directory Password: VMware1!
Site Name: virtuallyGhetto

For a vCenter Server Management Node only , you will need to edit the following (below is an example of the data input):

Host Network IP Address: 192.168.110.100
Host Network IP Address Family: ipv4
Host Network DNS Servers: 192.168.110.10
Host Network Default Gateway: 192.168.110.1
Host Network Mode: static
Host Network Identity: vc-01a.corp.local
Host Network Prefix: 24
Tools-based Time Synchronization Enable: check OR NTP Servers
Platform Services Controller: psc-01a.corp.local
Root Password: VMware1!
SSH Enabled: check/uncheck
Directory Domain Name: vghetto.local
New Identity Domain: uncheck
Directory Password: VMware1!
Site Name: virtuallyGhetto

For a Platform Services Controller Node only, you will need to edit the following (below is an example of the data input):

Host Network IP Address: 192.168.110.110
Host Network IP Address Family: ipv4
Host Network DNS Servers: 192.168.110.10
Host Network Default Gateway: 192.168.110.1
Host Network Mode: static
Host Network Identity: psc-01a.corp.local
Host Network Prefix: 24
Tools-based Time Synchronization Enable: check OR NTP Servers
Root Password: VMware1!
SSH Enabled: check/uncheck
Directory Domain Name: vghetto.local
New Identity Domain: check
Directory Password: VMware1!
Site Name: virtuallyGhetto

If everything was deployed successfully, you should now have a VCSA 6.0 instance running in either your vCloud Director or vCloud Air environment.

11 thoughts on “How to deploy vSphere 6.0 (VCSA & ESXi) on vCloud Director and vCloud Air?

  1. I cannot get this to work on vCloud Air ondemand. I was able to upload the OVF, and the properties were filled in correctly. On startup, the vcsa (6.0.0-2656757) spits out warnings on the console regarding empty parameter values which are surely filled in right. For example, it displays a warning about network family or type value being empty. Seems like OVF values are not being read properly on first boot in my case. Using the UK datacenter btw.

    • Strange, it does sound like the OVF properties didn’t get read in. Did you run into any troubles either manually performing the OVF “surgery” or just using the template to upload. I’ve only verified this on US vCA On-Demand, so I don’t know if there’s any differences, I can see if I can try this on UK env. Which deployment type/size were you trying to deploy? I would love to see us provide these images OOTB within the vCA Catalog so customers can directly use it.

      • i did some manual surgery to the OVF file which get’s generated when extracting the vcsa OVA file using the OVF tool, but i double checked the steps you described in your arictle to be exactly the same. All goes well afterwards, except for the appliance OS not recognizing the filled in OVF values. I was trying the most basic/simple embedded/tiny deployment.

        I agree, deploying the vCA from some kind of VMware provided/filled catalog would be great.

      • i forgot to mention: When i’m importing the OVF into the catalog through the web interface, i can see the ovftool binaries installed with the client integration plugin are used. This client integration plugin is at version 6 for me at the moment, due to some local testing with vSphere 6. Could versions compatibility be an issue when it’s used against vCloud Air ?

    • I am facing the same issue, and from my past experience I believe it is VCD’s inability to use OVF customization parameters.
      I have tried many different ways to fix this and get this to work, but does not work!

      I think I will just wait for vCA guys to make the appliance available in the catalog, or fix the VCD issue.

  2. Hi there, first thanks for all your great articles! I am a bit confused about the versions used here.
    Your text and files seem to mix 3 different versions?

    Build 2562625, 2656757, 2656759.

    For me the current downloadable full VCSA is 2656757 (and the extracted files are named with that build number), and there is only a patch to 2656759.
    So I can’t use the pre-created OVF-files that you offer for downloading I think and have to modify them?

    Additionally in Step 2 it might be more clear to write, that one has to extract the generated ova file with 7zip to get the disks and other files.

    Sources:

    “VCSA 6.0 ISO (Build 2656757 was used)”

    “I have pre-created the following 3 OVFs (based on vSphere 6.0 GA release of VCSA 6.0) ”

    Then in the OVF-Files:
    <ovf:File ovf:href="VMware-vCenter-Server-Appliance-6.0.0.5110-2656759_OVF10-file1.json"

    In Step 3:
    <ovf:File ovf:href="VMware-vCenter-Server-Appliance-6.0.0.5100-2562625_OVF10-file1.json"
    <vmw:Requisite ovf:fileRef="VMware-vCenter-Server-Appliance-6.0.0.5110-2656759-upgrade-requirements.rpm_id"

    • I had updated the article a couple of times, so that’s why you’re seeing 2562625 build numbers but that actually doesn’t matter since these are just filenames and pointers. They can be named blah{1,2,3} and you do not need to use 7zip (that just gives you the friendly names) and as I mentioned you can use ovftool or 7zip to extract, the point is as long as you filenames match up to what you have on the filesystem, that is all that matters. Sorry if that wasn’t clear and I’ve gone and fixed the entries referencing 2562525 so you should only see the last two builds which represent the ISO build number and the internal contents which has a different build number. Hope this makes sense

  3. Thanks a lot for this great post !

    I used IP only when configuring vCenter (192.168.2.105 – no DNS name).
    I performed a DNAT on vCloud AIR edge service gateway (for instance public-IP:4431 -> 192.168.2.105:443).
    I can connected to the getting started page of vCenter (the one displaying: Getting Started – To access vSphere remotely, use the
    vSphere Web Client: Log in to vSphere Web Client – For help, see: vSphere Documentation) using https://public-IP:4431
    But when I click on the link “Log in to vSphere Web Client”, vCenter web UI does not appear after a long timeout. I see a message like cannot connect to 192.168.2.105.

    Do you know how I can resolve this ?

    globally speaking, if you can detail how you access the vCenter web UI from Internet, it would be great !

    thanks in advance

    • If you want to access it over the public internet, you need to deploy using a DNS name, this way you can map it locally on your desktop to point to the external IP in vCloud Air. If you use IP, then it’ll try to connect to your 192 address which won’t be accessible even with a DNAT. The better and more simplistic approach is to setup a “jump host”, could even be Linux as long as you have browser access and have it dual-home to point to your internal network this way you can remote to this “jump host” and reach the 192 address

      • I was thinking about the DNS solution as well. However, something puzzles me:
        vCenter server has 1 IP which is 192.168.2.105.
        let’s say DNS name is vCenter.abc.com

        public IP I get from VCA is aa.bb.cc.dd.

        if I create a DNS entry using:
        vCenter.abc.com – aa.bb.cc.dd

        vCenter web UI will use aa.bb.cc.dd as “binding” address when I will try to access it from internet ?
        I was rather thinking it will try to bind to 192.168.2.105 and then I will face the same issue than before (i.e page not responding and then error message “The server at 192.168.2.105 is taking too long to respond” showing up on the screen).

        thanks for your appreciated help

  4. Hi William ..thanks for a great article .. I wonder if you could possibly provide some guidance in deploying the VCVA onto both a vcenter or esxi host .. I am really having issues with the dpoly scripts in 6 update 1A .. and it’s sooo slow using the scripts.
    Any info would be greatly appreciated

Thanks for the comment!