For those of you who follow me on Twitter, you may have seen a few tweets from me hinting at a vSphere deployment script that I have been working on. This was something I had initially built for my own lab use but I figured it probably could benefit the larger VMware community, especially around testing and evaluational purposes. Today, I am please to announce the release of my vGhetto vSphere Lab Deployment (VVLD) scripts which leverages the new PowerCLI 6.5 release which is partly why I needed to wait until it was available before publishing.

There are literally hundreds if not more ways to build and configure a vSphere lab environment. Over the years, I have noticed that some of these methods can be quite complex simply due to their requirements, or incomplete as they only handle specific portion of the deployment or add additional constraints and complexity because they are composed of several other tools and scripts which can make it hard to manage. One of my primary goals for the project was to be able to stand up a fully functional vSphere environment, not just deploying a vCenter Server Appliance (VCSA) or a couple of Nested ESXi VMs, but rather the entire vSphere stack and have it fully configured and ready for use. I also wanted to develop the scripts using a single scripting language that was not only easy to use, so that others could enhance or extend it further but also with the broadest support into the various vSphere APIs. Lastly, as a stretch goal, I would love to be able to run this script across the different OS platforms.

With these goals in mind, I decided to build these scripts using the latest PowerCLI 6.5 release. Not only is PowerCLI super easy to use, but I was able to immediately benefit from some of the new functionality that was added in the latest PowerCLI release such as the native VSAN cmdlets which I could use a sub-set of the cmdlets against prior releases of vSphere like 6.0 Update 2 for example. Although, not all functionality in PowerCLI has been ported over to PowerCLICore, you can see where VMware is going with it and my hope is that in the very near future, what I have created can one day be executed across all OS platforms whether that is Windows, Linux or Mac OS X and potentially even ARM-based platforms 🙂

Changelog:

  • 11/22/16
    • Automatically handle Nested ESXi on vSAN
  • 01/20/17
    • Resolved "Another task in progress" thanks to Jason M
  • 02/12/17
    • Support for deploying to VC Target
    • Support for enabling SSH on VCSA
    • Added option to auto-create vApp Container for VMs
    • Added pre-check for required files
  • 02/17/17
    • Added missing dvFilter param to eth1 (missing in Nested ESXi OVA)
  • 02/21/17 (All new features added only to the vSphere 6.5 Std deployment)
    • Support for deploying NSX 6.3 & registering with vCenter Server
    • Support for updating Nested ESXi VM to ESXi 6.5a (required for NSX 6.3)
    • Support for VDS + VXLAN VMkernel configuration (required for NSX 6.3)
    • Support for "Private" Portgroup on eth1 for Nested ESXi VM used for VXLAN traffic (required for NSX 6.3)
    • Support for both Virtual & Distributed Portgroup on $VMNetwork
    • Support for adding ESXi hosts into VC using DNS name (disabled by default)
    • Added CPU/MEM/Storage resource requirements in confirmation screen

Requirements:

  • 1 x Physical ESXi host OR vCenter Server running at least ESXi 6.0 Update 2 or greater
  • PowerCLI 6.5 R1 installed on a Window system
  • Nested ESXi 6.0 or 6.5 Virtual Appliance OVA
  • vCenter Server Appliance (VCSA) 6.0 or 6.5 extracted ISO
  • NSX 6.3 OVA (optional)
    • ESXi 6.5a offline patch bundle

Supported Deployments:

The scripts support deploying both a vSphere 6.0 Update 2 as well as vSphere 6.5 environment and there are two types of deployments for each:

  • Standard - All VMs are deployed directly to the physical ESXi host
  • Self Managed - Only the Nested ESXi VMs are deployed to physical ESXi host. The VCSA is then bootstrapped onto the first Nested ESXi VM

Below is a quick diagram to help illustrate the two deployment scenarios. The pESXi in gray is what you already have deployed which must be running at least ESXi 6.0 Update 2. The rest of the boxes is what the scripts will deploy. In the "Standard" deployment, three Nested ESXi VMs will be deployed to the pESXi host and configured with vSAN. The VCSA will also be deployed directly to the pESXi host and the vCenter Server will be configured to add the three Nested ESXi VMs into its inventory. This is a pretty straight forward and basic deployment, it should not surprise anyone. The "Self Managed" deployment is simliar, however the biggest difference is that rather than the VCSA being deployed directly to the pESXi host like the "Standard" deployment, it will actually be running within the Nested ESXi VM. The way that this deployment scenario works is that we will still deploy three Nested ESXi VM onto the pESXi host, however, the first Nested ESXi VM will be selected as a "Bootstrap" node which we will then construct a single-node vSAN to then deploy the VCSA. Once the vCenter Server is setup, we will then add the remainder Nested ESXi VMs into its inventory.

vsphere-6-5-vghetto-lab-deployment-0
For most users, I expect the "Standard" deployment to be more commonly used but for other advanced workflows such as evaluating the new vCenter Server High Availability feature in vSphere 6.5, you may want to use the "Self Managed" deployment option. Obviously, if you select the latter deployment, the provisioning will take longer as you are now "double nested" and depending on your underlying physical resources, this can take quite a bit more time to deploy as well as consume more physical resources as your Nested ESXi VMs must now be larger to accommodate the VCSA. In both scenarios, there is no reliance on additional shared storage, they will both create a three node vSAN Cluster which of course you can expand by simply editing the script.

Deployment Time:

Here is a table breaking down the deployment time for each scenario and vSphere version:

Deployment Type Duration
vSphere 6.5 Standard 36 min
vSphere 6.0 Standard 26 min
vSphere 6.5 Self Managed 47 min
vSphere 6.0 Self Managed 34 min

Obviously, your miles will vary based on your hardware configuration and the size of your deployment.

Scripts:

There are four different scripts which covers the scenarios we discussed above:

Instructions:

Please refer to the Github project here for detailed instructions.

Verification:

Once you have saved all your changes, you can then run the script. You will be provided with a summary of what will be deployed and you can verify that everything is correct before attempting the deployment. Below is a screenshot on what this would look like:

Sample Execution:

Here is an example of running a vSphere 6.5 "Standard" deployment:


Here is an example of running a vSphere 6.5 "Self Managed" deployment:

vsphere-6-5-vghetto-lab-deployment-2
If everything is succesful, you can now login to your new vCenter Server and you should either see the following for a "Standard" deployment:

vsphere-6-5-vghetto-lab-deployment-5
or the following for "Self Managed" deployment:

vsphere-6-5-vghetto-lab-deployment-6
I hope you find these scripts as useful as I do and feel free to enhance these scripts to perform additional functionality or extend them to cover other VMware product deployments such as NSX or vRealize products for example. Enjoy!

51 thoughts on “vGhetto Automated vSphere Lab Deployment for vSphere 6.0u2 & vSphere 6.5

  1. Hi William.

    This is great! – do you have any plans to release scripts to load a lab like this into workstation instead of ESXi?

    Thanks

    Graham

  2. William, looking forward to trying this out on my new Skull Canyon NUC (with two NVMe SSD drives). Will this work pointed at that? I know you had some previous posts about bootstrapping into such an environment but I thought perhaps you needed some non-flash storage also?

    • And, it doesn’t work. I believe I need storage or network (a dv switch?) configured on the physical host, which is a 6.0.0U2 booted from USB on my NUC. I think we need clarification on these two parameters:

      $VMNetwork = “dv-access333-dev”
      $VMDatastore = “himalaya-local-SATA-dc3500-2”

      For either config, or at least (I’m trying) the self-managed:
      Do these need to already exist as whatever we set them to be on the physical host? Which means we need physical storage at the lowest physical layer? This isn’t clear from your instructions, which (I gathered and could be completely wrong) said the script installs it all. Too much to wish for of course. 🙂

      I’ll try again with a USB datastore mounted and a dv switch created. But some confirmation would be wonderful, I’d love to get this working.

      Thanks much!

        • [11-22-2016_03-20-24] Deploying the VCSA …
          Task failed. Status: ERROR Progress: 5% Starting VMware Identity Management
          Service… Error: Problem Id: None Component key: idm Detail:
          Encountered an internal error. Traceback (most recent call last): File
          “/usr/lib/vmidentity/firstboot/vmidentity-firstboot.py”, line 2018, in main
          vmidentityFB.boot() File
          “/usr/lib/vmidentity/firstboot/vmidentity-firstboot.py”, line 349, in boot
          self.configureSTS(self.__stsRetryCount, self.__stsRetryInterval) File
          “/usr/lib/vmidentity/firstboot/vmidentity-firstboot.py”, line 1479, in
          configureSTS self.startSTSService() File
          “/usr/lib/vmidentity/firstboot/vmidentity-firstboot.py”, line 1141, in
          startSTSService returnCode = self.startService(self.__sts_service_name,
          self.__stsRetryCount * self.__stsRetryInterval) File
          “/usr/lib/vmidentity/firstboot/vmidentity-firstboot.py”, line 88, in
          startService return service_start(svc_name, wait_time) File
          “/usr/lib/vmware/site-packages/cis/utils.py”, line 784, in service_start
          raise ServiceStartException(svc_name) ServiceStartException: { “resolution”:

          null, “detail”: [ { “args”: [
          “vmware-stsd” ], “id”:
          “install.ciscommon.service.failstart”, “localized”: “An error
          occurred while starting service ‘vmware-stsd'”, “translatable”: “An

          error occurred while starting service ‘%(0)s'” } ],
          “componentKey”: null, “problemId”: null } Resolution: This is an
          unrecoverable error, please retry install. If you run into this error again,
          please collect a support bundle and open a support request.
          Collecting the support bundle from the deployed appliance…

          anyone can help with this error?

  3. Hi William,

    This is awesome…I’ve just binned my script for yours, it’s great!

    I had trouble getting the OVF configuration to stick to my nested ESXi hosts (I’m deploying to vCenter (VCSA 6.0u2)) using the “$vm.ExtensionData.ReconfigVM_Task($spec)” method.

    Instead I used Get-OvfConfiguration and set the properties before using Import-VApp with the -OvfConfiguration parameter:

    $ovfConfig = Get-OvfConfiguration -Ovf $NestedESXiApplianceOVA
    $ovfConfig.Common.guestinfo.hostname.Value = $VMName
    $ovfConfig.Common.guestinfo.ipaddress.Value = $VMIPAddress
    $ovfConfig.Common.guestinfo.netmask.Value = $VMNetmask
    $ovfConfig.Common.guestinfo.gateway.Value = $VMGateway
    $ovfConfig.Common.guestinfo.dns.Value = $VMDNS
    $ovfConfig.Common.guestinfo.domain.Value = $VMDomain
    $ovfConfig.Common.guestinfo.ntp.Value = $VMNTP
    $ovfConfig.Common.guestinfo.syslog.Value = $VMSyslog
    $ovfConfig.Common.guestinfo.password.Value = $VMPassword
    $ovfConfig.Common.guestinfo.ssh.Value = $VMSSH
    $ovfConfig.Common.guestinfo.createvmfs.Value = $VMVMFS
    $ovfConfig.NetworkMapping.VM_Network.Value = $network

    Write-Log “Deploying Nested ESXi VM $VMName …”
    $vm = Import-VApp -Server $vCenter -VMHost $pEsxi -Source $NestedESXiApplianceOVA -OvfConfiguration $ovfConfig -Name $VMName -Location $cluster -Datastore $datastore -DiskStorageFormat thin

    Hope that helps!

  4. Hi William, thanks a lot for this awesome work!

    Just for those folks like me who try to use the script with a system proxy set (even with local bypass), you’ll got this kind of error until you disable it :

    2016-11-24 16:15:37,447 – vCSACliInstallLogger – ERROR – An error occurred when connecting to “10.95.1.12”: Failed to login to host 10.95.1.12, as user root: [Errno socket error] [Errno 1] _ssl.c:510: error:140770FC:SSL routines:SSL23_GET_SERVER_HELLO:unknown protocol

    This looks like a limitation of the invoke-Expression cmdlet

  5. For those getting this error when running the script, you need to have the PortGroup set to Ephemeral connecting to the ESXi host.

    [11-27-2016_04:46:04] Deploying Nested ESXi VM vesxi65-1 …
    Set-NetworkAdapter : 11/27/2016 4:47:12 PM Set-NetworkAdapter The specified DV portgroup ‘LAB-701’ can not be used for vnic connection: Invalid portgroup type.
    At C:\Users\anthony.spiteri\Dropbox\Files\scripts\vsan_lab_deploy_2..ps1:196 char:51
    + … er $pEsxi | Set-NetworkAdapter -Portgroup $network -confirm:$false | …
    + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo : InvalidArgument: (:) [Set-NetworkAdapter], InvalidArgument
    + FullyQualifiedErrorId : Client20_VirtualNetworkServiceImpl_TryGetAvailableDVPortGroup_NotValidPortgroupType,VMware.VimAutomation.ViCore.Cmdlets.Commands.VirtualDevice.SetNetworkAdapter
    Set-NetworkAdapter : 11/27/2016 4:47:12 PM Set-NetworkAdapter The specified DV portgroup ‘LAB-701’ can not be used for vnic connection: Invalid portgroup type.
    At C:\Users\anthony.spiteri\Dropbox\Files\scripts\vsan_lab_deploy_2..ps1:196 char:51
    + … er $pEsxi | Set-NetworkAdapter -Portgroup $network -confirm:$false | …
    + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo : InvalidArgument: (:) [Set-NetworkAdapter], InvalidArgument
    + FullyQualifiedErrorId : Client20_VirtualNetworkServiceImpl_TryGetAvailableDVPortGroup_NotValidPortgroupType,VMware.VimAutomation.ViCore.Cmdlets.Commands.VirtualDevice.SetNetworkAdapter

  6. Great work. The deployment works great. My only problem is that the ESXi hosts deployet is not accessible through the vSphere Client or Web GUI. So, the add ESXi to cluster failed. But this is not new to me. The same thing happends to my fresh installed nested ESXi host (not with appliance). When it bootes up for the first time it is not possible to login through the vSphere Client or Web GUI with the given DHCP address. My only “fix” was to power o ff the VM (not shutdown) and power it back on. Then the password was set to blank and I could login to the address from the DHCP server. No difference if no DHCP is active and I have to configure static IP address for the first time, still unavailable. Any tips where I should start look to fix this? The pESXi have the Mac Learn dvFilter configured. This gives me headache..

    Again, great job.

    Cheers

  7. Having all sorts of issues… Physical host running 6.0u2. I get an error message when configuring the nested hosts:

    Set-NetworkAdapter : 12/5/2016 7:29:54 PM Set-NetworkAdapter The operation for the entity “labesx602” failed with the
    following message: “Another task is already in progress.”
    At C:\users\Marc\Desktop\Homelab-Build\vsphere-6.0-vghetto-standard-lab-deployment.ps1:196 char:51
    + … er $pEsxi | Set-NetworkAdapter -Portgroup $network -confirm:$false | …

    Then it continues and deploys the VCSA. at 25% done, it tries to continue the script and to connect to vcenter, and everything else fails (it’s not finished deploying yet).

    Any idea why these two issues arise?

    • Fourth reinstallation without any changes and I still get the Another task is already in progress, but this time the VCSA did install?! I don’t quite understand the logic behind it all… Need some ‘splaining 😉

        • Standalone. Tried on ESXi 6.5 and 6.0. Same result. Now the VCSA installs every time so no idea why it failed the first few times. Still have the other error however (“Another task is already in progress”).

        • First off, awesome script, thanks alot for sharing William!

          For those who are having the “Another task is already in progress.” problem. This occurs when more then one network adapter object is piped to Set-NetworkAdapter on line 204 of vsphere-6.5-vghetto-self-manage-lab-deployment.ps1.
          I solved it by replacing this:
          $vm | Get-NetworkAdapter -Server $pEsxi | Set-NetworkAdapter -Portgroup $network -confirm:$false | Out-File -Append -LiteralPath $verboseLogFile

          with this:
          $adapters = $vm | Get-NetworkAdapter -Server $pEsxi
          foreach ($adapter in $adapters) {
          $adapter | Set-NetworkAdapter -Portgroup $network -confirm:$false | Out-File -Append -LiteralPath $verboseLogFile
          }

          • Hi Peter,

            I actually had another individual who submitted the exact same fix but haven’t had time to update. I just pushed the changes this morning, so let me know if you have any issues.

  8. While this is a wonderful script, it would be terrific if it were updated to target a vCenter rather than a single host.

  9. thanks William.Much Appreciated.
    One quick one while i am setting this up on NUC with esxi 6.5, next step would be to install windows and powershell on a VM and then run the above scripts.

  10. I used this and it worked perfectly first time.

    One feature request – given this is for lab use it’d be good to add a variable to the script to set the webclient timeout to 0 (ie no timeout). If I find time I’ll have a look (I don’t imagine it’s difficult) but I imagine you can do it in your sleep.

  11. In your script comments in the script under the vCenter section, you say to change the hostname to the IP if you don’t have valid DNS. Have you ever had the script work without valid DNS? I tried several different configurations but it always failed during firstboot. I’m not sure that vCenter can be installed without valid DNS.

    • Hi Brandon,

      Yes. If you don’t have DNS at all, then you wouldn’t want to make up the value of DNS property as a lookup is performed (recently ran into this as well). Go ahead and set the DNS to the same IP as your VC. This should get you past the firstboot issue which I’m guessing has something to do with STS service.

  12. Thanks William. This is Awesome!

    This will save me some serious time whenever starting my environment from scratch (which I do quite frequently!)

    Great Job!

  13. I wonder how can I add the hosts to the vCenter by hostname instead of IPs. Anyone knows what I would need to edit to get this accomplished?

    Thanks

  14. Hi William,

    I have the following problem when running either the “Standard” or “Self-Managed” 6.0 scripts:-

    I want the VCSA to have be on a class B subnet and despite confirming that $VMNetmask = “255.255.0.0” I always end up with a VCSA having a “255.255.255.0” subnet mask. The IP address is correct but the mask is always wrong. This causes the script to fail to complete the post-deployment configurations.

    Once I log on to the VCSA I can manually change the mask to establish communication. It appears that the script is ignoring the $VMNetmask variable and always defaulting to 255.255.255.0

    Any ideas as to what I’m doing wrong?

    Many thanks in advance.

  15. Using the latest script that adds vCenter deployment, the step to move VMs into a vApp fails if using an ESXI deployment. The moveVMsIntovApp variable is initialized to 1 at the top of the script and never changed. It should be set to 0 if DeploymentTarget -eq “ESXI” somewhere.

    Great to be able to deploy a whole environment in 29 minutes!

    • Right, the vApp is only applicable for when deploying to vCenter Server. I’ll add some logic to ensure that it only runs when deploymentTarget is VCENTER. Thanks for feedback

  16. [02-15-2017_10:08:09] Disconnecting from new VCSA …
    [02-15-2017_10:08:09] vSphere 6.5 Lab Deployment Complete!
    [02-15-2017_10:08:09] StartTime: 02/15/2017 09:45:43
    [02-15-2017_10:08:09] EndTime: 02/15/2017 10:08:09
    [02-15-2017_10:08:09] Duration: 22.44 minutes

    And I can mess it up, and redeploy if needed. This is beyond awesome!

  17. Hi William,

    I want to install esxi servers in another vlan, how can we specify vlan id in script ?

    Do you have any plans to extend the scripts ? like deploying multiple vsan clusters using same scripts .

    Thanks,
    Surya

  18. The Nested ESXi VMs only support native VLANs, this is the most common way folks are deploying this in the lab. Not saying other method isn’t valid, but I chose to optimize on the 80%. If you need to provide a VLAN ID, then you can deploy the Nested ESXi VM to a known native VLAN, then using the vSphere APIs (includes PowerCLI), you can create a new VMkernel which accepts VLAN ID (basically, handle this as part of a custom post-deploy)

    In terms of multiple vSAN Clusters, there’s no plan atm

  19. Hi William, could you point me in the direction within the .ovf where I could use “vSphere 6.0 Standard” but with multiple nics?

  20. Finally got some time to test your script – awesome work, William. You saved me a heaps of time.

    Had a minor issue with it. If I configure script to update hosts to 6.5, but don’t choose to deploy NSX manager the update process fails because vESXi servers haven’t booted yet. The script tries to update the first server right after it completed deployment of the last. In my case I had only 3 vESXI and the first one was ready yet for an update.

    I also deployed the nested lab using NSX virtual switches in my physical setup. So, network is nested as well 🙂 to make it more interesting it is possible to enable VLAN tagging on virtual switches.

    Are you planning to continue developing this script? It would be great to have an option to deploy 2 sites with vCenters in ELM.

    I am also thinking to re-use some parts of this script https://powernsx.github.io/2016/11/14/NSX-Full-Stack-Deployment.html to automate configuration of NSX – preparing hosts, creating logical switches, configuring DRL and deploying 3-tier web app.

    Thanks again. that’s gonna be my favourite tool for the next year 🙂

    • I threw in a “start-sleep -seconds 120” after the OVA deploy loop to fix the issue with the upgrade failing.

  21. Just tired this script. Errored out me. Any ideas?

    Import-VApp : 04/04/2017 19:05:54 Import-VApp A general system error
    occurred:
    At C:\Users\x\Desktop\vLamAuto\vsphere-6.5-vghetto-standard-lab-deployment.ps1:
    446 char:19
    + $vm = Import-VApp -Server $viConnection -Source
    $NestedESXiAppliance …
    + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    ~~~
    + CategoryInfo : NotSpecified: (:) [Import-VApp], SystemError
    + FullyQualifiedErrorId : Client20_VappServiceImpl_ExportVApp_Error,VMware
    .VimAutomation.ViCore.Cmdlets.Commands.ImportVApp

    • …er…fixed the issue…had the ESXI in maintenance mode…I was playing with another script and got over excited.

Thanks for the comment!