There is not a whole lot of changes for kickstart configurations between ESXi 5.1 and ESXi 5.0, majority of the tips and tricks noted in the ESXi 5.0 kickstart guide are still relevant for ESXi 5.1. Below are a few new tips and tricks (some old) as well as a complete working ESXi 5.1 kickstart example that can be used as a reference.

Tip #1

There are 82 new ESXCLI commands, number of which are new as well as enhancements to existing commands and operations. The kickstart sample below converts many of the legacy esxcfg-* and vim-cmd/vsish commands over to ESXCLI such as, here are just a few:

  • esxcli network ip route [ipv4|ipv6] (VMkernel routes)
  • esxcli system snmp (SNMP)
  • esxcli system maintenanceMode (maintenance mode)
  • esxcli network ip interface tag (tag VMkernel traffic types)

Please refer to the vCLI/ESXCLI release notes for all new ESXCLI commands.

Tip #2

In previous releases of ESXi, you could add custom commands in /etc/rc.local which will automatically execute after all startup scripts have finished. With the latest release of ESXi 5.1, this functionality has been moved to /etc/rc.local.d/local.sh. If you try to edit the old file, you will find that it does not allow you to write any changes. This will be important as you migrate to ESXi 5.1 kickstart if you make use of this file for any custom startup commands.

Tip #3

To run nested ESXi and other hypervisors in ESXi 5.1, you need to to specify new vhv.enable parameter, please take a look at this article for more details.

Tip #4

There is a new ESXi Advanced Setting in ESXi 5.1 that allows you to control when an interactive ESXi Shell session will automatically logout based on configured idle time (in seconds). You can find more details in this blog article by Kyle Gleed.

esxcli system settings advanced set -o /UserVars/ESXiShellInteractiveTimeOut -i 3600

Tip #5

By default, an ESXi host will automatically grant root permission to the "ESX Admins" group for use when a host is joined to an Active Directory domain. You can alter the default group name if you already have an AD group defined by using the following command:

vim-cmd hostsvc/advopt/update Config.HostAgent.plugins.hostsvc.esxAdminsGroup string "Ghetto ESXi Admins"

Tip #6

A really neat feature in ESXi 5.1 is the ability to control which local users have full admin privileges to the DCUI, this is really useful for troubleshooting and you want to provide DCUI console access but not administrative permissions on the ESXi host itself. You can specify a list of local users by using the following command:

vim-cmd hostsvc/advopt/update DCUI.Access string root,william,tuan

Tip #7

If you wish to prevent VMs from sending out BPDU (Bridge Protocol Data Unit) packets, there is a new global configuration on an ESXi 5.1 host which you can set. By default, this setting is disabled and you will need to configure this on every ESXi host if you wish to block VM guests from sending out BPDU packets.

esxcli system settings advanced set -o /Net/BlockGuestBPDU -i 1

Tip #8

Here's an article about enabling/disabling IPv6 using ESXCLI

Tip #9

Here's an article about creating custom VIB for ESXi 5.1

Here is a complete working example of an ESXi 5.1 kickstart that can help you convert your existing ESX(i) 4.x/5.x to ESXi 5.1:

35 thoughts on “Automating ESXi 5.1 Kickstart Tips & Tricks

  1. The line GATEWAY=$(localcli network ip interface ipv4 get | grep “${VMK_INT}” didn’t work for me, because it returned the ip of the host and not the gw. I’ve replaced it with

    GATEWAY=$(localcli network ip route ipv4 list | grep default | awk ‘{print $3}’)

  2. I think you don’t have to specify “–level 345″ to chkconfig.
    I can see in my log:
    /sbin/chkconfig –level 345 ntpd on
    /sbin/chkconfig error: Warning: –level argument is not implemented

  3. thanks a lot for William’s great post.

    Is there a default network security settings exists? I mean the settings of Promiscuous/MAC Address Changes/Forged Transmits. So when I create a vswitch, the default settings would be applied to.

    if there is ,how to set it by cli?

    thanks

    • Sean,

      There is a default (create a vSwitch and you’ll see what they are) but this is not something you can configure. So you’ll still need to set the appropriate security settings based on your requirements.

    • Hi William

      I see. thank you. I asked the question because both in vSwitch and PortGroup properties, there are “Security” tab and displays “Policy Exception”. Generally, the word exception indicates inheritance blocked, so I guess there should be a default settings:) Unfortunately, it can not be configured.

  4. Hi William

    I would like to deploy the latest ESXi 5.1 (I mean it include the newest patches) via PXE.

    I plan to include some commands in kickstart.cfg, for example,
    esxcli software vib install -d /vmfs/volumes/$(hostname -s)-local-storage-1/patches/ESXi510-201212001.zip

    I am not quite familiar with applying patches to ESXi 5.x, maybe i have 2 ways to make the goals.
    1, Integrate the newest patches to installation resource.
    2, Controlled by kickstart.cfg, includes commands to apply patch after installation in the %firstboot stage.

    kindly please provide me some suggestion. thanks

    • Sean,

      ESXi patches are cumulative and replaces the entire ESXi image (unlike classic ESX, where you had individual patches). This makes patching extremely easy for ESXi. So if you’re going to use kickstart, I would recommend getting the latest patch and then generate a new ESXi ISO image which you can then use for your kickstart. Take a look at this article for more details on creating a new ISO Image with a patch http://blogs.vmware.com/vsphere/2012/08/creating-an-iso-image-from-a-vmware-patch-file.html

    • Hi William

      many thanks for you introduced the way to build ISO with lasted patches. it’s very useful. Considering my deployment scenario, the maintenance efforts for ESXi PXE deployment is a little complex. Besides the further patches download, i also need to do
      1, Use PowerCLI to re-build lasted ISO
      2, extract the ISO to PXE depot
      3, verify or modify chainloader configuration file (BOOT.CFG), because in my environment, I use the modified BOOT.CFG, something like below

      ———————————
      kernel=/ESXi51/tboot.b00
      kernelopt=runweasel ks=http://10.254.196.6/pxeboot/ESXi51/answerfiles/ks_esxi51_default.cfg
      modules=/ESXi51/b.b00 — /ESXi51/useropts.gz ……… more
      ———————————-

      so if considering my specific application, is there any other way to always make sure users get newest ESXi 5.x deployed with my less maintenance efforts?

      I checked your another great post (http://www.virtuallyghetto.com/2012/09/a-pretty-cool-method-of-upgrading-to.html) and tried this way, it seems the resource is required to be downloaded from internet. My question is how to build a local depot, which could help ESXi upgrade more efficiently?

      thank you in advanced. :) Nice Day!

  5. Nice work, William. Here’s a suggestion to skip extra pipes (grep | awk) & take advantage of awk’s powerful pattern matching. Above where you have the command…

    GATEWAY=$(localcli network ip route ipv4 list | grep default | awk ‘{print $3}’)

    …you can shorten it w/ simple awk pattern matching…

    GATEWAY=$(localcli network ip route ipv4 list | awk ‘/default/ {print $3}’)

  6. Hi.

    I was wondering, is it possible to install vibs from the ks.cfg? I have tryed this:
    esxcli software vib install -d /path where the vib is.. in the ks, but it does not work. Any idea of what I’m missing? It works manually after the install is finished.

    Also, I want to run a .sh from the ks, but after changing the .sh to be executable, the ks.cfg ignores the part where i try to run it… And after ESXi 5.1 is installed, i still have a problem running the .sh manually, something happens to the file in the installation, and it ends with an ?. The same goes for .vim files, when i write this in the ks: wget ftp:///template.vmx -O /tmp/template.vmx the file looks like this after the install: template.vmx?

    I can’t figure out what the problem is. Hopefully you have an answer for me :)

    Thanks in advanced ;)

    • Hi, thank you for answering me.

      I tried to add the %post section, but it failed.

      Error (see log for more info):
      User-supplied script failed. (Error code 512)

      This is how the %post section of my kickstart looks like..

      %post –interpreter=busybox –ignorefailure=false

      mkdir /depot

      # Installing hardware monitoring…
      wget ftp://ip/hp-esxi5.0uX-bundle-1.1.2-4.zip -O /depot/hp-esxi5.0uX-bundle-1.1.2-4.zip
      esxcli software vib install -d /depot/hp-esxi5.0uX-bundle-1.1.2-4.zip

      # Install esxcli-shell plugin
      esxcli software acceptance set –level CommunitySupported
      wget ftp://ip/esxcli-shell-1.1.0-15-offline_bundle.zip -O /depot/esxcli-shell-1.1.0-15-offline_bundle.zip
      esxcli software vib install -d /depot/esxcli-shell-1.1.0-15-offline_bundle.zip

      wget ftp://ip/newVM.sh -O /depot/newVM.sh

      wget ftp://ip/template.vmx -O /depot/template.vmx

      chmod +x /depot/newVM.sh
      chmod +x /depot/template.vmx

      sh /depot/newVM.sh

    • First off with any type of kickstart/script development, make sure you try out the above commands to ensure they work. I would not recommend putting patches in the root directory, usually they can get pretty large and you may fill up the main filesystem. You should store the files in a local datastore. In terms of logs, you should take a look at /var/log/esxi_install.log which you should get familiar with as it provides all the details of your install including %pre & %post section and you’ll probably want to copy that off to local datastore as it is not persisted over a reboot (take a look in the kickstart example I have).

  7. Please check that your password doesn’t contain # (hash) or = (equal) for starters

    the password #=8#=7a99Jj didn’t work

  8. Hi.

    If I wanted to install on a SD card, what would the command be then?

    I have used the following to install on a USB:
    install –firstdisk=usb-storage –overwritevmfs –novmfsondisk

    But instead of usb-storage, I want sd-card, but it doesn’t work. What do I write?

    Thanks in advance :)

  9. Hi,

    I tried to install using this it is working, one question about assigning a license key, i saw there is no %post section written on this kickstart, when i moved the license key related step to %post, it is unable to apply license during post, so i am getting warning message about evaluation. however it applies the license key during %firstboot but i don’t want message about evaluation. How to get rid of that?

    Thanks

    • It’s a best practice to do all post configurations in %firstboot if it isn’t needed during %post as hostd isn’t up and running vim-cmd will not work. Once the license is assigned, then you should not get that message. Are you saying that you’re still seeing the message even after assigning it?

    • Thanks William,

      i am not getting evaluation message after %firstboot(if i apply license in %firstboot)

      but if i assign license in %firstboot, it is giving me evaluation related warning after esxi installation(before machine gets rebooted and execute %firstboot), i want to get rid of the warning message i am getting just before machine reboot.

      is that any other way to assign a license key using some different command? that i can use during %post ?

      just a thought, if it is ok to assign a license during %firstboot why it is checking the license during %post (or before %firstboot)

  10. Hi.

    I would also know how to install on a SD card. My USB’s cause many warnings, so it would be nice to try SD card..

    And another thing; I try do copy the installation log files to a different partition, but with no luck. After a reboot, they get deleted!? Any thoughts on how to fix this? I need the log files…

    Thanks in advance

    • In our environment, we install to SD cards using kickstart all the time. I use a usb key for my install media and ks files.

      Try this:
      install –firstdisk=local –overwritevmfs –novmfsondisk

      One thing to note, due to the order of devices at boot, I have found that you MUST place your USB stick in the rear of an R720 for this to work, otherwise it will install esxi overtop of your install media.

  11. Hi. Great post!

    I’m trying to create a standard kickstart that reads from separate files.I want to install esxi 5.1 several times, but with different ip’s, name and configs.. (diskname, ip, domainname, license key, packages to install etc)

    I tried your example with the %include /../.. and extracting the information, but with no luck. Is there other ways to achieve this?

    I’m using the busybox interpreter, and I have trouble with the HDD’s which results in having to know the exact name of the HDD, so I want to have this read from a file so I don’t have to edit the kickstart each time…

    Hope you understand what I’m asking and have an answer for me.

    Thank you in advance!

    • warning:/tmp/cfg:line 1: argument “–bootproto” to command “network” set but an invalid value was specified.
      warning:/tmp/cfg:line 1: argument “–hostname” to command “network” is missing a value.
      warning:/tmp/cfg:line 1: argument “–ip” to command “network” is missing a value.
      warning:/tmp/cfg:line 1: argument “–netmask” to command “network” is missing a value.
      warning:/tmp/cfg:line 1: argument “–gateway” to command “network” is missing a value.
      warning:/tmp/cfg:line 1: argument “–nameserver” to command “network” is missing a value.
      warning:/tmp/cfg:line 1: no bootproto set. Defaulting to DHCP.

      I followed your example. But it didn’t work. Any thoughts on why?

      This is what my pre section looks like. I also have tryed bootprot=static, but then the installation fails..

      %pre –unsupported –interpreter=busybox

      CMDLINE_FILE=/tmp/cmdline
      ESXI_INSTALL_LOG=/var/log/esxi_install.log

      # Extract boot options
      vsish -e get /system/bootCmdLine > ${CMDLINE_FILE}

      # Extract and set variables
      HOSTNAME=$(cat ${CMDLINE_FILE} | grep hostname | sed -e ‘s/.*hostname=\([^ ]*\).*/\1/’)
      IP=$(cat ${CMDLINE_FILE} | grep ip | sed -e ‘s/.*ip=\([^ ]*\).*/\1/’)
      NETMASK=$(cat ${CMDLINE_FILE} | grep netmask | sed -e ‘s/.*netmask=\([^ ]*\).*/\1/’)
      GATEWAY=$(cat ${CMDLINE_FILE} | grep gateway | sed -e ‘s/.*gateway=\([^ ]*\).*/\1/’)
      NAMESERVER=$(cat ${CMDLINE_FILE} | grep nameserver | sed -e ‘s/.*nameserver=\([^ ]*\).*/\1/’)
      VIRTUAL_SWITCH_GROUP=$(cat ${CMDLINE_FILE} | grep virtual_switch_group | sed -e ‘s/.*virtual_switch_group=\([^ ]*\).*/\1/’)

      # Set the network configuration on the first network adapter
      echo “network –bootproto=none –hostname=${HOSTNAME} –ip=${IP} –netmask=${NETMASK} –gateway=${GATEWAY} –nameserver=${NAMESERVER} –addvmportgroup=0″ > /tmp/cfg

  12. # enable & start remote ESXi Shell (SSH)
    vim-cmd hostsvc/enable_ssh
    vim-cmd hostsvc/start_ssh

    does not work in my enviroment, does kickstart leave any log to examine?

    Kevin

  13. Truly an awesome article that helped me out tons! The best documentation and example I’ve found for the ESXi automated install! Thank you! Thank you! Thank you!

  14. I want to copy the OVA’s to the datastore and deploy them offline. But I am not able to do it

    I copied the files in ova directory and used the following command:

    cp -R /vmfs/volumes/CDROM/OVA/* /vmfs/volumes/datastore1/

    but did not succed, can you help me on this?

Thanks for the comment!