ova

Building your own Virtual Appliances using OVF properties Part 3

03/19/2019 by William Lam 3 Comments

To conclude this three-part blog series, we are now going take a look at reference implementation for building your own Microsoft Windows Virtual Appliance (VA). Similar to the Linux VA build, the Windows OVA will also support the ability to customize basic networking configuration including the use of a static or DHCP option.

In addition, to demonstrate the endless possibilities for building your own VA, I have also included an option to automatically join a Microsoft Active Directory Domain as part of the OVA deployment, which is a fairly common operation after deploying a Windows-based system. In the example below, I am using Windows Server 2016 and PowerShell to perform all the required automation.

Step 1 - Create a new VM in vCenter Server and then install Window Server 2016 using the ISO. Once you have completed the OS installation, you may want to apply any patches or packages that you want included as part of your VA. Once that is done, go ahead and shut down the VM.

Step 2 - Select the VM in the vSphere Inventory and then click on Configure->vApp and then check the Enable vApp Options. Once enabled, select OVF environment for the IP allocation scheme. In the OVF Details tab, select VMware Tools for the OVF environment transport. (Optionally) You can specify some additional metadata including appliance name and URLs to help others who maybe consuming your VA once it has been exported to an OVF/OVA.

Step 3 - Next, add the following 9 OVF properties which will be used as input to configure networking within PhotonOS. Click Add and provide a Label, Key and optional Category.

Label	Key	Category
Hostname	guestinfo.hostname	Networking
IP Address	guestinfo.ipaddress	Networking
Netmask	guestinfo.netmask	Networking
Gateway	guestinfo.gateway	Networking
DNS Server	guestinfo.dns	Networking
DNS Domain	guestinfo.domain	Networking
AD Domain	guestinfo.ad_domain	Active Directory
AD Username	guestinfo.ad_username	Active Directory
AD Password	guestinfo.ad_password	Active Directory

Step 3 - Power back on the VM and once it is available on the network (assuming DHCP), download and copy the sample first boot script customize-guest.ps1 to C:\Users\Administrator\Desktop. This script is where all the magic happens and will process the OVF property input and then configure the network settings and if specified, it will also perform the Active Directory domain join. Right now it assumes the networking fields are optional, meaning if they are left blank, it will default the system to DHCP. If you provide all input properties, then it will go ahead and configure a static network address.

Building your own Virtual Appliances using OVF properties Part 2

02/19/2019 by William Lam 2 Comments

In the previous article, we reviewed the concepts and basic approach to building your own VMware Virtual Appliance (OVF/OVA). In Part 2, we are now going to take a look at a reference implementation for building a Linux VA using VMware PhotonOS. Although I am using PhotonOS as the guest, you can apply these same techniques to any other Linux distribution of your choice.

Step 1 - Create a new VM in vCenter Server and then install PhotonOS using the ISO format. Once you have completed the OS installation, you may want to apply any patches or packages that you want included as part of your VA. Once that is done, go ahead and shut down the VM.

Step 3 - Next, add the following 6 OVF properties which will be used as input to configure networking within PhotonOS. Click Add and provide a Label, Key and optional Category.

Label	Key	Category
Hostname	guestinfo.hostname	Networking
IP Address	guestinfo.ipaddress	Networking
Netmask	guestinfo.netmask	Networking
Gateway	guestinfo.gateway	Networking
DNS Server	guestinfo.dns	Networking
DNS Domain	guestinfo.domain	Networking

Step 4 - Power back on the VM and once it is available on the network (assuming DHCP), download and copy the sample first boot script rc.local to /etc/rc.d/rc.local. This script is where all the magic happens and will process the OVF property input and then configure the network settings. Right now it assumes these fields are optional, meaning if they left blank, it will default the system to DHCP. If you provide all input properties, then it will go ahead and configure a static network address.

Building your own Virtual Appliances using OVF properties Part 1

02/01/2019 by William Lam 3 Comments

This has been a topic I have been wanting to write about for quite some time, especially as I get asked about this on fairly regular basis from both partners and customers. I normally point folks over to our official Virtual Appliance (VA) authoring tool, VMware Studio which includes a number of development resources to help get started. Studio is used by many of our partners when creating their VA offerings, although it may not be the easiest thing to get started with, it does provide a complete end-to-end solution.

Most recently, I found myself building out a couple of VAs for my own day to day use, including a custom PhotonOS OVA that allows me to configure a static network address during deployment through the use of custom OVF properties. The official PhotonOS OVA that VMware ships does not provide this option and automatically defaults to DHCP. If you want to setup a static IP Address, you would need to first deploy the VM and then login to the console or SSH (if you have DHCP enabled) and then manually update the networking settings.

For my use case, Studio was going to be overkill and not to mention it may not even support PhotonOS or other modern OSes in general. However, everything that is needed to build your own VA is actually available right in vCenter Server. This was the perfect opportunity and excuse for me to finally document *my* process, in case it can help others wanting to do the same, especially for a home lab setup. In Part 1, I will take you through the two important concepts of building your own VA and then in Part 2 and Part 3, we will take a look at building both a Linux and Windows VA. I will also publish a reference Linux and Windows implementation so that you can use that as a basis to build your own VA, which is not limited to just Linux or Windows, it can be ANY GuestOS that vSphere supports.

Quick Tip – Import OVF/OVA as VM Template using OVFTool 4.3 Update 1

01/29/2019 by William Lam 5 Comments

OVFTool is an extremely versatile command-line utility for importing and exporting Virtual Machines to and from the OVF/OVA format and it supports a number of VMware platforms including VMware Cloud on AWS (VMC), vSphere (vCenter Server and ESXi), Fusion, Workstation, Player and even vCloud Director (vCD).

An infrequent asks that I have seen from customers is the ability to deploy an OVF/OVA as a VM Template rather than just a Virtual Machine in a vSphere-based environment. OVFTool has had the ability to deploy to vAppTemplate for vCD-based environments, so it would make sense to also support vCenter Server VM Templates as well. Today, the workflow is a two-step process, deploy the OVF/OVA and then use the vSphere API to convert the VM to a VM Template.

With the latest OVFTool 4.3 Update 1 which was a minor release last year, we now have a new parameter called importAsTemplate which will allow customers to easily import an OVF/OVA directly into as a VM Template. Below is a quick sample using this new option and I am deploying to a VMC-based environment (see this article for requirements when using OVFTool with VMC)

ovftool.exe `
--acceptAllEulas `
--allowAllExtraConfig `
--name=PhotonOS-Template `
--datastore=WorkloadDatastore `
--net:None=sddc-cgw-network-1 `
--vmFolder=Templates `
--importAsTemplate `
C:\Users\william\Desktop\photon-hw13_uefi-3.0-49fd219.ova `
'vi://cloudadmin@vmc.local@vcenter.sddc-a-b-c-d.vmwarevmc.com/SDDC-Datacenter/host/Cluster-1/Resources/Compute-ResourcePool/'

ovftool.exe `

--acceptAllEulas `

--allowAllExtraConfig `

--name=PhotonOS-Template `

--datastore=WorkloadDatastore `

--net:None=sddc-cgw-network-1 `

--vmFolder=Templates `

--importAsTemplate `

C:\Users\william\Desktop\photon-hw13_uefi-3.0-49fd219.ova `

'vi://cloudadmin@vmc.local@vcenter.sddc-a-b-c-d.vmwarevmc.com/SDDC-Datacenter/host/Cluster-1/Resources/Compute-ResourcePool/'

Once the upload has completed, we can take a look at our vSphere UI and see that our imported OVA been automatically been converted to a VM Template!

Default hashing algorithm changed in OVFTool 4.2 preventing OVF/OVA import using vSphere C# Client

11/18/2016 by William Lam 10 Comments

After upgrading my home lab recently to vSphere 6.5, I also updated some of the related utilities such as the various SDKs and CLIs. One of the CLIs that I had updated was the latest version of OVFTool which is now at 4.2. I use OVFTool extensively to automate various Virtual Machine deployments (import/export). While testing out a new OVA that I had been working on, I needed to verify that it also worked with previous release of vSphere like vSphere 6.0 Update 2. I happen to have the vSphere C# Client open and connected to a vCenter Server and when I tried to import the newly created OVA, but it failed with the following error message:

The following manifest file entry(line1) is invalid: SHA256

I was pretty surprised by this since I went through this exact same workflow a couple of days ago without any problems. The only change that had happened was OVFTool and error seems to indicate an issue with the hashing algorithm. I ran OVFTool again using just the --help option to check what the default SHA hashing algorithm was, it was SHA256. I then compared that to an older version of OVFTool and it looks like the default had changed from SHA1 to SHA256.

From a security standpoint, this is a positive change as SHA1 is no longer considered a secure hashing algorithm and a stronger version should be used. It also turns out that the vSphere C# Client can only support SHA1 which is why I received the error after upgrading to the new version of OVFTool. Luckily, this is NOT a problem when using the vSphere Web Client or the vSphere HTML5 Client and only affects the vSphere C# Client. If you do need to use the vSphere C# Client for importing OVF/OVAs exported from the latest version of OVFTool, the workaround is quite simple, just override the default hashing algorithim when exporting by adding the additional CLI option:

--shaAlgorithm=sha1

Heads Up: OVF/OVA always deployed as Thick on VSAN when using vSphere Web Client

06/03/2016 by William Lam 24 Comments

Just wanted to give folks a heads up on an issue that a colleague of mines recently identified when provisioning Virtual Appliances (OVF/OVA) onto a VSAN datastore when using the vSphere Web Client. He found that regardless of the VSAN Storage Policy that was selected, whether it is the default VSAN Storage Policy or a custom one, the Virtual Appliance will always be Thick provisioned.

This behavior only occurs when using the vSphere Web Client and is not observed when using either the vSphere C# Client or the ovftool CLI. My understanding of the issue is that there are two ways in which a VM can get provisioned as Thin. The "old" method which was to explicitly specify the disk allocation type (Thin vs Thick) and the "new" method which uses VM Storage Policies. To ensure that we maintain backwards compatibility for older clients, if a client specifies Thick provisioned, it would actually override the VM Storage Policy even if the Object Space Reservation capability was set to 0 (Thin provisioned). Since you can no longer specify the disk allocation type in the vSphere Web Client, the default behavior is to not Thin provision and hence the current Thick provisioning result even though the default VSAN Storage Policy has OSR set to 0.

Note: When referring to Thick provisioned in VSAN (proportionalCapacity = 100), It is defined as provisioned Thin with a reservation so there is a guarantee that space is available for the object. It is not accurate to compare this to Zeroed Thick or Eager Zeroed Thick in the VMFS/NFS world as VSAN is an Object Store.

Engineering has already been engaged and is currently investigating the issue. We have also asked for a VMware KB to be published, so hopefully once that goes up, folks can subscribe to that for more details and updates.

In the meantime, since it is actually pretty difficult to see if you have been affected by issue, I have created a simple PowerCLI script called Get-VSANPolicy.ps1 which will allow you to quickly scan through your VM(s) to identify whether you have any VMs that have been Thick provision residing on a VSAN Datastore. You can either get all VMs by piping Get-VM * or a specific set of VMs into the script.

The following example retrieves all VMs that start with "Photon-Deployed-From-*" and extracts their current VSAN VM Storage Policy for both VM Home and individual VMDKs. Here, we can see that both VMs are using the default VSAN VM Storage Policy.

Get-VM "Photon-Deployed-From-*" | Get-VSANPolicy -datastore "vsanDatastore"

Lets now only search for VMs that have been Thick provisioned by using the -thick option and setting that to true. Here we can see that the OVF we provisioned through the vSphere Web Client is the only VM listed.

Get-VM "Photon-Deployed-From-*" | Get-VSANPolicy -datastore "vsanDatastore" -thick $true

If we want to get more details on the underlying VM Storage Policy that was applied, we can also specify the -details option to true. Here we can clearly see that the 2nd VM has proportionalCapacity=100 which means Thick provision.

Get-VM "Photon-Deployed-From-*" | Get-VSANPolicy -datastore "vsanDatastore" -thick $true -details $true

Luckily, the fix is quite easy thanks to Paudie O'Riordan who found out that it was as simple as just re-applying the VSAN VM Storage Policy! (Policy Based Management FTW!) This means there is no need to perform unnecessary Storage vMotions to be able to convert the VM from Thick to Thin, it is literally a couple of clicks in the UI.

UPDATE (07/15/16) - Thanks to reader Jose, it looks like using the vSphere Web Client to re-apply the VSAN VM Storage Policy will correctly apply the policy to the VM/VMDKs, but does not reclaim the underlying storage. It is recommended that you use the PowerCLI script below to re-apply the policy which will then properly reclaim the underlying storage and will properly reflect the storage utilization.

As with anything, I still prefer Automation and with that, I have created a secondary script to help with the remediation. This is also a PowerCLI script called Set-VSANPolicy.ps1 which accepts a list of VMs and the name of the VSAN VM Storage Policy that you wish to re-apply.

Here is an example of running the script and remediating two VMs that contains multiple VMDKs:

Set-VSANPolicy -listofvms $listofvms -policy $vsanpolicy

If you now re-run the first script, you should see that you no longer have VMs that are provisioned Thick anymore (this may take some time depending on the size of your VMs).