William Lam

How to deploy the vCenter Server Appliance (VCSA) with a custom MAC Address?

02/20/2020 by William Lam Leave a Comment

I recently had a question that came in from our field where a customer needed to deploy the vCenter Server Appliance (VCSA) with a specific MAC Address which was a requirement to ensure property connectivity within their network. This type of network requirement is not really new or unique, it is a common practice used to ensure only valid VMs with a static DHCP reservation can actually connect to a specific network but it certainly was the first time I had heard of this request for the VCSA.

With the default VCSA installer workflow, there is currently not a way to modify the network MAC Address which is automatically generated after the deployment of the OVA. Having said that, I have spent quite a bit of time exploring the various non-standard methods of deploying the VCSA in the past (see here, here and here) and with that information, you definitely can affect the MAC Address while still maintaining a valid VCSA deployment. With a bit of trial/error, there are two options depending if you are deploying the VCSA directly to an ESXi host (for initial setup) or to an existing vCenter Server. To demonstrate how this works, I have created a basic shell script called VCSAStaticMACAddress.sh which you can easily adapt to for a Windows-based environment.

The trick is that when you deploy to a vCenter Server endpoint, the required OVF properties are persisted which would allow you to only deploy the VCSA but not actually power it on and there you can easily augment a number of settings including the MAC Address. In the case of deploying directly to an ESXi host, OVF properties are not persisted and hence a challenge if you wish to make changes prior to powering on the VM. In earlier versions, it was possible to set these OVF properties by way of using the extraConfig property of the VM but it looks like this trick no longer works and requires a slight variation of the workflow which is described in the instructions below.

How to automate the creation multiple routable VLANs on single L2 network using VyOS

02/12/2020 by William Lam 4 Comments

My personal homelab has a very simple network topology, everything is connected to a single flat network. This has served me well over the years, but sometimes it can prevent me from deploying more complex scenarios. Most recently while working with NSX-T and Project Pacific, I had a need for additional VLANs which my home router does not support. There are a number of software solutions that can be used including the popular pfSense, which I have used before.

Over the Winter break, a colleague introduced me to VyOS, which is another popular software firewall and router solution. I had not heard of VyOS before but later realized it was derived from Vyatta, which I had heard of, but development of that solution had stopped and VyOS is now the open source version of that software. Having never played with VyoS before, I thought this might be a good learning opopournity and started to dabble with VyOS over the holiday. At a high level, I have VyOS connected to two networks: Outside network as VyOS refers which is your local LAN and Inside network as VyOS refers which is an is an isolated vSphere Portgroup (VSS/VDS) that is not connected to anything and configured to pass all traffic (4095). From here, you can create multiple VLANs in VyOS which can then be untagged using Virtual Guest Tagging (VGT) by placing a Nested ESXi VM on the same isolated portgroup and then creating the respective portgroups within the Nested ESXi VM mapping to the VyOS VLANs you have created.

One of the nice benefits of this solution is that you can create multiple "Isolated" yet routable networks that can still reach your primary LAN network and still have to access core infrastructure services running like Active Directory, DNS, etc. which was one of my requirements. After figuring out how VyOS works and applying that to my specific use case, I thought why not build some basic automation to setup this solution as I probably will forget how I setup everything. Initially I was using the VyOS OVA but later found out it was an extremely out of date there was no public version of the latest VyOS release in OVA form. I decided to use their latest rolling release and apply some vSphere API Automation to not only install VyOS but also fully configure based on template containing VyOS commands. I know the latest version of VyOS now includes a REST API but its a bit of a chicken/egg to enable and not very friendly to use compared to the solution I have built.

VMware Community Homelabs Project

02/11/2020 by William Lam 1 Comment

On a weekly basis, I easily get at least half a dozen inquires ranging from our customers, partners and even internal VMware employees on the topic of VMware Homelabs. The most common requests I receive is whether hardware X will work and whether I have tried it to recommended known build-of-materials (BOM). Funny enough, just last week I was asked to review our CTO's (Greg Lavender) BOM as he was also interested in building his own personal VMware homelab which goes to show just how popular this topic really is 😊

Although I have written a TON of content regarding VMware Homelabs, I definitely can not cover every single permutation. Having seen so many awesome VMware Homelabs over the years from the VMware Community, why not leverage the power of our community to crowdsource all the different homelab configurations into a single location which can then be shared with the rest of the community? This idea was kicked off about two weeks ago and I have put together a simple Google Form which you can find the link below to submit your information.

Submit VMware Community Homelab: https://www.virtuallyghetto.com/homelab

As of writing this blog post today, I have received a total of 48 valid submissions (there were a handful that had invalid URLs and/or did not follow directions and published a publicly accessible URL to their homelab BOM which I had to remove from the submission). The submissions have been pretty interesting to see and just how different each homelab is, especially from a cost perspective ranging from $800 up to $150,000 🤯At that price, this is a full blown datacenter and I am sure folks have an ideas on who owns those labs (hint, its not me 😉). I want to thank everyone who has submitted to the project and help get the word out, hopefully we will see even more submissions in 2020! The results have been pretty interesting and it is great to see how different each homelab is, especially on the price

For now, you can view the complete results in the short URL below and periodically I will process any additional submissions and publish them to the Github repo.

VMware Community Homelab Results: http://vmwa.re/homelab

How to exclude VCSA UI/CLI Installer from MacOS Catalina Security Gatekeeper?

02/08/2020 by William Lam 1 Comment

A couple of weeks ago I had upgraded my personal home computer to the latest MacOS Catalina (10.15) and one of the first issues I ran into was being able to access my vCenter Server. It turned out this was due to changes to MacOS security (which is a good thing) but certainly caught me and others off guard. In fact, I spent quite some time searching online and eventually found this workaround here.

After sharing this tidbit online (which several others also ran into) I came to learn that both Duncan Epping blogged about this issue back in Nov 2019 here and Christian Mohn blogged about this in Dec 2019 here. Sadly I did not come across either of their blogs using "NET::ERR_CERT_REVOKED macos catalina" in Google. I had assumed this was a Chrome issue and simply landed on the first few links and looking back, I now see Duncan's blog was #6 in the search results (doh!)

Today, I ran into another issue when attempting to use the VCSA CLI Installer, the following error was thrown:

“vcsa-deploy.bin” cannot be opened because the developer cannot be verified

This is again due to a security change in MacOS Catalina which now prevents terminal-based applications which are not notarized from running. For a single application/binary, you can go into System Preferences->Security & Privacy and allow anyway. For more complex applications like the VCSA CLI Installer which has a number of libraries and scripts, this will take awhile and end up frustrating end users. The updated security enhancement is actually a good thing and I did not want to disable the Gatekeeper service but I was interested in disabling it for the VCSA CLI Installer. While searching online, I came across this Hashicorp Terraform thread where folks were having the exact same issue and I found out there was a way to disable the MacOS Security Gatekeeper for a specific application.

To do so, we just need to recursively remove the metadata attribute "com.apple.quarantine" for the extracted VCSA ISO by running the following command:

sudo xattr -r -d com.apple.quarantine VMware-VCSA-all-6.7.0-Update-15132721

After the quarantine attribute has been removed, you can now run the VCSA CLI Installer (including UI Installer) without being prompted with an error. Hopefully VMware will consider notarizing future releases of the VCSA Installer and I will be sharing this feedback internally if it has not already.

Using PowerCLI to automate the retrieval of VCSA Password Policies

02/06/2020 by William Lam Leave a Comment

I hope that every vSphere administrator or operator by now is familiar with the extremely powerful vSphere Guest Operations API functionality (details here and here), which can easily be consumed using PowerCLI's Invoke-VMScript cmdlet. If not, highly recommend you check out the links referenced. I know the GuestOps API is certainly my top favorite with sending VM keystrokes capability a very close second!

Not only does the GuestOps API unlock functionality that simply may not be possible (e.g. there's no API or automation interface) but it also enables automation within a VM without requiring any type of remote management services enabled (e.g. SSH or WinRM) or even networking to the VM for that matter!

The reason I am bringing all this up is that although there is not an API for managing and retrieving vCenter Single Sign-On (SSO) configurations which includes password policies, there is a way in which customers can still automate and retrieve this and other information by leveraging the GuestOps API. In fact, back in 2015 I demonstrated on how you can retrieve VCSA SSO password policy and configurations and we can simply apply the GuestOps API to help us automate this task. In addition, most customers do not enable SSH by default and we can still apply the GuestOps API technique and perform automation tasks to VSCSA without requiring SSH as described in this blog post back in 2016.

Workspace One Access (vIDM) Powershell Module to automate creating 3rd Party Identity Provider

02/05/2020 by William Lam Leave a Comment

One of the projects I am currently working on involves Workspace One Access (formally VMware Identity Manager) and configuring a 3rd Party Identity Provider for Identity Federation. As with anything, using the UI for the first time to validate the workflow is perfectly fine for me but after that, I normally prefer to automate, especially as I was rebuilding this particular setup a few times. I saw that Workspace One Access (WSO Access) had a REST API but I was surprised that there were no APIs for actually managing the configurations.

I figured before giving up, I should see at least see how the UI was performing these operations as "some API" should exists and started up one of my favorite browser tools Chrome Developer Console to inspect the HTTP requests. I came to learn there were an additional set of "Jersey" APIs (no background on the Jersey name, but its part of the API URI) that might do exactly what I was looking for. After a bit of trial/error, I was able to fully automate the creation of both a WSO Access Directory as well as 3rd Party Identity Provider.