Kubernetes

Guest Customization support for Instant Clone in vSphere 7

05/14/2020 by William Lam 2 Comments

vSphere Instant Clone was re-architected back in vSphere 6.7 and has been enhanced to be made more powerful and flexible. These enhancements not only power solutions like VMware Horizon but it also unlocks new customer use cases including things like Instant Cloning of Nested ESXi and Apple MacOS Guests.

Although the possibilities are truly endless with Instant Clone, this also means that any customization including basic guest identity such as hostname and networking must now use an alternative workflow. For application-level customization, it is expected that customers will create and manage these custom scripts but for basic networking configuration, it would be ideal to leverage the existing and well known vSphere Guest Customization Engine.

While downloading a file from MyVmware the other day, I came across an interesting set of packages called Guest Customization Engine for Instant Clone. Upon further investigation, I came to learn that these guest packages actually enable support for native vSphere Guest Customization for Instant Clone in vSphere 7 for the following Linux guest OSes:

CentOS 7.4 or higher
RHEL 6.8 or higher
RHEL 7.4 or higher
Ubuntu 16.04
SUSE 11SP4
SUSE 12SP3 or higher

In addition, there is also new set of vSphere (SOAP) APIs that you will need to interact with to use the new Instant Clone Guest Customization feature. The GuestCustomizationManager is a new vSphere 7.0 API which includes the following three API methods:

AbortCustomization_Task
CustomizeGuest_Task
StartGuestNetwork_Task

If you are interested in taking advantage of the new Instant Clone Guest Customization in vSphere 7, you can refer to the official VMware documentation which has step by step instructions.

Tanzu Kubernetes Grid (TKG) Demo Appliance for VMC and vSphere

05/11/2020 by William Lam 8 Comments

As some of you can probably tell from my recent Twitter updates and blog posts (here and here) that I have been spending some time lately with both vSphere with Kubernetes and Tanzu Kubernetes Grid (TKG). Like many of you in the community, I am still pretty new to Kubernetes (K8s) and I am still learning about what it has to offer both from an infrastructure standpoint but more importantly how it can be used to deliver new and modern applications. I am also very lucky to be part of the the VMware Event Broker Appliance Open Source Fling project which builds and runs on top K8s and this project has allowed me to really get hands on which is how I learn best.

A couple of months back I was asked to put together a workshop to demonstrate how to deploy TKG Clusters running on VMware Cloud on AWS (VMC) and while developing the workshop, I thought it would be really cool if I could make it even easier for anyone that is brand new to K8s to quickly get started with TKG. I wanted to have a solution that can literally be dropped into any supported vSphere-based environment with basic networking to go from Zero to Kubernetes in less than 30 minutes!

Enter the Demo Appliance for Tanzu Kubernetes Grid (TKG) Fling

A Virtual Appliance that pre-bundles all required dependencies to help customers in learning and deploying standalone Tanzu Kubernetes Grid (TKG) clusters running on either VMware Cloud on AWS and/or vSphere 6.7 Update 3 environment for Proof of Concept, Demo and Dev/Test purposes. This appliance will enable you to quickly go from zero to Kubernetes in less than 30 minutes with just an SSH client and a web browser!

In addition to the appliance, I have also put together a step by step workshop-style guide which not only walks you through in deploying your first TKG Cluster but also provide some example demos and references which you can explore further. Below are some of the highlights of the Demo Appliance for TKG:

Configure non-secure Harbor registry with Tanzu Kubernetes Grid (TKG)

05/09/2020 by William Lam 3 Comments

In an earlier blog post, I shared the steps to to configure Harbor with a proper signed SSL certificate that would serve as private container registry for Tanzu Kubernetes Grid (TKG) CLI running in an air-gapped environment.

Although Harbor can easily be configured to support custom CA signed certificate, self-sign certificate and even just using HTTP, there are several additional steps and dependencies that is required if you wish to use a non-secure container registry with TKG CLI. This definitely was a bunch of trial/error and hopefully this can be made easier in the future to easily enable non-secure registry support with TKG CLI out of the box for development and testing purpose.

I also want to give a huge thanks to Jun Wang from our Modern Application Business Unit (MAPU), he was instrumental in helping me out and ultimately his tip on updating the containerd configuration was the last piece to the puzzle so that the K8s images deployed would use our insecure Harbor registry for pulling container images.

Troubleshooting tips for configuring vSphere with Kubernetes

05/05/2020 by William Lam 9 Comments

With more and more folks trying out the new vSphere with Kubernetes capability, I have seen an uptick in questions both internally and externally around the initial setup of the infrastructure required for vSphere with Kubernetes but also during the configuration of a vSphere Cluster for Workload Management.

One of the most common question is why are there no vSphere Clusters listed or why a specific vSphere Cluster is showing up as Incompatible? There are a number of reasons that this can occur including vCenter Server not being able to communicate with NSX-T Manager to retrieve the list of NSX pre-checks which would cause the list to either be empty or listed as incompatible. Not having proper time sync between vCenter Server and NSX-T which can also manifest in a similar behavior among other infrastructure issues.

Having ran into some of these issues myself when developing my automation script, I figure it might be useful to share some of the troubleshooting tips I have used when trying to figure out what is going on whether that is during the initial setup or actually deploying workloads using vSphere with Kubernetes.

Configure log forwarding from Tanzu Kubernetes Grid (TKG) to vRealize Log Insight Cloud

04/27/2020 by William Lam 1 Comment

As much as I enjoy kubectl'ing logs in real time for troubleshooting and debugging purposes, this usually does not scale beyond a couple of Kubernetes (K8s) Clusters if you are lucky. Even then, you will not retain any of the historical logs which may be required for deeper analysis or for auditing purposes. This is usually solved by having a centralized log management platform and while working with Tanzu Kubernetes Grid (TKG) running on VMware Cloud on AWS, a solution like vRealize Log Insight Cloud (vRLIC) makes a lot of sense.

While browsing through the vRLIC console, I noticed that it supports a number of log sources including K8s which was exactly what I was looking for. However, after going through the instructions in configuring fluentd on my TKG Cluster, I found that that nothing was being sent. After a bit of debugging, I realized a few steps were actually missing that was required to setup this up on TKG Cluster.

I eventually figured it out and will be sharing this feedback with the vRLIC folks but in the meantime, you can follow the instructions below on how to forward both system and application logs from your TKG Cluster or any K8s deployment for that matter which has outbound connectivity to connect to vRLIC.

Deploy Harbor in an Air-Gapped environment for Tanzu Kubernetes Grid (TKG)

04/24/2020 by William Lam 1 Comment

When using Tanzu Kubernetes Grid (TKG) and the new TKG CLI, outbound internet connectivity is required as part of the initial setup on the machine running TKG CLI but also on the TKG Management Cluster which is automatically stood up as part of the deployment. For demo and testing purposes, this is usually not a problem but for anyone looking to run this in a Production or datacenter environment, direct internet access is generally not available.

TKG does support air-gapped environments today by requiring a private container registry that has been configured with all the required containers. Once your registry has been setup, you will also need to update the TKG YAML manifest files to specify your private registry as by default, it will point to registry.tkg.vmware.run. You can use any container registry that is supported with Kubernetes including the popular Harbor solution. One thing to note is that your private registry must have a proper signed SSL certificate, custom CA certificates or self-signed certificates are not officially supported today with TKG.

Since I recently had to set this up for a project I am working on, which I hope to talk about in a future blog post, I thought it would be useful to share the instructions on how to setup and configure Harbor to be used in-conjunction with TKG as well as any other solution that requires a container registry running in your own environment. In my deployment, I will be using Let's Encrypt for generating the required SSL certificate, but you can use any existing service for performing this operation. I will also be installing Harbor on Photon OS, but you can use any operating system of your choice that Harbor is supported on.

Pre-Requisites

Access to a public DNS domain which you have ownership of (e.g. adding new records)
Access to your internal DNS server to add a custom DNS zone lookup entry (e.g. registry.<yourdomain>.com)