Kubernetes

Configure log forwarding from Tanzu Kubernetes Grid (TKG) to vRealize Log Insight Cloud

04/27/2020 by William Lam Leave a Comment

As much as I enjoy kubectl'ing logs in real time for troubleshooting and debugging purposes, this usually does not scale beyond a couple of Kubernetes (K8s) Clusters if you are lucky. Even then, you will not retain any of the historical logs which may be required for deeper analysis or for auditing purposes. This is usually solved by having a centralized log management platform and while working with Tanzu Kubernetes Grid (TKG) running on VMware Cloud on AWS, a solution like vRealize Log Insight Cloud (vRLIC) makes a lot of sense.

While browsing through the vRLIC console, I noticed that it supports a number of log sources including K8s which was exactly what I was looking for. However, after going through the instructions in configuring fluentd on my TKG Cluster, I found that that nothing was being sent. After a bit of debugging, I realized a few steps were actually missing that was required to setup this up on TKG Cluster.

I eventually figured it out and will be sharing this feedback with the vRLIC folks but in the meantime, you can follow the instructions below on how to forward both system and application logs from your TKG Cluster or any K8s deployment for that matter which has outbound connectivity to connect to vRLIC.

Deploy Harbor in an Air-Gapped environment for Tanzu Kubernetes Grid (TKG)

04/24/2020 by William Lam Leave a Comment

When using Tanzu Kubernetes Grid (TKG) and the new TKG CLI, outbound internet connectivity is required as part of the initial setup on the machine running TKG CLI but also on the TKG Management Cluster which is automatically stood up as part of the deployment. For demo and testing purposes, this is usually not a problem but for anyone looking to run this in a Production or datacenter environment, direct internet access is generally not available.

TKG does support air-gapped environments today by requiring a private container registry that has been configured with all the required containers. Once your registry has been setup, you will also need to update the TKG YAML manifest files to specify your private registry as by default, it will point to registry.tkg.vmware.run. You can use any container registry that is supported with Kubernetes including the popular Harbor solution. One thing to note is that your private registry must have a proper signed SSL certificate, custom CA certificates or self-signed certificates are not officially supported today with TKG.

Since I recently had to set this up for a project I am working on, which I hope to talk about in a future blog post, I thought it would be useful to share the instructions on how to setup and configure Harbor to be used in-conjunction with TKG as well as any other solution that requires a container registry running in your own environment. In my deployment, I will be using Let's Encrypt for generating the required SSL certificate, but you can use any existing service for performing this operation. I will also be installing Harbor on Photon OS, but you can use any operating system of your choice that Harbor is supported on.

Pre-Requisites

Access to a public DNS domain which you have ownership of (e.g. adding new records)
Access to your internal DNS server to add a custom DNS zone lookup entry (e.g. registry.<yourdomain>.com)

How to deploy Tanzu Kubernetes Grid (TKG) Cluster with Antrea CNI

04/20/2020 by William Lam Leave a Comment

I have been working with Tanzu Kubernetes Grid (TKG) quite a bit lately and using their new slick TKG CLI for deploying standalone Tanzu Kubernetes Clusters (TKC) which can run in both VMware Cloud on AWS as well as your on-premises vSphere 6.7 Update 3 environment. If you have vSphere 7 and the vSphere with Kubernetes capability, it also supports TKG deployments natively as part of that solution but you can also use TKG CLI to deploy TKC's.

Out of the box, TKG includes all the necessary software components to deploy a production grade, upstream and conformant Kubernetes distribution. For most customers, the "batteries included" type of offering is more than sufficient but for some customers who may wish to customize some of these components further when running the standalone distribution. One such example is swapping out the default Container Network Interface (CNI) which uses Calico for a different CNI with more capabilities.

As you may have guess from the title of this post, we will be replacing Calico with Antrea which is another open source CNI. In fact, Antrea was started by VMware last year and uses Open vSwitch (OVS) to provide network and security capabilities to Kubernetes. You can read more about Project Antrea here and more details about its architecture can be found here.

Disclaimer: This is currently not officially supported by VMware. I do know the TKG team is looking at Antrea support in the future.

Automated vSphere 7 and vSphere with Kubernetes Lab Deployment Script

04/13/2020 by William Lam 40 Comments

I know many of you have been asking me about my vSphere with Kubernetes automation script which I had been sharing snippets of on Twitter. For the past couple of weeks, I have been hard at work making the required changes between the vSphere 7 Beta and GA workflows, some additional testing and of course documentation. Hopefully the wait was worth it (I think it is) and if you enjoy the script or have benefited, please consider adding 🌟to the Github repo to show your support! Thanks and enjoy

Had to make some updates to one of my vGhetto Automated Lab Deployment Scripts

💥44min to automate all required #vSphere7 infrastructure! 🤛🎤🥳

1 x VCSA 7.0
3 x ESXi + vSAN 7.0
1 x NSX-T 3.0 UA
1 x NSX-T Edge

Need to clean up #ProjectPacific wording but its working great! pic.twitter.com/ZInPgVgbGS

— William Lam (@lamw) April 4, 2020

The Github repository:

https://github.com/lamw/vghetto-vsphere-with-kubernetes-external-nsxt-automated-lab-deployment

Before getting started, please carefully read through the requirements section along with the complete sample end-to-end execution if you are new to vSphere with Kubernetes. You will need to have a VMware Cloud Foundation (VCF) 4.0 license before you can get started and specifically an NSX-T Advance license which is one of the required parameters within the script. If you do not have access to a VCF 4 license, I strongly recommend taking part in the recent VMUG Advantage Homelab Group Buy effort which I had started to easily get access to the latest VMware releases along with a nice 15% discount!

The script supports deploying both a standard vSphere 7 environment with just VCSA, ESXi and vSAN as well as the complete solution which includes NSX-T to support vSphere with Kubernetes. For more details, please refer to the FAQ.

Useful Interactive Terminal and Graphical UI Tools for Kubernetes

04/05/2020 by William Lam 9 Comments

I recently ran an internal hands-on workshop where I demonstrated to our field, marketing, support and engineering on just how easy it is to deploy and manage Tanzu Kubernetes Grid (TKG) Clusters running on VMware Cloud on AWS. While developing the lab which has been several weeks in the making, I did not want to assume everyone was familiar with Kubernetes (K8s) and as part of the workshop, I thought it would be useful to include some additional utilities that would provide a better lab experience for those that are just getting started in their K8s journey.

I included tools like bat, kube-ps1 and Octant as an example and the weekend before the workshop, I came to learn about a really neat terminal-based UI tool called kubelive and I knew I had to include that in the workshop. Interestingly, after the workshop, several folks shared with me that they had not heard of this tool either and others on Twitter had the same feedback. Given the level of interests with K8s in the VMware community, especially with the launch of vSphere with Kubernetes (Project Pacific), I thought it might be useful to share some of the terminal-based and graphical UI tools that I had been evaluating and learning about.

As of writing this blog post, my current two favorites is Octant for a graphical-based UI (browser) and K9s for a terminal-based UI.

Octant

Octant is a browser-based UI aimed at application developers giving them visibility into how their application is running. I also think this tool can really benefit anyone using K8s, especially if you forget the various options to kubectl to inspect your K8s Cluster and/or workloads. Octant is also a VMware Open Source project and it is supported on Windows, Mac and Linux (including ARM) and runs locally on a system that has access to a K8S Cluster. After installing Octant, just type octant and it will start listening on localhost:7777 and you just launch your web browser to access the UI.

One thing I really like about Octant is how easy it is to switch context between different K8s Cluster with a simple drop down along with namespace filtering which is quite helpful in narrowing down a specific deployments, usually for informational or troubleshooting purposes.

Most of my workflows generally involves a specific K8s pod and the Resource View tab is super handy to give me a quick overview of the different resources that is associated whether that is a deployment, secret, service, etc.

The YAML tab beats using cat and bat, especially for really large and complex deployments and you can search right in the browser.
[Read more...] about Useful Interactive Terminal and Graphical UI Tools for Kubernetes

Sneak peek at deploying Tanzu Kubernetes Grid on vSphere & VMware Cloud on AWS

03/16/2020 by William Lam 1 Comment

Last week, VMware had its huge launch which included VMware Cloud Foundation 4, vSphere 7, vSAN 7 and the brand new VMware Tanzu Portfolio that will help organizations build, run and manage modern applications for their business. Although we still have a couple of more months before general availability, the level of excitement for these upcoming releases was pretty amazing to see on social media, especially for the highly anticipated Kubernetes with vSphere capability, formally known as Project Pacific.

UPDATE (04/10/20) - TKG 1.0 has officially GA'ed, you can now deploy TKG Clusters using the new TKG CLI/UI as demonstrated in this blog post.

When vSphere 7 is available, Kubernetes with vSphere is just one way in which customers will be able to deploy upstream and conformant Open Source Kubernetes also referred to as Tanzu Kubernetes Grid (TKG). For customers that require a solution today or for those that may not able to upgrade to vSphere 7 immediately, VMware has another option called Tanzu Kubernetes Grid Plus for vSphere and VMware Cloud on AWS which was also announced last week but albeit not many folks took notice.

There are few things worth mentioning about Tanzu Kubernetes Grid Plus:

Tanzu Kubernetes Grid Plus is a new offering that is comprised of the core TKG software and it includes support from VMware's Customer Reliability Engineering (CRE), which is a team made up of Kubernetes Architects that can help customers design and architect their Cloud Native Applications running on the VMware Tanzu Platform
For customers that do not require CRE support and just want standard VMware GSS Support, then you can simply use Tanzu Kubernetes Grid (TKG) which includes VMware Support and Subscription (SnS)
The core TKG software which Tanzu Kubernetes Grid Plus includes can deploy Tanzu Kubernetes Cluster (upstream K8s running in VMs). Customers can already do this right now by using Cluster API (CAPI) and specifically the Cluster API Provider for vSphere (CAPV) which provides lifecycle management of Tanzu Kubernetes Cluster running on any valid vSphere endpoint including VMware Cloud on AWS. The latest CAPV v1alpha3 release also introduces a new workflow that reduces the number of steps compared to v1alpha2 which is now deprecated

For customers that want to further customize how Tanzu Kubernetes Cluster are deployed and configured including tools that they are already using, CAPV will be the most flexible option. Having gone through the CAPV workflows myself, I think it can still be daunting if you are new to this space, not to mention the different CLI tools that you will have to learn to have a successful deployment. Luckily, the VMware TKG team has also been thinking about the overall user experience and how they can build on top of CAPV to provide a much more simpler and more intuitive interface for customers that is just looking for a turnkey option.

Here is a sneak peek (Technical Preview) of the upcoming Tanzu Kubernetes Grid CLI or TKG CLI for short which will make deploying Tanzu Kubernetes Cluster on ANY vSphere endpoint incredibly EASY!