Kubernetes

Running sk8s (Simple Kubernetes) on VMC with an AWS Elastic Load Balancer

02/27/2019 by William Lam Leave a Comment

Last week I wrote about a really nifty Virtual Appliance called sk8s which can be used to quickly setup a Kubernetes (k8s) cluster for development and testing purposes. If you have not checked out that article, be sure to give that a read first to get the full context. As mentioned in the previous article, sk8s runs great on any vSphere deployment but it can also run on VMware Cloud on AWS (VMC) which adds an additional capability where an AWS Elastic Load Balancer (ELB) can automatically be provisioned and configured to front-end the k8s control plane as part of the deployment for external access.

The nice benefit of this is that you only need to configure access to the ELB and not directly to the underlying VMs running within the SDDC, both simplifying the setup but also reducing the need to expose the VMs directly to the internet. The write-up below is similar to that of the previous article, but it does expand into greater detail when deploying to VMC and all the required configuration changes within the VPC using the AWS Console and the Network and Security changes using the VMC Console.

Note: If you decide to use the integrated AWS ELB integration, please be aware that you will be charged for the consumption. For pricing, please see the AWS documentation here.

Prerequisites:

Access to the VMC Console and VMC SDDC
NSX-T Logical Network with DHCP enabled
AWS Access & Secret Key for automatically creating ELB (Optional)
govc

Step 1 - Install govc on your local desktop which has access to your VMC vSphere environment. If you have not installed govc, the quickest way is to simply download the latest binary, below is an example of installing the latest MacOS version:

curl -L https://github.com/vmware/govmomi/releases/download/v0.20.0/govc_darwin_amd64.gz | gunzip > /usr/local/bin/govc
chmod +x /usr/local/bin/govc

Step 2 - We need to verify a few settings in the AWS Console to ensure that the VPC that is connected to your SDDC is properly configured so that the provisioning of the ELB will be successful.

sk8s – Simple Kubernetes (k8s) Virtual Appliance

02/22/2019 by William Lam 14 Comments

I recently had a need for a for a basic Kubernetes (k8s) Cluster which I also needed to have running locally in my vSphere Home Lab for testing purposes. I know there are a number of great blog articles out there that shows you how to setup your own k8s from scratch, including a recent blog series from Myles Gray. However, I was looking for something quick that I could consume without requiring any setup. To be honest, installing your own k8s from scratch is so 2017 😉

If you ask most people, they simply just want to consume k8s as an integrated solution that just works and not have to worry about installing and managing the underlying components that make up k8s. VMware PKS and PKS Cloud are two great examples of this where Pivotal and VMware provides a comprehensive solution (including Software Defined Networking) for managing the complete lifecycle (Day 0 to Day N) for running Enterprise K8s, whether that is within your own datacenter or running as a public cloud service. For my exploratory use case, PKS was overkill and I also did not have the required infrastructure setup in this particular environment, so I had to rule that out for now.

While searching online, I accidentally stumbled onto a recent VMware Open Source project called sk8s, short for Simple Kubernetes (k8s) which looked really interesting. At first glance, a few things stood out to me immediately. This project was created by none other than Andrew Kutz, for those not familiar with Andrew's work, he famously created the Storage vMotion UI plugin for the vSphere C# Client before VMware had native UI for the feature. He was also the creator of the first vCenter Simulator back in the day called simDK that was also widely used by a number of customers including myself. I knew Andrew had joined our Cloud Native Business Unit (CNABU), but I was not sure what he was up to these days, guess I now know 🙂 and is helping both VMware and the OSS community in k8s development.

vGhetto Automated Pivotal Container Service (PKS) Lab Deployment

06/12/2018 by William Lam 3 Comments

While working on my Getting started with VMware Pivotal Container Service (PKS) blog series awhile back, one of the things I was also working on was some automation to help build out the required infrastructure NSX-T (Manager, Controller & Edge), Nested ESXi hosts configured with VSAN for the Compute vSphere Cluster and Pivotal Ops Manager. This was not only useful for my own learning purposes, but that I could easily rebuild my lab if I had messed something up and allowed me to focus more on the PKS solution rather than standing up the infrastructure itself.

To be honest, I had about 95% of the script done but I was not able to figure out one of the NSX-T APIs and I got busy and had left the script on the back burner. This past weekend while cleaning out some of my PKS research documents, I came across the script and funny enough, in about 30minutes I was able to solve the problem which I was stuck for weeks prior. I just finished putting the final touches on the script along with adding some documentation. Simliar to my other vGhetto Lab Automation scripts, I have created a Github repo vGhetto Automated PKS Lab Deployment

UPDATE (06/19/18) - I have just updated the script to also include the deployment and configuration of the PKS components (Ops Manager, BOSH Director, Harbor & Stemcell). The script by default will now configure everything end-2-end and you will have a fully functional PKS environment that you can start playing around with. For complete details, please see the Github repo which has the updated requirements and documentation. Below is a screenshot of the PKS deployment and configuration which requires the use of the Ops Manager CLI (OM).

The script will deploy the following components which will be placed inside of a vApp as shown in the screenshot below:

NSX-T Manager
NSX-T Controller x 3 (though you technically only need one for lab/poc purposes)
NSX-T Edge
Nested ESXi VMs x 3 (VSAN will be configured)
Ops Manager

The script follows my PKS blog series and automates Part 3 (NSX-T) and the start of Part 4 (Ops Manager deploy), please refer to these individual blog posts for more information. The goal of the script is to enable folks to jump right into the PKS configuration workflows and not have to worry about setting up the actual infrastructure that is needed for PKS. Once the script has finished, you can jump right into Ops Manager and start your PKS journey.

Here is a sample execution of the script which took ~29 minutes to complete.

The full requirements for using the script be found on the Github repo and below are the software versions that I had used to deploy and configure PKS:

Getting started with VMware Pivotal Container Service (PKS) Part 11: Application Monitoring

05/03/2018 by William Lam Leave a Comment

In this blog post, we will take a look at using VMware Wavefront, a recent acquisition, that is a leading SaaS based metrics and monitoring solutions for Cloud Native Applications. Wavefront supports monitoring Kuberenetes (K8S) and many other applications, but what is really neat about Wavefront is that it not only does it give us deeper metrics about our K8S infrastructure (cluster, pod, namespaces and containers), but it can also be used to help developers instrument their application for metric collection and monitoring and provide a complete end-to-end view. The Wavefront team also recently published this blog post which has a nice video demo of their integration with VMware PKS.

If you missed any of the previous articles, you can find the complete list here:

Getting started with VMware Pivotal Container Service (PKS) Part 10: Infrastructure Monitoring

05/01/2018 by William Lam 1 Comment

In this blog post, we go through and configure VMware vRealize Operations Manager (vROps) which Cloud Admins/Platforms Operators can take advantage of to help with monitoring and alerting on the underlying PKS infrastructure (compute, storage & networking). In addition, through the use of vROps Management Packs, users can also get additional visibility into the individually deployed PKS managed Kubernetes (K8S) Clusters which can be useful when debugging and with your development teams.

If you missed any of the previous articles, you can find the complete list here:

Configuring Active Directory integration with VMware PKS Ops Manager using VMware Identity Manager (vIDM)

04/27/2018 by William Lam 1 Comment

When configuring Ops Manager for VMware Pivotal Container Service (PKS) from an Authentication standpoint, you can either chose local authentication or use an external identity provider. The former means you are managing local users that reside within the User Account and Authentication (UAA) component of Ops Manager, which may be okay for a lab or proof of concept environment. However, for a Production deployment, most customers prefer to use their enterprise directory services which is typically Microsoft Active Directory.

Ops Manager can integrate with a number of external identity providers as long as it can speak SAML. For VMware customers, the preferred identity provider solution is VMware Identity Manager (vIDM) which not only supports Active Directory, but can also support a number of other directory service integrations like Active Directory Federation Services (ADFS) as example. Since vIDM supports SAML-based authentication, we can configure Ops Manager to use vIDM which also means we benefit from all of the enterprise Single Sign-On capabilities that vIDM delivers, including things like multi-factor authentication which can provide an additional layer of security when connecting to your PKS infrastructure.

Since there is currently no documentation on how to set this up, with the help of my colleague Blair Fritz and Assaf from the vIDM Engineering team, we have documented the process below which outline the required steps to integrate Ops Manager with vIDM.