Docker

Configure non-secure Harbor registry with Tanzu Kubernetes Grid (TKG)

05/09/2020 by William Lam 3 Comments

In an earlier blog post, I shared the steps to to configure Harbor with a proper signed SSL certificate that would serve as private container registry for Tanzu Kubernetes Grid (TKG) CLI running in an air-gapped environment.

Although Harbor can easily be configured to support custom CA signed certificate, self-sign certificate and even just using HTTP, there are several additional steps and dependencies that is required if you wish to use a non-secure container registry with TKG CLI. This definitely was a bunch of trial/error and hopefully this can be made easier in the future to easily enable non-secure registry support with TKG CLI out of the box for development and testing purpose.

I also want to give a huge thanks to Jun Wang from our Modern Application Business Unit (MAPU), he was instrumental in helping me out and ultimately his tip on updating the containerd configuration was the last piece to the puzzle so that the K8s images deployed would use our insecure Harbor registry for pulling container images.

Deploy Harbor in an Air-Gapped environment for Tanzu Kubernetes Grid (TKG)

04/24/2020 by William Lam Leave a Comment

When using Tanzu Kubernetes Grid (TKG) and the new TKG CLI, outbound internet connectivity is required as part of the initial setup on the machine running TKG CLI but also on the TKG Management Cluster which is automatically stood up as part of the deployment. For demo and testing purposes, this is usually not a problem but for anyone looking to run this in a Production or datacenter environment, direct internet access is generally not available.

TKG does support air-gapped environments today by requiring a private container registry that has been configured with all the required containers. Once your registry has been setup, you will also need to update the TKG YAML manifest files to specify your private registry as by default, it will point to registry.tkg.vmware.run. You can use any container registry that is supported with Kubernetes including the popular Harbor solution. One thing to note is that your private registry must have a proper signed SSL certificate, custom CA certificates or self-signed certificates are not officially supported today with TKG.

Since I recently had to set this up for a project I am working on, which I hope to talk about in a future blog post, I thought it would be useful to share the instructions on how to setup and configure Harbor to be used in-conjunction with TKG as well as any other solution that requires a container registry running in your own environment. In my deployment, I will be using Let's Encrypt for generating the required SSL certificate, but you can use any existing service for performing this operation. I will also be installing Harbor on Photon OS, but you can use any operating system of your choice that Harbor is supported on.

Pre-Requisites

Access to a public DNS domain which you have ownership of (e.g. adding new records)
Access to your internal DNS server to add a custom DNS zone lookup entry (e.g. registry.<yourdomain>.com)

Configuring Github Actions self-hosted runners on PhotonOS

12/17/2019 by William Lam Leave a Comment

Ever since Github announced Github Actions, which is now generally available for everyone, I have been a huge fan of the service. I even shared a blog post earlier this year on how you can easily incorporate automated application deployment to a vSphere or VMware Cloud on AWS based environment, which can automatically be triggered by native developer workflows directly from Github. This can be a really powerful and enabling capability for your developers, especially when taking advantage of an on-demand solution like VMware Cloud on AWS.
Right before VMworld Barcelona, I saw a tweet from the Github Twitter account announcing another cool feature which is the ability to run your own self-hosted runners. By default, when you use Github Actions, the runners are hosted by Github and when a Docker Container is launched, it is running within their infrastructure. During the beta, I had noticed some inconsistencies on how long it would take my Github Actions to kickoff which is usually within a minute or so but I have seen cases where it has gone up 5 to 10 minutes.

I was told that this was an infrastructure issue, but it did raise an interesting question in my mind on SLAs. As far as I know, nothing is publicly documented and Github also mentioned they did not have an SLA for the service. If you need a more predictable experience, you now have the option of running the "runners" in your own infrastructure which can be on-premises environment or even a public cloud where you have available compute capacity.

I finally got a chance to explore this capability and of course, I had to figure out how to get this working with our very own VMware PhotonOS. With a bit of trial and error, I was able to get everything working. In fact, I was able to run my Github runner directly in my VMware Cloud on AWS environment which can be quite useful for customers with development and CI/CD-based workloads and being able to leverage Github Actions.

Integrating Github Actions with vSphere and VMware Cloud on AWS

04/01/2019 by William Lam 2 Comments

I have always been a fan of event-driven automation, the idea where you can automatically trigger a workflow or an operation based on a specific event. In the consumer world, the most popular example is the If This, Then That (IFTTT) service, which I use on a regular basis to automate the sharing of new articles from virtuallyGhetto to different Social Media channels.

For the Enterprise, this is also not a new idea and many folks including myself have been doing this for years in vSphere using vCenter Server Alarms. In fact, one example I still reference on a regular basis is from 2012 where you automatically apply a set of vSphere Security Hardening configurations to a Virtual Machine when a new VM Create Event is published by vCenter.

There are countless more examples of this concept beyond VMware but the general idea is to be able to subscribe to specific events and then automatically do something when a given event occurs. When Github Actions (Beta) was announced last year, I was really interested as I think this could open the door for a ton of interesting possibilities, especially from a VMware perspective around Continuous Integration/Development (CI/CD). I quickly registered for the Beta but did not get access until the start of this year. If you want to know what Github Actions can do, check out some of these demos that have been built by various folks from the community. The really exciting thing about Github Actions is that you can literally execute any workflow as long as you can containerized your business logic within a Docker Container. This means, you can use any language or tool that you are familiar with and make this work with Github Actions, pretty powerful stuff!

It was only recently while working on a personal project, which I hope will make its way to VMworld, that I finally got a chance to dig into Github Actions. I noticed in many of the online Github Action examples, that it included ways to deploy applications and containers to a Public Cloud but there was nothing that I found related to VMware. I figured, this would be a good learning opopournity for myself and I could even learn how to build my own Actions which can be useful for others to use or extend further.

Quick Tip – Creating a multiline Dockerfile using heredoc w/variable substitution

04/26/2017 by William Lam 1 Comment

I was helping out a fellow colleague yesterday who was having some troubles handling a multiline echo statement within his Dockerfile. There are multiple ways in which you can create multiline Dockerfiles, the web is full of examples from using multiple echo statements (pretty ugly) to using heredocs which is easier to read and manage. The challenge was that he also wanted to substitute some variables into his multiline statement which apparently there were no examples online, at least neither of us could find.

Taking a closer look, I found that we can just leverage Bash's ANSI-C Quoting syntax $'string' to do what we want, which was actually something new to me as well. You can then pass in the variable like you normally would between the strings and that would give you the readability of heredocs and still be able to use Docker variables. I am sure there are other methods with more extensive escapes with single-ticks, but I also prefer a solution that is easy to read and use in case others need to manage it.

Here is a quick sample Dockerfile which demonstrates how this works:

FROM photon:1.0

ARG BASEURL="https://vmware.bintray.com/powershell"

RUN echo $'[powershell]\n\
name=VMware Photon Linux 1.0(x86_64)\n\
baseurl='$BASEURL$'\n\
gpgcheck=0\n\
enabled=1\n\
skip_if_unavailable=True\n '\
>> /etc/yum.repos.d/powershell.repo

CMD ["/bin/bash"]

FROM photon:1.0

ARG BASEURL="https://vmware.bintray.com/powershell"

RUN echo $'[powershell]\n\

name=VMware Photon Linux 1.0(x86_64)\n\

baseurl='$BASEURL$'\n\

gpgcheck=0\n\

enabled=1\n\

skip_if_unavailable=True\n '\

>> /etc/yum.repos.d/powershell.repo

CMD ["/bin/bash"]

Basically the echo statement has $'SOME-STRING'$VARIABLE$'SOME-STRING'

If we build and run this Docker image, we can see that we have properly substituted the BASEURL variable into our file as seen in the screenshot below.

docker build -t sample .
docker run --rm -it sample cat /etc/yum.repos.d/powershell.repo

I personally prefer to keep such logic within a separate script which the Dockerfile can reference, but I was also sympathetic to that fact that my colleague wanted to keep things simple and just have everything within the Dockerfile. I figure I would share this in case other comes across this problem as well as benefiting myself as I will probably forget in a months time 🙂

Easily try out vSAN 6.6 Encryption feature using KMIP Docker Container

04/14/2017 by William Lam 4 Comments

One of biggest feature introduced in the upcoming vSAN 6.6 release is the native vSAN Data-at-Rest Encryption capability. My good friend Duncan Epping even posted a video recently demo'ing the feature and showing how easy it is to enable with just a couple of clicks. Just like VM Encryption which was introduced in vSphere 6.5, vSAN Encryption also requires a Key Management Interoperability Protocol (KMIP) Server which needs to be associated with your vCenter Server.

The really nice thing about this is that because both VM Encryption and vSAN Encryption uses the exact same encryption library, as long as you have a supported KMS (which you can find over on the VMware KMS HCL here, more are being certified and added), you can actually leverage the same KMS for both types of encryption across different vSphere Clusters with different requirements. For the ultra paranoid, you could even "double" encrypt by running Encrypted VMs on top of a vSAN Encrypted Datastore 😉

As with any feature that relies on 3rd party tools, it can take some time to acquire evaluational licenses. For those of you who would like to try out either vSAN or VM Encryption from a functional standpoint, you can quickly get started in under a few minutes by using the KMIP Docker Container that I had built last year. This is a great way to familiarize yourself with the workflow or even try out some of the new vSphere and vSAN APIs if you plan to automate the KMIP configuration or even deployment of encrypted VMs. Another great use case for this is doing live demos and all you need is just a couple of Nested ESXi VMs and a Docker Container Host like Photon OS or even just your laptop for example. Below are the instructions on how to get started.

Disclaimer: It is also very important to note that you should NOT be using this for any production workloads or any VMs that you care about. For actual production deployments of VM Encryption or vSAN Encryption, you should be leveraging a production grade KMIP Server as PyKMIP stores the encryption keys in memory and will be lost upon a restart. This will also be true even for the virtual appliance, so this is really for quick evaluational purposes, do NOT run anything important that you care about due to the risks mentioned earlier.