virtuallyGhetto

William Lam

Setup custom login banner when logging into a vSphere with Kubernetes Cluster

by Leave a Comment

While working on my PowerCLI module for enabling workload management for a vSphere with Kubernetes (K8s) Cluster, I came to discover a pretty cool feature that is only available when using the vSphere with K8s API to enable Workload Management on a vSphere Cluster.

As part of the enablement spec, there is a new property called login_banner. Taking a closer look, this property allows you to specify a custom message that would be displayed as part of the initial login to your vSphere with K8s Cluster using the vSphere kubectl plugin. This is similar to an SSH login banner which can be used to provide internal disclaimers and/or additional instructions for your end users.

Here is an example of what the login banner can look like. Yup, vSphere with K8s supports emojis or rather the terminal you are using to login can potentially render emojis 😀


The good news is that I have already added this feature into the new New-WorkloadManagement function and you can specify a message by adding the -LoginBanner parameter.

For those interested in rendering emojis within their banner, you can take a look at the following example and you can find the complete list of emoji unicodes here.

Workload Management PowerCLI Module for automating vSphere with Kubernetes

by Leave a Comment

One of the last things on my to-do list after creating my Automated vSphere 7 and vSphere with Kubernetes Lab Deployment Script which is still the quickest and most reliable way to have a fully deployed and configured environment to try out vSphere with Kubernetes using Nested ESXi, was to also automate the enablement of Workload Management for a given vSphere Cluster.

There are two new vCenter Server REST APIs to be aware of as it pertains to vSphere with Kubernetes:

  • namespaces = Manages the lifecycle and access control to a vSphere Namespace
  • namespace-management = Despite the name, this refers to lifecycle and management of a Workload Management Cluster

I also have to mention that Vikas Shitole, who works on vCenter Server, has fantastic blog series covering various parts of the new vSphere with Kubernetes API along with Python examples if you want to dive further. Since Vikas has done a great job covering Python, I figure I will demonstrate how to consume these new vSphere with Kubernetes API using PowerCLI, which many of our customers use to automate.

I have created a new WorkloadManagement.psm1 PowerCLI module which includes following functions:

  • Get-WorkloadManagement
  • New-WorkloadManagement
  • Remove-WorkloadManagement

Below are the two steps required to get started with the Workload Management PowerCLI Module.

Step 1 - Download and import the WorkloadManagement PowerCLI Module by running the following command:

Import-Module ./VMware.WorkloadManagement.psm1

Step 2 - A connection to the vCenter REST API endpoint using the Connect-CisServer cmdlet is required for enabling and disabling Workload Management Cluster

Connect-CisServer -Server pacific-vcsa-2.cpbu.corp -User *protected email* -Password VMware1!

A connection to vCenter Server using Connect-VIServer cmdlet is only required if you wish to retrieve information about an existing Workload Management Cluster

Connect-VIServer -Server pacific-vcsa-2.cpbu.corp -User *protected email* -Password VMware1!

[Read more...] about Workload Management PowerCLI Module for automating vSphere with Kubernetes

How to configure network proxy with Tanzu Kubernetes Grid (TKG)?

by Leave a Comment

Network Proxies are commonly used by customers to provide connectivity from internal servers/services to access external networks like the Internet in a controlled and secured manner. While working on a recent network proxy enhancement for our VMware Event Broker Appliance (VEBA) Fling, I had setup a Squid server which is a popular network proxy solution.

I had noticed a couple of folks were asking about network proxy configuration for Standalone Tanzu Kubernetes Grid (TKG) and figure this might be interesting to explore, especially for my recently released TKG Demo Appliance Fling which enables folks to quickly go from zero to Kubernetes in just 30 minutes! I figured this would be another good opportunity to learn a bit more about TKG as well as Kubernetes (K8s) and I jokingly said to myself, how hard could this be!? 😉 Apparently it was not trivial and took a bit of trial/error to figure out the correct combination and below is the procedure that can be followed for both standard deployment of TKG as well as the TKG Demo Appliance Fling.

Proxy Setting configurations for TKG CLI

The TKG CLI uses KinD (Kubernetes in Docker) under the hood to setup the initial K8s bootstrap cluster to deploy the TKG Management Cluster. If you have not already downloaded KinD node image (registry.tkg.vmware.run/kind/node:v1.17.3_vmware.2) or if you need to go through a network proxy to do so, then the following instructions can be followed to make your Docker Client aware of a network proxy.

Here is an example of the error if Docker Client can not download the image:

# docker pull registry.tkg.vmware.run/kind/node:v1.17.3_vmware.2
Error response from daemon: Get https://registry.tkg.vmware.run/v2/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)

If you are not using a private container registry with TKG, then you also need to also ensure that the KinD Cluster can connect to your network proxy when it pulls down the required containers from the internet. Luckily, KinD can simply detect the network proxy settings of your operating system. You can either set the proxy using traditional environmental variables (http_proxy, https_proxy and no_proxy) during your use of TKG CLI or you can simply set it globally so you do not forget.

In my setup, TKG CLI is running in a Photon OS VM and global proxy settings are configured in /etc/sysconfig/proxy Proxy settings will vary across operating systems and you should check with the vendor documentation for specific instructions. The following command will set both HTTP and HTTPS proxy variables to use my proxy server and you will also want to make sure you whitelist all networks and addresses which you want to by-pass the proxy.

cat > /etc/sysconfig/proxy << EOF
PROXY_ENABLED="yes"
HTTP_PROXY="http://192.168.1.3:3128"
HTTPS_PROXY="http://192.168.1.3:3128"
NO_PROXY="localhost,192.168.1.0/24,192.168.2.0/24,registry.rainpole.io,10.2.224.4,.svc,100.64.0.0/13,100.96.0.0/11"
EOF

Note: If you are using the TKG Demo Appliance, you only need to configure the Photon OS global proxy settings. In my example, I have white listed my local 192.168.* addresses, registry.rainpole.io which is the embedded Harbor registry, 10.2.224.4 which is the internal IP Address of VMC vCenter Server, *.svc addresses which all the internal K8s services and 100.64.0.0/13 which is the CIDR range used by TKG for the Service networks and 100.96.0.0/11 which is the CIDR range used by TKG Cluster networks.

[Read more...] about How to configure network proxy with Tanzu Kubernetes Grid (TKG)?

Guest Customization support for Instant Clone in vSphere 7

by 1 Comment

vSphere Instant Clone was re-architected back in vSphere 6.7 and has been enhanced to be made more powerful and flexible. These enhancements not only power solutions like VMware Horizon but it also unlocks new customer use cases including things like Instant Cloning of Nested ESXi and Apple MacOS Guests.

Although the possibilities are truly endless with Instant Clone, this also means that any customization including basic guest identity such as hostname and networking must now use an alternative workflow. For application-level customization, it is expected that customers will create and manage these custom scripts but for basic networking configuration, it would be ideal to leverage the existing and well known vSphere Guest Customization Engine.

While downloading a file from MyVmware the other day, I came across an interesting set of packages called Guest Customization Engine for Instant Clone. Upon further investigation, I came to learn that these guest packages actually enable support for native vSphere Guest Customization for Instant Clone in vSphere 7 for the following Linux guest OSes:

  • CentOS 7.4 or higher
  • RHEL 6.8 or higher
  • RHEL 7.4 or higher
  • Ubuntu 16.04
  • SUSE 11SP4
  • SUSE 12SP3 or higher

In addition, there is also new set of vSphere (SOAP) APIs that you will need to interact with to use the new Instant Clone Guest Customization feature. The GuestCustomizationManager is a new vSphere 7.0 API which includes the following three API methods:

  • AbortCustomization_Task
  • CustomizeGuest_Task
  • StartGuestNetwork_Task

If you are interested in taking advantage of the new Instant Clone Guest Customization in vSphere 7, you can refer to the official VMware documentation which has step by step instructions.

How to passthrough USB Keyboard/Mouse HID and CCID devices to VM in ESXi?

by 10 Comments

About a month back I had received an interesting tidbit from Darius Davis (VMware Engineer) after helping a customer solve an interesting problem and Darius thought this could be a useful blog post to share. Funny enough, a couple of weeks after that conversation, a simliar issue was being faced by another customer and luckily I was able to share with them the solution and also validate the specific configuration that was needed.

The customer that Darius was helping out had two VMs running on ESXi which they wanted to configure several passthrough devices. In addition to a PCI passthrough of a GPU, they also wanted to passthrough independent USB keyboard and mouse to each individual VM. PCI passthrough to a VM is nothing new but passing through a USB keyboard/mouse also known as Human Interface Devices (HID) to a VM is generally not expected. The physical ESXi host just assumes these type of USB devices are meant for it to consume.

In addition to HID USB devices, there are also Chip Card Interface Devices (CCID) USB devices like a smart card reader which customers may also want to passthrough to a VM. The latter use case was what I ended up helping the customer out with. To passthrough HID/CCID USB devices, the following steps are required which will include changes to the ESXi host.

Step 1 - Add the following two VM Advanced Settings for all USB CCID/HID devices that you wish to enable passthrough:

HID USB Devices:

usb.generic.allowHID = "TRUE"
usb.quirks.device0 = "0xXXXX:0xYYYY allow"

CCID USB Devices:

usb.generic.allowCCID = "TRUE"
usb.quirks.device0 = "0xXXXX:0xYYYY allow"

where 0xXXXX = vendorId and 0xYYYY = deviceId (e.g 0x03f0:0x0024) which was retrieved from the previous step

To list all USB devices and get their vendor and device ID, you can use the lsusb command found within the ESXi Shell and below is an example listing out both my USB Mouse and Keyboard.

Step 2 - We need to make the USB arbitrator service aware of these USB device quirks by adding the usb.quirks.deviceN string to /etc/vmware/config file. In my example above, I want to enable passthrough for both my Mouse and Keyboard device, so the following entries would be appended:

usb.quirks.device0 = "0x05fe:0x0011 allow"
usb.quirks.device1 = "0x046d:0xc31d allow"

Step 3 - Lastly, we need to add the following string to the ESXi boot option to disable the VMkernel from claiming HID USB devices.

CONFIG./USB/quirks=0xXXXX:0xYYYY::0xffff:UQ_KBD_IGNORE

where 0xXXXX = vendorId and 0xYYYY = deviceId (e.g 0x03f0:0x0024)

The easiest way to append this to the boot option is by editing /bootbank/boot.cfg rather than manually typing this during the initial boot up (SHIFT+O)

Note: This is not required for CCID devices or mouse devices as they are not claimed by ESXi

Step 4 - A system reboot of your ESXi host will be required for the changes to go into effect. Once your ESXi host is available, you can use the vSphere H5 Client or Embedded ESXi Host Client to attach the USB devices. For vCenter Server, click on "Add New Device" and select "Host USB Device" and for ESXi, click on "Add other device" and select "USB Device".

New VEBA release, new website and new mascot!

by Leave a Comment

Today I am very happy to share a number of updates with the community regarding the popular VMware Event Broker Appliance (VEBA) Fling. Each release has always been a team effort, but but I am especially proud of this release as it demonstrates how large the team has grown in the past 6 months and their impactful contributions to this solution to help our VMware customers and partners. Michael and I could not be more prouder and the feedback both internally and externally has been nothing but amazing and we are just getting started when it comes to event-driven automation for the SDDC! 

New VEBA Release

Here are some of the key features in our latest v0.4 release. If you wish to see a detailed change log, please refer to the VEBA github releases page.

Below are two features that I think is worth highlighting:

Thanks to Frankie Gold, we now have a slick new VEBA DCUI which replaces the old static /etc/issue entry which was only updated once after a successful deployment. If you decided to change the hostname, these changes would not be reflected. The new VEBA DCUI is dynamic and will display the latest configuration from the system including the configured system resources. In addition, it also uses the new /etc/veba-release file found within VEBA appliance which provides information about the version of VEBA, commit ID along with the event processor that was configured.


As part of the DCUI development planning, I was reminded of this fun little VMware Easter Egg. I thought it would be fun to include a few of our own and also give a nod back to this old school easter egg which sadly is no longer in the product. The default color scheme is green (cyan) and if you go into the VM Console and type "veba", you will activate an alternate color scheme as shown in the screenshot below. To return to the original color scheme, just type "veba" again to deactivate.


There is actually a couple more interesting 🐣easter eggs which I had asked Frankie to include ... I wonder who will be the first to find and share them? Maybe the first few folks who share details about the easter egg on it Twitter will get one of the new VEBA sticker!

UPDATE (05/13/20) - Congrats to Allan Kjaer on finding the first VEBA DCUI Easter Egg #1 which is typing the word "pride" (this is a nod back to the original easter egg found in Fusion, see reference above)

Congrats David Bibby on finding the second and final VEBA DCUI Easter Egg #2 which is typing the word "otto" (name of VEBA's mascot) which will activate VEBA DCUI console with rendering of Otto 🙂 To deactivate and return to the default screen, simply type "otto" again.

[Read more...] about New VEBA release, new website and new mascot!