Arm

VEBA + Knative + k3s on ESXi-Arm

01/26/2021 by William Lam Leave a Comment

In response to a customer request to add Arm64 support for our VMware Event Router, I have been spending some more time playing with k3s (lightweight Kubernetes distribution for Arm) running on ESXi-Arm using a Raspberry Pi. Not only was this a good learning experience that exposed to me to the broader Arm ecosystem, which is still maturing but it also took me down several 🐰🕳️ which got me exploring new tools that I had never used before such as Buildpacks and Docker buildx to name a few.

This past weekend, I was finally successful in setting up our VMware Event Router for Arm using the Knative processor on a k3s cluster using ESXi-Arm running on a Raspberry Pi 4b 8GB model! As of writing this, the following versions were used:

Knative Serving v0.20.0
Knative Net Contour v0.20.0
Knative Eventing v0.20.1
RabbitMQ Cluster Operator v0.5.0

Made some more progress w/@KnativeProject + @VMWEventBroker on k3s on @esxi_arm

✅ Knative Serving & Eventing
✅ @RabbitMQ Operator & Eventing
✅ @projectcontour
✅ @VMware Event Router

Just need to figure out @buildpacks_io for Arm64 - https://t.co/ChdkMLSXMp looks promising pic.twitter.com/XFWDiGONSB

— William Lam (@lamw) January 24, 2021

In addition, I was able to also convert the Knative python echo function that was originally created by my colleague Michael Gasch and build an Arm64 version of the Knative python echo function which demonstrates the integration of VEBA with the Knative processor connected to a vCenter Server as my event source.

🥳 Successfully deployed & verified my arm64 python echo func w/@VMWEventBroker (Event Router) using the @KnativeProject processor!

Awesome for lightweight testing/development purposes on small VM w/k3s on @esxi_arm

Heck, don’t even need real vCenter, can run vcsim locally! pic.twitter.com/DuI16fvXfs

— William Lam (@lamw) January 24, 2021

For those interested in just the VMware Event Router Arm64 image, you can access it here and we plan to make that an official image shortly. For those interested in setting up a fully functional Arm deployment of VEBA and Knative processor, you can find the detailed instructions below.

ESXi-Arm Licensing Options

12/12/2020 by William Lam Leave a Comment

The the adoption of the ESXi-Arm Fling has skyrocketed since its initial launch back in October and has already surpassed over 10K+ downloads in its first month of release!

Just got an update from the @vmwflings team that the #ESXionARM Fling has already surpassed 10K+ downloads since its release! 🤯

Thank you to everyone in the community who’s participated, especially those who’ve shared their feedback & those doing some really stuff with it!

— William Lam (@lamw) November 18, 2020

One interesting observation that I have noticed lately has been an increased in inquiries related to licensing ESXi-Arm. This week alone, I have seen this question come up at least a dozen plus times across various communication channels.

How to run Kali Linux on ESXi-Arm

11/30/2020 by William Lam 14 Comments

A few folks have been trying to get Kali Linux, a Debian-derived Linux distribution designed for digital forensics and penetration testing, to run on ESXi-Arm. Similiar to Raspberry Pi OS, the techniques outlined in this blog post can be used to convert the image-based system to a Virtual Machine that ESXi-Arm can then run natively. However, there is an issue when attempting to install grub during Step 16 where it can not locate an EFI partition.

With the help of Cyprien Laplace, we can now run Kali Linux on ESXi-Arm! You can find the detailed instructions below including installing VMware Tools for Kali Linux running on ESXi-Arm.

How to build a customizable Raspberry Pi OS Virtual Appliance (OVA)?

11/16/2020 by William Lam 6 Comments

After posting the instructions on how to install Raspberry Pi (rPI) OS into a Virtual Machine running on ESXi-Arm, I was already thinking about an easier consumption method that not only benefited VMware customers interested in running rPI OS as a VM but also the larger rPI OS development community. Just imagine, you can now easily deploy, build and test multiple rPI OS/application on a single physical rPI and get all the benefits of vSphere that many customers have enjoyed for the past two decades.

My goal was to build an rPI OS OVA that would enable some basic guest customization such as networking and configuring the password for the default pi user. As you can see from the screenshot below, I was able to accomplish this with minimal trial/error and works fantastic!

I was initially planning to release the rPI OS OVA as a VMware Fling which can then be made available to the community. However, due challenges in the way rPI OS is distributed today via an image file and the inclusion of packages that makes it difficult for redistribution, I decided to forgo the VMware Fling route and simply publish the instructions with some supplemental scripts that can be used to produce the same rPI OS OVA that I have built for my own personal use.

It would have been great if this could be made available and if anyone from Raspberry Pi organization is reading this and is interested in hosting the download, I would be more than welcome to provide you with OVA file.

Stateless ESXi-Arm with Raspberry Pi

11/03/2020 by William Lam 18 Comments

I am super excited to be able to finally share, what I think, is a really cool ESXi-Arm solution which has been an evolution of this and this. This solution also incorporates a number of automation techniques I have shared over the years when it comes to ESXi scripted installation aka Kickstart, so it was really neat to all those things get pulled into a single solution. Lastly, I also want to give huge thanks to Cyprien Laplace who threw the initial challenge my way after I had shared how to perform an ESXi-Arm scripted installation without using SD Card.

ESXi-x86 can be deployed using either a stateful or stateless installation. In the latter case, ESXi is booted over the network using the vSphere Auto Deploy feature in vCenter Server which does not require any local media for ESXi. Upon attaching itself to vCenter Server, Auto Deploy then leverages vSphere Host Profiles and its rules engine to determine which configurations or profiles should be applied to ensure the ESXi hosts are configured per their desired stated. Here is a quick video overview of how Auto Deploy and Host Profiles work.

Fundamentally, vSphere Auto Deploy and Host Profiles can also work with ESXi-Arm but today, vCenter Server would require some code modification for this to actually work.

OK, so am I teasing you with something that does not exists? Nope, but I just wanted to help set the context 🙂

The solution that I have created boots ESXi-Arm over the network in a "stateless" manner, so there is no need for an SD Card or USB device plugged into the Raspberry Pi (rPI). In addition to the ESXi-Arm files, it also includes a custom payload which runs to retrieve additional configurations which can automatically join a desired vCenter Server as well as apply further customizations of an ESXi-Arm host. As you can see, this solution behaves similar to that of vSphere Auto Deploy and Host Profiles but does not use either of these vSphere features and works with the ESXi-Arm Fling right now.

Technically speaking, these techniques can also be applied to ESXi-x86 but I will leave that to the reader for further exploration.

Automated network scripted installation of ESXi-Arm without SD Card

10/23/2020 by William Lam 5 Comments

A topic that I have been working on even before the release of the ESXi-Arm Fling is the ability to perform a network scripted installation (Kickstart) of ESXi-Arm for the Raspberry Pi (rPI) but doing so without the need of an SD Card plugged into the rPI itself. The latter point is critical as today, the SD Card is used to house the required UEFI files to allow the rPI to boot the ESXi-Arm installation files which can be served from a local USB device or remotely over the network using HTTP, NFS or even iSCSI boot, which Andrei had recently blogged about.

Running through the SD Card preparation is not a difficult process and if you only have a single ESXi-Arm host, this may not be all that interesting other than learning about how this works and setting up a basic Kickstart environment. However, if you have several rPI and maybe you do not have spare SD Card and you prefer to make it easy to deploy additional ESXi-Arm host, this is a pretty cool solution. The precursor to this work was actually from a blog post I had published a few weeks ago on copying the rPI UEFI files and booting ESXi-Arm off of a USB device.

Once I figured out how that worked, it was simply figuring out the automation required during the %post section of a scripted installation of ESXi-Arm to pull down a copy of the UEFI files which is then copied onto the first partition of the USB device and thus allowing you to completely boot off of the USB device after installation. This took some trial and error playing around with mcopy which is a tool I have written about to help copy files to and from a USB device using the ESXi Shell. The other trick that we are taking advantage of here is that the USB device that you intend to use are mostly the same from a UEFI point of view and by disabling all other boot option, we ensure that after the UEFI files are copied over to ESXi-Arm host, it will boot from USB device rather than over the network.