Arm

How to build a customizable Raspberry Pi OS Virtual Appliance (OVA)?

11/16/2020 by William Lam Leave a Comment

After posting the instructions on how to install Raspberry Pi (rPI) OS into a Virtual Machine running on ESXi-Arm, I was already thinking about an easier consumption method that not only benefited VMware customers interested in running rPI OS as a VM but also the larger rPI OS development community. Just imagine, you can now easily deploy, build and test multiple rPI OS/application on a single physical rPI and get all the benefits of vSphere that many customers have enjoyed for the past two decades.

My goal was to build an rPI OS OVA that would enable some basic guest customization such as networking and configuring the password for the default pi user. As you can see from the screenshot below, I was able to accomplish this with minimal trial/error and works fantastic!

I was initially planning to release the rPI OS OVA as a VMware Fling which can then be made available to the community. However, due challenges in the way rPI OS is distributed today via an image file and the inclusion of packages that makes it difficult for redistribution, I decided to forgo the VMware Fling route and simply publish the instructions with some supplemental scripts that can be used to produce the same rPI OS OVA that I have built for my own personal use.

It would have been great if this could be made available and if anyone from Raspberry Pi organization is reading this and is interested in hosting the download, I would be more than welcome to provide you with OVA file.

Stateless ESXi-Arm with Raspberry Pi

11/03/2020 by William Lam 11 Comments

I am super excited to be able to finally share, what I think, is a really cool ESXi-Arm solution which has been an evolution of this and this. This solution also incorporates a number of automation techniques I have shared over the years when it comes to ESXi scripted installation aka Kickstart, so it was really neat to all those things get pulled into a single solution. Lastly, I also want to give huge thanks to Cyprien Laplace who threw the initial challenge my way after I had shared how to perform an ESXi-Arm scripted installation without using SD Card.

ESXi-x86 can be deployed using either a stateful or stateless installation. In the latter case, ESXi is booted over the network using the vSphere Auto Deploy feature in vCenter Server which does not require any local media for ESXi. Upon attaching itself to vCenter Server, Auto Deploy then leverages vSphere Host Profiles and its rules engine to determine which configurations or profiles should be applied to ensure the ESXi hosts are configured per their desired stated. Here is a quick video overview of how Auto Deploy and Host Profiles work.

Fundamentally, vSphere Auto Deploy and Host Profiles can also work with ESXi-Arm but today, vCenter Server would require some code modification for this to actually work.

OK, so am I teasing you with something that does not exists? Nope, but I just wanted to help set the context 🙂

The solution that I have created boots ESXi-Arm over the network in a "stateless" manner, so there is no need for an SD Card or USB device plugged into the Raspberry Pi (rPI). In addition to the ESXi-Arm files, it also includes a custom payload which runs to retrieve additional configurations which can automatically join a desired vCenter Server as well as apply further customizations of an ESXi-Arm host. As you can see, this solution behaves similar to that of vSphere Auto Deploy and Host Profiles but does not use either of these vSphere features and works with the ESXi-Arm Fling right now.

Technically speaking, these techniques can also be applied to ESXi-x86 but I will leave that to the reader for further exploration.

Automated network scripted installation of ESXi-Arm without SD Card

10/23/2020 by William Lam 2 Comments

A topic that I have been working on even before the release of the ESXi-Arm Fling is the ability to perform a network scripted installation (Kickstart) of ESXi-Arm for the Raspberry Pi (rPI) but doing so without the need of an SD Card plugged into the rPI itself. The latter point is critical as today, the SD Card is used to house the required UEFI files to allow the rPI to boot the ESXi-Arm installation files which can be served from a local USB device or remotely over the network using HTTP, NFS or even iSCSI boot, which Andrei had recently blogged about.

Running through the SD Card preparation is not a difficult process and if you only have a single ESXi-Arm host, this may not be all that interesting other than learning about how this works and setting up a basic Kickstart environment. However, if you have several rPI and maybe you do not have spare SD Card and you prefer to make it easy to deploy additional ESXi-Arm host, this is a pretty cool solution. The precursor to this work was actually from a blog post I had published a few weeks ago on copying the rPI UEFI files and booting ESXi-Arm off of a USB device.

Once I figured out how that worked, it was simply figuring out the automation required during the %post section of a scripted installation of ESXi-Arm to pull down a copy of the UEFI files which is then copied onto the first partition of the USB device and thus allowing you to completely boot off of the USB device after installation. This took some trial and error playing around with mcopy which is a tool I have written about to help copy files to and from a USB device using the ESXi Shell. The other trick that we are taking advantage of here is that the USB device that you intend to use are mostly the same from a UEFI point of view and by disabling all other boot option, we ensure that after the UEFI files are copied over to ESXi-Arm host, it will boot from USB device rather than over the network.

Packer reference for PhotonOS Arm NFS Virtual Appliance using OVF properties for ESXi-Arm

10/21/2020 by William Lam 2 Comments

In case the title was not descriptive enough, I was curious if I could build an Arm Virtual Appliance (OVA) using OVF properties that would allow for all sorts of interesting guest customizations which I have blogged about before here, here, here and here using x86 PhotonOS as a reference implementation. My idea for this actually pre-dated the release of the ESXi on Arm Fling, but it was only until recently with support for VMware Tools for Photon OS Arm, was I able to finally piece together this solution.

It was also neat to see that I could build an Arm OVA using x86 tooling (Packer and OVFTool) which ran on my desktop and you simply needed an ESXi-Arm host. This really goes to show the level of compatibility from a management and vSphere API point of view that an ESXi-Arm host behaves just like a standard x86 ESXi host!

and successfully deployed maybe the “1st” OVA on #ESXionARM?

Just confirmed all guest customization via OVF properties executed correctly! Will be publishing reference Packer image in case you wish to build your own pic.twitter.com/PiSpceXtFF

— William Lam (@lamw) October 15, 2020

To demonstrate a more interesting use case than just basic network customization for the ?first? Arm OVA, I thought it would be useful to setup a simple NFS appliance that would take input from the user such the size of the exported volume (default 60GB) and then the name of the mount point. Upon first boot up, there is a guest customization script that would read in the OVF properties and configure the networking, OS password and NFS server configuration which you can certainly use to host your Arm VMs.

Kubernetes on ESXi-Arm using k3s

10/16/2020 by William Lam 10 Comments

The tiny form factor of a Raspberry Pi (rPI) is a fantastic hardware platform to start playing with the ESXi-Arm Fling. You can already do a bunch of fun VMware things like running a lightweight vSAN Witness Node to setting up basic automation environment for PowerCLI, Terraform and Packer to running rPI OS as VM, enabling some neat use cases like consolidating your physical rPI assets which might be running RetroPi and Pi-Hole which many home labbers are doing.

In addition to VMware solutions, its is also a great platform to learn and tinker with new technologies like Kubernetes (K8s) which I am sure many of you have been hearing about 🙂 Although our vSphere with Tanzu and Tanzu Kubernetes Grid (TKG) does not currently work with the ESXi-Arm Fling, I have actually been meaning to try out a super lightweight K8s distribution designed for IoT/Edge called k3s (pronounced k-3-s) which also recently joined Cloud Native Computing Foundation (CNCF) Sandbox level.

k3s is supported on rPI and you normally would have multiple rPI devices to represent the number of nodes, for example if you want a basic 3-Node cluster, you would need three physical rPI devices. With ESXi-Arm, you can now create these nodes as VM, using just a single rPI. This opens up the door for all sorts of explorations, you can create HA cluster or try out more advanced features which might be more difficult if you needed several physical devices. If you mess up, you can simply re-deploy the VM without much pain or simply clone the VM.

In my setup, I am using 3 x Photon OS VMs. One for the primary node and two for k3s worker nodes. You can certainly install k3s on any other Arm-based OS including rPI OS (which can now run as a VM as mentioned earlier).

Installing VMware Tools on Raspberry Pi OS for ESXi-Arm

10/15/2020 by William Lam 3 Comments

Now that you have Raspberry Pi (rPI) OS running as a VM on ESXi-Arm, the next thing you will probably want to install is VMware Tools, especially useful to see the IP Address of your Guest if you are using DHCP and for enabling guest "soft" shutdown using the vSphere UI. Below are the instructions