instant clone

Which VM was this vSphere VM cloned from?

01/11/2021 by William Lam Leave a Comment

This was a question that I saw back in December on the VMware {code} Slack which was quickly answered by the always awesome Luc Dekens. The solution is to look at vCenter Server Events, which are super rich in information and can be used for a number of things including identifying the source VM that it was cloned from. When I was a customer, this was something I did all the time, using events for auditing purposes but also identifying who, what and when a certain operation was performed including source VMs for cloning operations.

Although this information maybe known to some, there is still not an elegant solution that can help someone quickly identify the source VM for a specific vSphere VM that was cloned. This topic also intrigued me as I have seen this question come up in the past. I figure I might as well add this to my random scripting backlog and take a look when I had some time.

Before taking a look at the solution, it is important to understand the different types of clones that exists in vSphere today and also the respective vCenter Server events that can help us correlate to both the source VM but also the specific clone type.

Cloning Types

Full Clone - An independent copy of a virtual machine that shares nothing with the parent virtual machine after the cloning operation. Ongoing operation of a full clone is entirely separate from the parent virtual machine
Linked Clone - A copy of a virtual machine that shares virtual disks with the parent virtual machine in an ongoing manner. This conserves disk space, and allows multiple virtual machines to use the same software installation
Instant Clone - An independent copy of a virtual machine that starts executing from the exact running state of the source powered on virtual machine. Instant Clone uses rapid in-memory cloning of a running parent virtual machine and copy-on-write, simliar to that of Linked Cloning to rapidly deploy virtual machines

Instant Clone Apple MacOS

03/28/2019 by William Lam 1 Comment

Whether you are a brand new startup working on the next hot mobile app or an established Enterprise or Consumer brand company, development and testing of Apple iOS and/or MacOS is simply a reality in todays world. The vast majority of these customers accomplish this by running Apple MacOS on vSphere, either within their own on-premises datacenter or leveraging MacStadium, the largest MacOS Cloud hosting provider, who also runs their Mac infrastructure using VMware vSphere.

The ability to quickly build/test and deploy your application (Continuous Integration and Continuous Delivery) can mean the difference of having an edge over you competitor or being able to keep up with the demands of your business. Many customers have benefited from using vSphere platform and with technologies like Linked Clones, which allows you to quickly spin up a new VM without having to perform a complete full clone, it means you can build and test your application even faster.

In vSphere 6.7, we introduced a major enhancement to Instant Clone, which you can read more about here and here. One of the questions I have been seeing lately is whether Instant Clone can be applied to MacOS guests? The answer is absolutely! In fact, Matt Moriarity, who works for TravisCI, recently shared some tidbits on how to get a MacOS Mojave guests to see the updated MAC Address to ensure that there are no network conflicts when performing an Instant Clone.

The majority of the "hard" work to use Instant Clone is really from within the GuestOS and the customization script that needs to be developed. In fact, Instant Clone is pretty OS agnostics and you can even Instant Clone Microsoft Windows 98 and 2000, if you really wanted to 😀

Instant Clone Microsoft Windows & VM Keystroke VMworld demo and code posted

01/08/2019 by William Lam 2 Comments

Apologies for the delay in getting my VMworld 2018 demo and code posted online, I know a number of you have been asking about the Windows Instant Clone samples to get an idea on how to create your own customization scripts for managing more "recent" Microsoft Windows releases 😉 and perhaps you might even consider submitting a pull request to share with the community. I have posted both the videos and code samples below. Enjoy and happy Automating!

Instant Clone Microsoft Windows

To demonstrate the power of the newly re-architected Instant Clone feature in vSphere 6.7 and to help make the point clear that the Instant Clone feature is really Guest Operating System agnostic, meaning you can Instant Clone any to Virtual Machine that can run VMware Tools, I thought it would be fun to see how old of a Microsoft OS that I could Instant Clone. After a bit of trial/error, that turned out to be Windows 98 and Windows 2000 🙂

Windows 98 Demo

Windows 2000 Demo

Leveraging Instant Clone in vSphere 6.7 for extremely fast Nested ESXi provisioning

05/17/2018 by William Lam 18 Comments

The idea of "Instant Cloning" a Nested ESXi VM (running ESXi in a VM) is not a new concept. In fact, I had shared a solution back in 2015 using the private VMFork APIs. However, what has changed is the ease of consumption, primarily due to the re-architecture of Instant Clone in vSphere 6.7 (more details here and here) which resulted in a public and simplified API. Some of you might ask, why not simply clone a Nested ESXi VM or create a Link Clone? What benefit would I get by using Instant Clone?

The answer is not only speed, but the fact that the instantiated VM is fully operational and ready to start executing where as a traditional full clone or linked clone requires a full OS boot up that can take up to several minutes to deploy and configure. This may not sound like much for a small number of Nested ESXi VMs, but as you increase the number of instances, Instant Clone really shines while still maintaining speed and the instant availability of the VM. As you can imagine, this definitely opens up for some interesting use cases whether it be for personal home lab or educational purposes like VMware HOL. In addition, we also have customers who deploy Nested ESXi not only at high scale but also with a high churn rate for development purposes, think CI/CD type of a workload who can also benefit from Instant Clone.

So how fast are we talking about? Lets say you wanted to test out the latest version of VSAN in vSphere 6.7, you would normally deploy 3 Nested ESXi VMs, power them up and wait for them to be ready on the network. With Instant Clone, you can deploy three fully functional Nested ESXi VMs in just 30seconds! As the VMs are instantly available for consumption, you can start the VSAN enablement workflow immediately and even parts of that can be baked into the Instant Clone workflow. With the ease of provisioning Nested ESXi VMs, you can simply maintain a catalog of ESXi templates which are in "frozen" states and then leverage Instant Clone to deploy just-in-time Nested ESXi environments and discard them once you are done. Pretty slick if you ask me! and something I plan on using going forward.

Disclaimer: Nested ESXi is still not officially supported by VMware. Please use at your own risk.

New Instant Clone Architecture in vSphere 6.7 – Part 2

04/30/2018 by William Lam 4 Comments

In the previous article, I provided an overview of the new "Parentless" Instant Clone feature which was introduced in vSphere 6.7 and some of the architectural differences between prior versions of Instant Clone. In this post, I will show you how to use the new Instant Clone feature, which is currently only available with the vSphere API.

There are two important parts when using Instant Clone:

A user defined script which runs within the GuestOS and is responsible for customizing the network identity of the Instant Clone. This script is not just limited to network configurations but can also be used to customize other OS and/or application settings.
The deployment script which runs outside of the GuestOS and instantiates new Instant Clones using the vSphere API. This script is responsible for passing in data (network configuration, OS and/or application settings) to the GuestOS which can then be accessed directly by the user defined script running within each Instant Clone for actual customization.

To demonstrate the use of the Instant Clone API, I will be using PowerCLI and have also created an Instant Clone PowerCLI module called InstantClone.psm1. However, the API is not limited to just PowerCLI and can be consumed using any of the new vSphere 6.7 SDKs. For our example, we will be exercising the creating Instant Clones from a "Frozen" Source VM workflow since that is the most efficient method, especially as you scale to larger number of Instant Clones. For those that wish to experiment with the other workflow of creating Instant Clones from a "Running" Source VM, feel free to take a look at the previous post for the required steps.

For my setup, I will be using an Ubuntu Server 16.04 system as my Source VM and I will create 30 Instant Clones from this system. Each Instant Clone will immediately be available for use and will be fully customized and reachable from a network perspective (e.g. unique IP and MAC Address). Here is a screenshot of one of my deployments to give you an idea of how blazing fast Instant Clone is 🙂

Here is a short video that I just recorded that demos the Instant Clone APIs using PowerCLI (this demo is using an Ubuntu VM configured with 2GB memory where as the original screenshot was 1GB)

Demo of Instant Clone in vSphere 6.7 from lamw on Vimeo. [Read more...] about New Instant Clone Architecture in vSphere 6.7 – Part 2

New Instant Clone Architecture in vSphere 6.7 – Part 1

04/24/2018 by William Lam 6 Comments

Instant Clone or VMFork (as it is referred internally) has been around for a number of years now. It was initially available as part of vSphere 6.0 with the primary consumer being Horizon View and their just-in-time desktop solution. Although Instant Clone was part of the core vSphere platform, public APIs were not available for external consumption. Many customers were interested in the technology to enable other non-VDI use cases such as Dev/Test, Continuous Integration/Continuous Development (CI/CD) and even Container workloads. Part of the reason for not exposing the API was partially due to the original Instant Clone architecture which has certain limitations and constraints.

In addition, VMware was also interested in getting feedback from customers on how they would like to consume Instant Clone from an Automation standpoint, this was important because the current workflows were also some what complex. This started out with the release of a PowerCLI Instant Clone Extension Fling that provided an abstraction on top of the private APIs. Based on that and other feedback, VMware followed that up by releasing Instant Clone for pyvmomi (vSphere SDK for Python) Fling which gave customers more programmatic access to the private APIs. Both Flings were a huge success and we even had customers using the pyvmomi Instant Clone modules in Production to deploy several hundred Instant Clone VMs per day for their CI/CD workloads.

Taking the learnings from both Horizon View and the feedback from customers using the Flings, the Instant Clone Product/Engineering team has been hard at work behind the scenes on simplifying the Instant Clone architecture and removing limitations and constraints that had existed in earlier versions. As you can imagine, this was a non-trivial amount of work that would need to be released in phases, especially as VM lifecycle management touches almost every part of the vSphere stack. The team really focused on ease of consumption, especially from an Automation standpoint which is how most customers prefer to consume Instant Clone.