I recently had to re-provision one of my VSAN lab environments using my recently shared ESXi 5.5 VSAN Kickstart. I usually specify a license key within the Kickstart so I do not have to license the ESXi host later. This actually got me wondering on whether VSAN would in fact work with Free ESXi aka vSphere Hypevisor? Being a curious person, I of course had to test this in the lab 🙂

Needless to say, if you want to properly evaluate or use VSAN in production, you should go through the supported method of using vCenter Server as it provides a simple and intuitive management interface for VSAN. More importantly, having the ability to create individual VM Storage Policies that can be applied on a per VMDK basis based on SLA's for your given application or Virtual Machine.

Disclaimer: This is not officially supported by VMware and running ESXi without a VSAN license is against VMware's EULA.

Since we do not have a vCenter Server, we will need to be able to fully configure VSAN without it. Luckily, we know of a way to "bootstraping" VSAN onto an ESXi host without vCenter Server and I will be leveraging that blog post to test this scenario with Free ESXi.

Prerequisite:

  • 3 ESXi 5.5 hosts already installed and licensed with vSphere Hypervisor (Free ESXi) License
  • SSH Enabled

Step 1 - SSH to the first ESXi host and run the following ESXCLI command to create a VSAN Cluster:

esxcli vsan cluster join -u $(python -c 'import uuid; print str(uuid.uuid4());')

configure-vsan-for-free-esxi-0
Step 2 - Run the following ESXCLI command to make a note of the VSAN Cluster UUID (highlighted in green in the screenshot above) which will be needed later:

esxcli vsan cluster get

Step 3 - Enable VSAN Traffic for VMkernel interface you plan on using for VSAN traffic by running the following ESXCLI command:

esxcli vsan network ipv4 add -i vmk0

Step 4 - Run the following command to view a list of disks that are eligible for use with VSAN. You will need a minimum of 1xSSD and 1xMD

vdq -q

configure-vsan-for-free-esxi-1
Step 5 - Using the information from vdq, we will now create our VSAN Disk Group which will contain the SSD/MD's to be used for VSAN. Use the following ESXCLI command and substituting in the SSD/MD Names (please refer to the screenshot above for an example):

esxcli vsan storage add -s [SSD] -d [MD]

Step 6 - To ensure you have properly configured a VSAN Disk Group, you can run the following ESXCLI command to confirm:

esxcli vsan storage list

configure-vsan-for-free-esxi-2
At this point, we now have a single ESXi host configured with VSAN Datastore, we can also confirm this by running the following ESXCLI command:

esxcli storage filesystem list

configure-vsan-for-free-esxi-3
Step 7 - Repeat Steps 3-6 on the remainder two ESXi hosts

Step 8 - Finally, we now need to join the remainder ESXi hosts to the VSAN Cluster. We will need the VSAN Cluster UUID that we recorded earlier and specify that in the following ESXCLI command on each of the remainder ESXi hosts:

esxcli vsan cluster join -u [VSAN-CLUSTER-UUID]

If we now login to all of our ESXi hosts using the vSphere C# Client, we will see a common VSAN Datastore that is shared among the three ESXi hosts. To prove that that VSAN is in fact working, we can create a Virtual Machine and ensure we can power it on as seen in the screenshot below. By default, VSAN has a "Default" policy which defines FTT (Number of host failures to tolerate) set to 1 and assuming you have at least 3 ESXi hosts, all Virtual Machines will be protected by default.

configure-vsan-for-free-esxi-4
Even though you can run VSAN using Free ESXi and leveraging the default VM Storage Policy that is built into VSAN for protecting Virtual Machines, you are only exercising a tiny portion of the potential that VSAN can bring when consuming it through vCenter Server. As mentioned earlier, you will not have the ability to create specific VM Storage Policies and assign them based on the specific SLAs and be able to easily monitor their compliance and remediation. The management of VSAN Cluster for adding additional capacity or serviceability is also quite limited without vCenter Server, though it can be definitely be done it is much easier with just a couple of clicks in the vSphere Web Client or a simple API call.

8 thoughts on “Does VSAN work with Free ESXi?

  1. mate, did U try get vSAN working (SSD caching functionality) on single ESXi host? or any other free/cheap solution to use SSD cache for local storage on ESXi server????

    • For me it works after changing the policy:
      esxcli vsan policy setdefault -c vdisk -p “((\”hostFailuresToTolerate\” i1) (\”forceProvisioning\” i1))”
      esxcli vsan policy setdefault -c vmnamespace -p “((\”hostFailuresToTolerate\” i1) (\”forceProvisioning\” i1))”

  2. I follow the above step and did it on ESXi free 6.0 I got the message below when I try to create a directories in datastore. any clue ?

    Call “FileManager.MakeDirectory” for object “ha-nfc-file-manager” on ESXi “172.1.1.11” failed.
    An unknown error has occurred.
    An unknown error has occurred.

  3. I’ve deployed the nested template 6.0 U02 to esxi 3 separate ovfs and added the advanced settings before starting for the first time.
    I think I’m doing something wrong in creating the cluster. When I do a “get” on each host say they are Master node and the total capacity of the vsanDS is that of single node. Also the Member count =1 on all the 3 nodes.
    What I’m I missing?
    Should I be using the “sub-cluster master UUID” of the first node to join?

    Node1
    [root@vasn-nst:~] esxcli vsan cluster get
    Cluster Information
    Enabled: true
    Current Local Time: 2016-05-12T15:06:48Z
    Local Node UUID: 572dc585-be2f-cc61-f7de-000c29a9ee16
    Local Node Type: NORMAL
    Local Node State: MASTER
    Local Node Health State: HEALTHY
    Sub-Cluster Master UUID: 572dc585-be2f-cc61-f7de-000c29a9ee16
    Sub-Cluster Backup UUID:

    Sub-Cluster UUID: 523c4ee7-48f4-8a97-d0d3-4184a033fdc2

    Sub-Cluster Membership Entry Revision: 0
    Sub-Cluster Member Count: 1
    Sub-Cluster Member UUIDs: 572dc585-be2f-cc61-f7de-000c29a9ee16
    Sub-Cluster Membership UUID: a6943457-3132-e089-ea0a-000c29a9ee16

    Node2:
    [root@vsan-nst2:~] esxcli vsan cluster join -u 523c4ee7-48f4-8a97-d0d3-4184a033fdc2

    Node3:
    [root@nst-vsan3:~] esxcli vsan cluster join -u 523c4ee7-48f4-8a97-d0d3-4184a033fdc2

    [root@nst-vsan3:~] esxcli vsan cluster get
    Cluster Information
    Enabled: true
    Current Local Time: 2016-05-12T15:36:13Z
    Local Node UUID: 57315802-da49-d4f7-7b81-000c29a3a076
    Local Node Type: NORMAL
    Local Node State: MASTER
    Local Node Health State: HEALTHY
    Sub-Cluster Master UUID: 57315802-da49-d4f7-7b81-000c29a3a076
    Sub-Cluster Backup UUID:
    Sub-Cluster UUID: 523c4ee7-48f4-8a97-d0d3-4184a033fdc2
    Sub-Cluster Membership Entry Revision: 0
    Sub-Cluster Member Count: 1
    Sub-Cluster Member UUIDs: 57315802-da49-d4f7-7b81-000c29a3a076
    Sub-Cluster Membership UUID: d49b3457-3085-366c-f9f2-000c29a3a076

    • You need to tell each host about the others – it’s no good creating copies of one and using that on three other machines, when installed they all think they are the only version…

Thanks for the comment!