There was an excellent blog post from Duncan a couple of weeks back going over the new vMotion capabilities in vSphere 6.0 which includes: Cross vSwitch vMotion, Cross vCenter vMotion (xVC-vMotion) and Long Distance vMotion (LD-vMotion). If you have not checked out his article, I highly recommend you give it a read before proceeding further. After reading through Duncan's article, I noticed he had missed out on one additional vMotion capability which might not be obvious as the option is no where to be found in the vSphere Web Client UI. In fact, I was only aware of this additional capability after hearing about it from Engineering during the development of vSphere 6.

The additional vMotion capability actually extends the Cross vCenter Server vMotion (xVC-vMotion) workflow which allows an administrator to live migrate a running virtual machine between two vCenter Servers that are part of the same SSO Domain. By virtue of being in the same SSO Domain using the new Enhanced Linked Mode feature, both vCenter Servers will be visible in the vSphere Web Client and will be available to be selected either as a source or destination for a vMotion operation.

Screen Shot 2015-02-07 at 10.34.53 AM
This extended Cross vCenter Server vMotion capability (unofficially I am calling it ExVC-vMotion) allows an administrator to live migrate a running virtual machine between two vCenter Servers which are NOT part of the same SSO Domain. How cool is that!? In my opinion, this is actually a pretty big deal because I think it truly removes any boundaries for a vSphere virtual machine and will open up an entire new class of mobility use cases that were never thought possible before. This will definitely make it interesting for customers who wish to migrate workloads from their on-premises datacenter into a completely different vSphere environment or even one that is hosted by a service provider or maybe even vCloud Air?

The ExVC-vMotion operation is currently only available today using the vSphere API, not because it is a private API but because there is no UI wizard for this operation. The reason the current xVC-vMotion is so seamless today is that both your source and destination vCenter Server is visible by being part of the same SSO Domain. If you have two completely different vCenter Servers which are not joined to the same SSO Domain or have completely different SSO Domains, then you will need to use the vSphere API to perform this operation.

All vMotion operations including vMotion without shared storage uses the vSphere API RelocateVM_Task() method. In vSphere 6.0, the method has been enhanced to accept a new property called ServiceLocator which provides a service endpoint to a vCenter Server where a VM can be migrated to. One important thing to note is that if you wish to migrate a VM between two vCenter Servers located in the same SSO Domain, there is an sslThumbprint property that is not required to be set. However, if the two vCenter Servers are NOT part of the same SSO Domain, then you need to set that property. In addition, if the VM is migrated to a different vCenter Server, additional properties such as the ESXi host, vSphere Cluster/Resource Pool and Datastore must be specified as part of the migration spec.

UPDATE (05/25/16) - I have re-factored and simplified my xVC-vMotion script which supports additional capabilities. Please have a look at the blog post here for more details.

To demonstrate this awesome ExVC-vMotion operation, I have created a simple PowerCLI script called run-cool-ExVC-vMotion.ps1 which accepts 12 command-line parameters which are described in more detail below:

Variable Description
sourceVC The hostname or IP Address of the source vCenter Server
sourceVCUsername The username to connect to source vCenter Server
sourceVCPassword The password to connect to source vCenter Server
destVC The hostname or IP Address of the destination vCenter Server
destVCUsername The username to connect to the destination vCenter Server
destVCPassword The password to connect to the destination vCenter Server
destVCThumbprint The SSL Thumbprint (SHA1) of the destination vCenter Server (can be retrieved using either this or this)
datastorename The destination vSphere Datastore where the VM will be migrated to
clustername The destination vSphere Cluster where the VM will be migrated to
vmhostname The destination vSphere ESXi host where the VM will be migrated to
vmnetworkname The destination vSphere VM Portgroup where the VM will be migrated to
vmname The name of the source VM to be migrated

In my lab environment, I have configured two vCenter Server's which are part of two different SSO Domains as seen in the screenshot below:

Screen Shot 2015-02-10 at 5.53.47 AM
I have tiny Linux VM (vMA) that I am using which I will be migrating from vcenter60-4 to vcenter60-5 which has a completely different datastore and VM portgroup (if you have stretched/extended L2, then the VM would stay online during this migration). I then execute the script using the following parameters based on my own environment and we can see the migration is kicking off:

.\run-cool-ExVC-vMotion.ps1 vcenter60-4.primp-industries.com administrator@vghetto.local VMware1! vcenter60-5.primp-industries.com administrator@vsphere.local VMware1! 82:D0:CF:B5:CC:EA:FE:AE:03:BE:E9:4B:AC:A2:B0:AB:2F:E3:87:49 vesxi60-8-local-storage NY-Cluster vesxi60-8.primp-industries.com NY-VM-Network vMA

Screen Shot 2015-02-10 at 6.05.46 AM
One the migration has completed, if we now take a look at our vSphere Web Client, we can see the VM has now been migrated to the other vCenter Server.

Screen Shot 2015-02-10 at 5.57.01 AM
I really hope to see the vSphere Web Client get enhanced to support this cool vMotion capability, but in the mean time you can easily perform this operation using the above PowerCLI script or any other scripting/programming language calling into the vSphere API. Set your VM free and let it migrate where your heart desires 🙂

54 thoughts on “Did you know of an additional cool vMotion capability in vSphere 6.0?

  1. Amazing explanation. One question though, does this mean that a VM which is not on an extended L2 network can live migrate between vCenter servers on same SSO domain?

  2. Maybe I missed it, how does this deal with potential duplicate MACs going forward ?
    vCenter 1 now thinks the MACs are free and may give them to another VM.

    • Troy,

      Great question, actually this won’t be a problem as there’s a special MAC Address “blacklist” which is maintained by source VC. This list is updated with the VM’s MAC Address(s) once it’s been successfully migrated to another VC to ensures that the source VC will not hand out that MAC Address again

  3. Nice script, thanks for sharing it
    But it does not work for me (using the RC). It ask for the VM Folder destination and stop the migration.

    If you add this line in the # Source VM to migrate part:
    $vmfolder = (Get-Folder -Server $destVCConn -Name “VM Folder Name…”)

    And then this line in the # Relocate Spec for Migration part:
    $spec.folder = $vmfolder.Id

    Then it works (at least for me)

  4. Hi William, thanks for sharing.

    By the way, any idea why it fails when machines are connected to vDS? and another one, on the script comments you say that its only looking for one network card, can you share the modifications for VM’s with 2 or more vnics.

    Thanks a lot for you help.

    Regards
    Paulo José

    • That’s an exercise I’ve left for the reader 🙂 You can always post in the PowerCLI community forums if you’re unable to figure it out but the script is provided as a sample

      • Problem solved, got help from a colleague of mine.
        So our “enhanced” version of your script now migrates machines with more then one nic, and also migrates machines connected to vDS.

        Thanks any way.

        Regards

  5. I was thinking of using this script to migrate to a new vSphere 6 environment. I was going to present the same datastores to both clusters and do an xVC-vMotion across. I would like to do a vMotion without moving the VMDK’s. From the above notes its not clear if I could set the datastorename on the source and destination the same to achieve this?

    • It sounds like you’re wanting to do just a vMotion (where the VMDKs don’t change) but across two different vCenter Server 6.0 env? You could try specifying the source/destination datastore to be the same name, not sure if it’ll work but worth a try.

  6. When we run this script in our environment, we are getting the following error:

    Wait-Task : 7/29/2015 10:41:48 AM Wait-Task The operation for the
    entity “KVTEST1” failed with the following message: “A specified parameter was
    not correct: ServiceLocator.instanceUuid”
    At c:\xvc_migrate.ps1:159 char:10
    + $task1 | Wait-Task -Verbose
    + ~~~~~~~~~~~~~~~~~~
    + CategoryInfo : NotSpecified: (:) [Wait-Task], InvalidArgument
    + FullyQualifiedErrorId : Client20_TaskServiceImpl_CheckServerSideTaskUpda
    tes_OperationFailed,VMware.VimAutomation.ViCore.Cmdlets.Commands.WaitTask

    I am getting the instanceUUID from the dest vcenter, so I don’t think that that is the problem.

    Any ideas?

  7. Which constrains have Cross VC migration for data relocate DRS/HA and Tasks of VM? Located in the same SSO domain in VCs, maybe?

  8. Hi William,

    is it possible to run such vMotion if the Source VM runs on an ESXi 5.5 and vDS 5.5 connected to a 6.0 vCenter? Or must all be on version 6.0 (ESXi host and vDS)

  9. do we have a full fledged working way of setting l3 vmotion on vsphere 6 yet? it s a pain to configure it manually on 100s of blades every day when you are a consultant.

    • Yes, L3 is fully supported. You can create a new Network Stack in ESXi which will allow you to use different networks to route out versus old method of adding static routes. Please refer to the vSphere documentation for more details

  10. Hi William,
    Awesome script, one thing though – it doesn’t work with Distributed Switches.
    I have a solution for anyone wanting to use Williams script, please see below.

    1) Open the script and locate the line that reads “$Spec.DeviceChange[0].Device.Backing.DeviceName = $vmnetworkname”
    2) Comment out that line, and add the following directly below it.

    ################################################################################################
    $vm2 = Get-View -ViewType VirtualMachine -Filter @{“Name” = $VMname}
    $vmhostView = Get-View -ID $vm2.Runtime.Host
    $esx = Get-View -Id $vmview.Runtime.Host
    foreach($netMoRef in $esx.Network){
    if($netMoRef.Type -eq “DistributedVirtualPortGroup”){
    $net = Get-View -Id $netMoRef
    if($net.Name -eq $destdvPortGroup){
    $dvPgKey = $net.MoRef.Value
    $dvSwitchUuid = (Get-View -Id $net.Config.DistributedVirtualSwitch).Summary.Uuid
    } # end if
    } # end if
    } # end foreach loop

    $spec.deviceChange[0].device.backing = New-Object VMware.Vim.VirtualEthernetCardDistributedVirtualPortBackingInfo
    $spec.deviceChange[0].device.backing.port = New-Object VMware.Vim.DistributedVirtualSwitchPortConnection
    $spec.deviceChange[0].device.backing.port.switchUuid = $dvSwitchUuid
    $spec.deviceChange[0].device.backing.port.portgroupKey = $dvPgKey
    ################################################################################################

    3) Scroll back up to the ‘debugging’ section of the script which is commented out, and add the following line, where $destdvPortGroup is the name of the destination port group you want to vMotion to.
    ################################################################################################
    $destdvPortGroup = “DPG – VLAN 77”
    ################################################################################################
    4) Done! It’s a bit of a hardcoded change but it works. Someone with better scripting skills could easily update Williams script to dynamically tell if it should be a normal switch or a distributed switch.

    Don’t normally reply to blog posts but this took a little while to figure out and couldn’t find it posted anywhere else! Hope it helps someone

    • Rumpole,

      First of all, thank you for posting code to help with a vDS Switch. I am running into an issue with your additions.
      Is this line correct?
      $esx = Get-View -Id $vmview.Runtime.Host

      I get the following error:
      Get-View : Cannot validate argument on parameter ‘Id’. The argument is null or empty. Supply an argument that is not null or empty and then try the command again.
      At D:\Scripts\0 Automate\run-cool-ExVC-vMotion.ps1:176 char:21
      + $esx = Get-View -Id $vmview.Runtime.Host
      + ~~~~~~~~~~~~~~~~~~~~
      + CategoryInfo : InvalidData: (:) [Get-View], ParameterBindingValidationException
      + FullyQualifiedErrorId : ParameterArgumentValidationError,VMware.VimAutomation.ViCore.Cmdlets.Commands.DotNetInterop.GetVIView

      Thanks
      Fred

      • Hi Fred,

        Did you ever get the answer to this or get this working? We are trying to accomplish the same thing.

        Thanks!
        Wes

  11. Great posting guys thanks so much. Meanwhile I’m having a hard time finding information pertaining to non-federated cross vCenter vMotion. Mr Google knows very little. The only reference I’m finding points to the SDK and mentions using additional (unspecified) parameters. Can anyone offer suggestions. Our objective is to collapse 4 vCenters with 4 separate SSO domains into 2 vCenters with a single SSO domain running in enhanced linked mode.

    • Ray,

      I’m a bit confused about your question, did you go through the blog article? What you’re asking for is exactly what I’ve blogged about which is be able to migrate a VM across two vCenter Server 6.0 that are not part of the same SSO Domain or as you called it “non-federated”

  12. Hi Lam,

    I am getting the error below.

    Exception calling “RelocateVM_Task” with “2” argument(s): ”
    Required property instanceUuid is missing from data object of type ServiceLocat
    or
    while parsing serialized DataObject of type vim.ServiceLocator
    at line 1, column 301
    while parsing property “service” of static type ServiceLocator
    while parsing serialized DataObject of type vim.vm.RelocateSpec
    at line 1, column 295
    while parsing call information for method RelocateVM_Task

    • Are you specifying the instanceUUID property? That seems to be the error that’s indicated … If you are, can you try both all lower and all upper case, I remember someone was having an issue and was related to the “case” of this property

  13. Hi William,

    I was trying to use your script to do a vMotion within the same SSO – Enhanced Linked Mode – but as I do not want to change datastores (stretched metro cluster with SRM 6.1) I don’t think it is appropriate. I’m running into an issue where I am unable to vMotion any VM within the GUI that has vmdks in more than one datastore. A single datastore VM works fine across vCenters but once I add another datastore to the VM, it will only show me the local hosts in the vCenter. I’ve never heard of such a restriction so I was trying to script the move to see if I could get a warning message as to why it is not displaying the other hosts. Is there another method like MigrateVM_Task that allows cross-vCenter with the same SSO? Thanks.

    • Drew,

      You’re not forced to change the datastore 🙂 Although most customers who may have a use for xVC-vMotion would more than likely change the various compute/storage resources as they’re usually not shared amongst VC. However, as I said this is not necessary required and what is possible is all documented in the vSphere API which you can see from the “spec” http://pubs.vmware.com/vsphere-60/index.jsp?topic=%2Fcom.vmware.wssdk.apiref.doc%2Fvim.vm.RelocateSpec.html you do not need to relocate the disks. You should be able to just leave that blank, if it throws an error, I’m curious if you’ve tried just specifying the exact same datastore (e.g. no change).

      MigrateVM_Task() is only for traditional vMotion which does not cross vCenter Server boundary, for all other types of migrations which included Storage vMotion/etc. you will need to use the RelocateVM_Task() as mentioned in the documentation

      • So this turned out to be 2 bugs in the VMware code concerning shared storage. The first is you cannot cross-vMotion with a VM that has multiple datastores. The second is you cannot cross-vMotion a VM if there is not 2x vmdk free space in the datastore – basically enough so that if you did a Storage vMotion it would succeed. VMware is working on bugs and hopefully a KB.

        • Hi Drew (and anyone that stumbles across this),

          I submitted a pull request to update William’s script (https://github.com/lamw/vghetto-scripts/pulls). I wrote in some logic to handle multi-nic VMs where the NICs are on different portgroups as well as VMDKs on different datastores. The 2x space requirement still exists (doh!) but I was able to move around 3k VMs between vCenters with the same shared storage. Hopefully others can make use of it too

          -Alex

  14. If you use a DVswitch on the target vcenter you have to change a little bit in your script, i have hardcoded my dvswitch name:
    I have added in Line 134:
    $destvds = Get-view (Get-VDSwitch -Server $destVCConn -name mydvswitch) -Property uuid
    $destpg = Get-View (Get-VDPortgroup -name $vmnetworkname) -Property key

    and changed line 165 from
    $spec.deviceChange[0].Device.backing.NetworkName = $vmnetworkname
    to
    $spec.deviceChange[0].Device.backing = new-object VMware.Vim.VirtualEthernetCardDistributedVirtualPortBackingInfo
    $spec.deviceChange[0].Device.backing.port = New-Object VMware.Vim.DistributedVirtualSwitchPortConnection
    $spec.deviceChange[0].Device.backing.port.portgroupkey = $destpg.key
    $spec.deviceChange[0].Device.backing.port.switchuuid = $destvds.uuid

  15. Hi folks,

    Given the interests for this script that includes both XvC-vMotion within the same SSO Domain (which the script does do today) as well as between different SSO Domain + variations between VSS/VDS and supporting multiple vNICs. I’ve actually re-written the script to easily support all these use cases. I’m doing some finalize testing but I will post an update which will be a new blog so that folks can easily find it as I know the title does not lend itself to Google searches 🙂

    Appreciate all the feedback

  16. This is awesome to hear! Do you know this will work while migrating from a vCenter 5.1 installation to a completely separate vCenter 6 installation?

    • No, Inter-vCenter vMotion only works between two vCenter 6 with ESXi 6 Hosts connected.

Thanks for the comment!