Running rsync on ESXi is not a new topic, there are a number of users in the community who have made this work. One well-known user, Dave Mishchenko, who runs vm-help and recently authored VMware ESXi: Planning, Implementation, and Security, provides an rsync binary that can be downloaded to run on ESXi. Not that I did not trust Dave and the source of his rsync binary, but I wanted to compile my own and understand the process Dave took to get to a compatible rsync for ESXi. In talking to Dave, I found out the binary was actually provided by a VMTN user, named glim who had noted some of the details in this VMTN thread.
The user basically created a statically linked versus a dynamically linked version of rsync which runs without relying on libraries that may or may not be present on ESXi. My initial attempts failed using both a 32/64bit CentOS 5.5 and latest version of rsync 3.0.7 and 3.0.3. The compilation of rsync was successful, but upon executing on ESXi, it immediately had a segmentation fault. I started to think there might be some dependencies in libraries that were used to compile rsync that I was missing. I was also in conversation with Alain Spineux, creator of MKSBackup regarding this same topic and he had found a way to compile rsync based on the article Hacking ESXi by Stjepan Groš.
Stejepan found that using CentOS 3.9, one could compile both statically and dynamically linked binaries without any modifications to ESXi. Though Stejepan did not provide the steps to compile rsync, it is relatively simple to do so. I also found that you do not need to use older 3.0.3 release of rsync, you can use the latest version which is 3.0.7 as of writing this article.
Before we get started, let me remind you of the disclaimer:
*** THIS IS PROBABLY NOT SUPPORTED BY VMWARE, USE AT YOUR OWN RISK ***
Download and install CentOS 3.9 i386 - You can find a list of download sites here
With a default installation of CentOS 3.9, you just need to get GCC installed
yum -y install gcc
Next you will need to download the latest version of rsync which is 3.0.7 here and SCP it to your CentOS system
Next you will now extract the contents of the rsync-3.0.7.tar.gz
tar -zxvf rsync-3.0.7.tar.gz
Now you will change into the rsync-3.0.7 directory and run the configure command
./configure
Now you will run the make command which will compile rsync, but you will also be passing in additional flags to create a statically linked binary
make CFLAGS="-static" EXEEXT="-static"
Finally, you will need to run the strip utility which discards all symbols from object files
strip rsync-static
You now have a self-contain rsync binary labeled rsync-static found in the current working directory
Next up, you will SCP the rsync-static binary to your ESX(i) host and set the appropriate permissions to execute. I decided to temporarily store it in /tmp, but if you want the binary to persist through a reboot, you may want to store it in either a local or remote VMFS/NFS volume.
I quickly verified that rsync was in functional, by rsyncing /etc directory in ESXi over to another host. I use the following command:
/tmp/rsync-static --compress --progress --stats -e dbclient -Rr /etc build-centos:/upload
Once the file transfer has completed, you should see the /etc directory replicated on the destination host.
You may have noticed in the command line we are specifying "dbclient", by default rsync will go over SSH but ESXi does not actually have the traditional ssh client that we are all used to. It actually uses dropbear which is tiny SSH server and client which is frequently used in embedded systems such as Busybox.
Since /bin/ssh path does not exists, rsync will fail unless you specify the dbclient to the -e parameter.
Alain also found that retrieving peer name may fail and the rsync code currently does not properly handle this failure and just exits. By looking at the clientname.c source code which is part of rsync, Alain noted that even the developers left a comment that states the error could probably be handled without exiting. The fix is to just comment out the rsynerr line which does not force the program to exit when hitting this problem and this can be found on line 171.
Once you have made the changes, you just need to follow the instructions above to compile your static rsync binary. This is not necessary required, but if you run into this issue, this is a quick hack that can help.
I would like to thank Alain for sharing this tidbit of information
Hi!
I did:
Install a centos 3.9 (Is it the ok version?)
Download a rsync 3.0.7.
I did compile it, with yours instructions.
I did copy then rsync executable to my esxi free 4.1.
I did execute the rsync, and I did receive:
Segmentation fault.
I’m thinking the problem is whe don’t have the centos’s libc in the esxi 4.1:
uidlist.o(.text+0xd2): In function
gid_to_name':name_to_gid’:: Using 'getgrgid' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
util.o(.text+0xc86): In function
: Using ‘getgrnam’ in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
util.o(.text+0xc27): In function
name_to_uid':uid_to_name’:: Using 'getpwnam' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
uidlist.o(.text+0x94): In function
: Using ‘getpwuid’ in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
socket.o(.text+0x413): In function `try_bind_local’:
: Using ‘getaddrinfo’ in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
Do you have a solution for this?
TIA
jorge infante
rosario – santa fe – argentina
which version of centos do you recommend using for an rsync binary in esxi 3.5? currently I receive this when running the rsync binary in esxi3.5.
rsync error: syntax or usage error (code 1) at main.c(1408) [client=3.0.3]
@Jesse,
I don’t know, the version I used is listed in the blog post, I didn’t test with other versions of ESXi. Remember this is all unsupported, so it may just not work with older releases or even newer release for that matter. Good Luck
Hi !
My comment deals with a similar issue : using a NFS server binary inside ESXi.
With the help of R Margar (http://rene.margar.fr/2010/03/client-nut-natif-pour-vmware-esxi/), i can use directly the unfs binary from CentOS 5.5, inside ESXi 4.0. With Ghetto VCB, it’s very interesting to backup VM. But no READDIRPLUS makes this NFS server quite limited for live NFS Disk :-(.
Did you already manage to create some kernel driver for ESXi, like an NFS server (which has READDIRPLUS procedure included) ?
Hi William,
I can confirm that on Centos 3/i386 I can compile (with warnings) and strip rsync-3.0.9 and it runs perfectly on ESXi 5.0 Update 1 | 15 MAR 2012 | Build 623860.
Many thanks for a great post.
Scott
Hi
It seems that CentOS 3.9 is not supported anymore 🙂
http://mirror.centos.org/centos/3.9/readme.txt
Luckily the Harvey Mudd College has a mirror 🙂
The ISO can be found there: http://mirror.hmc.edu/centos/3.9/isos/i386/CentOS-3.9-server-i386.iso
Also, you have to use that in the yum configuration too, because the CentOS site gives you a 404:
perl -pi -e “s/centos\.org/hmc.edu/g” /etc/yum.conf
Also load the GPG key:
rpm –import http://mirror.centos.org/centos/RPM-GPG-KEY-CentOS-3
I can confirm a working rsync 3.0.9 binary that seems to run OK on ESXi 5.1.0 build-799733.
Thank you, Dick!
or in /etc/yum.conf replace the source by :: http://vault.centos.org/3.9/os/i386/
[base]
name=CentOS-$releasever – Base
baseurl=http://vault.centos.org/3.9/os/$basearch/
#baseurl=http://mirror.centos.org/centos/$releasever/os/$basearch/
gpgcheck=1
#released updates
[update]
name=CentOS-$releasever – Updates
baseurl=http://vault.centos.org/3.9/updates/$basearch/
#baseurl=http://mirror.centos.org/centos/$releasever/updates/$basearch/
gpgcheck=1
#packages used/produced in the build but not released
[addons]
name=CentOS-$releasever – Addons
baseurl=http://vault.centos.org/3.9/addons/$basearch/
#baseurl=http://mirror.centos.org/centos/$releasever/addons/$basearch/
gpgcheck=1
#additional packages that may be useful
#[extras]
#name=CentOS-$releasever – Extras
#baseurl=http://mirror.centos.org/centos/$releasever/extras/$basearch/
#gpgcheck=1
#additional packages that extend functionality of existing packages
#[centosplus]
#name=CentOS-$releasever – Plus
#baseurl=http://mirror.centos.org/centos/$releasever/centosplus/$basearch/
#gpgcheck=1
#packages in testing
#[testing]
#name=CentOS-$releasever – Testing
#baseurl=http://mirror.centos.org/centos/$releasever/testing/$basearch/
#gpgcheck=1
#between point release supported updates
#[fasttrack]
#name=CentOS-$releasever – Fasttrack
#baseurl=http://mirror.centos.org/centos/$releasever/fasttrack/$basearch/
#gpgcheck=1
Dick,
Thank you VERY much for tracking down and sharing the Harvey Mudd hack. Very much appreciated 🙂
Why is CentOS 3.9 required?
That is just the version that I was able to get things working.
where can I get rsync 3.0.9 binary for ESXi 5.1.0 build-799733.
I do not succeed to compile on a ubuntu 12.04LTS
Thank you
uidlist.o: In function `gid_to_name’:
uidlist.c:(.text+0x10a): warning: Using ‘getgrgid’ in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
….
I am still looking at rsync 3.0.9 binary for ESXi 5.1
I find in https://bugzilla.samba.org/show_bug.cgi?id=8177
and
http://communities.vmware.com/thread/342492
happy new year
This comment has been removed by the author.
Pulled rsync HEAD-20121003 that calls it self 3.1.0dev which didn’t have molniev’s patch, so it was applied. The rsynced sparse -flat file reported a different side on the nfs share, when it was rsynced back to esxi 5.1, it reported the correct size and booted. not much of a test, but feel free to try http://rob.com/lancair/2013.01/rsync.gz ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), statically linked, for GNU/Linux 2.2.5, stripped
thank you for the binary
I have done few test
time rsync –sparse –progress –stats -avz /vmfs/volumes/datavm216C/sauvegarde/alpi91/alpi91-2013-01-02_01-05-01/ seerveur.distant.com:/vmfs/volumes/datavm229C/tmp
sending incremental file list
./
alpi91-2013-01-05_01-05-01 -> /vmfs/volumes/datavm216C/sauvegarde/alpi91/alpi91-2013-01-05_01-05-01
alpi91-flat.vmdk
42,949,672,960 100% 3.95MB/s 2:52:39 (xfr#1, to-chk=4/7)
alpi91.vmdk
542 100% 0.90kB/s 0:00:00 (xfr#2, to-chk=3/7)
alpi91.vmx
3,125 100% 4.72kB/s 0:00:00 (xfr#3, to-chk=2/7)
alpi91_1-flat.vmdk
21,474,836,480 100% 17.44MB/s 0:19:34 (xfr#4, to-chk=1/7)
alpi91_1.vmdk
544 100% 1.01kB/s 0:00:00 (xfr#5, to-chk=0/7)
Number of files: 7 (reg: 5, dir: 1, link: 1)
Number of created files: 6 (reg: 5, link: 1)
Number of deleted files: 0
Number of regular files transferred: 5
Total file size: 64,424,513,720 bytes
Total transferred file size: 64,424,513,651 bytes
Literal data: 64,424,513,651 bytes
Matched data: 0 bytes
File list size: 0
File list generation time: 0.174 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 15,645,990,678
Total bytes received: 117
sent 15,645,990,678 bytes received 117 bytes 1,356,098.88 bytes/sec
total size is 64,424,513,720 speedup is 4.12
real 3h 12m 17s
user 1h 15m 48s
sys 0m 0.00s
+ md5sum /vmfs/volumes/datavm216C/sauvegarde/alpi91/alpi91-2013-01-05_01-05-01/*
It look find between 2
VMkernel ordi216.cccp.local 5.1.0 #1 SMP Release build-799733 Aug 1 2012 20:03:00 x86_64 GNU/Linux
Thanks for the binary RobLogan,
I’ve just tried it on ESXi 5.5 and it appears to work fine still.
Cheers,
Andrew
I try rsync binary on ESXi-5.5.0-1331820 but it does not work!
rsync -a /C/fidit/ghettoVCB/conf/ ordi216.cccp.local:/C/tmp/
ERROR: out of memory in map_ptr [sender]
rsync error: error allocating core memory buffers (code 22) at util.c(119) [sender=3.1.0dev]
md5sum /bin/rsync
2a5e5a91de0cd00c8faa6f37d7172404 /bin/rsync
Andrew cant yon send your binary please ?
by cons on the destination machine Vphere not see that it’s thin?
but it work
GSCopy Pro v6.0 (RoboCopy Alternative) with Open File Agent
GSCopyPro is a single command-line tool (CLI) that can copy, replicate and move files from one folder to another. This folder can be on the same machine/ server or another server elsewhere. What makes GSCopyPro stand out from other competitors is the fact it works on 32-bit as well as 64-bit systems and has no restrictions. It can easily be scheduled to run as a scheduled task and fully automated. GSCopyPro also comes with an open file agent which can copy files that are locked/ opened by other processes. This feature is supported in all windows vSCersions from widows XP/ 2003 and later.
Go To:>> http://www.gurusquad.com/GOPYPRO
This comment has been removed by the author.
I compiled a new rsync 3.1.0 for ESXi 5.1. You can get it at http://damiendebin.net/blog/2013/12/06/esxi-5-dot-1-and-rsync/
merci beaucoup damien
je confirme que ton binaire marche sur ESXi 5.1 and 5.5
It work on ESXi 5.1 and 5.5
William,
Thanks for this well-written, very helpful post! I just used it to successfully compile rsync 3.1.0 for ESXi 4.1 on a CentOS 3.9/i386 VirtualBox VM, and have verified that I am now able to rsync to the host. Awesome!
Notes:
I used the process you described in your post, but made the following changes:
– As described in Dick Visser’s comment, above, I used his Harvey Mudd College mod to get yum working with their CentOS 3.9 repo, and his reminder to import the CentOS 3 RPM GPG key from the URL he specified.
– I had to use yum to install zlib-devel, in addition to gcc. This is probably because I installed a very basic pattern for CentOS 3.9.
– Although not required, for ease of use, I created a utils directory on one of our datastores, used scp to copy the rsync-static binary to the utils directory, and then symlinked it to /usr/bin/rsync.
Best,
Rob
(Using RobLogan’s download) rsync transfers work, except when a file name has dashes “-” in it. Trying to send such a file name yields this error:
building file list … done
ERROR: out of memory in map_ptr [generator]
rsync: connection unexpectedly closed (30 bytes received so far) [sender]
rsync error: error allocating core memory buffers (code 22) at /SourceCache/rsync/rsync-45/rsync/io.c(453) [sender=2.6.9]
Remove the dashes, even change them to underscores “_”, or even spaces ” “, and it transfers fine.
Configure and make commands for compiling rsync-3.1.1 on CentOS to run on ESXi 5.5 (possibly others, must test):
./configure –with-included-popt –with-included-zlib
make LIBS=”-Wl,-static -lattr -lacl -Wl,-Bdynamic”
The resultant binary is just ‘rsync’. This runs great on ESXi 5.5 and fixes the problem with DNS not resolving properly on the strictly statically linked versions. This is sort of a hybrid as it uses static versions of libacl and libattr but dynamic versions of glibc.
Hope this helps anyone having trouble.
So can I migrate a Live VM using this rsync method? I have a VPN server that needs to be moved to a different host. I want to do it while the server is online.
Thanks
Using the pre-built static rsync, version 3.1.0, copying large vmdk files I get this error:
rsync error: error allocating core memory buffers (code 22) at util2.c(102) [sender=3.1.0]
I’ve abandoned using rsync for backups. Using this combination of ssh and dd gets the job done:
ssh [email protected]_ESX dd bs=1M if=/vmfs/volumes/datastore1/vmname/vmname/vmname_0-flat.vmdk | dd conv=sparse of=vmname_0-flat.vmdk
dd on ESXi does not support conv=sparse. The source machine in this backup scenario is a Ubuntu Linux system. dd conv=sparse is supported on Linux, and any zero extent 512 bytes or longer are treated as sparse.
Below is the ls -l and du -scm size of a thin-provisioned ESXi vmdk backed up using the above ssh/dd pipeline:
-rw-r–r– 1 root root 51539607552 Apr 1 14:14 xxx.vmdk
7992 xxx.vmdk
The source vmdk file sizes are:
-rw——- 1 root root 51539607552 Mar 31 16:59 xxx.vmdk
8867 xxx.vmdk
It is obvious that dd conv=sparse is slightly more efficient at finding zero-extents than the VMFS filesystem.
I have run md5sum on the original vmdk and the copied disk, and the checksums match.