How to rsync files between two remotes?
pubsub.slavino.sk / planetdebian · Saturday, 6 November - 10:41 · 3 minutes
can copy files between two remote hosts through localhost.
This comes in handy when the two servers cannot communicate
directly or if they are unable to authenticate one to the
does not support such a feature.
Here is a trick to emulate the behavior of
with SSH tunnels.
When syncing with a remote host,
to spawn a
process. It interacts with it through its
standard input and output. The idea is to recreate the same setup
using SSH tunnels and
, a versatile tool to establish
bidirectional data transfers.
The first step is to connect to the source server and ask
command-line to spawn the remote
flag overrides the command to use to get a remote shell: instead of
, we use
$ ssh web04 $ rsync -e 'sh -c ">&2 echo $@" echo' -aLv /data/. web05:/data/. web05 rsync --server -vlogDtpre.iLsfxCIvu . /data/.rsync: connection unexpectedly closed (0 bytes received so far) [sender]rsync error: error in rsync protocol data stream (code 12) at io.c(228) [sender=3.2.3]
The second step is to connect to the destination server with
port forwarding. When connecting to the local port 5000, the TCP
connection is forwarded through SSH to the remote port 5000 and
. When receiving the connection,
command we got at the previous step and connects its
standard input and output to the incoming TCP socket.
$ ssh -L 127.0.0.1:5000:127.0.0.1:5000 web05 $ socat tcp-listen:5000,reuseaddr exec:"rsync --server -vlogDtpre.iLsfxCIvu . /data/."
The last step is to connect to the source with
is used in place of a regular SSH connection and
connects its standard input and output to a TCP socket connected to the
remote port 5000. Thanks to the remote port forwarding, SSH forwards
the data to the local port 5000. From there, it is relayed back to the
destination, as described in the previous step.
$ ssh -R 127.0.0.1:5000:127.0.0.1:5000 web04 $ rsync -e 'sh -c "socat stdio tcp-connect:127.0.0.1:5000"' -aLv /data/. remote:/data/. sending incremental file listhaproxy.debian.net/haproxy.debian.net/dists/buster-backports-1.8/Contents-amd64.bz2haproxy.debian.net/dists/buster-backports-1.8/Contents-i386.bz2[…]media.luffy.cx/videos/2021-frnog34-jerikan/progressive.mp4sent 921,719,453 bytes received 26,939 bytes 7,229,383.47 bytes/sectotal size is 7,526,872,300 speedup is 8.17
This little diagram may help understand how everything fits together:
rsync manual page
prohibits the use of
. Use this
hack at your own risk!
--senderare used internally by rsync, and should never be typed by a user under normal circumstances. Some awareness of these options may be needed in certain scenarios, such as when setting up a login that can only run an rsync command. For instance, the support directory of the rsync distribution has an example script named
rrsync(for restricted rsync) that can be used with a restricted ssh login.
I was hoping to get something similar with a one-liner. But this does not work!
$ socat \> exec:"ssh web04 rsync --server --sender -vlLogDtpre.iLsfxCIvu . /data/."\> exec:"ssh web05 rsync --server -vlogDtpre.iLsfxCIvu /data/. /data/."\over-long vstring received (511 > 255)over-long vstring received (511 > 255)rsync error: requested action not supported (code 4) at compat.c(387) [sender=3.2.3]rsync error: requested action not supported (code 4) at compat.c(387) [Receiver=3.2.3]socat E waitpid(): child 878292 exited with status 4socat E waitpid(): child 878293 exited with status 4