rsync snippets

Resume scp download

Tagged rsync, scp, resume download  Languages bash

Unfortunately there is no way for scp to resume a download. Don't despair though if your download was interrupted. Rsync is capable of resuming it. Be aware though, that rsync makes a temporary file that will become as large as the target file is so your disk may become full during the resume process.

rsync --partial --progress --rsh=ssh host:/work/source.tar.bz2 .

Using rsync to backup a remote machine to a local directory via SSH

Tagged rsync, backup, sync, ssh  Languages bash

This command copies the files on the remote machine 127.0.0.1 to the local directory /backups/127.0.0.1:

rsync --delete -r -ve "ssh -p 666" 127.0.0.1:/var/www/ /backups/127.0.0.1

Some useful parameters:

-r = recursive
-n = simulate
-v = verbose
-a = archive mode
-L = follow symbolic links
--size-only = 
--progress = show progress indicator
--numeric-ids = don't map user and group IDs to local user and group names
-e = use SSH and port 666
--bwlimit = limit bandwidth to 100 kilobytes/second
--exclude-from = exclude the files and directories listed in the configuration file
--delete = remove local files that have been removed on the remote server

SCP via an SSH jump host

Tagged jump host, rsync, scp, ssh  Languages bash

I prefer using rsync to scp when copying files via jump hosts, for example, to target-host via the intermediate host jump-host:

rsync -v -e 'ssh -A -t [email protected] ssh -A -t target-host' nasty.file :/tmp

rsync syntax is stable across versions and platforms while the syntax of SCP is different across platforms and versions.

man ssh:

     -A      Enables forwarding of the authentication agent connection.  This can also be specified on a per-host basis in a configuration
             file.

             Agent forwarding should be enabled with caution.  Users with the ability to bypass file permissions on the remote host (for the
             agent's UNIX-domain socket) can access the local agent through the forwarded connection.  An attacker cannot obtain key material
             from the agent, however they can perform operations on the keys that enable them to authenticate using the identities loaded into
             the agent.
             
     -t      Force pseudo-terminal allocation.  This can be used to execute arbitrary screen-based programs on a remote machine, which can be
             very useful, e.g. when implementing menu services.  Multiple -t options force tty allocation, even if ssh has no local tty.