rsync Cheat Sheet and How To Guide

Overview: What is rsync?

rsync (remote sync) is a powerful file synchronization tool used to copy and synchronize files and directories locally or remotely. It is commonly used for backup, mirroring data, or transferring files efficiently between systems. One of its key features is the ability to copy only the differences between source and destination (incremental backups), which makes it highly efficient.

Rsync  is  a  fast and extraordinarily versatile file copying tool.  It can copy locally, to/from  another  host  over  any  remote  shell,  or to/from  a  remote  rsync  daemon.  It offers a large number of options that control every aspect of its  behavior  and  permit  very  flexible specification  of  the set of files to be copied.  It is famous for its delta-transfer algorithm, which reduces the amount of  data  sent  over the  network  by  sending only the differences between the source files and the existing files in the destination.  Rsync is  widely  used  for backups and mirroring and as an improved copy command for everyday use.

Note: read the man page of the version of Unix you may be running as the options used below may differ slightly.

Package Installation

rsync generally comes standard with Solaris 11 else

# pkg install rsync

In Linux you may need to install the package and note the installation method for the appropriate distribution

# yum -y install rsync

or

# apt install rsync

General Command Syntax

The basic syntax for rsync is:

$ rsync [options] source destination

  • the source and destination can be:
    • file
    • directory
    • host:/path
  • source: The path to the file or directory you want to copy.
  • destination: The location to copy the file or directory to. This can be a local path or a remote system in the form of [user@]hostname:/path.

Sample usage:

$ rsync [OPTION…] SRC… [DEST]               – local host

$ rsync [OPTION…] SRC… SRC… [DEST]        – local host

$ rsync [OPTION…] [USER@]HOST:SRC… [DEST]   – remote pull

$ rsync [OPTION…] SRC… [USER@]HOST:DEST     – remote push

Commonly Used Options

  1. -a (archive mode)
  • Purpose: Preserves symbolic links, file permissions, timestamps, and recursive copy of directories.
  • Use when: You want a comprehensive copy that retains the structure of the original.
  1. -v (verbose)
  • Purpose: Provides detailed information during the operation.
  • Use when: You want to see a list of files being transferred.
  1. -z (compress)
  • Purpose: Compresses data during the transfer to reduce bandwidth usage.
  • Use when: You are transferring large files over slow networks.
  1. -r (recursive)
  • Purpose: Recursively copies entire directories.
  • Use when: Copying directories with multiple subdirectories.
  1. –dry-run
  • Purpose: Simulates the operation, showing what would happen without actually performing the copy.
  • Use when: You want to test a command before execution to avoid errors.
  1. –delete
  • Purpose: Deletes files from the destination that no longer exist in the source.
  • Use when: Keeping the destination exactly the same as the source (use with caution).
  1. -P
  • Purpose: Shows progress during transfer and handles partial transfers.
  • Use when: Generally used you manually execute rsync and avoid when scripting rsync and within cron.

Basic Usage Examples

Copying Files Locally

Scenario: You want to copy files from one directory to another on the same machine.

$ rsync -av /home/user/docs/ /home/user/backup_docs/

  • Explanation:
    • -a: Archive mode to preserve permissions and timestamps.
    • -v: Verbose mode to show the progress.
    • /home/user/docs/: Source directory to copy.
    • /home/user/backup_docs/: Destination directory.

Copying Files to a Remote Host

Scenario: You need to copy files to a remote server.

$ rsync -av /home/user/docs/ user@host1:/home/user/backup_docs/

  • Explanation:
    • user@host1: The remote system’s username and hostname.
    • The -av flags ensure all files, permissions, and timestamps are preserved.
    • /home/user/docs/: The local source directory.
    • /home/user/backup_docs/: The remote destination directory.

Copying Files from a Remote Host

Scenario: You want to copy files from a remote server to your local machine.

$ rsync -av user@host1:/home/user/docs/ /home/user/backup_docs/

 

  • Explanation:
    • user@host1:/home/user/docs/: the source is a user and directory
    • /home/user/backup_docs/: the destination is a local directory).

Advanced Usage Examples

 

Synchronizing Two Directories with Different Names

Scenario: You want to sync the contents of two directories, but their names differ.

$ rsync -av /home/user/docs/ /home/user/backup_docs/

  • Explanation: The trailing slash (/) on the source (/home/user/docs/) means the contents of that directory will be copied to the destination directory (/home/user/backup_docs/).

If you omit the trailing slash from the source:

$ rsync -av /home/user/docs /home/user/backup_docs/

  • Explanation: The entire docs directory will be copied into backup_docs/ as a subdirectory, rather than just its contents.

Using Remote Shell (SSH)

Scenario: You want to copy files between two remote servers using SSH for security.

$ rsync -av -e ssh user@host1:/home/user/docs/ user@host2:/home/user/backup_docs/

  • Explanation:
    • -e ssh: Specifies using SSH as the transport method for security.

Using Remote Shell (SSH) on a Specific Port

Scenario: Use a non-default SSH port for the transfer.

$ rsync -av -e ‘ssh -p 2222’ user@host1:/home/user/docs/ user@host2:/home/user/backup_docs/

  • Explanation:
    • -e ‘ssh -p 2222’ option specifies SSH to use port 2222.

Limiting Bandwidth Usage

Scenario: You want to limit the transfer speed to avoid overloading the network.

$ rsync -av –bwlimit=1000 /home/user/docs/ user@host1:/home/user/backup_docs/

  • Explanation:
    • –bwlimit=1000: Limits the transfer speed to 1000 KB/s.

Using –dry-run to Preview Changes

Scenario: You want to ensure the correct files will be copied before running the actual command.

$ rsync -avn /home/user/docs/ user@host1:/home/user/backup_docs/

 

  • Explanation:
    • –dry-run | -n : This will simulate the operation and show you which files would be transferred without actually copying them.

Compare the Source and Destination

Scenario: You want to Compare files in the source and destination without transferring.

$ rsync -avi /home/user/docs/ user@host1:/home/user/backup_docs/

 

  • Explanation:
    • -i option outputs a detailed list of changes, showing which files would be transferred.

Copying and Deleting Extra Files

Scenario: You want to synchronize two directories and remove any files on the destination that no longer exist on the source.

$ rsync -av –delete /home/user/docs/ user@host1:/home/user/backup_docs/

  • Explanation:
    • –delete: Deletes files in the destination that are no longer present in the source.

Using -z to Compress Data During Transfer

Scenario: You are transferring large files over a slow network, and you want to reduce bandwidth usage.

$ rsync -avz /home/user/large_files/ user@host2:/home/user/backup_files/

  • Explanation:
    • -z: Compresses the files during transfer to save bandwidth.

Using Exclude Patterns

Scenario: You want to copy a directory but exclude certain file types (e.g., .log files).

$ rsync -av –exclude=’*.log’ /home/user/docs/ user@host1:/home/user/backup_docs/

  • Explanation:
    • –exclude=’*.log’: Excludes files that end with .log from being transferred.

Using Include and Exclude Patterns

Scenario: similarly, you want to copy a directory but include some and exclude certain file types.

$ rsync -avz –include=’*.txt’ –exclude=’*’ /path/to/source/ user@host1:/path/to/destination/

  • Explanation:
  • –include=’*.txt’ option includes only .txt files, while –exclude=’*’ excludes everything else.

Preserve Hard Links

Scenario: Preserve hard links during the transfer.

$ rsync -avzH /path/to/source/ user@host1:/path/to/destination/

  • Explanation:
    • -H option preserves hard links, which is useful for complex directory structures.

Resume Failed Transfers

Scenario: Resume a partially completed transfer due to network or session disconnects.

$ rsync -avz –partial /path/to/source/ user@host1:/path/to/destination/

  • Explanation:
    • –partial option keeps partially transferred files, allowing you to resume the transfer later.

Log the Transfers

Scenario: Log the rsync operation to a file.

$ rsync -avz –log-file=/path/to/rsync.log /path/to/source/ user@host1:/path/to/destination/

  • Explanation:
    • –log-file option writes the transfer details to a log file.

Real-Life Examples

 

Backup a Website to a Remote Server

Backup a local website directory to a remote server for disaster recovery.

$ rsync -avz –delete /var/www/html/ user@host1:/backup/website/

  • Note: This syncs the local /var/www/html/ directory to the remote /backup/website/ directory, deleting any extraneous files on the remote server.

Mirror a Directory Between Two Servers

Mirror a directory from one remote server to another.

$ rsync -avz user@host1:/path/to/source/ user@host2:/path/to/destination/

  • Note: This syncs the contents of /path/to/source/ on host1 to /path/to/destination/ on host2.

Sync Logs with Compression and Bandwidth Limit

Sync log files from a remote server to a local machine, compressing data and limiting bandwidth.

$ rsync -avz –bwlimit=500 –include=’*.log’ –exclude=’*’ user@host1:/var/log/ /backup/logs/

  • Note: This transfers only .log files from /var/log/ on host1 to /backup/logs/ locally, with a bandwidth limit of 500 KB/s.

Resume a Failed Transfer

Resume a large file transfer that was interrupted.

$ rsync -avz –partial /path/to/largefile.tar.gz user@host1:/path/to/destination/

  • Note: The –partial option allows rsync to resume the transfer from where it left off.

Relative and Absolute Patterns

From the man page:

ANCHORING INCLUDE/EXCLUDE PATTERNS

As  mentioned  earlier, global include/exclude patterns are anchored at the “root of the transfer” (as opposed to per-directory patterns, which are anchored at the merge-file’s  directory). If  you  think of the transfer as  a subtree of names that are being sent from sender to re- ceiver, the transfer-root is where the tree starts to be duplicated in the destination directory. This root governs where patterns that start with a / match.

Because  the  matching  is relative to the transfer-root, changing the trailing slash on a source path or changing your use of the  –relative option  affects the path you need to use in your matching (in addition to changing how much of the file tree is duplicated on the  destination host). The following examples demonstrate this.

Let’s  say that we want to match two source files, one with an absolute path of “/home/me/foo/bar”, and one with a path of “/home/you/bar/baz”. Here is how the various command choices differ for a 2-source transfer:

Example cmd: rsync -a /home/me /home/you /dest

+/- pattern: /me/foo/bar

+/- pattern: /you/bar/baz

Target file: /dest/me/foo/bar

Target file: /dest/you/bar/baz

 

Example cmd: rsync -a /home/me/ /home/you/ /dest

+/- pattern: /foo/bar               (note missing “me”)

+/- pattern: /bar/baz               (note missing “you”)

Target file: /dest/foo/bar

Target file: /dest/bar/baz

 

Example cmd: rsync -a –relative /home/me/ /home/you /dest

+/- pattern: /home/me/foo/bar       (note full path)

+/- pattern: /home/you/bar/baz      (ditto)

Target file: /dest/home/me/foo/bar

Target file: /dest/home/you/bar/baz

 

Example cmd: cd /home; rsync -a –relative me/foo you/ /dest

+/- pattern: /me/foo/bar      (starts at specified path)

+/- pattern: /you/bar/baz     (ditto)

Target file: /dest/me/foo/bar

Target file: /dest/you/bar/baz

The easiest way to see what name you should filter is to just look  at the output when using –verbose and put a / in front of the name (use the –dry-run option if you’re not yet ready to copy any files).

If the source directory has a trailing slash, the command will copy only the directory contents to the destination directory.

When the trailing slash is omitted, rsync copies the source directory inside the destination directory. ie if using no trailing slash on either source or destination a directory named as per the source (eg /u01) will be created under the path specified at the destination (ie /u01/u01)

A  trailing  slash on the source changes this behavior to avoid creating an additional directory level at the destination.

You can think of a trailing / on a source as meaning “copy the contents of this directory” as opposed to “copy the directory by name”, but in both cases the attributes of the containing directory are transferred to the containing directory on the destination. 

# rsync -avz foo:src/bar/ /data/tmp

 

Each of the following commands copies the files in the same way, including their setting of the attributes of /dest/foo:

# rsync -av /src/foo /dest

# rsync -av /src/foo/ /dest/foo

 

Excluding / Including Files

From the man page:

    –exclude=PATTERN       exclude files matching PATTERN

    –exclude-from=FILE     read exclude patterns from FILE

    –include=PATTERN       don’t exclude files matching PATTERN

    –include-from=FILE     read include patterns from FILE

    –files-from=FILE       read list of source-file names from FILE

 

There are two options to exclude files and directories.

The first option is to use the –exclude argument and specify the files and directories you want to exclude on the command line.

When excluding files or directories , you need to use their relative paths to the source location.

The following example shows how to exclude the crash and tmp directories:

# rsync -a –exclude=crash –exclude=tmp /source_directory/ /destination_directory/

 

The second option is to use the –exclude-from option and specify the files and directories you want to exclude from a file.

# cat /exclude-file.txt

crash

tmp

# rsync -a –exclude-from=’/exclude-file.txt’ /source_directory/ /destination_directory/

 

You can also specify what files to include in a similar way

/tmp/files-to-include contains specific files to rsync relative to the source directory

# rsync -a –files-from=/tmp/files-to-include /u01 rem-host:/u01-backup

 

Files-from can be read from the remote host

Copy all the files specified in the /tmp/files-to-include file that was located on the remote “src” host.

# rsync -a –files-from=:/tmp/files-to-include rem-host:/ /tmp/copy

 

Summary of Key Options

 

Option Description Example Usage
-a Archive mode (preserves structure, permissions, etc.) rsync -av source/ destination/
-v Verbose output (shows details) rsync -av source/ destination/
-z Compresses files during transfer rsync -avz source/ destination/
–delete Deletes files on the destination that are missing from source rsync -av –delete source/ destination/
–dry-run Simulates the command without making changes rsync -av –dry-run source/ destination/
–exclude Excludes files or directories rsync -av –exclude=’*.log’ source/ destination/
-e ssh Use SSH for secure transfers rsync -av -e ssh source/ user@host:/path/
–bwlimit=KB/s Limits transfer speed rsync -av –bwlimit=1000 source/ destination/

rsync Output Explanation

 

The format of the rsync output is;

XYcstpoguax path/to/file

And it will look something like this in the log file:

>f.st…… access_top/access_r/sam/scs/restart
.d..t…… access_top/access_r/sam/scs/run/
>f+++++++++ access_top/arch_control/ACCESS-G_sam_2014031_1
.L..t…… ashare/ASX/um/cpt/my/UM_usiFC_4D -> UM_usiFC_4D.normal
>f..t…… msas/bin/mergs
cLc.t…… rto/scs/cgi/sam/sam -> /g/sc/home/access_top/tmp/sam_sam_split2
*deleting   tzg/D_msas/data/odb_obs/ABC2014030713/stcache.ss/

This is what the characters(XYcstpoguax) in front of the file mean:

X  is replaced by the type of update being done
                    Y  is replaced by the file-type

cstpoguax  these letters represent attributes that may be output if they are being modified

 

The update types that replace the X are as follows:

  • < means that a file is being transferred to the remote host (sent).
  • > means that a file is being transferred to the local host (received).
  • c means that a local change/creation is occurring for the item (such as the creation of a directory or the changing of a symlink, etc.).
  • h means that the item is a hard link to another item (requires –hard-links).
  • . means that the item is not being updated (though it might have attributes that are being modified).
  • * means that the rest of the itemized-output area contains a message (e.g. “deleting”).

The file-types that replace the Y are:

  • f for a file
  • d for a directory
  • L for a symlink
  • D for a device
  • S for a special file (e.g. named sockets and fifos).

cstpoguax  These letters are the actual letters that will be output if the associated attribute for the item is being updated or a “.” for no change.
Three exceptions to this are:

  1. a newly created item replaces each letter with a “+
  2. an identical item replaces the dots with spaces
  3. an unknown attribute replaces each letter with a “?” (this can happen when talking to an older rsync).

The attribute that is associated with each letter is as follows:

  • c means either that a regular file has a different checksum (requires –checksum) or that a symlink, device, or special file has a changed value. Note that if you are sending files to an rsync prior to 3.0.1, this change flag will be present only for checksum-differing regular files.
  • s means the size of a regular file is different and will be updated by the file transfer.
  • t means the modification time is different and is being updated to the sender’s value (requires –times). An alternate value of T means that the modification time will be set to the transfer time, which happens when a file/symlink/device is updated without –times and when a symlink is changed and the receiver can’t set its time. (Note: when using an rsync 3.0.0 client, you might see the s flag combined with t instead of the proper T flag for this time-setting failure.)
  • p means the permissions are different and are being updated to the sender’s value (requires –perms).
  • means the owner is different and is being updated to the sender’s value (requires –ownerand super-user privileges).
  • g means the group is different and is being updated to the sender’s value (requires –group and the authority to set the group).
  • u slot is reserved for future use.
  • a means that the ACL information changed.
  • x means that the extended attribute information changed.

One other output is possible:
when deleting files, the “%i” will output the string “*deleting” for each item that is being removed (assuming that you are talking to a recent enough rsync that it logs deletions instead of outputting them as a verbose message).

All of this can be simplified like this:

XYcstpoguax path/to/file

|||||||||||

`———– The type of update being done:

 ||||||||||   < file is being transferred to the remote host (sent).

 ||||||||||   > file is being transferred to the local host (received).

 ||||||||||   c local change/creation for the item, such as:

 ||||||||||    – the creation of a directory

 ||||||||||    – the changing of a symlink,

 ||||||||||    – etc.

 ||||||||||   h the item is a hard link to another item (requires –hard-links).

 ||||||||||   . the item is not being updated (though it might have attributes that are being modified).

 ||||||||||   * means that the rest of the itemized-output area contains a message (e.g. “deleting”).

 ||||||||||  

 `———- The file type:

  |||||||||   f for a file,

  |||||||||   d for a directory,

  |||||||||   L for a symlink,

  |||||||||   D for a device,

  |||||||||   S for a special file (e.g. named sockets and fifos).

  |||||||||  

  `———  c Different checksum(for regular files) or changed value(for symlink, device, and special file)

   `——–  s Size is different

    `——-  t Modification time is different

     `——  p Permission are different

      `—–  o Owner is different

       `—-  g Group is different

        `—  u The u slot is reserved for future use.

         `–  a The ACL information changed

          `-  x Extended attribute information changed

 

This cheat sheet should guide you through rsync from basic to advanced operations, making it easier to leverage its full capabilities for various tasks.

Check out our other Cheat Sheets and Blogs and if you would like us to write a cheat sheet for you, for FREE, (and we find it suitable) Contact Us.