Scheduled Linux rsync remote incremental file backup

rsync is a data mirroring and backup tool for Linux systems. Using the fast incremental backup tool Remote Sync allows for remote synchronization, supporting local copying or synchronization with other SSH or rsync hosts.

rsync can mirror an entire directory tree and file system. It can easily maintain the original file's permissions, timestamps, symbolic and hard links, and more. Installation does not require special permissions.

Fast: During the first synchronization, rsync will copy everything, but on subsequent synchronizations, only modified files will be transmitted. rsync can perform compression and decompression during data transmission, allowing for lower bandwidth usage.

Secure: Files can be transmitted using scp, ssh, and other methods, and a direct socket connection is also possible. It supports anonymous transmission for easy website mirroring.

Incremental backup, similar to committing code in git, means that during each synchronization, only the modified part of the files needs to be synchronized, avoiding a full backup of all files each time.

First, refer to the simple method for passwordless login in Linux at https://www.whuanle.cn/archives/1321 to add the local machine's key to the remote machine. This way, you won't need to enter a password when using ssh.

Command example:

rsync -rtv -e ssh ubuntu@192.168.3.3:/var/blog/www /opt/blog_bak/
# If you need to set the ssh port
rsync -rtv -e "ssh -p 1022" ubuntu@192.168.3.3:/var/blog/www /opt/blog_bak/

Here, -e ssh... indicates the backup method used, which is remote backup in this case. You can also use local backup to back up from one directory to another. The -rtv option provides detailed output of the files being synchronized, indicating whether to preserve the original creation time of the files, etc.

• -v, --verbose detailed output.
• -r, --recursive process subdirectories recursively.
• -t, --times preserve file time information.

You can also combine this with the built-in scheduling service in Linux to execute backup commands regularly. https://www.whuanle.cn/archives/1438

-v, --verbose detailed output.
-q, --quiet minimal output mode.
-c, --checksum enable checksum checking to force verification of file transfers.
-a, --archive archive mode, indicating to transfer files recursively and preserve all file attributes, equivalent to -rlptgoD.
-r, --recursive process subdirectories recursively.
-R, --relative use relative path information.
-b, --backup create backups; if the same filename already exists at the destination, rename the old file to ~filename. The --suffix option can be used to specify a different backup file prefix.
--backup-dir store backup files (like ~filename) in this directory.
-suffix=SUFFIX define backup file prefix.
-u, --update only update, skipping any files in the DST that already exist and have a later timestamp than the files being backed up, without overwriting the updated files.
-l, --links preserve symbolic links.
-L, --copy-links treat symbolic links as normal files.
--copy-unsafe-links only copy links that point to paths outside the SRC directory tree.
--safe-links ignore links that point outside the SRC directory tree.
-H, --hard-links preserve hard links.
-p, --perms preserve file permissions.
-o, --owner preserve file owner information.
-g, --group preserve file group information.
-D, --devices preserve device file information.
-t, --times preserve file time information.
-S, --sparse handle sparse files in a special way to save space in the DST.
-n, --dry-run show which files will be transferred.
-w, --whole-file copy files without performing incremental checks.
-x, --one-file-system do not cross filesystem boundaries.
-B, --block-size=SIZE define the block size used by the checksum algorithm, default is 700 bytes.
-e, --rsh=command specify the method to use for data synchronization, such as rsh or ssh.
--rsync-path=PATH specify the path to the rsync command on the remote server.
-C, --cvs-exclude use the same method as CVS to automatically ignore files, to exclude those not intended for transfer.
--existing only update files that already exist in the DST, and do not back up newly created files.
--delete remove files from DST that are not in SRC.
--delete-excluded also remove files from the receiving end that are excluded by this option.
--delete-after delete after the transfer is finished.
--ignore-errors delete even if I/O errors occur.
--max-delete=NUM delete at most NUM files.
--partial keep files that have not been fully transferred for faster re-transmission later.
--force forcibly delete directories, even if they are not empty.
--numeric-ids do not match numeric user and group IDs to usernames and group names.
--timeout=time set the IP timeout, in seconds.
-I, --ignore-times do not skip files that have the same timestamp and length.
--size-only when determining whether to back up files, only consider file size and not timestamp.
--modify-window=NUM set the timestamp window to decide if files are the same when timestamps match, default is 0.
-T --temp-dir=DIR create temporary files in DIR.
--compare-dest=DIR compare files from DIR to determine if they need to be backed up.
-P equivalent to --partial.
--progress show the backup process.
-z, --compress compress files during transfer.
--exclude=PATTERN specify patterns of files to exclude from transfer.
--include=PATTERN specify patterns of files to include for transfer.
--exclude-from=FILE exclude files specified by the patterns in FILE.
--include-from=FILE include files matching the patterns specified in FILE.
--version print version information.
--address bind to a specific address.
--config=FILE specify another configuration file, not using the default rsyncd.conf file.
--port=PORT specify a different rsync server port.
--blocking-io use blocking I/O for remote shell.
-stats give transfer status for certain files.
--progress show transfer progress.
--log-format=formAT specify the format of log files.
--password-file=FILE get the password from FILE.
--bwlimit=KBPS limit I/O bandwidth, KBytes per second.
-h, --help display help information.