January 17

Linux: Using rsync to mirror data between servers

This LinuxAnswer describes how to mirror 2 systems using rsync over ssh.
I will only talk about a live server and a backup server where the backup server will connect to the live server to pull the data that is to be backed up.

1) You know how to open up a terminal and type a few basic commands.
2) You have a working ssh server and client installed. If not then see:
3) You have private/public keys generated to allow passwordless logins to the live server form the backup server.

Why would you want to?
There are many reasons so I’ll just list a few:
1) Data transfer is fast as rsync only copies modified files
2) Running it over ssh encrypts the data transfer so it is more secure than other methods

The real howto
1) Decide on the directories you need to backup on the live server assuming it is a webserver this may be “/home/httpd”
2) Decide on the options you want. The most common I would use are:
-a Archive mode this is a combination of “-rlptgoD” basically it works recursively and maintains file information such as creation dates, permissions etc. See the man page for detailed info.
-v Increase the verbosity. This will let you see what is transferred
-z Compress data so that it is a quicker transfer
–delete-after Delete any files that have been deleted on the live server
-e ssh Most importantly, run the transfer over an ssh connection
A full list can be obtains from “man rsync”.
3) Try a dry run on the backup server with “-n” to make sure any typos don’t totally screw your system. This will just show what would be done:
rsync -e ssh -avzn –delete-after user@liveserver:/home/httpd /home
4) If everything went as expected you can give it a go without -n
rsync -e ssh -avz –delete-after user@liveserver:/home/httpd /home
You should get the info about the files being transferred. Running it again should be quicker as very little has probably changed.
5) That should be it, just try creating and deleting a few files and run rsync to make sure the changes occur

Automating the process
The obvious answer running the rsync commands on the backup server via cron.
A basic example being to mirror every hour on the hour:
0 * * * * rsync -e ssh -avz user@liveserver:/home/httpd /home 2>&1 > /var/log/hourly_backup.log
Then remove deleted files every night:
30 0 * * * rsync -e ssh –delete-after -avz user@liveserver:/home/httpd /home 2>&1 > /var/log/nightly_backup.log

By: D. Ross

Copyright 2021. All rights reserved.

Posted January 17, 2014 by Timothy Conrad in category "Linux

About the Author

If I were to describe myself with one word it would be, creative. I am interested in almost everything which keeps me rather busy. Here you will find some of my technical musings. Securely email me using - PGP: 4CB8 91EB 0C0A A530 3BE9 6D76 B076 96F1 6135 0A1B