Talkback for article: 326, March2004

Rsync: The Best Backup System Ever

Back to: http://cgi.linuxfocus.org/English/March2004/article326.shtml

From: Troels Arvin <troels(at)arvin.dk> [ date: 2004-03-02 ]
I have used rsync for backup purposes in the past. I have now switched to "rdiff-backup". rdiff-backup uses the same algorithms as rsync (by way of the "librsync" library) but adds a number of very nice features. Have a look at http://rdiff-backup.stanford.edu/
From: Jeff Childs <jeff at naihl dot net> [ date: 2004-03-02 ]
This looks like a great solution, and one I'm looking forward to trying soon. The nice thing about this solution is that I can still back my data up to tape and move it offsite, but instead of doing it on a bunch of servers, I can do it on just one.
From: Wulf <LFrsync.3.wulf(at)spamgourmet.com> [ date: 2004-03-19 ]
I've recently begun playing around with a Linux based backup system for my home network along these lines - this article will help me refine it. Thanks for helping bring things into focus :)

Wulf
From: Radoslaw Wisneiwski <radzimir(at)polbox.com> [ date: 2004-04-06 ]
And how you do you make backup for backup server?
Imagine you lost backup server AND one of the other servers. Then you data is lost.
The solution could be to make incremental backup containing actual version of the changed file or the new file. Then one full backup and set of incremental backups allow to survive such an disaster.
But in the article stands, incremental directories contains previous versions of the file, not actual.

Is it possible to change that?
From: femi solagbade <strange279(at)yahoo.com> [ date: 2004-04-10 ]
hi,

i read your article with great interest. I intend to make this my 1st linux project (i've been a windows guy).

your example talks about having 50-100 days backup on one disk. well my organisation needs to have daily backups removed from the premises and I've been trying to figure out how to achieve this. can you help?
From: Claus P fleger <claus DOT pfleger AT gw-stpoelten DOT com> [ date: 2004-04-24 ]
I wonder if it is possible to save & keep permissions and rights of files on a NT 4 fileserver with rsync?


From: Michael van Eeden [ date: 2004-04-29 ]
I have used rsync for years to do backup, it is very powerful. But about a year ago i found BackupPC and it is perfect, it does everything you ever wanted from a backup system and saves you from creating all the scripts to let rsync do exactly what you want. It can use rsync for the transfer of the files but also other methods.

http://backuppc.sourceforge.net/

From: swirly <laurent.cooper_AT_ac-lyon.fr> [ date: 2004-06-01 ]
a nice project for rsync backup is dirvish.
www.pegasys.ws/dirvish/
nice, functionnal and light.

From: Sven C. Merckens <merckenssc(at)bigfoot.com> [ date: 2004-06-19 ]
Hi,
scripts looks great, I was looking for exactly this functionality.

I use it only local to save up my data on a FireWire-HD.

So I modified the paths and tried it.

The scripts makes the backup correct, but the archiving of the modified files does not work.

It only creates the folder with the actual date and copies some data (only stuff from one folder) in it, but I get a lot of error-messages of that kind:

stat "/Volumes/G5Test/TESTZIEL/2004-06-19/Library/Safari/History.plist" failed: No such file or directory

The script is running with "sudo", so this canīt be a privilige-problem.

The folder "/Volumes/G5Test/TESTZIEL/2004-06-19/Library/" exists (it was created by the script).

What can be the problem here?
I also donīt understsand why the script does not send out a eMail-Message if errors have occured. Only after a "error-less" run, I get a report via eMail.

Many thanks in advance

Sven C.
From: Dwaine Castle <dcastle(at)carolina.rr.com> [ date: 2004-07-18 ]
Great article. Thank you.
From: Wil <support at sa-maastricht dot net> [ date: 2004-08-03 ]
Great solution,

Just 1 question (apparently 2):
how to preserve ACLs from a nmbfs share? Whats best policy ie mount a share with smbfs on the backup server and do a local rsync or pipe it? The backuped up data need to be a samba share also, accesable by the ntadmin group.

Thx for answers.
From: Erling [ date: 2004-09-02 ]
Nice article, thanks!

I use this solution on my home network. The important stuff on my iBook is
synced with my desktop pc, I keep one week of incremental backups.
As I have little data everyone fits nicely on a 4GB dds2 tape, which also
includes all important files on the desktop pc. This soution has saved me
several times.
From: carlg [ date: 2004-11-26 ]
Great article....I noticed in the script about the directory to be backed up:
<snip>
BACKUPDIR=directory_you_want_to_backup
<snip>

What if you had seven (7) different directories you wanted backed up with rsync?

Would you just add 6 additional BACKUPDIR=directory_you_want_to_backup statements that would cover all seven (7) directories????
From: Anonymous User [ date: 2004-12-18 ]
good question, I too was thinking the same thing regarding multi directories.

From: Oscar Santacreu [ date: 2005-03-02 ]
Multiple directories: Use function :-)

#!/bin/sh

function do_copy {

(all commands to do copy from $ORIGIN to $DESTINATION)

}

ORIGIN="/var"
DESTINATION="/backup/var"
do_copy

ORIGIN="/home"
DESTINATION="/backup/home"
do_copy

and so on...

And a very interesting page:
http://www.mikerubel.org/computers/rsync_snapshots/

Best
O.
From: Fab [ date: 2005-03-04 ]
Rsync is very practical but is not a backup software. To maintain a mirror is not the same as to backup data. The multi-localization is required : if you have got a fire on your backup server, all is destructed, which could be consistently a matter.

I agree with the utility of miroring, it is a good tool for crash recovery, but the comparison with bands is a quite skewed. Bands are always required if you do backup media dissemination.
From: Diccon Tesson [ date: 2005-04-16 ]
Was reassuring to find someone who came up with the same ideas as myself. The only thing i hadnt factored in was the incremental backups.
Its a great solution.
We have some NAS machines (snap servers) that are backed up with an autoloader, but if there was a problem with the auto loader then we would have no method of recovery, and the dam things are expensive to replace.
I dont know about anyone else but the most important thing is the current data, its handy to have a historical incremental backup but if the backup server crashes/burns/explodes then its on a different site to the NAS device so we still have our data. Visa versa too.
Great idea, the concept of having an instant stop gap seriously appeals too.
Many thanks
(going to check the first run of rsync, wahoo)
From: Taz <dominique(at)inetys.fr> [ date: 2005-05-27 ]
Smaching,
backup system is up after a couple of hours of brainstorming....it is a very good way to backup datas simply and fastly !

one question after all :
some of the log file i'm backing up are not readable (like auth.log or access.log...), it's like they where put in a binary format...all the file of the website are ok (ie htm and php extensions), but not those af the log directory... any clue about this phenomenon ?
From: Amit sharma <amitsharma_26(at)hotmail.com> [ date: 2005-10-20 ]
QUERIES:

Suppose we have about 1000 files at location 'A' & 500 files at location 'B' & we want them to be transmitted to each other (not exactly synchronizing two location). RSYNC can do that, no issues on that. But i have few queries further.

My first query:
1. Can we make some sort of queuing procedure alongwith or through RSYNC. Like in which we can provide priorities to particular JOBs in a queue. Even if we are not able to make a seperate system for queuing, Is there a way to change the priorities for my Jobs ( Like by default, jobs are processed alphabeticaly or numerically & that is the default priority order).

But what if i have been left with another 600 files to transmit & i want last 100 to come at top of this queue or i want these last 100 jobs to be transmitted on urgent priority. How to assign priorities to jobs.

Second query:
2. Is there a way to confirm online(not through log files) what exactly has been transmitted & what is left to be transmitted. (For example we have the --progress & -vvv verbosity options for telling me what exactly is happening at this moment)

Third query:
3. Also i wanted to know , is it possible for RSYNC to transmit more than 1 files at a same time ?


Please Oblige me with your suggestions.

19 talkbacks




Due to the increased amount of web spam we have deciced to removed the talkback posting possibility. You can read old talkbacks but you can no longer post new ones.

Back to http://cgi.linuxfocus.org/English/March2004/article326.shtml

Please contact webmaster(at)linuxfocus.org if you have any questions with regards to this talkback

lftalkback version 3.10