Recovering current_mirror

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Recovering current_mirror

Jacob Anawalt-2
Hi,

I've been using rdiff-backup for a while, via backupninja, and for the
most part things work very well. Recently though due to a upgrade
configuration error with the Linux Container (LXC) my backups are in
it lost it's current_mirror file. It does not have two, it has zero.
Yes, the thing that shouldn't happen [0] happened. My destination
directory is not corrupt. I do not have recovered files in lost+found.
I just had the container running itself twice and I suspect the
"delete one of the two current_mirror.* files" happened twice leaving
me with zero.

As you know, when the rdiff-backup-data folder has no current_mirror
file when you try to run most commands they complain and exit. The
messages vary:

rdiff-backup --list-increments /target

Fatal Error: Bad rdiff-backup-data dir on destination side
...
exists, but we cannot find a valid current_mirror marker
...

rdiff-backup --verify /target
Fatal Error: Could not get time of current mirror


I found and tried rdiff-backup-regress [1] but it also wanted that
current_mirror file. The message suggested creating a new
current_mirror stamp file and explained what it needed to have. This
was the key to getting things working. Listing the last two
mirror_metadata* files in the rdiff-backup-data directory I used their
timestamp as a guide and a fake PID to create my new current_miorror
stamp file:

# ls -ltr mirror_metadata*|tail -n2
-rw------- 1 root root 31221 Feb 16 01:00
mirror_metadata.2015-02-16T01:00:02-07:00.snapshot.gz
-rw------- 1 root root     0 Feb 16 01:00
mirror_metadata.2015-02-15T01:00:01-07:00.diff

#echo "PID 666" >
/rdiffs/halifax/rdiff-backup-data/current_mirror.2015-02-16T01\:00\:02-07\:00.data

This seemed to fix things up. list increments, verify, and the default
/src /target backup. The current_mirror $timestamp needs to exactly
match the mirror_metadata.{$timestamp}.snapshot.gz one or list and
verify will appear to work but backup will die in metadata.py on
assert inclist[-1].getinctime() == time, inclist[-1]

It seems that rdiff-backup knows what the stamp should be, and it
knows that mirror_metadata exists, and there are a plethera of other
files for it to look at like session_statistics.{$timestamp}.dat,
file_statistics.{$timestamp}.data.gz,
extended_attributes.{$timestamp}.snapshot,
error_log.{$timestamp}.data, and possibly most helpful, backup_log
that rdiff-backup could be a little more robust about the
current_mirror file or a little more helpful to the user when things
fail in suggesting where they might be able to restore back to.

Of course this is all mile-high observations without really
understanding the code and how hard it is but it is an area where I
feel rdiff-backup could use the most work. It is great that as a
mirror copy everything is fine even if you have to blow away
rdiff-backup-data when something goes wrong but then I almost may as
well just use rsync.

Are there commands that I have overlooked that can examine the data in
rdiff-backup-data and determine which last backup was successful and
set current_mirror back to that time or that would in some other way
recover current_mirror when it is missing and then validate the
reverse incremental backups to see which are valid?


0. http://www.nongnu.org/rdiff-backup/FAQ.html#regress_failure
1. http://lists.gnu.org/archive/html/rdiff-backup-users/2011-01/msg00030.html

Thanks,
--
Jacob

_______________________________________________
rdiff-backup-users mailing list at [hidden email]
https://lists.nongnu.org/mailman/listinfo/rdiff-backup-users
Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki
Reply | Threaded
Open this post in threaded view
|

Re: Recovering current_mirror

Alvin Starr
rdiff-backup keeps an the most current image of your backup in the top
level directory along with the rdiff-backup-data tree.

So you have the more current data sitting in that directory.

Use rsync or tar to backup that toplevel directory (less the
rdiff-backup-data) and then restore that on your system.

That data should be good so long as your last backup completed and did
not die half way.


On 02/18/2015 02:02 PM, Jacob Anawalt wrote:

> Hi,
>
> I've been using rdiff-backup for a while, via backupninja, and for the
> most part things work very well. Recently though due to a upgrade
> configuration error with the Linux Container (LXC) my backups are in
> it lost it's current_mirror file. It does not have two, it has zero.
> Yes, the thing that shouldn't happen [0] happened. My destination
> directory is not corrupt. I do not have recovered files in lost+found.
> I just had the container running itself twice and I suspect the
> "delete one of the two current_mirror.* files" happened twice leaving
> me with zero.
>
> As you know, when the rdiff-backup-data folder has no current_mirror
> file when you try to run most commands they complain and exit. The
> messages vary:
>
> rdiff-backup --list-increments /target
>
> Fatal Error: Bad rdiff-backup-data dir on destination side
> ...
> exists, but we cannot find a valid current_mirror marker
> ...
>
> rdiff-backup --verify /target
> Fatal Error: Could not get time of current mirror
>
>
> I found and tried rdiff-backup-regress [1] but it also wanted that
> current_mirror file. The message suggested creating a new
> current_mirror stamp file and explained what it needed to have. This
> was the key to getting things working. Listing the last two
> mirror_metadata* files in the rdiff-backup-data directory I used their
> timestamp as a guide and a fake PID to create my new current_miorror
> stamp file:
>
> # ls -ltr mirror_metadata*|tail -n2
> -rw------- 1 root root 31221 Feb 16 01:00
> mirror_metadata.2015-02-16T01:00:02-07:00.snapshot.gz
> -rw------- 1 root root     0 Feb 16 01:00
> mirror_metadata.2015-02-15T01:00:01-07:00.diff
>
> #echo "PID 666" >
> /rdiffs/halifax/rdiff-backup-data/current_mirror.2015-02-16T01\:00\:02-07\:00.data
>
> This seemed to fix things up. list increments, verify, and the default
> /src /target backup. The current_mirror $timestamp needs to exactly
> match the mirror_metadata.{$timestamp}.snapshot.gz one or list and
> verify will appear to work but backup will die in metadata.py on
> assert inclist[-1].getinctime() == time, inclist[-1]
>
> It seems that rdiff-backup knows what the stamp should be, and it
> knows that mirror_metadata exists, and there are a plethera of other
> files for it to look at like session_statistics.{$timestamp}.dat,
> file_statistics.{$timestamp}.data.gz,
> extended_attributes.{$timestamp}.snapshot,
> error_log.{$timestamp}.data, and possibly most helpful, backup_log
> that rdiff-backup could be a little more robust about the
> current_mirror file or a little more helpful to the user when things
> fail in suggesting where they might be able to restore back to.
>
> Of course this is all mile-high observations without really
> understanding the code and how hard it is but it is an area where I
> feel rdiff-backup could use the most work. It is great that as a
> mirror copy everything is fine even if you have to blow away
> rdiff-backup-data when something goes wrong but then I almost may as
> well just use rsync.
>
> Are there commands that I have overlooked that can examine the data in
> rdiff-backup-data and determine which last backup was successful and
> set current_mirror back to that time or that would in some other way
> recover current_mirror when it is missing and then validate the
> reverse incremental backups to see which are valid?
>
>
> 0. http://www.nongnu.org/rdiff-backup/FAQ.html#regress_failure
> 1. http://lists.gnu.org/archive/html/rdiff-backup-users/2011-01/msg00030.html
>
> Thanks,


--
Alvin Starr                   ||   voice: (905)513-7688
Netvel Inc.                   ||   Cell:  (416)806-0133
[hidden email]              ||


_______________________________________________
rdiff-backup-users mailing list at [hidden email]
https://lists.nongnu.org/mailman/listinfo/rdiff-backup-users
Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki