Repair repository

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

Repair repository

Patrik Dufresne
Hello,

One of my backup repository seams to be mess up alot. I think the backup have been interrupted multiple time. I do have a very long history for this backup and I don't want to lose it.

First, I encounter "regressing destination" error. So as usual, I've delete the current_mirror. But even then, the backup refused to start with the following exception:

# /usr/bin/rdiff-backup  --print-statistics --exclude '**/.wine' --exclude '**/.PlayOnLinux' --exclude '**/.dropbox' --exclude '**/.dropbox.cache' --exclude '**/.dropbox-dist' --exclude '**/.cache/' --exclude '**/.gnome2_private' --exclude '**/.config/deluge' --exclude '**/.gstreamer-0.10' --exclude '**/.gvfs' --exclude '**/.gtk-bookmarks' --exclude '**/.local/share/Trash/' --exclude '**/.local/share/zeitgeist' --exclude '**/.local/share/gvfs-metadata' --exclude '**/.cache' --exclude '**/.config/google-chrome' --exclude '**/VirtualBox VMs' --exclude '**/Downloads' --exclude '**/.gnupg' --exclude '**/.m2/repository' --exclude '**/.metadata/.plugins/org.eclipse.e4.workbench' --exclude '**/.metadata/.plugins/org.eclipse.m2e.core/' --exclude '**/*~' --exclude '**/Steam' --exclude '**/.xchat2' --exclude '**/.xbmc' --exclude '**/.swt' --exclude '**/.eclipse' --exclude '**/.vim' --exclude '**/.ekwos' --exclude '**/.glpk' --exclude '**/workspace/*/target/*' --exclude '**/.dropbox' --include '/home/ikus060' --include '/etc/backup.d/' --include '/home/ikus060/.local/share/hamster-applet' --include '/root/.ssh' --exclude '/*' / ikus060@fente::/home/ikus060/ikus060-rdiff
Exception 'Path: /home/ikus060/ikus060-rdiff/rdiff-backup-data/mirror_metadata.2015-09-14T19:00:02-04:00.snapshot.gz
Index: ('mirror_metadata.2015-09-14T19:00:02-04:00.snapshot.gz',)
Data: {'size': 2948234, 'ctime': 1442273220, 'perms': 384, 'inode': 6447109, 'mtime': 1442273220, 'ea': <rdiff_backup.eas_acls.ExtendedAttributes instance at 0x7f7fa4731248>, 'devloc': 23L, 'uname': 'ikus060', 'nlink': 1, 'gname': 'ldap-users', 'gid': 10000, 'atime': 1442273454, 'type': 'reg', 'uid': 10000}' raised of class '<type 'exceptions.AssertionError'>':
  File "/usr/lib/python2.7/dist-packages/rdiff_backup/Main.py", line 304, in error_check_Main
    try: Main(arglist)
  File "/usr/lib/python2.7/dist-packages/rdiff_backup/Main.py", line 324, in Main
    take_action(rps)
  File "/usr/lib/python2.7/dist-packages/rdiff_backup/Main.py", line 280, in take_action
    elif action == "backup": Backup(rps[0], rps[1])
  File "/usr/lib/python2.7/dist-packages/rdiff_backup/Main.py", line 343, in Backup
    backup.Mirror_and_increment(rpin, rpout, incdir)
  File "/usr/lib/python2.7/dist-packages/rdiff_backup/backup.py", line 48, in Mirror_and_increment
    DestS.set_rorp_cache(dest_rpath, source_rpiter, 1)
  File "/usr/lib/python2.7/dist-packages/rdiff_backup/connection.py", line 450, in __call__
    return apply(self.connection.reval, (self.name,) + args)
  File "/usr/lib/python2.7/dist-packages/rdiff_backup/connection.py", line 370, in reval
    if isinstance(result, Exception): raise result

Traceback (most recent call last):
  File "/usr/bin/rdiff-backup", line 30, in <module>
    rdiff_backup.Main.error_check_Main(sys.argv[1:])
  File "/usr/lib/python2.7/dist-packages/rdiff_backup/Main.py", line 304, in error_check_Main
    try: Main(arglist)
  File "/usr/lib/python2.7/dist-packages/rdiff_backup/Main.py", line 324, in Main
    take_action(rps)
  File "/usr/lib/python2.7/dist-packages/rdiff_backup/Main.py", line 280, in take_action
    elif action == "backup": Backup(rps[0], rps[1])
  File "/usr/lib/python2.7/dist-packages/rdiff_backup/Main.py", line 343, in Backup
    backup.Mirror_and_increment(rpin, rpout, incdir)
  File "/usr/lib/python2.7/dist-packages/rdiff_backup/backup.py", line 48, in Mirror_and_increment
    DestS.set_rorp_cache(dest_rpath, source_rpiter, 1)
  File "/usr/lib/python2.7/dist-packages/rdiff_backup/connection.py", line 450, in __call__
    return apply(self.connection.reval, (self.name,) + args)
  File "/usr/lib/python2.7/dist-packages/rdiff_backup/connection.py", line 370, in reval
    if isinstance(result, Exception): raise result
AssertionError: Path: /home/ikus060/ikus060-rdiff/rdiff-backup-data/mirror_metadata.2015-09-14T19:00:02-04:00.snapshot.gz
Index: ('mirror_metadata.2015-09-14T19:00:02-04:00.snapshot.gz',)
Data: {'size': 2948234, 'ctime': 1442273220, 'perms': 384, 'inode': 6447109, 'mtime': 1442273220, 'ea': <rdiff_backup.eas_acls.ExtendedAttributes instance at 0x7f7fa4731248>, 'devloc': 23L, 'uname': 'ikus060', 'nlink': 1, 'gname': 'ldap-users', 'gid': 10000, 'atime': 1442273454, 'type': 'reg', 'uid': 10000}

Do you have some good approach to repair a repository ? May I revert the backup destination to a stable point ?

--
Patrik Dufresne


_______________________________________________
rdiff-backup-users mailing list at [hidden email]
https://lists.nongnu.org/mailman/listinfo/rdiff-backup-users
Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki
Reply | Threaded
Open this post in threaded view
|

Re: Repair repository

Dominic Raferd-3
After deleting the current_mirror file did you then run rdiff-backup with --check-destination-dir? This performs the actual regression to the previous, hopefully consistent, backup.

On 15 September 2015 at 00:46, Patrik Dufresne <[hidden email]> wrote:
Hello,

One of my backup repository seams to be mess up alot. I think the backup have been interrupted multiple time. I do have a very long history for this backup and I don't want to lose it.

First, I encounter "regressing destination" error. So as usual, I've delete the current_mirror. But even then, the backup refused to start with the following exception:

# /usr/bin/rdiff-backup  --print-statistics --exclude '**/.wine' --exclude '**/.PlayOnLinux' --exclude '**/.dropbox' --exclude '**/.dropbox.cache' --exclude '**/.dropbox-dist' --exclude '**/.cache/' --exclude '**/.gnome2_private' --exclude '**/.config/deluge' --exclude '**/.gstreamer-0.10' --exclude '**/.gvfs' --exclude '**/.gtk-bookmarks' --exclude '**/.local/share/Trash/' --exclude '**/.local/share/zeitgeist' --exclude '**/.local/share/gvfs-metadata' --exclude '**/.cache' --exclude '**/.config/google-chrome' --exclude '**/VirtualBox VMs' --exclude '**/Downloads' --exclude '**/.gnupg' --exclude '**/.m2/repository' --exclude '**/.metadata/.plugins/org.eclipse.e4.workbench' --exclude '**/.metadata/.plugins/org.eclipse.m2e.core/' --exclude '**/*~' --exclude '**/Steam' --exclude '**/.xchat2' --exclude '**/.xbmc' --exclude '**/.swt' --exclude '**/.eclipse' --exclude '**/.vim' --exclude '**/.ekwos' --exclude '**/.glpk' --exclude '**/workspace/*/target/*' --exclude '**/.dropbox' --include '/home/ikus060' --include '/etc/backup.d/' --include '/home/ikus060/.local/share/hamster-applet' --include '/root/.ssh' --exclude '/*' / ikus060@fente::/home/ikus060/ikus060-rdiff
Exception 'Path: /home/ikus060/ikus060-rdiff/rdiff-backup-data/mirror_metadata.2015-09-14T19:00:02-04:00.snapshot.gz
Index: ('mirror_metadata.2015-09-14T19:00:02-04:00.snapshot.gz',)
Data: {'size': 2948234, 'ctime': 1442273220, 'perms': 384, 'inode': 6447109, 'mtime': 1442273220, 'ea': <rdiff_backup.eas_acls.ExtendedAttributes instance at 0x7f7fa4731248>, 'devloc': 23L, 'uname': 'ikus060', 'nlink': 1, 'gname': 'ldap-users', 'gid': 10000, 'atime': 1442273454, 'type': 'reg', 'uid': 10000}' raised of class '<type 'exceptions.AssertionError'>':
  File "/usr/lib/python2.7/dist-packages/rdiff_backup/Main.py", line 304, in error_check_Main
    try: Main(arglist)
  File "/usr/lib/python2.7/dist-packages/rdiff_backup/Main.py", line 324, in Main
    take_action(rps)
  File "/usr/lib/python2.7/dist-packages/rdiff_backup/Main.py", line 280, in take_action
    elif action == "backup": Backup(rps[0], rps[1])
  File "/usr/lib/python2.7/dist-packages/rdiff_backup/Main.py", line 343, in Backup
    backup.Mirror_and_increment(rpin, rpout, incdir)
  File "/usr/lib/python2.7/dist-packages/rdiff_backup/backup.py", line 48, in Mirror_and_increment
    DestS.set_rorp_cache(dest_rpath, source_rpiter, 1)
  File "/usr/lib/python2.7/dist-packages/rdiff_backup/connection.py", line 450, in __call__
    return apply(self.connection.reval, (self.name,) + args)
  File "/usr/lib/python2.7/dist-packages/rdiff_backup/connection.py", line 370, in reval
    if isinstance(result, Exception): raise result

Traceback (most recent call last):
  File "/usr/bin/rdiff-backup", line 30, in <module>
    rdiff_backup.Main.error_check_Main(sys.argv[1:])
  File "/usr/lib/python2.7/dist-packages/rdiff_backup/Main.py", line 304, in error_check_Main
    try: Main(arglist)
  File "/usr/lib/python2.7/dist-packages/rdiff_backup/Main.py", line 324, in Main
    take_action(rps)
  File "/usr/lib/python2.7/dist-packages/rdiff_backup/Main.py", line 280, in take_action
    elif action == "backup": Backup(rps[0], rps[1])
  File "/usr/lib/python2.7/dist-packages/rdiff_backup/Main.py", line 343, in Backup
    backup.Mirror_and_increment(rpin, rpout, incdir)
  File "/usr/lib/python2.7/dist-packages/rdiff_backup/backup.py", line 48, in Mirror_and_increment
    DestS.set_rorp_cache(dest_rpath, source_rpiter, 1)
  File "/usr/lib/python2.7/dist-packages/rdiff_backup/connection.py", line 450, in __call__
    return apply(self.connection.reval, (self.name,) + args)
  File "/usr/lib/python2.7/dist-packages/rdiff_backup/connection.py", line 370, in reval
    if isinstance(result, Exception): raise result
AssertionError: Path: /home/ikus060/ikus060-rdiff/rdiff-backup-data/mirror_metadata.2015-09-14T19:00:02-04:00.snapshot.gz
Index: ('mirror_metadata.2015-09-14T19:00:02-04:00.snapshot.gz',)
Data: {'size': 2948234, 'ctime': 1442273220, 'perms': 384, 'inode': 6447109, 'mtime': 1442273220, 'ea': <rdiff_backup.eas_acls.ExtendedAttributes instance at 0x7f7fa4731248>, 'devloc': 23L, 'uname': 'ikus060', 'nlink': 1, 'gname': 'ldap-users', 'gid': 10000, 'atime': 1442273454, 'type': 'reg', 'uid': 10000}

Do you have some good approach to repair a repository ? May I revert the backup destination to a stable point ?

--
Patrik Dufresne


_______________________________________________
rdiff-backup-users mailing list at [hidden email]
https://lists.nongnu.org/mailman/listinfo/rdiff-backup-users
Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki


_______________________________________________
rdiff-backup-users mailing list at [hidden email]
https://lists.nongnu.org/mailman/listinfo/rdiff-backup-users
Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki
Reply | Threaded
Open this post in threaded view
|

Re: Repair repository

Patrik Dufresne
Hello Dominic,

I did not run "--check-destination-dir" right after deleting the "current_mirror". I try to run it after making a backup.

This morning I had:
current_mirror.2015-08-24T01:12:31-04:00.data
current_mirror.2015-09-14T19:37:48-04:00.data

The last successful backup seams to be "2015-08-24T01:12:31-04:00".
I delete "current_mirror.2015-09-14T19:37:48-04:00.data". Then I ran "rdiff-backup --check-destination-dir":

$ rdiff-backup --check-destination-dir ~/ikus060-rdiff/
Fatal Error: Destination dir /home/ikus060/ikus060-rdiff does not need checking

I believe something is wrong...

BTW, I see alot of files related to "2015-09-14T19:37:48-04:00" in rdiff-backup-data: file_statistics, increments, mirror_metadata. I also see some files related to "2015-09-14T19:00:02-04:00". Should I delete all of them ? I know the last successfule backup is "2015-08-24".

--
Patrik Dufresne


On Tue, Sep 15, 2015 at 3:05 AM, Dominic Raferd <[hidden email]> wrote:
After deleting the current_mirror file did you then run rdiff-backup with --check-destination-dir? This performs the actual regression to the previous, hopefully consistent, backup.

On 15 September 2015 at 00:46, Patrik Dufresne <[hidden email]> wrote:
Hello,

One of my backup repository seams to be mess up alot. I think the backup have been interrupted multiple time. I do have a very long history for this backup and I don't want to lose it.

First, I encounter "regressing destination" error. So as usual, I've delete the current_mirror. But even then, the backup refused to start with the following exception:

# /usr/bin/rdiff-backup  --print-statistics --exclude '**/.wine' --exclude '**/.PlayOnLinux' --exclude '**/.dropbox' --exclude '**/.dropbox.cache' --exclude '**/.dropbox-dist' --exclude '**/.cache/' --exclude '**/.gnome2_private' --exclude '**/.config/deluge' --exclude '**/.gstreamer-0.10' --exclude '**/.gvfs' --exclude '**/.gtk-bookmarks' --exclude '**/.local/share/Trash/' --exclude '**/.local/share/zeitgeist' --exclude '**/.local/share/gvfs-metadata' --exclude '**/.cache' --exclude '**/.config/google-chrome' --exclude '**/VirtualBox VMs' --exclude '**/Downloads' --exclude '**/.gnupg' --exclude '**/.m2/repository' --exclude '**/.metadata/.plugins/org.eclipse.e4.workbench' --exclude '**/.metadata/.plugins/org.eclipse.m2e.core/' --exclude '**/*~' --exclude '**/Steam' --exclude '**/.xchat2' --exclude '**/.xbmc' --exclude '**/.swt' --exclude '**/.eclipse' --exclude '**/.vim' --exclude '**/.ekwos' --exclude '**/.glpk' --exclude '**/workspace/*/target/*' --exclude '**/.dropbox' --include '/home/ikus060' --include '/etc/backup.d/' --include '/home/ikus060/.local/share/hamster-applet' --include '/root/.ssh' --exclude '/*' / ikus060@fente::/home/ikus060/ikus060-rdiff
Exception 'Path: /home/ikus060/ikus060-rdiff/rdiff-backup-data/mirror_metadata.2015-09-14T19:00:02-04:00.snapshot.gz
Index: ('mirror_metadata.2015-09-14T19:00:02-04:00.snapshot.gz',)
Data: {'size': 2948234, 'ctime': 1442273220, 'perms': 384, 'inode': 6447109, 'mtime': 1442273220, 'ea': <rdiff_backup.eas_acls.ExtendedAttributes instance at 0x7f7fa4731248>, 'devloc': 23L, 'uname': 'ikus060', 'nlink': 1, 'gname': 'ldap-users', 'gid': 10000, 'atime': 1442273454, 'type': 'reg', 'uid': 10000}' raised of class '<type 'exceptions.AssertionError'>':
  File "/usr/lib/python2.7/dist-packages/rdiff_backup/Main.py", line 304, in error_check_Main
    try: Main(arglist)
  File "/usr/lib/python2.7/dist-packages/rdiff_backup/Main.py", line 324, in Main
    take_action(rps)
  File "/usr/lib/python2.7/dist-packages/rdiff_backup/Main.py", line 280, in take_action
    elif action == "backup": Backup(rps[0], rps[1])
  File "/usr/lib/python2.7/dist-packages/rdiff_backup/Main.py", line 343, in Backup
    backup.Mirror_and_increment(rpin, rpout, incdir)
  File "/usr/lib/python2.7/dist-packages/rdiff_backup/backup.py", line 48, in Mirror_and_increment
    DestS.set_rorp_cache(dest_rpath, source_rpiter, 1)
  File "/usr/lib/python2.7/dist-packages/rdiff_backup/connection.py", line 450, in __call__
    return apply(self.connection.reval, (self.name,) + args)
  File "/usr/lib/python2.7/dist-packages/rdiff_backup/connection.py", line 370, in reval
    if isinstance(result, Exception): raise result

Traceback (most recent call last):
  File "/usr/bin/rdiff-backup", line 30, in <module>
    rdiff_backup.Main.error_check_Main(sys.argv[1:])
  File "/usr/lib/python2.7/dist-packages/rdiff_backup/Main.py", line 304, in error_check_Main
    try: Main(arglist)
  File "/usr/lib/python2.7/dist-packages/rdiff_backup/Main.py", line 324, in Main
    take_action(rps)
  File "/usr/lib/python2.7/dist-packages/rdiff_backup/Main.py", line 280, in take_action
    elif action == "backup": Backup(rps[0], rps[1])
  File "/usr/lib/python2.7/dist-packages/rdiff_backup/Main.py", line 343, in Backup
    backup.Mirror_and_increment(rpin, rpout, incdir)
  File "/usr/lib/python2.7/dist-packages/rdiff_backup/backup.py", line 48, in Mirror_and_increment
    DestS.set_rorp_cache(dest_rpath, source_rpiter, 1)
  File "/usr/lib/python2.7/dist-packages/rdiff_backup/connection.py", line 450, in __call__
    return apply(self.connection.reval, (self.name,) + args)
  File "/usr/lib/python2.7/dist-packages/rdiff_backup/connection.py", line 370, in reval
    if isinstance(result, Exception): raise result
AssertionError: Path: /home/ikus060/ikus060-rdiff/rdiff-backup-data/mirror_metadata.2015-09-14T19:00:02-04:00.snapshot.gz
Index: ('mirror_metadata.2015-09-14T19:00:02-04:00.snapshot.gz',)
Data: {'size': 2948234, 'ctime': 1442273220, 'perms': 384, 'inode': 6447109, 'mtime': 1442273220, 'ea': <rdiff_backup.eas_acls.ExtendedAttributes instance at 0x7f7fa4731248>, 'devloc': 23L, 'uname': 'ikus060', 'nlink': 1, 'gname': 'ldap-users', 'gid': 10000, 'atime': 1442273454, 'type': 'reg', 'uid': 10000}

Do you have some good approach to repair a repository ? May I revert the backup destination to a stable point ?

--
Patrik Dufresne


_______________________________________________
rdiff-backup-users mailing list at [hidden email]
https://lists.nongnu.org/mailman/listinfo/rdiff-backup-users
Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki



_______________________________________________
rdiff-backup-users mailing list at [hidden email]
https://lists.nongnu.org/mailman/listinfo/rdiff-backup-users
Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki
Reply | Threaded
Open this post in threaded view
|

Re: Repair repository

Robert Nichols-2
In reply to this post by Patrik Dufresne
On 09/14/2015 06:46 PM, Patrik Dufresne wrote:
> One of my backup repository seams to be mess up alot. I think the backup
> have been interrupted multiple time. I do have a very long history for
> this backup and I don't want to lose it.
>
> First, I encounter "regressing destination" error. So as usual, I've
> delete the current_mirror. But even then, the backup refused to start
> ...

So, rather than letting the regression proceed and return the archive
to a consistent state, you deleted the marker that said that recovery
was needed and pushed another backup into that damaged archive. And,
you have done this ("as usual") several times.

I would be surprised to discover any practical way to fix that.  I
suspect that your "very long history" has been corrupt for quite
some time.

--
Bob Nichols     "NOSPAM" is really part of my email address.
                 Do NOT delete it.


_______________________________________________
rdiff-backup-users mailing list at [hidden email]
https://lists.nongnu.org/mailman/listinfo/rdiff-backup-users
Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki
Reply | Threaded
Open this post in threaded view
|

Re: Repair repository

Dominic Raferd-3
In reply to this post by Patrik Dufresne
I suggest you try using my rdiff-backup-regress - at http://www.timedicer.co.uk/programs/help/rdiff-backup-regress.sh.php - this can force a regression (or multiple regressions).

On 15 September 2015 at 12:34, Patrik Dufresne <[hidden email]> wrote:
Hello Dominic,

I did not run "--check-destination-dir" right after deleting the "current_mirror". I try to run it after making a backup.

This morning I had:
current_mirror.2015-08-24T01:12:31-04:00.data
current_mirror.2015-09-14T19:37:48-04:00.data

The last successful backup seams to be "2015-08-24T01:12:31-04:00".
I delete "current_mirror.2015-09-14T19:37:48-04:00.data". Then I ran "rdiff-backup --check-destination-dir":

$ rdiff-backup --check-destination-dir ~/ikus060-rdiff/
Fatal Error: Destination dir /home/ikus060/ikus060-rdiff does not need checking

I believe something is wrong...

BTW, I see alot of files related to "2015-09-14T19:37:48-04:00" in rdiff-backup-data: file_statistics, increments, mirror_metadata. I also see some files related to "2015-09-14T19:00:02-04:00". Should I delete all of them ? I know the last successfule backup is "2015-08-24".

--
Patrik Dufresne


On Tue, Sep 15, 2015 at 3:05 AM, Dominic Raferd <[hidden email]> wrote:
After deleting the current_mirror file did you then run rdiff-backup with --check-destination-dir? This performs the actual regression to the previous, hopefully consistent, backup.

On 15 September 2015 at 00:46, Patrik Dufresne <[hidden email]> wrote:
Hello,

One of my backup repository seams to be mess up alot. I think the backup have been interrupted multiple time. I do have a very long history for this backup and I don't want to lose it.

First, I encounter "regressing destination" error. So as usual, I've delete the current_mirror. But even then, the backup refused to start with the following exception:

# /usr/bin/rdiff-backup  --print-statistics --exclude '**/.wine' --exclude '**/.PlayOnLinux' --exclude '**/.dropbox' --exclude '**/.dropbox.cache' --exclude '**/.dropbox-dist' --exclude '**/.cache/' --exclude '**/.gnome2_private' --exclude '**/.config/deluge' --exclude '**/.gstreamer-0.10' --exclude '**/.gvfs' --exclude '**/.gtk-bookmarks' --exclude '**/.local/share/Trash/' --exclude '**/.local/share/zeitgeist' --exclude '**/.local/share/gvfs-metadata' --exclude '**/.cache' --exclude '**/.config/google-chrome' --exclude '**/VirtualBox VMs' --exclude '**/Downloads' --exclude '**/.gnupg' --exclude '**/.m2/repository' --exclude '**/.metadata/.plugins/org.eclipse.e4.workbench' --exclude '**/.metadata/.plugins/org.eclipse.m2e.core/' --exclude '**/*~' --exclude '**/Steam' --exclude '**/.xchat2' --exclude '**/.xbmc' --exclude '**/.swt' --exclude '**/.eclipse' --exclude '**/.vim' --exclude '**/.ekwos' --exclude '**/.glpk' --exclude '**/workspace/*/target/*' --exclude '**/.dropbox' --include '/home/ikus060' --include '/etc/backup.d/' --include '/home/ikus060/.local/share/hamster-applet' --include '/root/.ssh' --exclude '/*' / ikus060@fente::/home/ikus060/ikus060-rdiff
Exception 'Path: /home/ikus060/ikus060-rdiff/rdiff-backup-data/mirror_metadata.2015-09-14T19:00:02-04:00.snapshot.gz
Index: ('mirror_metadata.2015-09-14T19:00:02-04:00.snapshot.gz',)
Data: {'size': 2948234, 'ctime': 1442273220, 'perms': 384, 'inode': 6447109, 'mtime': 1442273220, 'ea': <rdiff_backup.eas_acls.ExtendedAttributes instance at 0x7f7fa4731248>, 'devloc': 23L, 'uname': 'ikus060', 'nlink': 1, 'gname': 'ldap-users', 'gid': 10000, 'atime': 1442273454, 'type': 'reg', 'uid': 10000}' raised of class '<type 'exceptions.AssertionError'>':
  File "/usr/lib/python2.7/dist-packages/rdiff_backup/Main.py", line 304, in error_check_Main
    try: Main(arglist)
  File "/usr/lib/python2.7/dist-packages/rdiff_backup/Main.py", line 324, in Main
    take_action(rps)
  File "/usr/lib/python2.7/dist-packages/rdiff_backup/Main.py", line 280, in take_action
    elif action == "backup": Backup(rps[0], rps[1])
  File "/usr/lib/python2.7/dist-packages/rdiff_backup/Main.py", line 343, in Backup
    backup.Mirror_and_increment(rpin, rpout, incdir)
  File "/usr/lib/python2.7/dist-packages/rdiff_backup/backup.py", line 48, in Mirror_and_increment
    DestS.set_rorp_cache(dest_rpath, source_rpiter, 1)
  File "/usr/lib/python2.7/dist-packages/rdiff_backup/connection.py", line 450, in __call__
    return apply(self.connection.reval, (self.name,) + args)
  File "/usr/lib/python2.7/dist-packages/rdiff_backup/connection.py", line 370, in reval
    if isinstance(result, Exception): raise result

Traceback (most recent call last):
  File "/usr/bin/rdiff-backup", line 30, in <module>
    rdiff_backup.Main.error_check_Main(sys.argv[1:])
  File "/usr/lib/python2.7/dist-packages/rdiff_backup/Main.py", line 304, in error_check_Main
    try: Main(arglist)
  File "/usr/lib/python2.7/dist-packages/rdiff_backup/Main.py", line 324, in Main
    take_action(rps)
  File "/usr/lib/python2.7/dist-packages/rdiff_backup/Main.py", line 280, in take_action
    elif action == "backup": Backup(rps[0], rps[1])
  File "/usr/lib/python2.7/dist-packages/rdiff_backup/Main.py", line 343, in Backup
    backup.Mirror_and_increment(rpin, rpout, incdir)
  File "/usr/lib/python2.7/dist-packages/rdiff_backup/backup.py", line 48, in Mirror_and_increment
    DestS.set_rorp_cache(dest_rpath, source_rpiter, 1)
  File "/usr/lib/python2.7/dist-packages/rdiff_backup/connection.py", line 450, in __call__
    return apply(self.connection.reval, (self.name,) + args)
  File "/usr/lib/python2.7/dist-packages/rdiff_backup/connection.py", line 370, in reval
    if isinstance(result, Exception): raise result
AssertionError: Path: /home/ikus060/ikus060-rdiff/rdiff-backup-data/mirror_metadata.2015-09-14T19:00:02-04:00.snapshot.gz
Index: ('mirror_metadata.2015-09-14T19:00:02-04:00.snapshot.gz',)
Data: {'size': 2948234, 'ctime': 1442273220, 'perms': 384, 'inode': 6447109, 'mtime': 1442273220, 'ea': <rdiff_backup.eas_acls.ExtendedAttributes instance at 0x7f7fa4731248>, 'devloc': 23L, 'uname': 'ikus060', 'nlink': 1, 'gname': 'ldap-users', 'gid': 10000, 'atime': 1442273454, 'type': 'reg', 'uid': 10000}

Do you have some good approach to repair a repository ? May I revert the backup destination to a stable point ?

--
Patrik Dufresne


_______________________________________________
rdiff-backup-users mailing list at [hidden email]
https://lists.nongnu.org/mailman/listinfo/rdiff-backup-users
Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki




_______________________________________________
rdiff-backup-users mailing list at [hidden email]
https://lists.nongnu.org/mailman/listinfo/rdiff-backup-users
Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki
Reply | Threaded
Open this post in threaded view
|

Re: Repair repository

Patrik Dufresne
I've tried the script. I had issues with it. 
It searchs 'current_mirror' file recursively. For some reash, the backup contains files named 'current_mirror'. I add `-maxdepth 1` to fix this.

Regression also failed:

$ ./rdiff-backup-regress.sh ikus060-rdiff

rdiff-backup-regress.sh v0.7 [09 Dec 2014] by Dominic (-h for help)
=======================

Using repository: ikus060-rdiff
Started Tue Sep 22 16:56:52 EDT 2015
Note: rdiff-backup recognises this archive as damaged
About to regress ikus060-rdiff archive from 2015-09-15T08:00:02-04:00
2015-08-24T01:12:31-04:00 to 2015-09-14T19:00:02-04:00: ok (y/-)? y
Regression 1 of 1: from 2015-09-15T08:00:02-04:00
2015-08-24T01:12:31-04:00 to 2015-09-14T19:00:02-04:00...
Exception 'Too many recent increments' raised of class '<type 'exceptions.AssertionError'>':
  File "/usr/lib/python2.7/dist-packages/rdiff_backup/Main.py", line 304, in error_check_Main
    try: Main(arglist)
  File "/usr/lib/python2.7/dist-packages/rdiff_backup/Main.py", line 324, in Main
    take_action(rps)
  File "/usr/lib/python2.7/dist-packages/rdiff_backup/Main.py", line 282, in take_action
    elif action == "check-destination-dir": CheckDest(rps[0])
  File "/usr/lib/python2.7/dist-packages/rdiff_backup/Main.py", line 872, in CheckDest
    dest_rp.conn.regress.Regress(dest_rp)
  File "/usr/lib/python2.7/dist-packages/rdiff_backup/regress.py", line 71, in Regress
    for rf in iterate_meta_rfs(mirror_rp, inc_rpath): ITR(rf.index, rf)
  File "/usr/lib/python2.7/dist-packages/rdiff_backup/regress.py", line 195, in iterate_meta_rfs
    raw_rfs = iterate_raw_rfs(mirror_rp, inc_rp)
  File "/usr/lib/python2.7/dist-packages/rdiff_backup/regress.py", line 164, in iterate_raw_rfs
    root_rf = RegressFile(mirror_rp, inc_rp, restore.get_inclist(inc_rp))
  File "/usr/lib/python2.7/dist-packages/rdiff_backup/regress.py", line 218, in __init__
    self.set_regress_inc()
  File "/usr/lib/python2.7/dist-packages/rdiff_backup/regress.py", line 233, in set_regress_inc
    assert len(newer_incs) <= 1, "Too many recent increments"

Traceback (most recent call last):
  File "/usr/bin/rdiff-backup", line 30, in <module>
    rdiff_backup.Main.error_check_Main(sys.argv[1:])
  File "/usr/lib/python2.7/dist-packages/rdiff_backup/Main.py", line 304, in error_check_Main
    try: Main(arglist)
  File "/usr/lib/python2.7/dist-packages/rdiff_backup/Main.py", line 324, in Main
    take_action(rps)
  File "/usr/lib/python2.7/dist-packages/rdiff_backup/Main.py", line 282, in take_action
    elif action == "check-destination-dir": CheckDest(rps[0])
  File "/usr/lib/python2.7/dist-packages/rdiff_backup/Main.py", line 872, in CheckDest
    dest_rp.conn.regress.Regress(dest_rp)
  File "/usr/lib/python2.7/dist-packages/rdiff_backup/regress.py", line 71, in Regress
    for rf in iterate_meta_rfs(mirror_rp, inc_rpath): ITR(rf.index, rf)
  File "/usr/lib/python2.7/dist-packages/rdiff_backup/regress.py", line 195, in iterate_meta_rfs
    raw_rfs = iterate_raw_rfs(mirror_rp, inc_rp)
  File "/usr/lib/python2.7/dist-packages/rdiff_backup/regress.py", line 164, in iterate_raw_rfs
    root_rf = RegressFile(mirror_rp, inc_rp, restore.get_inclist(inc_rp))
  File "/usr/lib/python2.7/dist-packages/rdiff_backup/regress.py", line 218, in __init__
    self.set_regress_inc()
  File "/usr/lib/python2.7/dist-packages/rdiff_backup/regress.py", line 233, in set_regress_inc
    assert len(newer_incs) <= 1, "Too many recent increments"
AssertionError: Too many recent increments
Error 1 occurred when attempting to regress archive...
Ended Tue Sep 22 16:57:02 EDT 2015





--
Patrik Dufresne


On Tue, Sep 15, 2015 at 12:08 PM, Dominic Raferd <[hidden email]> wrote:
I suggest you try using my rdiff-backup-regress - at http://www.timedicer.co.uk/programs/help/rdiff-backup-regress.sh.php - this can force a regression (or multiple regressions).

On 15 September 2015 at 12:34, Patrik Dufresne <[hidden email]> wrote:
Hello Dominic,

I did not run "--check-destination-dir" right after deleting the "current_mirror". I try to run it after making a backup.

This morning I had:
current_mirror.2015-08-24T01:12:31-04:00.data
current_mirror.2015-09-14T19:37:48-04:00.data

The last successful backup seams to be "2015-08-24T01:12:31-04:00".
I delete "current_mirror.2015-09-14T19:37:48-04:00.data". Then I ran "rdiff-backup --check-destination-dir":

$ rdiff-backup --check-destination-dir ~/ikus060-rdiff/
Fatal Error: Destination dir /home/ikus060/ikus060-rdiff does not need checking

I believe something is wrong...

BTW, I see alot of files related to "2015-09-14T19:37:48-04:00" in rdiff-backup-data: file_statistics, increments, mirror_metadata. I also see some files related to "2015-09-14T19:00:02-04:00". Should I delete all of them ? I know the last successfule backup is "2015-08-24".

--
Patrik Dufresne


On Tue, Sep 15, 2015 at 3:05 AM, Dominic Raferd <[hidden email]> wrote:
After deleting the current_mirror file did you then run rdiff-backup with --check-destination-dir? This performs the actual regression to the previous, hopefully consistent, backup.

On 15 September 2015 at 00:46, Patrik Dufresne <[hidden email]> wrote:
Hello,

One of my backup repository seams to be mess up alot. I think the backup have been interrupted multiple time. I do have a very long history for this backup and I don't want to lose it.

First, I encounter "regressing destination" error. So as usual, I've delete the current_mirror. But even then, the backup refused to start with the following exception:

# /usr/bin/rdiff-backup  --print-statistics --exclude '**/.wine' --exclude '**/.PlayOnLinux' --exclude '**/.dropbox' --exclude '**/.dropbox.cache' --exclude '**/.dropbox-dist' --exclude '**/.cache/' --exclude '**/.gnome2_private' --exclude '**/.config/deluge' --exclude '**/.gstreamer-0.10' --exclude '**/.gvfs' --exclude '**/.gtk-bookmarks' --exclude '**/.local/share/Trash/' --exclude '**/.local/share/zeitgeist' --exclude '**/.local/share/gvfs-metadata' --exclude '**/.cache' --exclude '**/.config/google-chrome' --exclude '**/VirtualBox VMs' --exclude '**/Downloads' --exclude '**/.gnupg' --exclude '**/.m2/repository' --exclude '**/.metadata/.plugins/org.eclipse.e4.workbench' --exclude '**/.metadata/.plugins/org.eclipse.m2e.core/' --exclude '**/*~' --exclude '**/Steam' --exclude '**/.xchat2' --exclude '**/.xbmc' --exclude '**/.swt' --exclude '**/.eclipse' --exclude '**/.vim' --exclude '**/.ekwos' --exclude '**/.glpk' --exclude '**/workspace/*/target/*' --exclude '**/.dropbox' --include '/home/ikus060' --include '/etc/backup.d/' --include '/home/ikus060/.local/share/hamster-applet' --include '/root/.ssh' --exclude '/*' / ikus060@fente::/home/ikus060/ikus060-rdiff
Exception 'Path: /home/ikus060/ikus060-rdiff/rdiff-backup-data/mirror_metadata.2015-09-14T19:00:02-04:00.snapshot.gz
Index: ('mirror_metadata.2015-09-14T19:00:02-04:00.snapshot.gz',)
Data: {'size': 2948234, 'ctime': 1442273220, 'perms': 384, 'inode': 6447109, 'mtime': 1442273220, 'ea': <rdiff_backup.eas_acls.ExtendedAttributes instance at 0x7f7fa4731248>, 'devloc': 23L, 'uname': 'ikus060', 'nlink': 1, 'gname': 'ldap-users', 'gid': 10000, 'atime': 1442273454, 'type': 'reg', 'uid': 10000}' raised of class '<type 'exceptions.AssertionError'>':
  File "/usr/lib/python2.7/dist-packages/rdiff_backup/Main.py", line 304, in error_check_Main
    try: Main(arglist)
  File "/usr/lib/python2.7/dist-packages/rdiff_backup/Main.py", line 324, in Main
    take_action(rps)
  File "/usr/lib/python2.7/dist-packages/rdiff_backup/Main.py", line 280, in take_action
    elif action == "backup": Backup(rps[0], rps[1])
  File "/usr/lib/python2.7/dist-packages/rdiff_backup/Main.py", line 343, in Backup
    backup.Mirror_and_increment(rpin, rpout, incdir)
  File "/usr/lib/python2.7/dist-packages/rdiff_backup/backup.py", line 48, in Mirror_and_increment
    DestS.set_rorp_cache(dest_rpath, source_rpiter, 1)
  File "/usr/lib/python2.7/dist-packages/rdiff_backup/connection.py", line 450, in __call__
    return apply(self.connection.reval, (self.name,) + args)
  File "/usr/lib/python2.7/dist-packages/rdiff_backup/connection.py", line 370, in reval
    if isinstance(result, Exception): raise result

Traceback (most recent call last):
  File "/usr/bin/rdiff-backup", line 30, in <module>
    rdiff_backup.Main.error_check_Main(sys.argv[1:])
  File "/usr/lib/python2.7/dist-packages/rdiff_backup/Main.py", line 304, in error_check_Main
    try: Main(arglist)
  File "/usr/lib/python2.7/dist-packages/rdiff_backup/Main.py", line 324, in Main
    take_action(rps)
  File "/usr/lib/python2.7/dist-packages/rdiff_backup/Main.py", line 280, in take_action
    elif action == "backup": Backup(rps[0], rps[1])
  File "/usr/lib/python2.7/dist-packages/rdiff_backup/Main.py", line 343, in Backup
    backup.Mirror_and_increment(rpin, rpout, incdir)
  File "/usr/lib/python2.7/dist-packages/rdiff_backup/backup.py", line 48, in Mirror_and_increment
    DestS.set_rorp_cache(dest_rpath, source_rpiter, 1)
  File "/usr/lib/python2.7/dist-packages/rdiff_backup/connection.py", line 450, in __call__
    return apply(self.connection.reval, (self.name,) + args)
  File "/usr/lib/python2.7/dist-packages/rdiff_backup/connection.py", line 370, in reval
    if isinstance(result, Exception): raise result
AssertionError: Path: /home/ikus060/ikus060-rdiff/rdiff-backup-data/mirror_metadata.2015-09-14T19:00:02-04:00.snapshot.gz
Index: ('mirror_metadata.2015-09-14T19:00:02-04:00.snapshot.gz',)
Data: {'size': 2948234, 'ctime': 1442273220, 'perms': 384, 'inode': 6447109, 'mtime': 1442273220, 'ea': <rdiff_backup.eas_acls.ExtendedAttributes instance at 0x7f7fa4731248>, 'devloc': 23L, 'uname': 'ikus060', 'nlink': 1, 'gname': 'ldap-users', 'gid': 10000, 'atime': 1442273454, 'type': 'reg', 'uid': 10000}

Do you have some good approach to repair a repository ? May I revert the backup destination to a stable point ?

--
Patrik Dufresne


_______________________________________________
rdiff-backup-users mailing list at [hidden email]
https://lists.nongnu.org/mailman/listinfo/rdiff-backup-users
Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki





_______________________________________________
rdiff-backup-users mailing list at [hidden email]
https://lists.nongnu.org/mailman/listinfo/rdiff-backup-users
Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki
Reply | Threaded
Open this post in threaded view
|

Re: Repair repository

Dominic Raferd-3
Hi Patrik

On 22/09/2015 22:00, Patrik Dufresne wrote:
> I've tried the script. I had issues with it.
> It searchs 'current_mirror' file recursively. For some reash, the
> backup contains files named 'current_mirror'. I add `-maxdepth 1` to
> fix this.

Good point, I have updated the script with this, which speeds it up
> Regression also failed:
>
> $ ./rdiff-backup-regress.sh ikus060-rdiff
> ...

I'm afraid I can only conclude that, as Robert suggested, your archive
is beyond repair, sorry.

I realise this suggestion is a bit late for you, but I do a daily
verification of all* my archives (repositories) and only proceed with
offsite synchronisation if they all pass. Very occasionally one of them
fails verification (coincidentally, one failed last night) and then I
fix it with --check-destination-dir, or rdiff-backup-regress, or recover
it from offsite backup. For each archive, I verify the latest backup
(the one stored 'in the clear') and the one preceding it (which uses diffs).

* Actually there is one archive that takes too long to verify so in this
one case I just wing it, which is not ideal...

_______________________________________________
rdiff-backup-users mailing list at [hidden email]
https://lists.nongnu.org/mailman/listinfo/rdiff-backup-users
Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki
Reply | Threaded
Open this post in threaded view
|

Re: Repair repository

Patrik Dufresne
I finally manage to repair the archive ! I've browse the code to determine where it's failing. I add some debug log for my self to determine where the error was coming from. It look like recovery was failing because some "increments" file was left over by failed backup. I've search them and delete them.

This might be something to add to your script. Basically, when trying to restore 2015-08-21T01:12:31-04:00, we need to delete every newer file from increments directory. Then --check-destination-dir is working.

--
Patrik Dufresne


On Wed, Sep 23, 2015 at 1:44 AM, Dominic Raferd <[hidden email]> wrote:
Hi Patrik

On 22/09/2015 22:00, Patrik Dufresne wrote:
I've tried the script. I had issues with it.
It searchs 'current_mirror' file recursively. For some reash, the backup contains files named 'current_mirror'. I add `-maxdepth 1` to fix this.

Good point, I have updated the script with this, which speeds it up
Regression also failed:

$ ./rdiff-backup-regress.sh ikus060-rdiff
...

I'm afraid I can only conclude that, as Robert suggested, your archive is beyond repair, sorry.

I realise this suggestion is a bit late for you, but I do a daily verification of all* my archives (repositories) and only proceed with offsite synchronisation if they all pass. Very occasionally one of them fails verification (coincidentally, one failed last night) and then I fix it with --check-destination-dir, or rdiff-backup-regress, or recover it from offsite backup. For each archive, I verify the latest backup (the one stored 'in the clear') and the one preceding it (which uses diffs).

* Actually there is one archive that takes too long to verify so in this one case I just wing it, which is not ideal...


_______________________________________________
rdiff-backup-users mailing list at [hidden email]
https://lists.nongnu.org/mailman/listinfo/rdiff-backup-users
Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki
Reply | Threaded
Open this post in threaded view
|

Re: Repair repository

Dominic Raferd-3
I'm delighted it worked for you - good work! I guess it is because deleting the increments files tricks rdiff-backup into thinking that there are no later backups than 2015-08-21T01:12:31-04:00, and in your case this was ok because all the subsequent backup attempts hadn't really changed any of the repository data (except metadata)?

Can you successfully verify the repository back to 2015-08-21T01:12:31-04:00 or (better) one backup earlier?

Dominic

On 23/09/2015 23:48, Patrik Dufresne wrote:
I finally manage to repair the archive ! I've browse the code to determine where it's failing. I add some debug log for my self to determine where the error was coming from. It look like recovery was failing because some "increments" file was left over by failed backup. I've search them and delete them.

This might be something to add to your script. Basically, when trying to restore 2015-08-21T01:12:31-04:00, we need to delete every newer file from increments directory. Then --check-destination-dir is working.

--
Patrik Dufresne


On Wed, Sep 23, 2015 at 1:44 AM, Dominic Raferd <[hidden email]> wrote:
Hi Patrik

On 22/09/2015 22:00, Patrik Dufresne wrote:
I've tried the script. I had issues with it.
It searchs 'current_mirror' file recursively. For some reash, the backup contains files named 'current_mirror'. I add `-maxdepth 1` to fix this.

Good point, I have updated the script with this, which speeds it up
Regression also failed:

$ ./rdiff-backup-regress.sh ikus060-rdiff
...

I'm afraid I can only conclude that, as Robert suggested, your archive is beyond repair, sorry.

I realise this suggestion is a bit late for you, but I do a daily verification of all* my archives (repositories) and only proceed with offsite synchronisation if they all pass. Very occasionally one of them fails verification (coincidentally, one failed last night) and then I fix it with --check-destination-dir, or rdiff-backup-regress, or recover it from offsite backup. For each archive, I verify the latest backup (the one stored 'in the clear') and the one preceding it (which uses diffs).

* Actually there is one archive that takes too long to verify so in this one case I just wing it, which is not ideal...



_______________________________________________
rdiff-backup-users mailing list at [hidden email]
https://lists.nongnu.org/mailman/listinfo/rdiff-backup-users
Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki