CRC Check Failed

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

CRC Check Failed

Mike Bydalek-3
Hello.

I've been trying to backup for a couple of days now and I'm having some
problems that I cannot get around.  For some reason, rdiff-backup dies
while trying to revert every single time.

I've tried to delete the directory on both the server and the client
without any luck.  At this point, I'm pretty stuck, so any suggestions
are highly appreciated.

For versions I'm using rdiff-backup 1.0.4

-Mike

Setting time of
server-root/afs/contentconnections.com/home/steve/ContentConnections/src/repos/php/survey/trunk/templates/edit/survey/.svn/prop-base
to 1141948062
Sending back exception CRC check failed of type exceptions.IOError:
  File "/usr/lib/python2.4/site-packages/rdiff_backup/connection.py",
line 334, in answer_request
    result = apply(eval(request.function_string), argument_list)
  File "/usr/lib/python2.4/site-packages/rdiff_backup/regress.py", line
70, in Regress
    for rf in iterate_meta_rfs(mirror_rp, inc_rpath): ITR(rf.index, rf)
  File "/usr/lib/python2.4/site-packages/rdiff_backup/regress.py", line
162, in iterate_meta_rfs
    for raw_rf, metadata_rorp in collated:
  File "/usr/lib/python2.4/site-packages/rdiff_backup/rorpiter.py", line
100, in Collate2Iters
    try: relem2 = riter2.next()
  File "/usr/lib/python2.4/site-packages/rdiff_backup/metadata.py", line
255, in iterate
    next_pos = self.get_next_pos()
  File "/usr/lib/python2.4/site-packages/rdiff_backup/metadata.py", line
246, in get_next_pos
    newbuf = self.fileobj.read(self.blocksize)
  File "/usr/lib/python2.4/gzip.py", line 225, in read
    self._read(readsize)
  File "/usr/lib/python2.4/gzip.py", line 273, in _read
    self._read_eof()
  File "/usr/lib/python2.4/gzip.py", line 309, in _read_eof
    raise IOError, "CRC check failed"

Traceback (most recent call last):
  File "/usr/bin/rdiff-backup", line 23, in ?
    rdiff_backup.Main.Main(sys.argv[1:])
  File "/usr/lib/python2.4/site-packages/rdiff_backup/Main.py", line
285, in Main
    take_action(rps)
  File "/usr/lib/python2.4/site-packages/rdiff_backup/Main.py", line
255, in take_action
    elif action == "backup": Backup(rps[0], rps[1])
  File "/usr/lib/python2.4/site-packages/rdiff_backup/Main.py", line
299, in Backup
    backup_final_init(rpout)
  File "/usr/lib/python2.4/site-packages/rdiff_backup/Main.py", line
396, in backup_final_init
    checkdest_if_necessary(rpout)
  File "/usr/lib/python2.4/site-packages/rdiff_backup/Main.py", line
903, in checkdest_if_necessary
    dest_rp.conn.regress.Regress(dest_rp)
  File "/usr/lib/python2.4/site-packages/rdiff_backup/connection.py",
line 445, in __call__
    return apply(self.connection.reval, (self.name,) + args)
  File "/usr/lib/python2.4/site-packages/rdiff_backup/connection.py",
line 367, in reval
    if isinstance(result, Exception): raise result
IOError: CRC check failed
illed by signal 1.



_______________________________________________
rdiff-backup-users mailing list at [hidden email]
http://lists.nongnu.org/mailman/listinfo/rdiff-backup-users
Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki
Reply | Threaded
Open this post in threaded view
|

Re: CRC Check Failed

dean gaudet-4
On Tue, 14 Mar 2006, Mike Bydalek wrote:

> I've been trying to backup for a couple of days now and I'm having some
> problems that I cannot get around.  For some reason, rdiff-backup dies
> while trying to revert every single time.
>
> I've tried to delete the directory on both the server and the client
> without any luck.  At this point, I'm pretty stuck, so any suggestions
> are highly appreciated.

which directory did you delete?

>   File "/usr/lib/python2.4/gzip.py", line 309, in _read_eof
>     raise IOError, "CRC check failed"

can you try running "gzip --test" on all the .gz files from the last
failed backup?

it's likely rdiff-backup needs to be more graceful in the event that gzip
finds a crc error in one of its files... but it'd be nice to verify this
is what's causing your problem first.

i assume you did have a failed backup for some reason -- power failure or
other crash during a backup maybe?

-dean


_______________________________________________
rdiff-backup-users mailing list at [hidden email]
http://lists.nongnu.org/mailman/listinfo/rdiff-backup-users
Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki
Reply | Threaded
Open this post in threaded view
|

Re: CRC Check Failed

dean gaudet-4
On Tue, 14 Mar 2006, dean gaudet wrote:

> On Tue, 14 Mar 2006, Mike Bydalek wrote:
>
> > I've been trying to backup for a couple of days now and I'm having some
> > problems that I cannot get around.  For some reason, rdiff-backup dies
> > while trying to revert every single time.
> >
> > I've tried to delete the directory on both the server and the client
> > without any luck.  At this point, I'm pretty stuck, so any suggestions
> > are highly appreciated.
>
> which directory did you delete?
>
> >   File "/usr/lib/python2.4/gzip.py", line 309, in _read_eof
> >     raise IOError, "CRC check failed"
>
> can you try running "gzip --test" on all the .gz files from the last
> failed backup?

er if it wasn't clear i meant to do this in dest/rdiff-backup-data...
perhaps something like this:

        find dest/rdiff-backup-data -type f -name \*.gz -print0 \
                | xargs -0r gzip --test

it could take a while but should do the trick...

-dean

>
> it's likely rdiff-backup needs to be more graceful in the event that gzip
> finds a crc error in one of its files... but it'd be nice to verify this
> is what's causing your problem first.
>
> i assume you did have a failed backup for some reason -- power failure or
> other crash during a backup maybe?
>
> -dean
>
>
> _______________________________________________
> rdiff-backup-users mailing list at [hidden email]
> http://lists.nongnu.org/mailman/listinfo/rdiff-backup-users
> Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki
>


_______________________________________________
rdiff-backup-users mailing list at [hidden email]
http://lists.nongnu.org/mailman/listinfo/rdiff-backup-users
Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki
Reply | Threaded
Open this post in threaded view
|

Re: CRC Check Failed

Mike Bydalek-3


dean gaudet wrote:
On Tue, 14 Mar 2006, dean gaudet wrote:

  
On Tue, 14 Mar 2006, Mike Bydalek wrote:

    
I've been trying to backup for a couple of days now and I'm having some
problems that I cannot get around.  For some reason, rdiff-backup dies
while trying to revert every single time.

I've tried to delete the directory on both the server and the client
without any luck.  At this point, I'm pretty stuck, so any suggestions
are highly appreciated.
      
which directory did you delete?
    
    
  File "/usr/lib/python2.4/gzip.py", line 309, in _read_eof
    raise IOError, "CRC check failed"
      
can you try running "gzip --test" on all the .gz files from the last 
failed backup?
    

er if it wasn't clear i meant to do this in dest/rdiff-backup-data... 
perhaps something like this:

	find dest/rdiff-backup-data -type f -name \*.gz -print0 \
		| xargs -0r gzip --test

it could take a while but should do the trick...
  

Here's the results of that:

$ find server-root/rdiff-backup-data -type f -name \*.gz -print0 | xargs -0r gzip --test

gzip: server-root/rdiff-backup-data/mirror_metadata.2006-03-12T21:46:00-07:00.snapshot.gz: unexpected end of file

Should I delete this file and re-run the backup then?


it's likely rdiff-backup needs to be more graceful in the event that gzip 
finds a crc error in one of its files... but it'd be nice to verify this 
is what's causing your problem first.

i assume you did have a failed backup for some reason -- power failure or 
other crash during a backup maybe?

    

I'm not 100% sure why it died, but yes, it failed for some reason.

Thanks,
Mike

_______________________________________________
rdiff-backup-users mailing list at [hidden email]
http://lists.nongnu.org/mailman/listinfo/rdiff-backup-users
Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki
Reply | Threaded
Open this post in threaded view
|

Re: CRC Check Failed

dean gaudet-4
On Tue, 14 Mar 2006, Mike Bydalek wrote:

> Here's the results of that:
>
> $ find server-root/rdiff-backup-data -type f -name \*.gz -print0 | xargs
> -0r gzip --test
>
> gzip:
> server-root/rdiff-backup-data/mirror_metadata.2006-03-12T21:46:00-07:00.snapshot.gz:
> unexpected end of file
>
> Should I delete this file and re-run the backup then?

hmm... but no CRC error... odd... i wonder if rdiff-backup requires the
contents of that file to regress the failed backup.

if you deleted it you might have to do further surgery on the increments
subdir ... i'm not 100% sure what surgery would be required... but you
*could* try something like:

        find dest/rdiff-backup-data -name '*.2006-03-12T21:46:00-07:00*' -type f -print0 | xargs -0 rm
        find dest/rdiff-backup-data -depth -name '*.2006-03-12T21:46:00-07:00*' -type d -print0 | xargs -0 rmdir

which should remove all files created during that backup... that *might*
leave the mirror in a state where you could perform another backup and
things would be OK again... but i suspect it will leave some things
incorrect (been a while since i understood this part of the code).

another option is to lose your increment history entirely -- do "rm -rf
dest/rdiff-backup-data" and then use the rdiff-backup --force option to
start again (it'll use the files already in the dest).  i know this method
works... i've used it many times.

rdiff-backup really needs to handle this one gracefully i think ... it
should be easy to reproduce by using "kill -9" at random times with an
in-progress backup (which would be a good regression test anyhow).

-dean


_______________________________________________
rdiff-backup-users mailing list at [hidden email]
http://lists.nongnu.org/mailman/listinfo/rdiff-backup-users
Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki
Reply | Threaded
Open this post in threaded view
|

Re: CRC Check Failed

Mike Bydalek-3


dean gaudet wrote:
On Tue, 14 Mar 2006, Mike Bydalek wrote:

  
Here's the results of that:

$ find server-root/rdiff-backup-data -type f -name \*.gz -print0 | xargs
-0r gzip --test

gzip:
server-root/rdiff-backup-data/mirror_metadata.2006-03-12T21:46:00-07:00.snapshot.gz:
unexpected end of file

Should I delete this file and re-run the backup then?
    

hmm... but no CRC error... odd... i wonder if rdiff-backup requires the 
contents of that file to regress the failed backup.
  
This may be an error in the error handling.  If you look at the original message, gzip threw an eof, but raised the IOError "CRC check failed" - false positive?
if you deleted it you might have to do further surgery on the increments 
subdir ... i'm not 100% sure what surgery would be required... but you 
*could* try something like:

	find dest/rdiff-backup-data -name '*.2006-03-12T21:46:00-07:00*' -type f -print0 | xargs -0 rm
	find dest/rdiff-backup-data -depth -name '*.2006-03-12T21:46:00-07:00*' -type d -print0 | xargs -0 rmdir
  
Well, I haven't deleted the snapshot.gz file yet, so I would like to hold off on rm(dir)'ing stuff, unless it's a last resort.  If you're sure that this will *only* delete that failed back up files, then I guess that's not too much of a loss.  Preferably I'd like to resolve this gracefully.

another option is to lose your increment history entirely -- do "rm -rf 
dest/rdiff-backup-data" and then use the rdiff-backup --force option to 
start again (it'll use the files already in the dest).  i know this method 
works... i've used it many times.

  
To be honest, I *really* don't want to do this at all.  We keep 30 days of stuff for our people, and starting over just isn't a viable option.
rdiff-backup really needs to handle this one gracefully i think ... it 
should be easy to reproduce by using "kill -9" at random times with an 
in-progress backup (which would be a good regression test anyhow).
  
I agree.  Time to fire up vmware and do some testing ;)

-Mike

_______________________________________________
rdiff-backup-users mailing list at [hidden email]
http://lists.nongnu.org/mailman/listinfo/rdiff-backup-users
Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki