Remove (large) files from backup without generating negative diff

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Remove (large) files from backup without generating negative diff

Marcin Zajączkowski
Hi,

I have a long established backup with some large files which I would
like to move to another directory withing the same backup (or just
remove entirely from the backup) as a clean up of the original disk
space/organization. I would like to avoid extra many GBs in the next
incremental backup session when those files are moved/removed. I don't
need their history - they will be backed up in another way/place.

Firstly, I was plying with removing just related information from
"increments" directory. Unfortunately it made restore/verification
lopped with 100% CPU utilization. I needed to remove all information
about affected files also from extended_attributes.*, file_statistics.*,
etc. I feel it like asking for trouble.

Two questions:
1. Do you know any better way of removing files from a (rdiff-)backup
without generating a "negative" diff in the next backup?
2. Do you see any additional risk with manual manipulation of
rdiff-backup files if --verify and --restore (checked selectively) seem
to work fine?

Marcin

_______________________________________________
rdiff-backup-users mailing list at [hidden email]
https://lists.nongnu.org/mailman/listinfo/rdiff-backup-users
Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki
Reply | Threaded
Open this post in threaded view
|

Re: Remove (large) files from backup without generating negative diff

Dominic Raferd-3
On 11 June 2017 at 11:13, Marcin Zajączkowski <[hidden email]> wrote:

> Hi,
>
> I have a long established backup with some large files which I would
> like to move to another directory withing the same backup (or just
> remove entirely from the backup) as a clean up of the original disk
> space/organization. I would like to avoid extra many GBs in the next
> incremental backup session when those files are moved/removed. I don't
> need their history - they will be backed up in another way/place.
>
> Firstly, I was plying with removing just related information from
> "increments" directory. Unfortunately it made restore/verification
> lopped with 100% CPU utilization. I needed to remove all information
> about affected files also from extended_attributes.*, file_statistics.*,
> etc. I feel it like asking for trouble.
>
> Two questions:
> 1. Do you know any better way of removing files from a (rdiff-)backup
> without generating a "negative" diff in the next backup?
> 2. Do you see any additional risk with manual manipulation of
> rdiff-backup files if --verify and --restore (checked selectively) seem
> to work fine?
>


​Regarding your second question, I don't know, but I would advise against.
Regarding your first, if you haven't had these large files in your backup
for very long it might be worth regressing your archive to a time before it
held these files using my script at
https://www.timedicer.co.uk/programs/help/rdiff-backup-regress.sh.php - it
must be run on the machine that holds the repository.​ Otherwise it might
be best to accept the bloat in your repository.
_______________________________________________
rdiff-backup-users mailing list at [hidden email]
https://lists.nongnu.org/mailman/listinfo/rdiff-backup-users
Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki
Reply | Threaded
Open this post in threaded view
|

Re: Remove (large) files from backup without generating negative diff

Robert Nichols-2
In reply to this post by Marcin Zajączkowski
On 06/11/2017 05:13 AM, Marcin Zajączkowski wrote:
> Hi,
>
> I have a long established backup with some large files which I would
> like to move to another directory withing the same backup (or just
> remove entirely from the backup) as a clean up of the original disk
> space/organization. I would like to avoid extra many GBs in the next
> incremental backup session when those files are moved/removed. I don't
> need their history - they will be backed up in another way/place.

I have a rather horrific shell script that does that, at least for the case where both the source and the archive are on Linux filesystems and there is no long_filename_data in the archive. I'll attach the top comments and usage output. E-mail me directly if you want the script. I've been using it successfully for several years (hasn't needed any changes since 2012-12-22), but it's never been tested in any other environment, and I'd rather not distribute it widely.

--
Bob Nichols     "NOSPAM" is really part of my email address.
                 Do NOT delete it.

_______________________________________________
rdiff-backup-users mailing list at [hidden email]
https://lists.nongnu.org/mailman/listinfo/rdiff-backup-users
Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki

comments.txt (3K) Download Attachment