Only sync remote metadata from primary backup chain

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Only sync remote metadata from primary backup chain

duplicity-talk mailing list
Hello,

I've been using duplicity to backup since 2014 with a full backup every 3 months, so I have some secondary backup chains. Additionally backups go to multiple separate places (each having multiple backup chains). My local cache folder is now over 70gb big and I had the idea to delete the cache for the secondary chains to free up space - since they are only needed when restoring files, but not for new backups (to the primary chains).

So I deleted the cache files (for example duplicity-full-signatures.20141106T121637Z.sigtar.gpg) and reclaimed a lot of space.

But after the next backup run, I saw this in the logs:

Synchronizing remote metadata to local cache...
Copying duplicity-full-signatures.20141106T121637Z.sigtar.gpg to local cache.
Copying duplicity-full-signatures.20150505T234205Z.sigtar.gpg to local cache.
Copying duplicity-full-signatures.20150904T234123Z.sigtar.gpg to local cache.
Copying duplicity-full-signatures.20151205T004128Z.sigtar.gpg to local cache.
...

Is this normal? If yes, is there an option to skip synchronizing the secondary backup chain's metadata I don't find in the man page? If I understand it correctly, I only need them to restore files from those chains, since I will not do new incremental backups to that secondary backup chains. I'd imagine duplicity to only sync the metadata in the case I want to restore files from those chains.

Or is there any other way to reduce the cache usage (besides completely deleting the remote backups which I don't want).

For reference: I'm using duplicity with duply (http://duply.net/), so maybe this triggers the behavior I describe if this is not normal behavior.

-- Markus

_______________________________________________
Duplicity-talk mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/duplicity-talk
Reply | Threaded
Open this post in threaded view
|

Re: Only sync remote metadata from primary backup chain

duplicity-talk mailing list
OK, I found a "workaround": Before I create a new full backup, I simply move the remote files to a new location. So for duplicity it looks like the backup location is fresh and it even deletes the local cache automatically, starting with a full backup (which I wanted anyways).

If I later want to restore from that backup I will have to move the files back into place on that remote.

-- Markus

On 01.07.17 13:57, Markus Doits via Duplicity-talk wrote:

> Hello,
>
> I've been using duplicity to backup since 2014 with a full backup every 3 months, so I have some secondary backup chains. Additionally backups go to multiple separate places (each having multiple backup chains). My local cache folder is now over 70gb big and I had the idea to delete the cache for the secondary chains to free up space - since they are only needed when restoring files, but not for new backups (to the primary chains).
>
> So I deleted the cache files (for example duplicity-full-signatures.20141106T121637Z.sigtar.gpg) and reclaimed a lot of space.
>
> But after the next backup run, I saw this in the logs:
>
> Synchronizing remote metadata to local cache...
> Copying duplicity-full-signatures.20141106T121637Z.sigtar.gpg to local cache.
> Copying duplicity-full-signatures.20150505T234205Z.sigtar.gpg to local cache.
> Copying duplicity-full-signatures.20150904T234123Z.sigtar.gpg to local cache.
> Copying duplicity-full-signatures.20151205T004128Z.sigtar.gpg to local cache.
> ...
>
> Is this normal? If yes, is there an option to skip synchronizing the secondary backup chain's metadata I don't find in the man page? If I understand it correctly, I only need them to restore files from those chains, since I will not do new incremental backups to that secondary backup chains. I'd imagine duplicity to only sync the metadata in the case I want to restore files from those chains.
>
> Or is there any other way to reduce the cache usage (besides completely deleting the remote backups which I don't want).
>
> For reference: I'm using duplicity with duply (http://duply.net/), so maybe this triggers the behavior I describe if this is not normal behavior.
>
> -- Markus
>
> _______________________________________________
> Duplicity-talk mailing list
> [hidden email]
> https://lists.nongnu.org/mailman/listinfo/duplicity-talk
>

--
Stellenticket GmbH
Oderberger Straße 56
10435 Berlin
Tel.: +49 (0)30 / 57707 4480
Fax: +49 (0)30 / 57707 4489
E-Mail: [hidden email]
Web: www.stellenticket.de

Geschäftsführer:
Christoph Nefzger, Markus Doits

Handelsregister:
Registergericht: Amtsgericht Charlottenburg (Berlin) HRB 121809 B
Umsatzsteueridentifikationsnummer: DE268748270

_______________________________________________
Duplicity-talk mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/duplicity-talk
Reply | Threaded
Open this post in threaded view
|

Re: Only sync remote metadata from primary backup chain

duplicity-talk mailing list
Please register this in https://bugs.launchpad.net/duplicity as a bug report so it does not get lost.  It seems like a good optimization.

...Ken


On Wed, Jul 5, 2017 at 10:45 AM, Markus Doits via Duplicity-talk <[hidden email]> wrote:
OK, I found a "workaround": Before I create a new full backup, I simply move the remote files to a new location. So for duplicity it looks like the backup location is fresh and it even deletes the local cache automatically, starting with a full backup (which I wanted anyways).

If I later want to restore from that backup I will have to move the files back into place on that remote.

-- Markus

On 01.07.17 13:57, Markus Doits via Duplicity-talk wrote:
> Hello,
>
> I've been using duplicity to backup since 2014 with a full backup every 3 months, so I have some secondary backup chains. Additionally backups go to multiple separate places (each having multiple backup chains). My local cache folder is now over 70gb big and I had the idea to delete the cache for the secondary chains to free up space - since they are only needed when restoring files, but not for new backups (to the primary chains).
>
> So I deleted the cache files (for example duplicity-full-signatures.20141106T121637Z.sigtar.gpg) and reclaimed a lot of space.
>
> But after the next backup run, I saw this in the logs:
>
> Synchronizing remote metadata to local cache...
> Copying duplicity-full-signatures.20141106T121637Z.sigtar.gpg to local cache.
> Copying duplicity-full-signatures.20150505T234205Z.sigtar.gpg to local cache.
> Copying duplicity-full-signatures.20150904T234123Z.sigtar.gpg to local cache.
> Copying duplicity-full-signatures.20151205T004128Z.sigtar.gpg to local cache.
> ...
>
> Is this normal? If yes, is there an option to skip synchronizing the secondary backup chain's metadata I don't find in the man page? If I understand it correctly, I only need them to restore files from those chains, since I will not do new incremental backups to that secondary backup chains. I'd imagine duplicity to only sync the metadata in the case I want to restore files from those chains.
>
> Or is there any other way to reduce the cache usage (besides completely deleting the remote backups which I don't want).
>
> For reference: I'm using duplicity with duply (http://duply.net/), so maybe this triggers the behavior I describe if this is not normal behavior.
>
> -- Markus
>
> _______________________________________________
> Duplicity-talk mailing list
> [hidden email]
> https://lists.nongnu.org/mailman/listinfo/duplicity-talk
>

--
Stellenticket GmbH
Oderberger Straße 56
10435 Berlin
Tel.: <a href="tel:%2B49%20%280%2930%20%2F%2057707%204480" value="+4930577074480">+49 (0)30 / 57707 4480
Fax: <a href="tel:%2B49%20%280%2930%20%2F%2057707%204489" value="+4930577074489">+49 (0)30 / 57707 4489
E-Mail: [hidden email]
Web: www.stellenticket.de

Geschäftsführer:
Christoph Nefzger, Markus Doits

Handelsregister:
Registergericht: Amtsgericht Charlottenburg (Berlin) HRB 121809 B
Umsatzsteueridentifikationsnummer: DE268748270

_______________________________________________
Duplicity-talk mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/duplicity-talk


_______________________________________________
Duplicity-talk mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/duplicity-talk