Duplicity/Duply: Multiple questions about backup restores in Duply/Duplicity

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Duplicity/Duply: Multiple questions about backup restores in Duply/Duplicity

duplicity-talk mailing list
Hello,

My company is using Duply to back up data in the following manner to S3:

Monday-Saturday: incremental backup
Sunday: full backup

Each time a full backup is made, a signature file is created and placed in the server's local cache as well as the S3 bucket.

Our S3 bucket policy is to hold backups in standard storage for 9 days, then move the backups to Glacier. After roughly 2 years, the data is deleted from Glacier.

With this in mind, I have the following questions:

1. Let's say I lose the signature file from the first week of September of last year. Can I still restore backups from the previous to and following that week, provided I have all the other signature files?

2. We never expire (purge?) our backups, so based on our S3 aging policy, we will start erasing backups in two years, including signature files. Can we still restore backups even with these older files being gone? For instance, can I restore a backup from two weeks ago if the signature files from two years ago have been deleted?

3. Something bad happens to a server and we lose the original .cache files stored locally on that server. We have the backup files and signature file for the last week in standard storage in a S3 bucket, with the rest of the files for older backups residing in Glacier. Can I restore from a backup from yesterday with what's already in standard storage, or do I need to restore all previous backups to standard storage first?

4. Does a full backup contain only the state of files at the time the full backup was made, or are the all the changes to those files since the last full also contained in the full? For instance, I have a file named spam.txt that contains the word "spam" on Tuesday and on Wednesday I add the line "and eggs" to it. Does the full backup of that file contain both versions, or only the "spam and eggs" version?

5. Do I need to keep al the incremental backups of all previous backups to be able to successfully restore from the most recent backup if we're not using purge?

Sorry, lots of questions, but Google isn't being my friend on the answers.

Thank you for your time,


-- Mike Beebe
--
Mike Beebe

 DevOps Engineer, LabKey
 

www.LabKey.com

_______________________________________________
Duplicity-talk mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/duplicity-talk
Reply | Threaded
Open this post in threaded view
|

Re: Duplicity/Duply: Multiple questions about backup restores in Duply/Duplicity

duplicity-talk mailing list
I can try a few of these.  Not sure about #1

> On Apr 6, 2018, at 5:26 PM, Mike Beebe via Duplicity-talk <[hidden email]> wrote:
>
> 2. We never expire (purge?) our backups, so based on our S3 aging policy, we will start erasing backups in two years, including signature files. Can we still restore backups even with these older files being gone? For instance, can I restore a backup from two weeks ago if the signature files from two years ago have been deleted?

Yes, you need the signature file from the previous full backup and all the incremental backups up to the one you need to restore.

So if you had a full backup on Sunday 2/4/18 and want to restore a file from Wednesday 2/7 then you need the 4 signature files from 2/4, 2/5, 2/6 and 2/7.  Older signature files previous to the full backup are irrelevant.

> 3. Something bad happens to a server and we lose the original .cache files stored locally on that server. We have the backup files and signature file for the last week in standard storage in a S3 bucket, with the rest of the files for older backups residing in Glacier. Can I restore from a backup from yesterday with what's already in standard storage, or do I need to restore all previous backups to standard storage first?
You will have to restore the backups to standard storage back to the last full backup.  To restore you need access to the full backup chain up to the version you are restoring.  That is the full and all intermediate incremental backups.

> 4. Does a full backup contain only the state of files at the time the full backup was made, or are the all the changes to those files since the last full also contained in the full? For instance, I have a file named spam.txt that contains the word "spam" on Tuesday and on Wednesday I add the line "and eggs" to it. Does the full backup of that file contain both versions, or only the "spam and eggs" version?
The full backup is the snapshot of the file on Sunday.  The next incremental backup after you add “and eggs” on Wednesday will get saved Wednesday night.  To restore that modified version you need access to the Sunday full backup and the rest of the chain (Monday, Tuesday and Wednesday) incrementally.

> 5. Do I need to keep al the incremental backups of all previous backups to be able to successfully restore from the most recent backup if we're not using purge?
Only the incremental backups back to the last full backup.  If you have

Full 1
inc 1A
inc 1B
inc 1C
Full 2
Inc 2A
Inc 2B
Inc 2C

And you delete Inc 1A, 1B and 1C you will lose any files that were modified after Full 1 and Modified again by the time Full 2.

So if you had Spam at Full 1, modified to to “Spam and eggs” by Inc 1B then modified it again to “Green Spam and Eggs” at Full 2.  And then delete Inc 1A, 1B and 1C you will lose “Spam and eggs”.  You will still have “Spam” in Full 1 and “Green  Spam and Eggs” in Full 2.

Clear as mud?

HTH,
Scott



_______________________________________________
Duplicity-talk mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/duplicity-talk
Reply | Threaded
Open this post in threaded view
|

Re: Duplicity/Duply: Multiple questions about backup restores in Duply/Duplicity

duplicity-talk mailing list
Scott,

Thank you greatly for your reply.

Yes, your explanations are very clear and very much appreciated -- you've helped clarify my understanding of how Duplicity works.

-- Mike Beebe

On Fri, Apr 6, 2018 at 2:46 PM, Scott Hannahs <[hidden email]> wrote:
I can try a few of these.  Not sure about #1

> On Apr 6, 2018, at 5:26 PM, Mike Beebe via Duplicity-talk <[hidden email]> wrote:
>
> 2. We never expire (purge?) our backups, so based on our S3 aging policy, we will start erasing backups in two years, including signature files. Can we still restore backups even with these older files being gone? For instance, can I restore a backup from two weeks ago if the signature files from two years ago have been deleted?

Yes, you need the signature file from the previous full backup and all the incremental backups up to the one you need to restore.

So if you had a full backup on Sunday 2/4/18 and want to restore a file from Wednesday 2/7 then you need the 4 signature files from 2/4, 2/5, 2/6 and 2/7.  Older signature files previous to the full backup are irrelevant.

> 3. Something bad happens to a server and we lose the original .cache files stored locally on that server. We have the backup files and signature file for the last week in standard storage in a S3 bucket, with the rest of the files for older backups residing in Glacier. Can I restore from a backup from yesterday with what's already in standard storage, or do I need to restore all previous backups to standard storage first?
You will have to restore the backups to standard storage back to the last full backup.  To restore you need access to the full backup chain up to the version you are restoring.  That is the full and all intermediate incremental backups.

> 4. Does a full backup contain only the state of files at the time the full backup was made, or are the all the changes to those files since the last full also contained in the full? For instance, I have a file named spam.txt that contains the word "spam" on Tuesday and on Wednesday I add the line "and eggs" to it. Does the full backup of that file contain both versions, or only the "spam and eggs" version?
The full backup is the snapshot of the file on Sunday.  The next incremental backup after you add “and eggs” on Wednesday will get saved Wednesday night.  To restore that modified version you need access to the Sunday full backup and the rest of the chain (Monday, Tuesday and Wednesday) incrementally.

> 5. Do I need to keep al the incremental backups of all previous backups to be able to successfully restore from the most recent backup if we're not using purge?
Only the incremental backups back to the last full backup.  If you have

Full 1
inc 1A
inc 1B
inc 1C
Full 2
Inc 2A
Inc 2B
Inc 2C

And you delete Inc 1A, 1B and 1C you will lose any files that were modified after Full 1 and Modified again by the time Full 2.

So if you had Spam at Full 1, modified to to “Spam and eggs” by Inc 1B then modified it again to “Green Spam and Eggs” at Full 2.  And then delete Inc 1A, 1B and 1C you will lose “Spam and eggs”.  You will still have “Spam” in Full 1 and “Green  Spam and Eggs” in Full 2.

Clear as mud?

HTH,
Scott





--
Mike Beebe

 DevOps Engineer, LabKey
 
www.LabKey.com

_______________________________________________
Duplicity-talk mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/duplicity-talk