Restarting from wrong place

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

Restarting from wrong place

duplicity-talk mailing list
Hi y'all. Thanks for this handy tool! I'm running into a problem trying to back up about 1 TB to Backblaze B2, using duplicity 0.7.12 on Ubuntu 17.10  (since that's the most recent packaged version).

Something has gone wrong in the middle of my backup. There should be 4703 volumes in the backup, but there are only 4669. When I examine what's actually present in B2, I see that the 34 missing volumes are from the middle of the set:

2450, 2746, 3126, 3127, 3128, 3129, 3130, 3131, 3132, 3133, 3134, 3135, 3136, 3137, 3138, 3139, 3140, 3141, 3276, 3277, 3278, 3279, 3280, 3281, 3730, 3732, 3733, 3735, 3878, 3879, 3880, 3883, 3884, 3885

duplicity notices that the backup set is incomplete...

> Last full backup left a partial set, restarting.

But then it picks up the backup from volume #4669, not from the first missing volume:

> Found 4703 volumes in manifest
> RESTART: Volumes 4669 to 4703 failed to upload before termination.
>          Restarting backup at volume 4669.

Is there a way to get it to fill in the gaps instead?

Thanks,
Aneel

_______________________________________________
Duplicity-talk mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/duplicity-talk
Reply | Threaded
Open this post in threaded view
|

Re: Restarting from wrong place

duplicity-talk mailing list
I just upgraded to 0.7.16 and see the same behavior.

On Fri, Feb 16, 2018, at 11:08 PM, backup--- via Duplicity-talk wrote:

> Hi y'all. Thanks for this handy tool! I'm running into a problem trying
> to back up about 1 TB to Backblaze B2, using duplicity 0.7.12 on Ubuntu
> 17.10  (since that's the most recent packaged version).
>
> Something has gone wrong in the middle of my backup. There should be
> 4703 volumes in the backup, but there are only 4669. When I examine
> what's actually present in B2, I see that the 34 missing volumes are
> from the middle of the set:
>
> 2450, 2746, 3126, 3127, 3128, 3129, 3130, 3131, 3132, 3133, 3134, 3135,
> 3136, 3137, 3138, 3139, 3140, 3141, 3276, 3277, 3278, 3279, 3280, 3281,
> 3730, 3732, 3733, 3735, 3878, 3879, 3880, 3883, 3884, 3885
>
> duplicity notices that the backup set is incomplete...
>
> > Last full backup left a partial set, restarting.
>
> But then it picks up the backup from volume #4669, not from the first
> missing volume:
>
> > Found 4703 volumes in manifest
> > RESTART: Volumes 4669 to 4703 failed to upload before termination.
> >          Restarting backup at volume 4669.
>
> Is there a way to get it to fill in the gaps instead?
>
> Thanks,
> Aneel
>
> _______________________________________________
> Duplicity-talk mailing list
> [hidden email]
> https://lists.nongnu.org/mailman/listinfo/duplicity-talk

_______________________________________________
Duplicity-talk mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/duplicity-talk
Reply | Threaded
Open this post in threaded view
|

Re: Restarting from wrong place

duplicity-talk mailing list
The B2 backend was rewritten in 0.7.15 to use the B2 API and not the web interface.  It's considerably faster and supports the progress indicator.

Some questions:
 - Has the backup size increased since you last ran a full backup?
 - What is the volume size you use (default is 200MB)? 

I'm guessing that, as the number of volumes increased, then B2 truncated the list being returned when querying for the list of files.  If I'm right, then it will repeat each time.

One thing to try, double your volume size, say 200MB to 400MB, start a new backup, and that should keep it from happening for a long while (just a workaround).

Please answer back and let me know if that works around the problem.

...Thanks,
...Ken


On Fri, Feb 16, 2018 at 11:18 PM, backup--- via Duplicity-talk <[hidden email]> wrote:
I just upgraded to 0.7.16 and see the same behavior.

On Fri, Feb 16, 2018, at 11:08 PM, backup--- via Duplicity-talk wrote:
> Hi y'all. Thanks for this handy tool! I'm running into a problem trying
> to back up about 1 TB to Backblaze B2, using duplicity 0.7.12 on Ubuntu
> 17.10  (since that's the most recent packaged version).
>
> Something has gone wrong in the middle of my backup. There should be
> 4703 volumes in the backup, but there are only 4669. When I examine
> what's actually present in B2, I see that the 34 missing volumes are
> from the middle of the set:
>
> 2450, 2746, 3126, 3127, 3128, 3129, 3130, 3131, 3132, 3133, 3134, 3135,
> 3136, 3137, 3138, 3139, 3140, 3141, 3276, 3277, 3278, 3279, 3280, 3281,
> 3730, 3732, 3733, 3735, 3878, 3879, 3880, 3883, 3884, 3885
>
> duplicity notices that the backup set is incomplete...
>
> > Last full backup left a partial set, restarting.
>
> But then it picks up the backup from volume #4669, not from the first
> missing volume:
>
> > Found 4703 volumes in manifest
> > RESTART: Volumes 4669 to 4703 failed to upload before termination.
> >          Restarting backup at volume 4669.
>
> Is there a way to get it to fill in the gaps instead?
>
> Thanks,
> Aneel
>
> _______________________________________________
> Duplicity-talk mailing list
> [hidden email]
> https://lists.nongnu.org/mailman/listinfo/duplicity-talk

_______________________________________________
Duplicity-talk mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/duplicity-talk


_______________________________________________
Duplicity-talk mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/duplicity-talk
Reply | Threaded
Open this post in threaded view
|

Re: Restarting from wrong place

duplicity-talk mailing list
On Sat, Feb 17, 2018, at 5:36 AM, Kenneth Loafman wrote:
The B2 backend was rewritten in 0.7.15 to use the B2 API and not the web interface.  It's considerably faster and supports the progress indicator.

Great! Glad I upgraded.


Some questions:
 - Has the backup size increased since you last ran a full backup?

It shouldn't have. This is a fairly static set of data (archived digital photos).


 - What is the volume size you use (default is 200MB)?

I was using the default 200MB


I'm guessing that, as the number of volumes increased, then B2 truncated the list being returned when querying for the list of files.  If I'm right, then it will repeat each time.

Well, it definitely repeats each time. It always starts from vol 4669, and seems to succeed at uploading all of the volumes it tries, but those are 4669-4703, which were already present at the destination.


One thing to try, double your volume size, say 200MB to 400MB, start a new backup, and that should keep it from happening for a long while (just a workaround).

Okay. I've started a new backup (to a new B2 bucket). It took about a month for a full backup last time, but the progress bar says that it'll finish in ~6 days now, which would be great.


Please answer back and let me know if that works around the problem.

I'll report in again in a week or so.

Thanks for your reply!



...Thanks,
...Ken


On Fri, Feb 16, 2018 at 11:18 PM, backup--- via Duplicity-talk <[hidden email]> wrote:
I just upgraded to 0.7.16 and see the same behavior.


On Fri, Feb 16, 2018, at 11:08 PM, backup--- via Duplicity-talk wrote:
> Hi y'all. Thanks for this handy tool! I'm running into a problem trying
> to back up about 1 TB to Backblaze B2, using duplicity 0.7.12 on Ubuntu
> 17.10  (since that's the most recent packaged version).
>
> Something has gone wrong in the middle of my backup. There should be
> 4703 volumes in the backup, but there are only 4669. When I examine
> what's actually present in B2, I see that the 34 missing volumes are
> from the middle of the set:
>
> 2450, 2746, 3126, 3127, 3128, 3129, 3130, 3131, 3132, 3133, 3134, 3135,
> 3136, 3137, 3138, 3139, 3140, 3141, 3276, 3277, 3278, 3279, 3280, 3281,
> 3730, 3732, 3733, 3735, 3878, 3879, 3880, 3883, 3884, 3885
>
> duplicity notices that the backup set is incomplete...
>
> > Last full backup left a partial set, restarting.
>
> But then it picks up the backup from volume #4669, not from the first
> missing volume:
>
> > Found 4703 volumes in manifest
> > RESTART: Volumes 4669 to 4703 failed to upload before termination.
> >          Restarting backup at volume 4669.
>
> Is there a way to get it to fill in the gaps instead?
>
> Thanks,
> Aneel
>
> _______________________________________________
> Duplicity-talk mailing list

_______________________________________________
Duplicity-talk mailing list


_______________________________________________
Duplicity-talk mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/duplicity-talk
Reply | Threaded
Open this post in threaded view
|

Re: Restarting from wrong place

duplicity-talk mailing list
To update: starting a new backup from scratch seems to have solved the problem. Thanks!


On Sat, Feb 17, 2018, at 12:40 PM, backup--- via Duplicity-talk wrote:
On Sat, Feb 17, 2018, at 5:36 AM, Kenneth Loafman wrote:
The B2 backend was rewritten in 0.7.15 to use the B2 API and not the web interface.  It's considerably faster and supports the progress indicator.

Great! Glad I upgraded.


Some questions:
 - Has the backup size increased since you last ran a full backup?

It shouldn't have. This is a fairly static set of data (archived digital photos).


 - What is the volume size you use (default is 200MB)?

I was using the default 200MB


I'm guessing that, as the number of volumes increased, then B2 truncated the list being returned when querying for the list of files.  If I'm right, then it will repeat each time.

Well, it definitely repeats each time. It always starts from vol 4669, and seems to succeed at uploading all of the volumes it tries, but those are 4669-4703, which were already present at the destination.


One thing to try, double your volume size, say 200MB to 400MB, start a new backup, and that should keep it from happening for a long while (just a workaround).

Okay. I've started a new backup (to a new B2 bucket). It took about a month for a full backup last time, but the progress bar says that it'll finish in ~6 days now, which would be great.


Please answer back and let me know if that works around the problem.

I'll report in again in a week or so.

Thanks for your reply!



...Thanks,
...Ken


On Fri, Feb 16, 2018 at 11:18 PM, backup--- via Duplicity-talk <[hidden email]> wrote:
I just upgraded to 0.7.16 and see the same behavior.


On Fri, Feb 16, 2018, at 11:08 PM, backup--- via Duplicity-talk wrote:
> Hi y'all. Thanks for this handy tool! I'm running into a problem trying
> to back up about 1 TB to Backblaze B2, using duplicity 0.7.12 on Ubuntu
> 17.10  (since that's the most recent packaged version).
>
> Something has gone wrong in the middle of my backup. There should be
> 4703 volumes in the backup, but there are only 4669. When I examine
> what's actually present in B2, I see that the 34 missing volumes are
> from the middle of the set:
>
> 2450, 2746, 3126, 3127, 3128, 3129, 3130, 3131, 3132, 3133, 3134, 3135,
> 3136, 3137, 3138, 3139, 3140, 3141, 3276, 3277, 3278, 3279, 3280, 3281,
> 3730, 3732, 3733, 3735, 3878, 3879, 3880, 3883, 3884, 3885
>
> duplicity notices that the backup set is incomplete...
>
> > Last full backup left a partial set, restarting.
>
> But then it picks up the backup from volume #4669, not from the first
> missing volume:
>
> > Found 4703 volumes in manifest
> > RESTART: Volumes 4669 to 4703 failed to upload before termination.
> >          Restarting backup at volume 4669.
>
> Is there a way to get it to fill in the gaps instead?
>
> Thanks,
> Aneel
>
> _______________________________________________
> Duplicity-talk mailing list

_______________________________________________
Duplicity-talk mailing list

_______________________________________________
Duplicity-talk mailing list


_______________________________________________
Duplicity-talk mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/duplicity-talk