Asynchronous upload and par2 creation ?

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Asynchronous upload and par2 creation ?

duplicity-talk mailing list
Hi,

I use Duplicity 0.7.18.1 with par2+multi:// as destination of my backup
(B2 & S3).

I set --asynchronous-upload too.

When I run Duplicity with --verbosity 9, par2 creation & uploads does
not seem really asynchronous...

I can see steps like "Create Par2 recovery files" or "Uploading
s3://..." and it seem to block Duplicity running, there is no other log
lines during those steps.

Is it really asynchronous ?

Thank you.

Flo


_______________________________________________
Duplicity-talk mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/duplicity-talk
Reply | Threaded
Open this post in threaded view
|

Re: Asynchronous upload and par2 creation ?

duplicity-talk mailing list
On 12.10.2018 09:23, Florent B via Duplicity-talk wrote:

> Hi,
>
> I use Duplicity 0.7.18.1 with par2+multi:// as destination of my backup
> (B2 & S3).
>
> I set --asynchronous-upload too.
>
> When I run Duplicity with --verbosity 9, par2 creation & uploads does
> not seem really asynchronous...
>
> I can see steps like "Create Par2 recovery files" or "Uploading
> s3://..." and it seem to block Duplicity running, there is no other log
> lines during those steps.
>
> Is it really asynchronous ?
>

no, it isn't. the par2 backend currently merely wraps put/get and runs par2 in addition to the underlying backend operations. you can check here
  https://bazaar.launchpad.net/~duplicity-team/duplicity/0.7-series/view/head:/duplicity/backends/par2backend.py

it's not asynchronous in any way.. sorry ede/duply.net

_______________________________________________
Duplicity-talk mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/duplicity-talk
Reply | Threaded
Open this post in threaded view
|

Re: Asynchronous upload and par2 creation ?

duplicity-talk mailing list
On 12.10.2018 12:27, Florent B wrote:

> On 12/10/2018 11:45, [hidden email] wrote:
>>
>> no, it isn't. the par2 backend currently merely wraps put/get and runs par2 in addition to the underlying backend operations. you can check here
>>   https://bazaar.launchpad.net/~duplicity-team/duplicity/0.7-series/view/head:/duplicity/backends/par2backend.py
>>
>> it's not asynchronous in any way.. sorry ede/duply.net
>
> Thank you for your answer. OK par2 is not asynchronous, but does upload
> is in this case ?
>

it should, see
  https://bazaar.launchpad.net/~duplicity-team/duplicity/0.7-series/view/head:/bin/duplicity#L418

but it's not completely unsynced as the man page states (we should probably remove EXPERIMENTAL)
"
--asynchronous-upload
(EXPERIMENTAL) Perform file uploads asynchronously in the background, with respect to volume creation. This means that duplicity can upload a volume while, at the same time, preparing the next volume for upload. The intended end-result is a faster backup, because the local CPU and your bandwidth can be more consistently utilized. Use of this option implies additional need for disk space in the temporary storage location; rather than needing to store only one volume at a time, enough storage space is required to store two volumes.
"

..ede/duply.net

_______________________________________________
Duplicity-talk mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/duplicity-talk