Connection reset by peer

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Connection reset by peer

duplicity-talk mailing list
Hi,

At some point during the night, my box is losing its internet
connection. Duplicity says:

Giving up after 5 attempts. error: [Errno 104] Connection reset by peer

Is there any way I can increase the number of attempts, or get duplicity
to wait indefinitely until it can make a connection?

Oli


_______________________________________________
Duplicity-talk mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/duplicity-talk
Reply | Threaded
Open this post in threaded view
|

Re: Connection reset by peer

duplicity-talk mailing list
On 31.05.2018 09:53, Oliver Cole via Duplicity-talk wrote:
> Hi,
>
> At some point during the night, my box is losing its internet connection. Duplicity says:
>
> Giving up after 5 attempts. error: [Errno 104] Connection reset by peer
>
> Is there any way I can increase the number of attempts, or get duplicity to wait indefinitely until it can make a connection?

you can try setting the parameters '--backend-retry-delay <number>', '--num-retries <number>' as documented in the man page http://duplicity.nongnu.org/duplicity.1.html .

..ede/duply.net

_______________________________________________
Duplicity-talk mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/duplicity-talk
Reply | Threaded
Open this post in threaded view
|

Re: Connection reset by peer

duplicity-talk mailing list
Apologies, I should have seen that.

However - this has merely revealed the problem is somewhere else - as I
am getting "Connection reset by peer" at all times of day.

This is all I have in logs:

DEBUG 1
. Uploaded s3://[...]//dataduplicity-full.[...].vol5247.difftar.gpg to
STANDARD Storage at roughly 1111424.204075 bytes/second

INFO 1
. Deleting [...]/tmp/duplicity-a7RYkf-tempdir/mktemp-GJuTPE-4

DEBUG 1
. Forgetting temporary file
[...]/tmp/duplicity-a7RYkf-tempdir/mktemp-GJuTPE-4

INFO 13
. AsyncScheduler: task completed successfully

INFO 2 1133987256723
. Processed volume 5247

INFO 1
. Writing duplicity-full-signatures.[...].sigtar.gpg

INFO 1
. Uploading s3://[...]//duplicity-full-signatures.[...].sigtar.gpg to
STANDARD Storage

DEBUG 1
. Backtrace of previous error: Traceback (innermost last):
.   File "/usr/lib/python2.7/dist-packages/duplicity/backend.py", line
369, in inner_retry
.     return fn(self, *args)
.   File "/usr/lib/python2.7/dist-packages/duplicity/backend.py", line
545, in move
.     self.__do_put(source_path, remote_filename)
.   File "/usr/lib/python2.7/dist-packages/duplicity/backend.py", line
515, in __do_put
.     self.backend._put(source_path, remote_filename)
.   File
"/usr/lib/python2.7/dist-packages/duplicity/backends/_boto_single.py",
line 237, in _put
.     self.upload(source_path.name, key, headers)
.   File
"/usr/lib/python2.7/dist-packages/duplicity/backends/_boto_single.py",
line 290, in upload
.     num_cb=(max(2, 8 * globals.volsize / (1024 * 1024)))
[...]
.   File
"/home/oli/.local/lib/python2.7/site-packages/boto/connection.py", line
1030, in _mexe
.     raise ex
.  error: [Errno 104] Connection reset by peer

Oddly, volume 5247 seems only partly uploaded - and always almost
exactly the same size - 167MB out of 200MB, to within a few hundred
bytes - even if I delete it from AWS and restart the backup. I don't
think it's the last volume, so wouldn't it be odd for the size to be wrong?

I also don't understand why it's failing during this signature upload,
and none of the previous ones? Is it just too big?

$ du -h duplicity-full-signatures.[...].sigtar.gpg
7.1G    duplicity-full-signatures.[...].sigtar.gpg

Oli


On 01/06/2018 11:56, [hidden email] wrote:

> On 31.05.2018 09:53, Oliver Cole via Duplicity-talk wrote:
>> Hi,
>>
>> At some point during the night, my box is losing its internet connection. Duplicity says:
>>
>> Giving up after 5 attempts. error: [Errno 104] Connection reset by peer
>>
>> Is there any way I can increase the number of attempts, or get duplicity to wait indefinitely until it can make a connection?
>
> you can try setting the parameters '--backend-retry-delay <number>', '--num-retries <number>' as documented in the man page http://duplicity.nongnu.org/duplicity.1.html .
>
> ..ede/duply.net
>

_______________________________________________
Duplicity-talk mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/duplicity-talk
Reply | Threaded
Open this post in threaded view
|

Re: Connection reset by peer

duplicity-talk mailing list
On 6/8/2018 17:54, Oliver Cole wrote:
> I also don't understand why it's failing during this signature upload, and none of the previous ones? Is it just too big?
>
> $ du -h duplicity-full-signatures.[...].sigtar.gpg
> 7.1G    duplicity-full-signatures.[...].sigtar.gpg
>
> Oli

it says 5TB here
  https://docs.aws.amazon.com/AmazonS3/latest/dev/UploadingObjects.html

so yeah, probably.. ede/duply.net

_______________________________________________
Duplicity-talk mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/duplicity-talk
Reply | Threaded
Open this post in threaded view
|

Re: Connection reset by peer

duplicity-talk mailing list
So, obviously 7.1GB < 5TB - I assume you're talking about the single
operation 5GB upload limit?

So for the archives, I seem to have solved this problem with
--s3-use-multiprocessing:

"If enabled, files duplicity uploads to S3 will be split into chunks and
uploaded in parallel. Useful if you want to saturate your bandwidth or
if large files are failing during upload."

Thanks Edgar!

On 10/06/2018 14:25, [hidden email] wrote:

> On 6/8/2018 17:54, Oliver Cole wrote:
>> I also don't understand why it's failing during this signature upload, and none of the previous ones? Is it just too big?
>>
>> $ du -h duplicity-full-signatures.[...].sigtar.gpg
>> 7.1G    duplicity-full-signatures.[...].sigtar.gpg
>>
>> Oli
>
> it says 5TB here
>    https://docs.aws.amazon.com/AmazonS3/latest/dev/UploadingObjects.html
>
> so yeah, probably.. ede/duply.net
>

_______________________________________________
Duplicity-talk mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/duplicity-talk
Reply | Threaded
Open this post in threaded view
|

Re: Connection reset by peer

duplicity-talk mailing list
no problem Oli. obviously i wanted write 5GB below though.

to clarify for the ml archives. just reread the link below and it says
"
Upload objects in a single operation—With a *single PUT operation*, you can upload objects up to *5 GB* in size.
"
and
"
You can use a *multipart upload* for objects from *5 MB to 5 TB* in size.
"

so for files >5GB you will need to enable --s3-use-multiprocessing as you did.. regards ede/duply.net

PS: it would be interesting to know what happens if the signature is <5MB as it says above that there is this minimum size.

On 26.06.2018 23:00, Oliver Cole wrote:

> So, obviously 7.1GB < 5TB - I assume you're talking about the single operation 5GB upload limit?
>
> So for the archives, I seem to have solved this problem with --s3-use-multiprocessing:
>
> "If enabled, files duplicity uploads to S3 will be split into chunks and uploaded in parallel. Useful if you want to saturate your bandwidth or if large files are failing during upload."
>
> Thanks Edgar!
>
> On 10/06/2018 14:25, [hidden email] wrote:
>> On 6/8/2018 17:54, Oliver Cole wrote:
>>> I also don't understand why it's failing during this signature upload, and none of the previous ones? Is it just too big?
>>>
>>> $ du -h duplicity-full-signatures.[...].sigtar.gpg
>>> 7.1G    duplicity-full-signatures.[...].sigtar.gpg
>>>
>>> Oli
>>
>> it says 5TB here
>>    https://docs.aws.amazon.com/AmazonS3/latest/dev/UploadingObjects.html
>>
>> so yeah, probably.. ede/duply.net
>>


_______________________________________________
Duplicity-talk mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/duplicity-talk