Duplicity full backup has become slow

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Duplicity full backup has become slow

duplicity-talk mailing list
Hi,

I've been happy using duplicity to backup my Ubuntu PC to my NAS for around 18 months now, using rsync.

I do a full back up every 2 weeks, and and incremental daily.

I noticed in August that the full backup started taking around twice as long, from

--------------[ Backup Statistics ]--------------
StartTime 1502355166.95 (Thu Aug 10 09:52:46 2017)
EndTime 1502373067.73 (Thu Aug 10 14:51:07 2017)
ElapsedTime 17900.78 (4 hours 58 minutes 20.78 seconds)
SourceFiles 1361618
SourceFileSize 212181777813 (198 GB)
NewFiles 1361618
NewFileSize 212181777813 (198 GB)
DeletedFiles 0
ChangedFiles 0
ChangedFileSize 0 (0 bytes)
ChangedDeltaSize 0 (0 bytes)
DeltaEntries 1361618
RawDeltaSize 211041882531 (197 GB)
TotalDestinationSizeChange 166597720163 (155 GB)
Errors 0
-------------------------------------------------

to 

--------------[ Backup Statistics ]--------------
StartTime 1503684459.99 (Fri Aug 25 19:07:39 2017)
EndTime 1503718307.60 (Sat Aug 26 04:31:47 2017)
ElapsedTime 33847.61 (9 hours 24 minutes 7.61 seconds)
SourceFiles 1361798
SourceFileSize 212042512113 (197 GB)
NewFiles 1361798
NewFileSize 212042512113 (197 GB)
DeletedFiles 0
ChangedFiles 0
ChangedFileSize 0 (0 bytes)
ChangedDeltaSize 0 (0 bytes)
DeltaEntries 1361798
RawDeltaSize 210902807446 (196 GB)
TotalDestinationSizeChange 166567853868 (155 GB)
Errors 0
-------------------------------------------------

As you can see, pretty much the same statistics except for the time taken.

Now in between these 2 full backups, probably the most significant change to my system was an upgrade from Ubuntu 16.10 to 17.04.

I don't think network speed is an issue because some of the (4GB) vol files can take as little as 5 minutes (looking at the timestamps) which I assume includes creation and transfer time.
I've noticed that the slowest archive files to be produced took around 1h 25m, and either consisted of lots and lots of small files, or very large binary files.

Is anyone aware of any change in the versions of the Ubuntu packages that duplicity relies on, between Ubuntu 16.10 and 17.04 that might have caused this slowdown?

I haven't looked yet at adding extra logging to help see if my thoughts on creation vs transfer time are correct.

On the plus side, this has nudged me to look more closely at what unnecessary cruft I'm backing up :D

Many thanks, and best regards,

Matthew.


_______________________________________________
Duplicity-talk mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/duplicity-talk
Reply | Threaded
Open this post in threaded view
|

Re: Duplicity full backup has become slow

duplicity-talk mailing list
On 11/12/2017 19:19, Matthew Harrison via Duplicity-talk wrote:

> Hi,
>
> I've been happy using duplicity to backup my Ubuntu PC to my NAS for around
> 18 months now, using rsync.
>
> I do a full back up every 2 weeks, and and incremental daily.
>
> I noticed in August that the full backup started taking around twice as
> long, from
>
> --------------[ Backup Statistics ]--------------
> StartTime 1502355166.95 (Thu Aug 10 09:52:46 2017)
> EndTime 1502373067.73 (Thu Aug 10 14:51:07 2017)
> ElapsedTime 17900.78 (4 hours 58 minutes 20.78 seconds)
> SourceFiles 1361618
> SourceFileSize 212181777813 (198 GB)
> NewFiles 1361618
> NewFileSize 212181777813 (198 GB)
> DeletedFiles 0
> ChangedFiles 0
> ChangedFileSize 0 (0 bytes)
> ChangedDeltaSize 0 (0 bytes)
> DeltaEntries 1361618
> RawDeltaSize 211041882531 (197 GB)
> TotalDestinationSizeChange 166597720163 (155 GB)
> Errors 0
> -------------------------------------------------
>
> to
>
> --------------[ Backup Statistics ]--------------
> StartTime 1503684459.99 (Fri Aug 25 19:07:39 2017)
> EndTime 1503718307.60 (Sat Aug 26 04:31:47 2017)
> ElapsedTime 33847.61 (9 hours 24 minutes 7.61 seconds)
> SourceFiles 1361798
> SourceFileSize 212042512113 (197 GB)
> NewFiles 1361798
> NewFileSize 212042512113 (197 GB)
> DeletedFiles 0
> ChangedFiles 0
> ChangedFileSize 0 (0 bytes)
> ChangedDeltaSize 0 (0 bytes)
> DeltaEntries 1361798
> RawDeltaSize 210902807446 (196 GB)
> TotalDestinationSizeChange 166567853868 (155 GB)
> Errors 0
> -------------------------------------------------
>
> As you can see, pretty much the same statistics except for the time taken.
>
> Now in between these 2 full backups, probably the most significant change
> to my system was an upgrade from Ubuntu 16.10 to 17.04.
>
> I don't think network speed is an issue because some of the (4GB) vol files
> can take as little as 5 minutes (looking at the timestamps) which I assume
> includes creation and transfer time.
> I've noticed that the slowest archive files to be produced took around 1h
> 25m, and either consisted of lots and lots of small files, or very large
> binary files.
>
> Is anyone aware of any change in the versions of the Ubuntu packages that
> duplicity relies on, between Ubuntu 16.10 and 17.04 that might have caused
> this slowdown?
>
> I haven't looked yet at adding extra logging to help see if my thoughts on
> creation vs transfer time are correct.
>
> On the plus side, this has nudged me to look more closely at what
> unnecessary cruft I'm backing up :D
>
> Many thanks, and best regards,

Matthew,

what are the respective duplicity versions that were/are used?

..ede/duply.net

_______________________________________________
Duplicity-talk mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/duplicity-talk
Reply | Threaded
Open this post in threaded view
|

Re: Duplicity full backup has become slow

duplicity-talk mailing list
Hi,

sorry, I realised just after sending that I'd left that out.

Currently, duplicity is on 0.7.12. (I'm now at Ubuntu 17.10) 

In Ubuntu 16.10 and 17.04 it was (as far as I can tell) 0.7.06

(It possibly went from 0.7.06-2ubuntu2 to 0.7.06-2ubuntu3, looking at launchpad.net, but I haven't found where to find more detail in /var/log/apt/history.log

I'll try to see what version changes happened with the dependencies.

I'm running with no-encryption, and Volsize set to 4096.
(I've also now added -v9 now to get more debug.)

Many thanks,

Matthew.


On 12 November 2017 at 18:25, edgar.soldin--- via Duplicity-talk <[hidden email]> wrote:
On 11/12/2017 19:19, Matthew Harrison via Duplicity-talk wrote:
> Hi,
>
> I've been happy using duplicity to backup my Ubuntu PC to my NAS for around
> 18 months now, using rsync.
>
> I do a full back up every 2 weeks, and and incremental daily.
>
> I noticed in August that the full backup started taking around twice as
> long, from
>
> --------------[ Backup Statistics ]--------------
> StartTime 1502355166.95 (Thu Aug 10 09:52:46 2017)
> EndTime 1502373067.73 (Thu Aug 10 14:51:07 2017)
> ElapsedTime 17900.78 (4 hours 58 minutes 20.78 seconds)
> SourceFiles 1361618
> SourceFileSize 212181777813 (198 GB)
> NewFiles 1361618
> NewFileSize 212181777813 (198 GB)
> DeletedFiles 0
> ChangedFiles 0
> ChangedFileSize 0 (0 bytes)
> ChangedDeltaSize 0 (0 bytes)
> DeltaEntries 1361618
> RawDeltaSize 211041882531 (197 GB)
> TotalDestinationSizeChange 166597720163 (155 GB)
> Errors 0
> -------------------------------------------------
>
> to
>
> --------------[ Backup Statistics ]--------------
> StartTime 1503684459.99 (Fri Aug 25 19:07:39 2017)
> EndTime 1503718307.60 (Sat Aug 26 04:31:47 2017)
> ElapsedTime 33847.61 (9 hours 24 minutes 7.61 seconds)
> SourceFiles 1361798
> SourceFileSize 212042512113 (197 GB)
> NewFiles 1361798
> NewFileSize 212042512113 (197 GB)
> DeletedFiles 0
> ChangedFiles 0
> ChangedFileSize 0 (0 bytes)
> ChangedDeltaSize 0 (0 bytes)
> DeltaEntries 1361798
> RawDeltaSize 210902807446 (196 GB)
> TotalDestinationSizeChange 166567853868 (155 GB)
> Errors 0
> -------------------------------------------------
>
> As you can see, pretty much the same statistics except for the time taken.
>
> Now in between these 2 full backups, probably the most significant change
> to my system was an upgrade from Ubuntu 16.10 to 17.04.
>
> I don't think network speed is an issue because some of the (4GB) vol files
> can take as little as 5 minutes (looking at the timestamps) which I assume
> includes creation and transfer time.
> I've noticed that the slowest archive files to be produced took around 1h
> 25m, and either consisted of lots and lots of small files, or very large
> binary files.
>
> Is anyone aware of any change in the versions of the Ubuntu packages that
> duplicity relies on, between Ubuntu 16.10 and 17.04 that might have caused
> this slowdown?
>
> I haven't looked yet at adding extra logging to help see if my thoughts on
> creation vs transfer time are correct.
>
> On the plus side, this has nudged me to look more closely at what
> unnecessary cruft I'm backing up :D
>
> Many thanks, and best regards,

Matthew,

what are the respective duplicity versions that were/are used?

..ede/duply.net

_______________________________________________
Duplicity-talk mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/duplicity-talk


_______________________________________________
Duplicity-talk mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/duplicity-talk
Reply | Threaded
Open this post in threaded view
|

Re: Duplicity full backup has become slow

duplicity-talk mailing list
On 11/12/2017 20:07, Matthew Harrison via Duplicity-talk wrote:
> I'll try to see what version changes happened with the dependencies.

please do

> I'm running with no-encryption, and Volsize set to 4096.
> (I've also now added -v9 now to get more debug.)

is it possible that your system run's out of memory during the backup and starts swapping? please check your ram usage during a typical run. there was a change that raised the size of the manifest significantly.

..ede/duply.net

_______________________________________________
Duplicity-talk mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/duplicity-talk
Reply | Threaded
Open this post in threaded view
|

Re: Duplicity full backup has become slow

duplicity-talk mailing list


On 12 November 2017 at 19:16, edgar.soldin--- via Duplicity-talk <[hidden email]> wrote:
On 11/12/2017 20:07, Matthew Harrison via Duplicity-talk wrote:
> I'll try to see what version changes happened with the dependencies.

please do

> I'm running with no-encryption, and Volsize set to 4096.
> (I've also now added -v9 now to get more debug.)

is it possible that your system run's out of memory during the backup and starts swapping? please check your ram usage during a typical run. there was a change that raised the size of the manifest significantly.

I have 16G installed, but it may be possible, I suppose... I have never noticed a general performance problem. I will have to check next time.
Is the full manifest kept in memory the whole time, or only that for the current volume?

best wishes,

M.

_______________________________________________
Duplicity-talk mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/duplicity-talk
Reply | Threaded
Open this post in threaded view
|

Re: Duplicity full backup has become slow

duplicity-talk mailing list
Duplicity 0.7.15 was just released.  Please try that if possible.

There are three options:
NOTE: UNinstall duplicity first if it was installed via the distribution repository.  For Ubuntu, that would be "sudo apt-get purge duplicity". 

The builds for the PPAs are running now and should be ready in about an hour.

...Thanks,
...Ken


On Sun, Nov 12, 2017 at 3:01 PM, Matthew Harrison via Duplicity-talk <[hidden email]> wrote:


On 12 November 2017 at 19:16, edgar.soldin--- via Duplicity-talk <[hidden email]> wrote:
On 11/12/2017 20:07, Matthew Harrison via Duplicity-talk wrote:
> I'll try to see what version changes happened with the dependencies.

please do

> I'm running with no-encryption, and Volsize set to 4096.
> (I've also now added -v9 now to get more debug.)

is it possible that your system run's out of memory during the backup and starts swapping? please check your ram usage during a typical run. there was a change that raised the size of the manifest significantly.

I have 16G installed, but it may be possible, I suppose... I have never noticed a general performance problem. I will have to check next time.
Is the full manifest kept in memory the whole time, or only that for the current volume?

best wishes,

M.

_______________________________________________
Duplicity-talk mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/duplicity-talk



_______________________________________________
Duplicity-talk mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/duplicity-talk