duplicity and compressed archives

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

duplicity and compressed archives

Noel Kelly-2
Hi

I have been running some tests using duplicity.  I have found that
compressed archives (eg .tar.gz) do not seem to transfer incrementally
using librsync.

If I transfer a compressed archive of a database dump using rsync then
only the changes are transferred after the initial full transfer.

When I try the same experiment using duplicity it simply transfers the
entire file again.  It is as though librsync is not being called/used.

Below is a copy of my last two duplicity commands - note the
ChangeFileSize is 51Mb each time.

Any suggestions or explanations much appreciated!

Cheers
Noel



n@medulla:/backups/20091003/$ sudo duplicity --scp-command "scp -l 1024"
./com.sql.bz2  scp://[hidden email]/com/
Password:
GnuPG passphrase:
No signatures found, switching to full backup.
tmpLUSD-P
...
100%   24KB  24.1KB/s   00:00  
--------------[ Backup Statistics ]--------------
StartTime 1254935682.83 (Wed Oct  7 18:14:42 2009)
EndTime 1254936124.95 (Wed Oct  7 18:22:04 2009)
ElapsedTime 442.13 (7 minutes 22.13 seconds)
SourceFiles 1
SourceFileSize 52987804 (50.5 MB)
NewFiles 1
NewFileSize 52987804 (50.5 MB)
DeletedFiles 0
ChangedFiles 0
ChangedFileSize 0 (0 bytes)
ChangedDeltaSize 0 (0 bytes)
DeltaEntries 1
RawDeltaSize 5204480 (4.96 MB)
TotalDestinationSizeChange 53615229 (51.1 MB)
Errors 0
-------------------------------------------------

n@medulla:/backups/20091006/$ sudo duplicity --scp-command "scp -l 1024"
./com.sql.bz2  scp://[hidden email]/com/
Password:
GnuPG passphrase:
duplicity-full.2009-10-07T18:14:34+01:00.manifest.gpg                              
100%  464     0.5KB/s   00:00  
duplicity-full-signatures.2009-10-07T18:14:34+01:00.sigtar.gpg                    
100%   24KB  24.1KB/s   00:01  
tmpxMoIR1
...
  100%   24KB  24.0KB/s   00:01  
--------------[ Backup Statistics ]--------------
StartTime 1254937169.04 (Wed Oct  7 18:39:29 2009)
EndTime 1254937617.68 (Wed Oct  7 18:46:57 2009)
ElapsedTime 448.64 (7 minutes 28.64 seconds)
SourceFiles 1
SourceFileSize 52903512 (50.5 MB)
NewFiles 0
NewFileSize 0 (0 bytes)
DeletedFiles 0
ChangedFiles 1
ChangedFileSize 52903512 (50.5 MB)
ChangedDeltaSize 0 (0 bytes)
DeltaEntries 1
RawDeltaSize 5204480 (4.96 MB)
TotalDestinationSizeChange 53540930 (51.1 MB)
Errors 0
-------------------------------------------------



_______________________________________________
Duplicity-talk mailing list
[hidden email]
http://lists.nongnu.org/mailman/listinfo/duplicity-talk
Reply | Threaded
Open this post in threaded view
|

Re: duplicity and compressed archives

edgar.soldin
Did you verify on the storage space that the incremental really is 51M
sized? It seemsto me that the value you are looking for ist the
RawDeltaSize 5204480 (4.96 MB)
also the file is detected as new, probably because the time stamp
changed ...

... ede
duply.net

On 07.10.2009 20:16, Noel Kelly wrote:

> Hi
>
> I have been running some tests using duplicity.  I have found that
> compressed archives (eg .tar.gz) do not seem to transfer incrementally
> using librsync.
>
> If I transfer a compressed archive of a database dump using rsync then
> only the changes are transferred after the initial full transfer.
>
> When I try the same experiment using duplicity it simply transfers the
> entire file again.  It is as though librsync is not being called/used.
>
> Below is a copy of my last two duplicity commands - note the
> ChangeFileSize is 51Mb each time.
>
> Any suggestions or explanations much appreciated!
>
> Cheers
> Noel
>
>
>
> n@medulla:/backups/20091003/$ sudo duplicity --scp-command "scp -l
> 1024" ./com.sql.bz2  scp://[hidden email]/com/
> Password:
> GnuPG passphrase:
> No signatures found, switching to full backup.
> tmpLUSD-P
> ...
> 100%   24KB  24.1KB/s   00:00   --------------[ Backup Statistics
> ]--------------
> StartTime 1254935682.83 (Wed Oct  7 18:14:42 2009)
> EndTime 1254936124.95 (Wed Oct  7 18:22:04 2009)
> ElapsedTime 442.13 (7 minutes 22.13 seconds)
> SourceFiles 1
> SourceFileSize 52987804 (50.5 MB)
> NewFiles 1
> NewFileSize 52987804 (50.5 MB)
> DeletedFiles 0
> ChangedFiles 0
> ChangedFileSize 0 (0 bytes)
> ChangedDeltaSize 0 (0 bytes)
> DeltaEntries 1
> RawDeltaSize 5204480 (4.96 MB)
> TotalDestinationSizeChange 53615229 (51.1 MB)
> Errors 0
> -------------------------------------------------
>
> n@medulla:/backups/20091006/$ sudo duplicity --scp-command "scp -l
> 1024" ./com.sql.bz2  scp://[hidden email]/com/
> Password:
> GnuPG passphrase:
> duplicity-full.2009-10-07T18:14:34+01:00.manifest.gpg                              
> 100%  464     0.5KB/s   00:00  
> duplicity-full-signatures.2009-10-07T18:14:34+01:00.sigtar.gpg                    
> 100%   24KB  24.1KB/s   00:01   tmpxMoIR1
> ...
>  100%   24KB  24.0KB/s   00:01   --------------[ Backup Statistics
> ]--------------
> StartTime 1254937169.04 (Wed Oct  7 18:39:29 2009)
> EndTime 1254937617.68 (Wed Oct  7 18:46:57 2009)
> ElapsedTime 448.64 (7 minutes 28.64 seconds)
> SourceFiles 1
> SourceFileSize 52903512 (50.5 MB)
> NewFiles 0
> NewFileSize 0 (0 bytes)
> DeletedFiles 0
> ChangedFiles 1
> ChangedFileSize 52903512 (50.5 MB)
> ChangedDeltaSize 0 (0 bytes)
> DeltaEntries 1
> RawDeltaSize 5204480 (4.96 MB)
> TotalDestinationSizeChange 53540930 (51.1 MB)
> Errors 0
> -------------------------------------------------
>
>
>
> _______________________________________________
> Duplicity-talk mailing list
> [hidden email]
> http://lists.nongnu.org/mailman/listinfo/duplicity-talk



_______________________________________________
Duplicity-talk mailing list
[hidden email]
http://lists.nongnu.org/mailman/listinfo/duplicity-talk