Re: When modifying a file under the backup dir, duplicity still thinks that there are no differences with the backup (stored on GDrive)

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

Re: When modifying a file under the backup dir, duplicity still thinks that there are no differences with the backup (stored on GDrive)

edgar.soldin
On 22.04.2015 03:48, Kostas Papadopoulos wrote:> Hi,

>
> After ~1yr of hiatus from duplicity, I've tried doing some quick testing of the latest duply 1.9.1 /duplicity 0.7.02.
>
> But *when I add or modify a file under the backup dir, duplicity still thinks that there are no differences* with the backup (stored on GDrive). although I hadn't run a backup for several weeks.
>
>     root@backup:~/src/duplicity-0.7.02# ~/bin/duply mytest verify
>     Start duply v1.9.1, time is 2015-04-22 01:32:26.
>     Using profile '/root/.duply/mytest'.
>     Using installed duplicity version 0.7.02, python 2.7.3, gpg 1.4.12 (Home: ~/.gnupg), awk 'mawk 1.3.3 Nov 1996, Copyright (C) Michael D. Brennan', bash '4.2.37(1)-release (x86_64-pc-linux-gnu)'.
>     Autoset found secret key of first GPG_KEY entry '0841EA59' for signing.
>     Checking TEMP_DIR '/tmp' is a folder (OK)
>     Checking TEMP_DIR '/tmp' is writable (OK)
>     TODO: reimplent tmp space check
>     Test - Encrypt to '0841EA59' & Sign with '0841EA59' (OK)
>     Test - Decrypt (OK)
>     Test - Compare (OK)
>     Cleanup - Delete '/tmp/duply.2138.1429666346_*'(OK)
>
>     --- Start running command VERIFY at 01:32:27.075 ---
>     Reading globbing filelist /root/.duply/mytest/exclude
>     Local and Remote metadata are synchronized, no sync needed.
>     Last full backup date: Sun Apr  5 15:55:20 2015
>     Verify complete: 73 files compared, 0 differences found.
>     --- Finished state OK at 01:33:47.897 - Runtime 00:01:20.821 ---
>     root@backup:~/src/duplicity-0.7.02#
>
>     root@backup:~/src/duplicity-0.7.02# grep -v ^# ~/.duply/mytest/conf|grep -v ^$
>     GPG_KEY='REMOVED'
>     GPG_PW='REMOVED'
>     TARGET='gdocs://[hidden email]/duplicity-backup-test'
>     TARGET_PASS='REMOVED'
>     SOURCE='/root/docs'
>     MAX_AGE=7D
>     MAX_FULL_BACKUPS=3
>     MAX_FULLS_WITH_INCRS=2
>     MAX_FULLBKP_AGE=2D
>     DUPL_PARAMS="$DUPL_PARAMS --full-if-older-than $MAX_FULLBKP_AGE "
>     root@backup:~/src/duplicity-0.7.02#
>
>     root@backup:~/src/duplicity-0.7.02# cat /root/.duply/mytest/exclude
>     /root/docs/dir4
>     /root/docs/dir6
>     root@backup:~/src/duplicity-0.7.02#
>
>     root@backup:~/src/duplicity-0.7.02# ls -la /root/docs/
>     total 48
>     drwxr-xr-x 12 root root 4096 Apr 22 00:58 .
>     drwx------  9 root root 4096 Apr 22 01:00 ..
>     drwxr-xr-x  2 root root 4096 Nov 16  2013 dir2
>     drwxr-xr-x  2 root root 4096 Nov 17  2013 dir3
>     drwxr-xr-x  2 root root 4096 Nov 18  2013 dir4
>     drwxr-xr-x  2 root root 4096 Apr 22 01:00 dir5
>     drwxr-xr-x  2 root root 4096 Nov 18  2013 dir6
>     drwxr-xr-x  2 root root 4096 Apr 22 00:58 dir7
>     drwxr-xr-x  2 root root 4096 Apr  5 16:59 DOKIMH-AGGLIKA-directory
>     drwxr-xr-x  2 root root 4096 Nov 17  2013 thin-client
>     drwxr-xr-x  2 root root 4096 Apr  5 16:32 ΔΟΚΙΜΗ-ΕΛΛΗΝΙΚΟ-directory
>     root@backup:~/src/duplicity-0.7.02#
>
>

please post the output of something along the lines

1. duplicity backup
2. touch a file
3. duplicity incr backup (here the summary log should show one changed file)

note that 'verify' is not a backup but a temp restore with comparision to the original local file.

also: use the duplicity-talk mailing list, that's what it's there for. we don't do one on one support.

..ede

_______________________________________________
Duplicity-talk mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/duplicity-talk
Reply | Threaded
Open this post in threaded view
|

Re: When modifying a file under the backup dir, duplicity still thinks that there are no differences with the backup (stored on GDrive)

Kenneth Loafman
Actually, he has to use --compare-data for verify to look at the local data, otherwise verify just validates that the backup is intact and is not corrupted.

On Tue, Apr 21, 2015 at 10:14 PM, <[hidden email]> wrote:
On <a href="tel:22.04.2015%2003" value="+12204201503">22.04.2015 03:48, Kostas Papadopoulos wrote:> Hi,
>
> After ~1yr of hiatus from duplicity, I've tried doing some quick testing of the latest duply 1.9.1 /duplicity 0.7.02.
>
> But *when I add or modify a file under the backup dir, duplicity still thinks that there are no differences* with the backup (stored on GDrive). although I hadn't run a backup for several weeks.
>
>     root@backup:~/src/duplicity-0.7.02# ~/bin/duply mytest verify
>     Start duply v1.9.1, time is 2015-04-22 01:32:26.
>     Using profile '/root/.duply/mytest'.
>     Using installed duplicity version 0.7.02, python 2.7.3, gpg 1.4.12 (Home: ~/.gnupg), awk 'mawk 1.3.3 Nov 1996, Copyright (C) Michael D. Brennan', bash '4.2.37(1)-release (x86_64-pc-linux-gnu)'.
>     Autoset found secret key of first GPG_KEY entry '0841EA59' for signing.
>     Checking TEMP_DIR '/tmp' is a folder (OK)
>     Checking TEMP_DIR '/tmp' is writable (OK)
>     TODO: reimplent tmp space check
>     Test - Encrypt to '0841EA59' & Sign with '0841EA59' (OK)
>     Test - Decrypt (OK)
>     Test - Compare (OK)
>     Cleanup - Delete '/tmp/duply.2138.1429666346_*'(OK)
>
>     --- Start running command VERIFY at 01:32:27.075 ---
>     Reading globbing filelist /root/.duply/mytest/exclude
>     Local and Remote metadata are synchronized, no sync needed.
>     Last full backup date: Sun Apr  5 15:55:20 2015
>     Verify complete: 73 files compared, 0 differences found.
>     --- Finished state OK at 01:33:47.897 - Runtime 00:01:20.821 ---
>     root@backup:~/src/duplicity-0.7.02#
>
>     root@backup:~/src/duplicity-0.7.02# grep -v ^# ~/.duply/mytest/conf|grep -v ^$
>     GPG_KEY='REMOVED'
>     GPG_PW='REMOVED'
>     TARGET='gdocs://mytest@.../duplicity-backup-test'
>     TARGET_PASS='REMOVED'
>     SOURCE='/root/docs'
>     MAX_AGE=7D
>     MAX_FULL_BACKUPS=3
>     MAX_FULLS_WITH_INCRS=2
>     MAX_FULLBKP_AGE=2D
>     DUPL_PARAMS="$DUPL_PARAMS --full-if-older-than $MAX_FULLBKP_AGE "
>     root@backup:~/src/duplicity-0.7.02#
>
>     root@backup:~/src/duplicity-0.7.02# cat /root/.duply/mytest/exclude
>     /root/docs/dir4
>     /root/docs/dir6
>     root@backup:~/src/duplicity-0.7.02#
>
>     root@backup:~/src/duplicity-0.7.02# ls -la /root/docs/
>     total 48
>     drwxr-xr-x 12 root root 4096 Apr 22 00:58 .
>     drwx------  9 root root 4096 Apr 22 01:00 ..
>     drwxr-xr-x  2 root root 4096 Nov 16  2013 dir2
>     drwxr-xr-x  2 root root 4096 Nov 17  2013 dir3
>     drwxr-xr-x  2 root root 4096 Nov 18  2013 dir4
>     drwxr-xr-x  2 root root 4096 Apr 22 01:00 dir5
>     drwxr-xr-x  2 root root 4096 Nov 18  2013 dir6
>     drwxr-xr-x  2 root root 4096 Apr 22 00:58 dir7
>     drwxr-xr-x  2 root root 4096 Apr  5 16:59 DOKIMH-AGGLIKA-directory
>     drwxr-xr-x  2 root root 4096 Nov 17  2013 thin-client
>     drwxr-xr-x  2 root root 4096 Apr  5 16:32 ΔΟΚΙΜΗ-ΕΛΛΗΝΙΚΟ-directory
>     root@backup:~/src/duplicity-0.7.02#
>
>

please post the output of something along the lines

1. duplicity backup
2. touch a file
3. duplicity incr backup (here the summary log should show one changed file)

note that 'verify' is not a backup but a temp restore with comparision to the original local file.

also: use the duplicity-talk mailing list, that's what it's there for. we don't do one on one support.

..ede


_______________________________________________
Duplicity-talk mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/duplicity-talk
Reply | Threaded
Open this post in threaded view
|

Re: When modifying a file under the backup dir, duplicity still thinks that there are no differences with the backup (stored on GDrive)

edgar.soldin
nope, am pretty sure that verify compares timestamps and existence of the file in the source path. see it regularly in my backup logs.

btw. something that annoys me since the beginning as i would expect a verify to only check consistence but an actual compare command to do the comparing with the source path ;)

..ede
 

On 22.04.2015 20:51, Kenneth Loafman wrote:

> Actually, he has to use --compare-data for verify to look at the local data, otherwise verify just validates that the backup is intact and is not corrupted.
>
> On Tue, Apr 21, 2015 at 10:14 PM, <[hidden email] <mailto:[hidden email]>> wrote:
>
>     On 22.04.2015 03 <tel:22.04.2015%2003>:48, Kostas Papadopoulos wrote:> Hi,
>     >
>     > After ~1yr of hiatus from duplicity, I've tried doing some quick testing of the latest duply 1.9.1 /duplicity 0.7.02.
>     >
>     > But *when I add or modify a file under the backup dir, duplicity still thinks that there are no differences* with the backup (stored on GDrive). although I hadn't run a backup for several weeks.
>     >
>     >     root@backup:~/src/duplicity-0.7.02# ~/bin/duply mytest verify
>     >     Start duply v1.9.1, time is 2015-04-22 01:32:26.
>     >     Using profile '/root/.duply/mytest'.
>     >     Using installed duplicity version 0.7.02, python 2.7.3, gpg 1.4.12 (Home: ~/.gnupg), awk 'mawk 1.3.3 Nov 1996, Copyright (C) Michael D. Brennan', bash '4.2.37(1)-release (x86_64-pc-linux-gnu)'.
>     >     Autoset found secret key of first GPG_KEY entry '0841EA59' for signing.
>     >     Checking TEMP_DIR '/tmp' is a folder (OK)
>     >     Checking TEMP_DIR '/tmp' is writable (OK)
>     >     TODO: reimplent tmp space check
>     >     Test - Encrypt to '0841EA59' & Sign with '0841EA59' (OK)
>     >     Test - Decrypt (OK)
>     >     Test - Compare (OK)
>     >     Cleanup - Delete '/tmp/duply.2138.1429666346_*'(OK)
>     >
>     >     --- Start running command VERIFY at 01:32:27.075 ---
>     >     Reading globbing filelist /root/.duply/mytest/exclude
>     >     Local and Remote metadata are synchronized, no sync needed.
>     >     Last full backup date: Sun Apr  5 15:55:20 2015
>     >     Verify complete: 73 files compared, 0 differences found.
>     >     --- Finished state OK at 01:33:47.897 - Runtime 00:01:20.821 ---
>     >     root@backup:~/src/duplicity-0.7.02#
>     >
>     >     root@backup:~/src/duplicity-0.7.02# grep -v ^# ~/.duply/mytest/conf|grep -v ^$
>     >     GPG_KEY='REMOVED'
>     >     GPG_PW='REMOVED'
>     >     TARGET='gdocs://[hidden email]/duplicity-backup-test <http://mytest@.../duplicity-backup-test>'
>     >     TARGET_PASS='REMOVED'
>     >     SOURCE='/root/docs'
>     >     MAX_AGE=7D
>     >     MAX_FULL_BACKUPS=3
>     >     MAX_FULLS_WITH_INCRS=2
>     >     MAX_FULLBKP_AGE=2D
>     >     DUPL_PARAMS="$DUPL_PARAMS --full-if-older-than $MAX_FULLBKP_AGE "
>     >     root@backup:~/src/duplicity-0.7.02#
>     >
>     >     root@backup:~/src/duplicity-0.7.02# cat /root/.duply/mytest/exclude
>     >     /root/docs/dir4
>     >     /root/docs/dir6
>     >     root@backup:~/src/duplicity-0.7.02#
>     >
>     >     root@backup:~/src/duplicity-0.7.02# ls -la /root/docs/
>     >     total 48
>     >     drwxr-xr-x 12 root root 4096 Apr 22 00:58 .
>     >     drwx------  9 root root 4096 Apr 22 01:00 ..
>     >     drwxr-xr-x  2 root root 4096 Nov 16  2013 dir2
>     >     drwxr-xr-x  2 root root 4096 Nov 17  2013 dir3
>     >     drwxr-xr-x  2 root root 4096 Nov 18  2013 dir4
>     >     drwxr-xr-x  2 root root 4096 Apr 22 01:00 dir5
>     >     drwxr-xr-x  2 root root 4096 Nov 18  2013 dir6
>     >     drwxr-xr-x  2 root root 4096 Apr 22 00:58 dir7
>     >     drwxr-xr-x  2 root root 4096 Apr  5 16:59 DOKIMH-AGGLIKA-directory
>     >     drwxr-xr-x  2 root root 4096 Nov 17  2013 thin-client
>     >     drwxr-xr-x  2 root root 4096 Apr  5 16:32 ΔΟΚΙΜΗ-ΕΛΛΗΝΙΚΟ-directory
>     >     root@backup:~/src/duplicity-0.7.02#
>     >
>     >
>
>     please post the output of something along the lines
>
>     1. duplicity backup
>     2. touch a file
>     3. duplicity incr backup (here the summary log should show one changed file)
>
>     note that 'verify' is not a backup but a temp restore with comparision to the original local file.
>
>     also: use the duplicity-talk mailing list, that's what it's there for. we don't do one on one support.
>
>     ..ede
>
>

_______________________________________________
Duplicity-talk mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/duplicity-talk
Reply | Threaded
Open this post in threaded view
|

Re: When modifying a file under the backup dir, duplicity still thinks that there are no differences with the backup (stored on GDrive)

Kenneth Loafman
OK, I confused the addition of tests in 0.7.01 with the actual changes.  Those are still in progress.


On Wed, Apr 22, 2015 at 3:23 PM, <[hidden email]> wrote:
nope, am pretty sure that verify compares timestamps and existence of the file in the source path. see it regularly in my backup logs.

btw. something that annoys me since the beginning as i would expect a verify to only check consistence but an actual compare command to do the comparing with the source path ;)

..ede


On <a href="tel:22.04.2015%2020" value="+12204201520">22.04.2015 20:51, Kenneth Loafman wrote:
> Actually, he has to use --compare-data for verify to look at the local data, otherwise verify just validates that the backup is intact and is not corrupted.
>
> On Tue, Apr 21, 2015 at 10:14 PM, <[hidden email] <mailto:[hidden email]>> wrote:
>
>     On <a href="tel:22.04.2015%2003" value="+12204201503">22.04.2015 03 <tel:22.04.2015%2003>:48, Kostas Papadopoulos wrote:> Hi,
>     >
>     > After ~1yr of hiatus from duplicity, I've tried doing some quick testing of the latest duply 1.9.1 /duplicity 0.7.02.
>     >
>     > But *when I add or modify a file under the backup dir, duplicity still thinks that there are no differences* with the backup (stored on GDrive). although I hadn't run a backup for several weeks.
>     >
>     >     root@backup:~/src/duplicity-0.7.02# ~/bin/duply mytest verify
>     >     Start duply v1.9.1, time is 2015-04-22 01:32:26.
>     >     Using profile '/root/.duply/mytest'.
>     >     Using installed duplicity version 0.7.02, python 2.7.3, gpg 1.4.12 (Home: ~/.gnupg), awk 'mawk 1.3.3 Nov 1996, Copyright (C) Michael D. Brennan', bash '4.2.37(1)-release (x86_64-pc-linux-gnu)'.
>     >     Autoset found secret key of first GPG_KEY entry '0841EA59' for signing.
>     >     Checking TEMP_DIR '/tmp' is a folder (OK)
>     >     Checking TEMP_DIR '/tmp' is writable (OK)
>     >     TODO: reimplent tmp space check
>     >     Test - Encrypt to '0841EA59' & Sign with '0841EA59' (OK)
>     >     Test - Decrypt (OK)
>     >     Test - Compare (OK)
>     >     Cleanup - Delete '/tmp/duply.2138.1429666346_*'(OK)
>     >
>     >     --- Start running command VERIFY at 01:32:27.075 ---
>     >     Reading globbing filelist /root/.duply/mytest/exclude
>     >     Local and Remote metadata are synchronized, no sync needed.
>     >     Last full backup date: Sun Apr  5 15:55:20 2015
>     >     Verify complete: 73 files compared, 0 differences found.
>     >     --- Finished state OK at 01:33:47.897 - Runtime 00:01:20.821 ---
>     >     root@backup:~/src/duplicity-0.7.02#
>     >
>     >     root@backup:~/src/duplicity-0.7.02# grep -v ^# ~/.duply/mytest/conf|grep -v ^$
>     >     GPG_KEY='REMOVED'
>     >     GPG_PW='REMOVED'
>     >     TARGET='gdocs://mytest@.../duplicity-backup-test <http://mytest@.../duplicity-backup-test>'
>     >     TARGET_PASS='REMOVED'
>     >     SOURCE='/root/docs'
>     >     MAX_AGE=7D
>     >     MAX_FULL_BACKUPS=3
>     >     MAX_FULLS_WITH_INCRS=2
>     >     MAX_FULLBKP_AGE=2D
>     >     DUPL_PARAMS="$DUPL_PARAMS --full-if-older-than $MAX_FULLBKP_AGE "
>     >     root@backup:~/src/duplicity-0.7.02#
>     >
>     >     root@backup:~/src/duplicity-0.7.02# cat /root/.duply/mytest/exclude
>     >     /root/docs/dir4
>     >     /root/docs/dir6
>     >     root@backup:~/src/duplicity-0.7.02#
>     >
>     >     root@backup:~/src/duplicity-0.7.02# ls -la /root/docs/
>     >     total 48
>     >     drwxr-xr-x 12 root root 4096 Apr 22 00:58 .
>     >     drwx------  9 root root 4096 Apr 22 01:00 ..
>     >     drwxr-xr-x  2 root root 4096 Nov 16  2013 dir2
>     >     drwxr-xr-x  2 root root 4096 Nov 17  2013 dir3
>     >     drwxr-xr-x  2 root root 4096 Nov 18  2013 dir4
>     >     drwxr-xr-x  2 root root 4096 Apr 22 01:00 dir5
>     >     drwxr-xr-x  2 root root 4096 Nov 18  2013 dir6
>     >     drwxr-xr-x  2 root root 4096 Apr 22 00:58 dir7
>     >     drwxr-xr-x  2 root root 4096 Apr  5 16:59 DOKIMH-AGGLIKA-directory
>     >     drwxr-xr-x  2 root root 4096 Nov 17  2013 thin-client
>     >     drwxr-xr-x  2 root root 4096 Apr  5 16:32 ΔΟΚΙΜΗ-ΕΛΛΗΝΙΚΟ-directory
>     >     root@backup:~/src/duplicity-0.7.02#
>     >
>     >
>
>     please post the output of something along the lines
>
>     1. duplicity backup
>     2. touch a file
>     3. duplicity incr backup (here the summary log should show one changed file)
>
>     note that 'verify' is not a backup but a temp restore with comparision to the original local file.
>
>     also: use the duplicity-talk mailing list, that's what it's there for. we don't do one on one support.
>
>     ..ede
>
>


_______________________________________________
Duplicity-talk mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/duplicity-talk
Reply | Threaded
Open this post in threaded view
|

Re: When modifying a file under the backup dir, duplicity still thinks that there are no differences with the backup (stored on GDrive)

Kenneth Loafman
It turns out it was fixed under 0.7.01 after all.  See https://bugs.launchpad.net/bugs/1354880

Thanks Aaron for pointing that out.  I need to reflect that in the changelog.

On Wed, Apr 22, 2015 at 3:31 PM, Kenneth Loafman <[hidden email]> wrote:
OK, I confused the addition of tests in 0.7.01 with the actual changes.  Those are still in progress.


On Wed, Apr 22, 2015 at 3:23 PM, <[hidden email]> wrote:
nope, am pretty sure that verify compares timestamps and existence of the file in the source path. see it regularly in my backup logs.

btw. something that annoys me since the beginning as i would expect a verify to only check consistence but an actual compare command to do the comparing with the source path ;)

..ede


On <a href="tel:22.04.2015%2020" value="+12204201520" target="_blank">22.04.2015 20:51, Kenneth Loafman wrote:
> Actually, he has to use --compare-data for verify to look at the local data, otherwise verify just validates that the backup is intact and is not corrupted.
>
> On Tue, Apr 21, 2015 at 10:14 PM, <[hidden email] <mailto:[hidden email]>> wrote:
>
>     On <a href="tel:22.04.2015%2003" value="+12204201503" target="_blank">22.04.2015 03 <tel:22.04.2015%2003>:48, Kostas Papadopoulos wrote:> Hi,
>     >
>     > After ~1yr of hiatus from duplicity, I've tried doing some quick testing of the latest duply 1.9.1 /duplicity 0.7.02.
>     >
>     > But *when I add or modify a file under the backup dir, duplicity still thinks that there are no differences* with the backup (stored on GDrive). although I hadn't run a backup for several weeks.
>     >
>     >     root@backup:~/src/duplicity-0.7.02# ~/bin/duply mytest verify
>     >     Start duply v1.9.1, time is 2015-04-22 01:32:26.
>     >     Using profile '/root/.duply/mytest'.
>     >     Using installed duplicity version 0.7.02, python 2.7.3, gpg 1.4.12 (Home: ~/.gnupg), awk 'mawk 1.3.3 Nov 1996, Copyright (C) Michael D. Brennan', bash '4.2.37(1)-release (x86_64-pc-linux-gnu)'.
>     >     Autoset found secret key of first GPG_KEY entry '0841EA59' for signing.
>     >     Checking TEMP_DIR '/tmp' is a folder (OK)
>     >     Checking TEMP_DIR '/tmp' is writable (OK)
>     >     TODO: reimplent tmp space check
>     >     Test - Encrypt to '0841EA59' & Sign with '0841EA59' (OK)
>     >     Test - Decrypt (OK)
>     >     Test - Compare (OK)
>     >     Cleanup - Delete '/tmp/duply.2138.1429666346_*'(OK)
>     >
>     >     --- Start running command VERIFY at 01:32:27.075 ---
>     >     Reading globbing filelist /root/.duply/mytest/exclude
>     >     Local and Remote metadata are synchronized, no sync needed.
>     >     Last full backup date: Sun Apr  5 15:55:20 2015
>     >     Verify complete: 73 files compared, 0 differences found.
>     >     --- Finished state OK at 01:33:47.897 - Runtime 00:01:20.821 ---
>     >     root@backup:~/src/duplicity-0.7.02#
>     >
>     >     root@backup:~/src/duplicity-0.7.02# grep -v ^# ~/.duply/mytest/conf|grep -v ^$
>     >     GPG_KEY='REMOVED'
>     >     GPG_PW='REMOVED'
>     >     TARGET='gdocs://mytest@.../duplicity-backup-test <http://mytest@.../duplicity-backup-test>'
>     >     TARGET_PASS='REMOVED'
>     >     SOURCE='/root/docs'
>     >     MAX_AGE=7D
>     >     MAX_FULL_BACKUPS=3
>     >     MAX_FULLS_WITH_INCRS=2
>     >     MAX_FULLBKP_AGE=2D
>     >     DUPL_PARAMS="$DUPL_PARAMS --full-if-older-than $MAX_FULLBKP_AGE "
>     >     root@backup:~/src/duplicity-0.7.02#
>     >
>     >     root@backup:~/src/duplicity-0.7.02# cat /root/.duply/mytest/exclude
>     >     /root/docs/dir4
>     >     /root/docs/dir6
>     >     root@backup:~/src/duplicity-0.7.02#
>     >
>     >     root@backup:~/src/duplicity-0.7.02# ls -la /root/docs/
>     >     total 48
>     >     drwxr-xr-x 12 root root 4096 Apr 22 00:58 .
>     >     drwx------  9 root root 4096 Apr 22 01:00 ..
>     >     drwxr-xr-x  2 root root 4096 Nov 16  2013 dir2
>     >     drwxr-xr-x  2 root root 4096 Nov 17  2013 dir3
>     >     drwxr-xr-x  2 root root 4096 Nov 18  2013 dir4
>     >     drwxr-xr-x  2 root root 4096 Apr 22 01:00 dir5
>     >     drwxr-xr-x  2 root root 4096 Nov 18  2013 dir6
>     >     drwxr-xr-x  2 root root 4096 Apr 22 00:58 dir7
>     >     drwxr-xr-x  2 root root 4096 Apr  5 16:59 DOKIMH-AGGLIKA-directory
>     >     drwxr-xr-x  2 root root 4096 Nov 17  2013 thin-client
>     >     drwxr-xr-x  2 root root 4096 Apr  5 16:32 ΔΟΚΙΜΗ-ΕΛΛΗΝΙΚΟ-directory
>     >     root@backup:~/src/duplicity-0.7.02#
>     >
>     >
>
>     please post the output of something along the lines
>
>     1. duplicity backup
>     2. touch a file
>     3. duplicity incr backup (here the summary log should show one changed file)
>
>     note that 'verify' is not a backup but a temp restore with comparision to the original local file.
>
>     also: use the duplicity-talk mailing list, that's what it's there for. we don't do one on one support.
>
>     ..ede
>
>



_______________________________________________
Duplicity-talk mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/duplicity-talk
Reply | Threaded
Open this post in threaded view
|

Re: When modifying a file under the backup dir, duplicity still thinks that there are no differences with the backup (stored on GDrive)

Kenneth Loafman
In reply to this post by Kenneth Loafman
The signatures in the sigtar files.  Think of them as uber-large hashes.

On Wed, Apr 22, 2015 at 5:25 PM, Kostas Papadopoulos <[hidden email]> wrote:
On 4/22/2015 2:51 PM, Kenneth Loafman wrote:
Actually, he has to use --compare-data for verify to look at the local data, otherwise verify just validates that the backup is intact and is not corrupted.


Thanks for the clarification, but then isn't the output of "verify" a bit misleading?
...
Last full backup date: Sun Apr  5 15:55:20 2015

Verify complete: 73 files compared, 0 differences found.
--- Finished state OK at 01:33:47.897 - Runtime 00:01:20.821 ---
Against what are those 73 files compared?

Br, KP



_______________________________________________
Duplicity-talk mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/duplicity-talk
Reply | Threaded
Open this post in threaded view
|

Re: Syntax of duplicity verify command

duplicity-talk mailing list
hey Kostas (again:)),

yeah duplicity's verify is actually a bastard command that verifies the possibility to restore _and_ allows to compare to the source folder. older versions did the second automatically, for newer versions you have to enable --compare-data for that.

the source folder argument is there and needed for the second part, that is why it is still needed.

the functionality should probably be separated into a new compare command in the future, but so far nobody did so.

..ede/duply.net

On 23.01.2017 21:16, Kostas Papadopoulos wrote:

> Dear duplicity devs & users,
>
> If the duplicity "verify" command just validates the integrity of the backup against the signatures of the sigtar files found at source_url, isn't then the requirement of a second argument (target_directory) unnecessary and confusing?
>
> Currently, if the specified target_dir doesn't exist, then duplicity "verify" exits with an error. If the specified target_dir is empty, it works fine. And if the specified target_dir has other random files (unrelated to duplicity), the number of "files compared" is somehow increased (e.g. from 75 to 140 in the example below)
>
> /root/docs was the backup original source dir
> /tmp/t is non-existant dir
> /tmp/tt is empty dir
> /tmp contains other stuff too
>
>     root@backup:/tmp# duplicity verify --name duply_test-local --encrypt-key AABBCCDDEEFFAA --sign-key AABBCCDDEEFFAA --verbosity 4 file:///tmp/backup-test-local /root/docs
>     Local and Remote metadata are synchronized, no sync needed.
>     Last full backup date: Mon Jan 23 19:00:45 2017
>     GnuPG passphrase:
>     Verify complete: 77 files compared, 0 differences found.
>     root@backup:/tmp# duplicity verify --name duply_test-local --encrypt-key AABBCCDDEEFFAA --sign-key AABBCCDDEEFFAA --verbosity 4 file:///tmp/backup-test-local /tmp/t
>     Verify directory /tmp/t does not exist
>     root@backup:/tmp# duplicity verify --name duply_test-local --encrypt-key AABBCCDDEEFFAA --sign-key AABBCCDDEEFFAA --verbosity 4 file:///tmp/backup-test-local /tmp/tt
>     Local and Remote metadata are synchronized, no sync needed.
>     Last full backup date: Mon Jan 23 19:00:45 2017
>     GnuPG passphrase:
>     Verify complete: 75 files compared, 0 differences found.
>     root@backup:/tmp# duplicity verify --name duply_test-local --encrypt-key AABBCCDDEEFFAA --sign-key AABBCCDDEEFFAA --verbosity 4 file:///tmp/backup-test-local /tmp/
>     Local and Remote metadata are synchronized, no sync needed.
>     Last full backup date: Mon Jan 23 19:00:45 2017
>     GnuPG passphrase:
>     Verify complete: 140 files compared, 0 differences found.
>
> I would imagine that a more readily understandable syntax would be a duplicity "verify" command with just one argument (duplicity verify ... source_url), and another duplicity "compare" command that would take two arguments (duplicity compare ... source_url target_directory [--compare-data]) and compare backup against the target_directory's files size/date and optionally content (when used with --compare-data).
>
> Thank you in advance for your consideration,
> KP
>
> On 04/23/2015 03:12 AM, Kenneth Loafman wrote:
>> The signatures in the sigtar files.  Think of them as uber-large hashes.
>>
>> On Wed, Apr 22, 2015 at 5:25 PM, Kostas Papadopoulos <[hidden email] <mailto:[hidden email]>> wrote:
>>
>>     On 4/22/2015 2:51 PM, Kenneth Loafman wrote:
>>>     Actually, he has to use --compare-data for verify to look at the local data, otherwise verify just validates that the backup is intact and is not corrupted.
>>>
>>
>>     Thanks for the clarification, but then isn't the output of "verify" a bit misleading?
>>
>>         ...
>>         Last full backup date: Sun Apr  5 15:55:20 2015
>>         Verify complete: 73 files compared, 0 differences found.
>>         --- Finished state OK at 01:33:47.897 - Runtime 00:01:20.821 ---
>>
>>     Against what are those 73 files compared?
>>
>>     Br, KP
>>
>>
>

_______________________________________________
Duplicity-talk mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/duplicity-talk
Reply | Threaded
Open this post in threaded view
|

Re: duplicity use of MD4/BLAKE2 and splitting of "verify" command

duplicity-talk mailing list
In reply to this post by Kenneth Loafman
On 18.02.2017 23:19, Kostas Papadopoulos wrote:
> Hi duplicity devs,
>
> May I ask if it'd be possible for duplicity v0.8.x to support librsync's new (well not so new, they switched ~2 years ago) BLAKE2 format, instead of the old (and broken according to some) MD4-based checksumming ? Please see "SECURITY: MD4 collision/preimage attacks (CVE-2014-8242)" [1] [2]

agreed, raising the version number would be a good opportunity to make a clean cut.

> And while I'm asking, I'd like to request that duplicity would split current "verify" into two commands: "verify" and "compare" (note: the current syntax causes confusion and I've found this specific request repeated in many questions/posts in the duplicity-talk list since 2010). Please see output of sample test runs attached below.

this is a long standing issue. so agreed again, but that would need someone volunteering to implement it. maybe you'd like to offer a bounty or start a coding bounty campaign for this on bountysource or similar.

..ede/duply.net

> Thank you in advance for your consideration, KP
>
>
> [1] https://github.com/librsync/librsync/issues/5
>
> [2] https://bugs.launchpad.net/duplicity/+bug/1342721
>
> On 01/23/2017 10:16 PM, Kostas Papadopoulos wrote:
>> Dear duplicity devs & users,
>>
>> If the duplicity "verify" command just validates the integrity of the backup against the signatures of the sigtar files found at source_url, isn't then the requirement of a second argument (target_directory) unnecessary and confusing?
>>
>> Currently, if the specified target_dir doesn't exist, then duplicity "verify" exits with an error. If the specified target_dir is empty, it works fine. And if the specified target_dir has other random files (unrelated to duplicity), the number of "files compared" is somehow increased (e.g. from 75 to 140 in the example below)
>>
>> /root/docs was the backup original source dir
>> /tmp/t is non-existant dir
>> /tmp/tt is empty dir
>> /tmp contains other stuff too
>>
>>     root@backup:/tmp# duplicity verify --name duply_test-local
>>     --encrypt-key AABBCCDDEEFFAA --sign-key AABBCCDDEEFFAA --verbosity
>>     4 file:///tmp/backup-test-local /root/docs
>>     Local and Remote metadata are synchronized, no sync needed.
>>     Last full backup date: Mon Jan 23 19:00:45 2017
>>     GnuPG passphrase:
>>     Verify complete: 77 files compared, 0 differences found.
>>     root@backup:/tmp# duplicity verify --name duply_test-local
>>     --encrypt-key AABBCCDDEEFFAA --sign-key AABBCCDDEEFFAA --verbosity
>>     4 file:///tmp/backup-test-local /tmp/t
>>     Verify directory /tmp/t does not exist
>>     root@backup:/tmp# duplicity verify --name duply_test-local
>>     --encrypt-key AABBCCDDEEFFAA --sign-key AABBCCDDEEFFAA --verbosity
>>     4 file:///tmp/backup-test-local /tmp/tt
>>     Local and Remote metadata are synchronized, no sync needed.
>>     Last full backup date: Mon Jan 23 19:00:45 2017
>>     GnuPG passphrase:
>>     Verify complete: 75 files compared, 0 differences found.
>>     root@backup:/tmp# duplicity verify --name duply_test-local
>>     --encrypt-key AABBCCDDEEFFAA --sign-key AABBCCDDEEFFAA --verbosity
>>     4 file:///tmp/backup-test-local /tmp/
>>     Local and Remote metadata are synchronized, no sync needed.
>>     Last full backup date: Mon Jan 23 19:00:45 2017
>>     GnuPG passphrase:
>>     Verify complete: 140 files compared, 0 differences found.
>>
>> I would imagine that a more readily understandable syntax would be a duplicity "verify" command with just one argument (duplicity verify ... source_url), and another duplicity "compare" command that would take two arguments (duplicity compare ... source_url target_directory [--compare-data]) and compare backup against the target_directory's files size/date and optionally content (when used with --compare-data).
>>
>> Thank you in advance for your consideration,
>> KP
>>
>> On 04/23/2015 03:12 AM, Kenneth Loafman wrote:
>>> The signatures in the sigtar files.  Think of them as uber-large hashes.
>>>
>>> On Wed, Apr 22, 2015 at 5:25 PM, Kostas Papadopoulos <[hidden email] <mailto:[hidden email]>> wrote:
>>>
>>>     On 4/22/2015 2:51 PM, Kenneth Loafman wrote:
>>>>     Actually, he has to use --compare-data for verify to look at the
>>>>     local data, otherwise verify just validates that the backup is
>>>>     intact and is not corrupted.
>>>>
>>>
>>>     Thanks for the clarification, but then isn't the output of
>>>     "verify" a bit misleading?
>>>
>>>         ...
>>>         Last full backup date: Sun Apr  5 15:55:20 2015
>>>         Verify complete: 73 files compared, 0 differences found.
>>>         --- Finished state OK at 01:33:47.897 - Runtime 00:01:20.821 ---
>>>
>>>     Against what are those 73 files compared?
>>>
>>>     Br, KP
>>>
>>>
>>
>
>

_______________________________________________
Duplicity-talk mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/duplicity-talk
Reply | Threaded
Open this post in threaded view
|

Re: duplicity use of MD4/BLAKE2 and splitting of "verify" command

duplicity-talk mailing list
New librsync was part of the plan.  The issue with the new librsync is that it doubles the size of each signature from 8 to 16.  That aggravates the problem of the non-split sigtar files since a lot of people are on the edge of the 5GB limit imposed by their storage provider.  0.8.0 will be backwards incompatible because of the increase.  We'll still be able to read old sigtar files, but older versions will not be able to read the new ones.

As to breaking verify into two commands, please make this into a bug report and I'll assign it to 0.8.

...Thanks,
...Ken


On Sun, Feb 19, 2017 at 6:28 AM, edgar.soldin--- via Duplicity-talk <[hidden email]> wrote:
On 18.02.2017 23:19, Kostas Papadopoulos wrote:
> Hi duplicity devs,
>
> May I ask if it'd be possible for duplicity v0.8.x to support librsync's new (well not so new, they switched ~2 years ago) BLAKE2 format, instead of the old (and broken according to some) MD4-based checksumming ? Please see "SECURITY: MD4 collision/preimage attacks (CVE-2014-8242)" [1] [2]

agreed, raising the version number would be a good opportunity to make a clean cut.

> And while I'm asking, I'd like to request that duplicity would split current "verify" into two commands: "verify" and "compare" (note: the current syntax causes confusion and I've found this specific request repeated in many questions/posts in the duplicity-talk list since 2010). Please see output of sample test runs attached below.

this is a long standing issue. so agreed again, but that would need someone volunteering to implement it. maybe you'd like to offer a bounty or start a coding bounty campaign for this on bountysource or similar.

..ede/duply.net

> Thank you in advance for your consideration, KP
>
>
> [1] https://github.com/librsync/librsync/issues/5
>
> [2] https://bugs.launchpad.net/duplicity/+bug/1342721
>
> On 01/23/2017 10:16 PM, Kostas Papadopoulos wrote:
>> Dear duplicity devs & users,
>>
>> If the duplicity "verify" command just validates the integrity of the backup against the signatures of the sigtar files found at source_url, isn't then the requirement of a second argument (target_directory) unnecessary and confusing?
>>
>> Currently, if the specified target_dir doesn't exist, then duplicity "verify" exits with an error. If the specified target_dir is empty, it works fine. And if the specified target_dir has other random files (unrelated to duplicity), the number of "files compared" is somehow increased (e.g. from 75 to 140 in the example below)
>>
>> /root/docs was the backup original source dir
>> /tmp/t is non-existant dir
>> /tmp/tt is empty dir
>> /tmp contains other stuff too
>>
>>     root@backup:/tmp# duplicity verify --name duply_test-local
>>     --encrypt-key AABBCCDDEEFFAA --sign-key AABBCCDDEEFFAA --verbosity
>>     4 file:///tmp/backup-test-local /root/docs
>>     Local and Remote metadata are synchronized, no sync needed.
>>     Last full backup date: Mon Jan 23 19:00:45 2017
>>     GnuPG passphrase:
>>     Verify complete: 77 files compared, 0 differences found.
>>     root@backup:/tmp# duplicity verify --name duply_test-local
>>     --encrypt-key AABBCCDDEEFFAA --sign-key AABBCCDDEEFFAA --verbosity
>>     4 file:///tmp/backup-test-local /tmp/t
>>     Verify directory /tmp/t does not exist
>>     root@backup:/tmp# duplicity verify --name duply_test-local
>>     --encrypt-key AABBCCDDEEFFAA --sign-key AABBCCDDEEFFAA --verbosity
>>     4 file:///tmp/backup-test-local /tmp/tt
>>     Local and Remote metadata are synchronized, no sync needed.
>>     Last full backup date: Mon Jan 23 19:00:45 2017
>>     GnuPG passphrase:
>>     Verify complete: 75 files compared, 0 differences found.
>>     root@backup:/tmp# duplicity verify --name duply_test-local
>>     --encrypt-key AABBCCDDEEFFAA --sign-key AABBCCDDEEFFAA --verbosity
>>     4 file:///tmp/backup-test-local /tmp/
>>     Local and Remote metadata are synchronized, no sync needed.
>>     Last full backup date: Mon Jan 23 19:00:45 2017
>>     GnuPG passphrase:
>>     Verify complete: 140 files compared, 0 differences found.
>>
>> I would imagine that a more readily understandable syntax would be a duplicity "verify" command with just one argument (duplicity verify ... source_url), and another duplicity "compare" command that would take two arguments (duplicity compare ... source_url target_directory [--compare-data]) and compare backup against the target_directory's files size/date and optionally content (when used with --compare-data).
>>
>> Thank you in advance for your consideration,
>> KP
>>
>> On 04/23/2015 03:12 AM, Kenneth Loafman wrote:
>>> The signatures in the sigtar files.  Think of them as uber-large hashes.
>>>
>>> On Wed, Apr 22, 2015 at 5:25 PM, Kostas Papadopoulos <[hidden email] <mailto:[hidden email]>> wrote:
>>>
>>>     On 4/22/2015 2:51 PM, Kenneth Loafman wrote:
>>>>     Actually, he has to use --compare-data for verify to look at the
>>>>     local data, otherwise verify just validates that the backup is
>>>>     intact and is not corrupted.
>>>>
>>>
>>>     Thanks for the clarification, but then isn't the output of
>>>     "verify" a bit misleading?
>>>
>>>         ...
>>>         Last full backup date: Sun Apr  5 15:55:20 2015
>>>         Verify complete: 73 files compared, 0 differences found.
>>>         --- Finished state OK at 01:33:47.897 - Runtime 00:01:20.821 ---
>>>
>>>     Against what are those 73 files compared?
>>>
>>>     Br, KP
>>>
>>>
>>
>
>

_______________________________________________
Duplicity-talk mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/duplicity-talk


_______________________________________________
Duplicity-talk mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/duplicity-talk
Reply | Threaded
Open this post in threaded view
|

Re: Syntax of duplicity verify command

duplicity-talk mailing list
In reply to this post by duplicity-talk mailing list

Thanks Kostas,

I agree with you (and ede). I have created a bug report for this:

https://bugs.launchpad.net/duplicity/+bug/1688657

I am about to do some work on the command line parsing side (Bug #1480565), so can add this to my list.

As ede said, the verify command has evolved unhelpfully over time and the need for a target in verify is a hangover from that.

Kind regards,

Aaron


On 25/01/17 11:05, edgar.soldin--- via Duplicity-talk wrote:
hey Kostas (again:)),

yeah duplicity's verify is actually a bastard command that verifies the possibility to restore _and_ allows to compare to the source folder. older versions did the second automatically, for newer versions you have to enable --compare-data for that.

the source folder argument is there and needed for the second part, that is why it is still needed.

the functionality should probably be separated into a new compare command in the future, but so far nobody did so.

..ede/duply.net

On 23.01.2017 21:16, Kostas Papadopoulos wrote:
Dear duplicity devs & users,

If the duplicity "verify" command just validates the integrity of the backup against the signatures of the sigtar files found at source_url, isn't then the requirement of a second argument (target_directory) unnecessary and confusing?

Currently, if the specified target_dir doesn't exist, then duplicity "verify" exits with an error. If the specified target_dir is empty, it works fine. And if the specified target_dir has other random files (unrelated to duplicity), the number of "files compared" is somehow increased (e.g. from 75 to 140 in the example below)

/root/docs was the backup original source dir
/tmp/t is non-existant dir
/tmp/tt is empty dir
/tmp contains other stuff too

    root@backup:/tmp# duplicity verify --name duply_test-local --encrypt-key AABBCCDDEEFFAA --sign-key AABBCCDDEEFFAA --verbosity 4 file:///tmp/backup-test-local /root/docs
    Local and Remote metadata are synchronized, no sync needed.
    Last full backup date: Mon Jan 23 19:00:45 2017
    GnuPG passphrase:
    Verify complete: 77 files compared, 0 differences found.
    root@backup:/tmp# duplicity verify --name duply_test-local --encrypt-key AABBCCDDEEFFAA --sign-key AABBCCDDEEFFAA --verbosity 4 file:///tmp/backup-test-local /tmp/t
    Verify directory /tmp/t does not exist
    root@backup:/tmp# duplicity verify --name duply_test-local --encrypt-key AABBCCDDEEFFAA --sign-key AABBCCDDEEFFAA --verbosity 4 file:///tmp/backup-test-local /tmp/tt
    Local and Remote metadata are synchronized, no sync needed.
    Last full backup date: Mon Jan 23 19:00:45 2017
    GnuPG passphrase:
    Verify complete: 75 files compared, 0 differences found.
    root@backup:/tmp# duplicity verify --name duply_test-local --encrypt-key AABBCCDDEEFFAA --sign-key AABBCCDDEEFFAA --verbosity 4 file:///tmp/backup-test-local /tmp/
    Local and Remote metadata are synchronized, no sync needed.
    Last full backup date: Mon Jan 23 19:00:45 2017
    GnuPG passphrase:
    Verify complete: 140 files compared, 0 differences found.

I would imagine that a more readily understandable syntax would be a duplicity "verify" command with just one argument (duplicity verify ... source_url), and another duplicity "compare" command that would take two arguments (duplicity compare ... source_url target_directory [--compare-data]) and compare backup against the target_directory's files size/date and optionally content (when used with --compare-data).

Thank you in advance for your consideration,
KP

On 04/23/2015 03:12 AM, Kenneth Loafman wrote:
The signatures in the sigtar files.  Think of them as uber-large hashes.

On Wed, Apr 22, 2015 at 5:25 PM, Kostas Papadopoulos <[hidden email] [hidden email]> wrote:

    On 4/22/2015 2:51 PM, Kenneth Loafman wrote:
    Actually, he has to use --compare-data for verify to look at the local data, otherwise verify just validates that the backup is intact and is not corrupted.

    Thanks for the clarification, but then isn't the output of "verify" a bit misleading?

        ...
        Last full backup date: Sun Apr  5 15:55:20 2015
        Verify complete: 73 files compared, 0 differences found.
        --- Finished state OK at 01:33:47.897 - Runtime 00:01:20.821 ---

    Against what are those 73 files compared?

    Br, KP



      
_______________________________________________
Duplicity-talk mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/duplicity-talk


_______________________________________________
Duplicity-talk mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/duplicity-talk