Duplicity does not cache gdocs verification code

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

Duplicity does not cache gdocs verification code

duplicity-talk mailing list
Hi all,

I am using Duplicity to perform a daily backup of a file server to a regular Google Drive account. I've written a basic script to perform the backup based on several blog posts (such as the one here: http://rockhopper.dk/linux/software/backing-up-to-google-drive-with-duplicity/) and the Duplicity man page.

The backup works perfectly and it stores 1GB encrypted files in the users Google Drive as per the below however everytime I run the backup I am always prompted for a verification code. The gdrive.cache file is never created and I have not been able to track down why this is. I've ensured the permissions are correct and duplicity can write to the directory and the script is run as root. I've searched everywhere but without joy. Could anyone shed some light on why the gdrive.cache file is not being created here? Or where I might start looking for errors? I see nothing obvious in the output of Duplicity when the backup runs.

Many thanks in advance for your help
Owen

The script:

#!/bin/bash

cd /root/.duplicity/

export PASSPHRASE=[Passphrase]
export GOOGLE_DRIVE_SETTINGS=/root/.duplicity/gdrive

duplicity incr --full-if-older-than 3M --volsize 1024 --asynchronous-upload --exclude-device-files --exclude-other-filesystems --allow-source-mismatch -v8 /home gdocs://user@.../Backup

unset PASSPHRASE
unset GOOGLE_DRIVE_SETTINGS

The gdrive settings file:

client_config_backend: settings
client_config:
client_id: [Client ID].apps.googleusercontent.com
client_secret: [Client Secret]
save_credentials: True
save_credentials_backend: file
save_credentials_file: gdrive.cache
get_refresh_token: True

_______________________________________________
Duplicity-talk mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/duplicity-talk
Reply | Threaded
Open this post in threaded view
|

Re: Duplicity does not cache gdocs verification code

duplicity-talk mailing list
On 05.01.2017 07:33, Owen Jacob via Duplicity-talk wrote:

> Hi all,
>
> I am using Duplicity to perform a daily backup of a file server to a regular Google Drive account. I've written a basic script to perform the backup based on several blog posts (such as the one here: http://rockhopper.dk/linux/software/backing-up-to-google-drive-with-duplicity/ <http://rockhopper.dk/linux/software/backing-up-to-google-drive-with-duplicity/>) and the Duplicity man page.
>
> The backup works perfectly and it stores 1GB encrypted files in the users Google Drive as per the below however everytime I run the backup I am always prompted for a verification code. The gdrive.cache file is never created and I have not been able to track down why this is. I've ensured the permissions are correct and duplicity can write to the directory and the script is run as root. I've searched everywhere but without joy. Could anyone shed some light on why the gdrive.cache file is not being created here? Or where I might start looking for errors? I see nothing obvious in the output of Duplicity when the backup runs.
>
> Many thanks in advance for your help
> Owen
>
> The script:
>
> #!/bin/bash
>
> cd /root/.duplicity/
>
> export PASSPHRASE=[Passphrase]
> export GOOGLE_DRIVE_SETTINGS=/root/.duplicity/gdrive
>
> duplicity incr --full-if-older-than 3M --volsize 1024 --asynchronous-upload --exclude-device-files --exclude-other-filesystems --allow-source-mismatch -v8 /home gdocs://[hidden email]/Backup <http://user@.../Backup>
>
> unset PASSPHRASE
> unset GOOGLE_DRIVE_SETTINGS
>
> The gdrive settings file:
>
> client_config_backend: settings
> client_config:
> client_id: [Client ID].apps.googleusercontent.com <http://apps.googleusercontent.com/>
> client_secret: [Client Secret]
> save_credentials: True
> save_credentials_backend: file
> save_credentials_file: gdrive.cache
> get_refresh_token: True
>

hey Owen,

1. what's your duplicity version?

2. did you do the dance as described in "a Note on Pydrive Backend"
    http://duplicity.nongnu.org/duplicity.1.html#sect22
?

3. why have you --allow-source-mismatch enabled?

..ede/duply.net

_______________________________________________
Duplicity-talk mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/duplicity-talk
Reply | Threaded
Open this post in threaded view
|

Re: Duplicity does not cache gdocs verification code

duplicity-talk mailing list
In reply to this post by duplicity-talk mailing list
Hi Edgar,

Thanks for your reply. To answer your questions:

1. I am using duplicity 0.7.10

2. I did do the dance as described in "a note on PyDrive". I am using a regular account to store the backup so I created the client ID and client secret for the Drive API. The bit I am missing is the last sentence of the note. I run my backup, it asks me to visit the URL to authorise access to my drive but it never cache's my verification so I must re-authenticate every time I run the backup.

3. I have --allow-source-mismatch on after I tided up our DNS server. The reverse lookup had a different hostname than the fileserver so Duplicity wouldn't increment the existing backup. Is using that flag bad practice?

Thanks again
Owen

_______________________________________________
Duplicity-talk mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/duplicity-talk
Reply | Threaded
Open this post in threaded view
|

Changing gpg keyring to use

duplicity-talk mailing list
Hi fellows,

does anyone by chance know how I can provide a different user's secret
keyring file to a restore process? I back up home directories with
system's private and the user's public keys (--encrypt-key=BBBEEECC),
thus I'd need the user's private key to restore. Since operation may
need sudo/root it seems that only private keys in root's keyring are
available to the duplicity or the gpg-agent. However, I must not provide
them to root for data privacy reasons. For the same reasons adding
root's key to a second --encrypt-key is not an option.

An example:
# sudo -c "export PASSPHRASE=whatever; duplicity  --use-agent
--ssh-options="-oIdentityFile=/root/.ssh/id_duplicity"
--encrypt-key=BBBEEECC --exclude-if-present .dupl_noBackup
--exclude-filelist /etc/duplicity/files2ignore /home/userx/
scp://system@192.168.5.5/BackUps/hostname.userx-BBBEEECC; unset PASSPHRASE"

The user may issue
$ sudo duplicity [verify|restore] --use-agent [--encrypt-secret-keyring
/home/userx/.gnupg/secring.gpg --encrypt-key BBBEEECC
--ssh-options="-oIdentityFile=/root/.ssh/id_duplicity"
scp://system@192.168.5.5/BackUps/hostname.userx-BBBEEECC /home/userx
duplicity 0.7.10 (August 20, 2016)
     :
Found primary backup chain with matching signature chain:
     :
          Incremental         Sat Jan  7 15:04:36 2017                 1
     :
GPGError: GPG Failed, see log below:
===== Begin GnuPG log =====
gpg: encrypted with 3072-bit RSA key, ID BBBEEECC, created 2013-12-15
"userx <userx@local>"
gpg: decryption failed: No secret key
===== End GnuPG log =====

The --encrypt-secret-keyring was just a test, according to manpage I did
not expect it really to work, but other attempts failed as well.

Any help is highly appreciated.

Best regards,
Christian



_______________________________________________
Duplicity-talk mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/duplicity-talk
Reply | Threaded
Open this post in threaded view
|

Re: Changing gpg keyring to use

duplicity-talk mailing list
Christian,

simply set duplicity parameter --gpg-options to tell gpg which folder to use as settings folder for this run instead of the current user's $HOME. eg. to use bernd's home

  duplicity --gpg-options="--homedir ~bernd/" ...

..ede/duply.net

On January 8, 2017 10:01:58 AM GMT+01:00, "C. Enzmann via Duplicity-talk" <[hidden email]> wrote:

>Hi fellows,
>
>does anyone by chance know how I can provide a different user's secret
>keyring file to a restore process? I back up home directories with
>system's private and the user's public keys (--encrypt-key=BBBEEECC),
>thus I'd need the user's private key to restore. Since operation may
>need sudo/root it seems that only private keys in root's keyring are
>available to the duplicity or the gpg-agent. However, I must not
>provide
>them to root for data privacy reasons. For the same reasons adding
>root's key to a second --encrypt-key is not an option.
>
>An example:
># sudo -c "export PASSPHRASE=whatever; duplicity  --use-agent
>--ssh-options="-oIdentityFile=/root/.ssh/id_duplicity"
>--encrypt-key=BBBEEECC --exclude-if-present .dupl_noBackup
>--exclude-filelist /etc/duplicity/files2ignore /home/userx/
>scp://system@192.168.5.5/BackUps/hostname.userx-BBBEEECC; unset
>PASSPHRASE"
>
>The user may issue
>$ sudo duplicity [verify|restore] --use-agent [--encrypt-secret-keyring
>
>/home/userx/.gnupg/secring.gpg --encrypt-key BBBEEECC
>--ssh-options="-oIdentityFile=/root/.ssh/id_duplicity"
>scp://system@192.168.5.5/BackUps/hostname.userx-BBBEEECC /home/userx
>duplicity 0.7.10 (August 20, 2016)
>     :
>Found primary backup chain with matching signature chain:
>     :
>         Incremental         Sat Jan  7 15:04:36 2017                 1
>     :
>GPGError: GPG Failed, see log below:
>===== Begin GnuPG log =====
>gpg: encrypted with 3072-bit RSA key, ID BBBEEECC, created 2013-12-15
>"userx <userx@local>"
>gpg: decryption failed: No secret key
>===== End GnuPG log =====
>
>The --encrypt-secret-keyring was just a test, according to manpage I
>did
>not expect it really to work, but other attempts failed as well.
>
>Any help is highly appreciated.
>
>Best regards,
>Christian
>
>
>
>_______________________________________________
>Duplicity-talk mailing list
>[hidden email]
>https://lists.nongnu.org/mailman/listinfo/duplicity-talk

_______________________________________________
Duplicity-talk mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/duplicity-talk
Reply | Threaded
Open this post in threaded view
|

Re: Duplicity does not cache gdocs verification code

duplicity-talk mailing list
In reply to this post by duplicity-talk mailing list
On 06.01.2017 01:47, Owen Jacob via Duplicity-talk wrote:
> Hi Edgar,
>
> Thanks for your reply. To answer your questions:
>
> 1. I am using duplicity 0.7.10

recent enough

> 2. I did do the dance as described in "a note on PyDrive". I am using a regular account to store the backup so I created the client ID and client secret for the Drive API. The bit I am missing is the last sentence of the note. I run my backup, it asks me to visit the URL to authorise access to my drive but it never cache's my verification so I must re-authenticate every time I run the backup.

2a.does the file configured as

  save_credentials_file: <filename to cache credentials>

exist, or is it created after a successful access? try deleting it, if it exists first.

2b. can you post the GOOGLE_DRIVE_SETTINGS settings file (private strings obfuscated)?

2c. maybe a bug in pydrive? what's your version? can you upgrade to a latest stable?

> 3. I have --allow-source-mismatch on after I tided up our DNS server. The reverse lookup had a different hostname than the fileserver so Duplicity wouldn't increment the existing backup. Is using that flag bad practice?

no, but it is used to resolve a precaution error that is raised in case you for some reason try to backup from another machine into the same remote folder.
so, simply using it one time resolves the error, if you know what you're doing ;). after that is is not needed any more and you are "protected" again.

..ede/duply.net

_______________________________________________
Duplicity-talk mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/duplicity-talk
Reply | Threaded
Open this post in threaded view
|

Re: Duplicity does not cache gdocs verification code

duplicity-talk mailing list
In reply to this post by duplicity-talk mailing list
Hi Edgar

2a. The file doesn't exist and it is not created after a successful access. I believe this is the root cause of my issue and it doesn't seem to be permissions related.

2b. Here are the contents of the GOOLE_DRIVE_SETTINGS:

client_config_backend: settings
client_config:
client_id: [Client ID].apps.googleusercontent.com
client_secret: [Client Secret]
save_credentials: True
save_credentials_backend: file
save_credentials_file: gdrive.cache
get_refresh_token: True

2c. I am using PyDrive 1.3.1 installed via PIP. I think I'll report an issue on their Git page since I feel it could be PyDrive and not Duplicity that is causing this.

3. Thanks for the information. I'll tidy my script up and remove that option.

Thanks.
Owen

_______________________________________________
Duplicity-talk mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/duplicity-talk
Reply | Threaded
Open this post in threaded view
|

Re: Duplicity does not cache gdocs verification code

duplicity-talk mailing list
Owen,

On 09.01.2017 03:41, Owen Jacob via Duplicity-talk wrote:
> save_credentials_file: gdrive.cache

did you ever try giving an absolute path for the file? eg. for testing

  save_credentials_file: /tmp/test.cache

?

..ede/duply.net


_______________________________________________
Duplicity-talk mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/duplicity-talk
Reply | Threaded
Open this post in threaded view
|

Re: Duplicity does not cache gdocs verification code

duplicity-talk mailing list
In reply to this post by duplicity-talk mailing list
Hi,

Adding the absolute file path to the config doesn't work either :(

Thanks
Owen

_______________________________________________
Duplicity-talk mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/duplicity-talk
Reply | Threaded
Open this post in threaded view
|

Re: Duplicity does not cache gdocs verification code

duplicity-talk mailing list
Please report this as a bug.

...Thanks,
...Ken


On Tue, Jan 10, 2017 at 4:23 AM, Owen Jacob via Duplicity-talk <[hidden email]> wrote:
Hi,

Adding the absolute file path to the config doesn't work either :(

Thanks
Owen

_______________________________________________
Duplicity-talk mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/duplicity-talk



_______________________________________________
Duplicity-talk mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/duplicity-talk