backing up using SFTP to a directory under my home directory

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

backing up using SFTP to a directory under my home directory

duplicity-talk mailing list
I have a Windows system running cygwin and duplicity (client) -- backup source
I have a Linux server capable of using SFTP -- backup destination

I am attempting to backup a folder from the client to the server. The
folder I want to place the backups is underneath my home directory on
the server, however:

/proj/backup_dir

When I issue a command like:

export FTP_PASSWORD=mypass
duplicity /cygdrive/c/folder_to_backup
pexpect+sftp://[hidden email]/proj/backup_dir

What ends up happening is that this backs up the contents to
~/proj/backup_dir which is not what I want... Moving the backup after
the fact would not work since I have a quota on my home directory
which exceeds the backup's size.

What I'd like it to do is to back up the contents to /proj/backup_dir

I attempted to use a symlink on my home directory to point to the
folder, and that works to an extent... This works:

duplicity /cygdrive/c/folder_to_backup
pexpect+sftp://[hidden email]/symlinked_dir

where symlinked_dir -> /proj/backup_dir

but this does not:

duplicity /cygdrive/c/folder_to_backup
pexpect+sftp://[hidden email]/symlinked_dir/subfolder

I get messages like this:
"bash-4.3$ cwdup.sh
Local and Remote metadata are synchronized, no sync needed.
      0 [main] python2.7 390948 child_info_fork::abort: address space
needed by '_librsync.dll' (0x520000) is already oc
cupied
      0 [main] python2.7 411620 child_info_fork::abort: address space
needed by '_librsync.dll' (0x520000) is already oc
cupied
Attempt 1 failed. ExceptionPexpect: pty.fork() failed: [Errno 11]
Resource temporarily unavailable"

I don't have control of this server, where I am backing up the data so
I can't change where the sftp command places me in my home directory
when I log in.

Is there another way I can accomplish backing up to some folder
underneath the server's root (/) directory, rather than on my home
(~/) directory?

Thanks!

_______________________________________________
Duplicity-talk mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/duplicity-talk
Reply | Threaded
Open this post in threaded view
|

Re: backing up using SFTP to a directory under my home directory

duplicity-talk mailing list
Luis,

Unfortunately since you don’t have control over the server it may be set up such that it does a chroot (change root) to your home directory in which case you can’t access file structure outside of that (aside from he symbolic link trick).

See https://askubuntu.com/questions/49271/how-to-setup-a-sftp-server-with-users-chrooted-in-their-home-directories#49284 for the settings that do this.  You might be able to read the settings on the server to see if it is set up that way.

This is a standard security setting to protect the server from unwanted access and probing.

-Scott


On Feb 7, 2018, at 7:47 PM, Luis Tirado via Duplicity-talk <[hidden email]> wrote:

I have a Windows system running cygwin and duplicity (client) -- backup source
I have a Linux server capable of using SFTP -- backup destination

I am attempting to backup a folder from the client to the server. The
folder I want to place the backups is underneath my home directory on
the server, however:

/proj/backup_dir

When I issue a command like:

export FTP_PASSWORD=mypass
duplicity /cygdrive/c/folder_to_backup
<a href="pexpect+sftp://username@host.edu/proj/backup_dir" class="">pexpect+sftp://username@.../proj/backup_dir

What ends up happening is that this backs up the contents to
~/proj/backup_dir which is not what I want... Moving the backup after
the fact would not work since I have a quota on my home directory
which exceeds the backup's size.

What I'd like it to do is to back up the contents to /proj/backup_dir

I attempted to use a symlink on my home directory to point to the
folder, and that works to an extent... This works:

duplicity /cygdrive/c/folder_to_backup
pexpect+sftp://[hidden email]/symlinked_dir

where symlinked_dir -> /proj/backup_dir

but this does not:

duplicity /cygdrive/c/folder_to_backup
pexpect+sftp://[hidden email]/symlinked_dir/subfolder

I get messages like this:
"bash-4.3$ cwdup.sh
Local and Remote metadata are synchronized, no sync needed.
     0 [main] python2.7 390948 child_info_fork::abort: address space
needed by '_librsync.dll' (0x520000) is already oc
cupied
     0 [main] python2.7 411620 child_info_fork::abort: address space
needed by '_librsync.dll' (0x520000) is already oc
cupied
Attempt 1 failed. ExceptionPexpect: pty.fork() failed: [Errno 11]
Resource temporarily unavailable"

I don't have control of this server, where I am backing up the data so
I can't change where the sftp command places me in my home directory
when I log in.

Is there another way I can accomplish backing up to some folder
underneath the server's root (/) directory, rather than on my home
(~/) directory?

Thanks!

_______________________________________________
Duplicity-talk mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/duplicity-talk


_______________________________________________
Duplicity-talk mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/duplicity-talk
Reply | Threaded
Open this post in threaded view
|

Re: backing up using SFTP to a directory under my home directory

duplicity-talk mailing list
Hi Scott,

Thanks for the reply. In this case, I can manually browse, place,
remove files from the /proj/backup_dir directory via sftp command line
when I connect manually via sftp commandline, via doing cd
/proj/backup_dir, put, etc.

It seems what I'm missing is how to add batched commands to change to
the correct backup folder before placing the backup there, or somehow
to de-couple the creation of the backup archive from the actual
upload, something along those lines.

On Wed, Feb 7, 2018 at 7:54 PM, Scott Hannahs <[hidden email]> wrote:

> Luis,
>
> Unfortunately since you don’t have control over the server it may be set up
> such that it does a chroot (change root) to your home directory in which
> case you can’t access file structure outside of that (aside from he symbolic
> link trick).
>
> See
> https://askubuntu.com/questions/49271/how-to-setup-a-sftp-server-with-users-chrooted-in-their-home-directories#49284
> for the settings that do this.  You might be able to read the settings on
> the server to see if it is set up that way.
>
> This is a standard security setting to protect the server from unwanted
> access and probing.
>
> -Scott
>
>
> On Feb 7, 2018, at 7:47 PM, Luis Tirado via Duplicity-talk
> <[hidden email]> wrote:
>
> I have a Windows system running cygwin and duplicity (client) -- backup
> source
> I have a Linux server capable of using SFTP -- backup destination
>
> I am attempting to backup a folder from the client to the server. The
> folder I want to place the backups is underneath my home directory on
> the server, however:
>
> /proj/backup_dir
>
> When I issue a command like:
>
> export FTP_PASSWORD=mypass
> duplicity /cygdrive/c/folder_to_backup
> pexpect+sftp://[hidden email]/proj/backup_dir
>
> What ends up happening is that this backs up the contents to
> ~/proj/backup_dir which is not what I want... Moving the backup after
> the fact would not work since I have a quota on my home directory
> which exceeds the backup's size.
>
> What I'd like it to do is to back up the contents to /proj/backup_dir
>
> I attempted to use a symlink on my home directory to point to the
> folder, and that works to an extent... This works:
>
> duplicity /cygdrive/c/folder_to_backup
> pexpect+sftp://[hidden email]/symlinked_dir
>
> where symlinked_dir -> /proj/backup_dir
>
> but this does not:
>
> duplicity /cygdrive/c/folder_to_backup
> pexpect+sftp://[hidden email]/symlinked_dir/subfolder
>
> I get messages like this:
> "bash-4.3$ cwdup.sh
> Local and Remote metadata are synchronized, no sync needed.
>      0 [main] python2.7 390948 child_info_fork::abort: address space
> needed by '_librsync.dll' (0x520000) is already oc
> cupied
>      0 [main] python2.7 411620 child_info_fork::abort: address space
> needed by '_librsync.dll' (0x520000) is already oc
> cupied
> Attempt 1 failed. ExceptionPexpect: pty.fork() failed: [Errno 11]
> Resource temporarily unavailable"
>
> I don't have control of this server, where I am backing up the data so
> I can't change where the sftp command places me in my home directory
> when I log in.
>
> Is there another way I can accomplish backing up to some folder
> underneath the server's root (/) directory, rather than on my home
> (~/) directory?
>
> Thanks!
>
> _______________________________________________
> Duplicity-talk mailing list
> [hidden email]
> https://lists.nongnu.org/mailman/listinfo/duplicity-talk
>
>

_______________________________________________
Duplicity-talk mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/duplicity-talk
Reply | Threaded
Open this post in threaded view
|

Re: backing up using SFTP to a directory under my home directory

duplicity-talk mailing list
In reply to this post by duplicity-talk mailing list
Luis,

aside from the possibility that your backend might be changerooted there is an official way to define an absolute vs. relative backend folder, which is described in the man page, section URL Format.

"
    scheme://[user[:password]@]host[:port]/[/]path
...
In protocols that support it, the path may be preceded by a single slash, ’/path’, to represent a relative path to the target home directory, or preceded by a double slash, ’//path’, to represent an absolute filesystem path.
"

any specific reason that you use pexpect+sftp:// instead of default paramiko?

..ede/duply.net

On 08.02.2018 01:47, Luis Tirado via Duplicity-talk wrote:

> I have a Windows system running cygwin and duplicity (client) -- backup source
> I have a Linux server capable of using SFTP -- backup destination
>
> I am attempting to backup a folder from the client to the server. The
> folder I want to place the backups is underneath my home directory on
> the server, however:
>
> /proj/backup_dir
>
> When I issue a command like:
>
> export FTP_PASSWORD=mypass
> duplicity /cygdrive/c/folder_to_backup
> pexpect+sftp://[hidden email]/proj/backup_dir
>
> What ends up happening is that this backs up the contents to
> ~/proj/backup_dir which is not what I want... Moving the backup after
> the fact would not work since I have a quota on my home directory
> which exceeds the backup's size.
>
> What I'd like it to do is to back up the contents to /proj/backup_dir
>
> I attempted to use a symlink on my home directory to point to the
> folder, and that works to an extent... This works:
>
> duplicity /cygdrive/c/folder_to_backup
> pexpect+sftp://[hidden email]/symlinked_dir
>
> where symlinked_dir -> /proj/backup_dir
>
> but this does not:
>
> duplicity /cygdrive/c/folder_to_backup
> pexpect+sftp://[hidden email]/symlinked_dir/subfolder
>
> I get messages like this:
> "bash-4.3$ cwdup.sh
> Local and Remote metadata are synchronized, no sync needed.
>       0 [main] python2.7 390948 child_info_fork::abort: address space
> needed by '_librsync.dll' (0x520000) is already oc
> cupied
>       0 [main] python2.7 411620 child_info_fork::abort: address space
> needed by '_librsync.dll' (0x520000) is already oc
> cupied
> Attempt 1 failed. ExceptionPexpect: pty.fork() failed: [Errno 11]
> Resource temporarily unavailable"
>
> I don't have control of this server, where I am backing up the data so
> I can't change where the sftp command places me in my home directory
> when I log in.
>
> Is there another way I can accomplish backing up to some folder
> underneath the server's root (/) directory, rather than on my home
> (~/) directory?
>
> Thanks!
>
> _______________________________________________
> Duplicity-talk mailing list
> [hidden email]
> https://lists.nongnu.org/mailman/listinfo/duplicity-talk
>


_______________________________________________
Duplicity-talk mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/duplicity-talk
Reply | Threaded
Open this post in threaded view
|

Re: backing up using SFTP to a directory under my home directory

duplicity-talk mailing list
Hi Scott/Edgar,

Switching to the default paramiko worked, had to end up installing
that module instead and changing to the default sftp:// that uses it
with the added / worked to access the correct path. For whatever
reason pexpect+sftp did not like the // syntax. Working smoothly now,
thanks!

-Luis

On Thu, Feb 8, 2018 at 6:03 AM, edgar.soldin--- via Duplicity-talk
<[hidden email]> wrote:

> Luis,
>
> aside from the possibility that your backend might be changerooted there is an official way to define an absolute vs. relative backend folder, which is described in the man page, section URL Format.
>
> "
>     scheme://[user[:password]@]host[:port]/[/]path
> ...
> In protocols that support it, the path may be preceded by a single slash, ’/path’, to represent a relative path to the target home directory, or preceded by a double slash, ’//path’, to represent an absolute filesystem path.
> "
>
> any specific reason that you use pexpect+sftp:// instead of default paramiko?
>
> ..ede/duply.net
>
> On 08.02.2018 01:47, Luis Tirado via Duplicity-talk wrote:
>> I have a Windows system running cygwin and duplicity (client) -- backup source
>> I have a Linux server capable of using SFTP -- backup destination
>>
>> I am attempting to backup a folder from the client to the server. The
>> folder I want to place the backups is underneath my home directory on
>> the server, however:
>>
>> /proj/backup_dir
>>
>> When I issue a command like:
>>
>> export FTP_PASSWORD=mypass
>> duplicity /cygdrive/c/folder_to_backup
>> pexpect+sftp://[hidden email]/proj/backup_dir
>>
>> What ends up happening is that this backs up the contents to
>> ~/proj/backup_dir which is not what I want... Moving the backup after
>> the fact would not work since I have a quota on my home directory
>> which exceeds the backup's size.
>>
>> What I'd like it to do is to back up the contents to /proj/backup_dir
>>
>> I attempted to use a symlink on my home directory to point to the
>> folder, and that works to an extent... This works:
>>
>> duplicity /cygdrive/c/folder_to_backup
>> pexpect+sftp://[hidden email]/symlinked_dir
>>
>> where symlinked_dir -> /proj/backup_dir
>>
>> but this does not:
>>
>> duplicity /cygdrive/c/folder_to_backup
>> pexpect+sftp://[hidden email]/symlinked_dir/subfolder
>>
>> I get messages like this:
>> "bash-4.3$ cwdup.sh
>> Local and Remote metadata are synchronized, no sync needed.
>>       0 [main] python2.7 390948 child_info_fork::abort: address space
>> needed by '_librsync.dll' (0x520000) is already oc
>> cupied
>>       0 [main] python2.7 411620 child_info_fork::abort: address space
>> needed by '_librsync.dll' (0x520000) is already oc
>> cupied
>> Attempt 1 failed. ExceptionPexpect: pty.fork() failed: [Errno 11]
>> Resource temporarily unavailable"
>>
>> I don't have control of this server, where I am backing up the data so
>> I can't change where the sftp command places me in my home directory
>> when I log in.
>>
>> Is there another way I can accomplish backing up to some folder
>> underneath the server's root (/) directory, rather than on my home
>> (~/) directory?
>>
>> Thanks!
>>
>> _______________________________________________
>> Duplicity-talk mailing list
>> [hidden email]
>> https://lists.nongnu.org/mailman/listinfo/duplicity-talk
>>
>
>
> _______________________________________________
> Duplicity-talk mailing list
> [hidden email]
> https://lists.nongnu.org/mailman/listinfo/duplicity-talk

_______________________________________________
Duplicity-talk mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/duplicity-talk