Re: rdiff to SMB share?

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
17 messages Options
Reply | Threaded
Open this post in threaded view
|

Re: rdiff to SMB share?

Cengiz Günay

Hi,

I have been trying to use rdiff-backup on a Linux system to back things up
to a SMB/CIFS share on some remote server. I have been getting the same
errors regardsless of the rdiff-backup version I'm using. I also tried
using CIFS versus the regular SMBFS. Searching the mailing list led me to
find this thread. I'd appreciate any input. Here's a listing of the error
with the latest stable version of rdiff-backup, followed by my system
info:

---
cengiz@laotse:~> rdiff-backup -v5 bin emory/cgunay/home_backup
Unable to import module xattr.
Extended attributes not supported on filesystem at bin
Unable to import module posix1e from pylibacl package.
ACLs not supported on filesystem at bin
-----------------------------------------------------------------
Detected abilities for source (read only) file system:
  Access control lists                         Off
  Extended attributes                          Off
  Mac OS X style resource forks                Off
  Mac OS X Finder information                  Off
-----------------------------------------------------------------
Warning: hard linking not supported by filesystem at
emory/cgunay/home_backup/rdiff-backup-data
Directories on file system at
emory/cgunay/home_backup/rdiff-backup-data/rdiff-backup.tmp.3 are not
fsyncable.
Assuming it's unnecessary.
Unable to import module xattr.
Extended attributes not supported on filesystem at
emory/cgunay/home_backup/rdiff-backup-data/rdiff-backup.tmp.3
Unable to import module posix1e from pylibacl package.
ACLs not supported on filesystem at
emory/cgunay/home_backup/rdiff-backup-data/rdiff-backup.tmp.3
Traceback (most recent call last):
  File "/usr/bin/rdiff-backup", line 23, in ?
    rdiff_backup.Main.Main(sys.argv[1:])
  File "/usr/lib64/python2.3/site-packages/rdiff_backup/Main.py", line
285, in Main
    take_action(rps)
  File "/usr/lib64/python2.3/site-packages/rdiff_backup/Main.py", line
255, in take_action
    elif action == "backup": Backup(rps[0], rps[1])
  File "/usr/lib64/python2.3/site-packages/rdiff_backup/Main.py", line
296, in Backup
    backup_set_fs_globals(rpin, rpout)
  File "/usr/lib64/python2.3/site-packages/rdiff_backup/Main.py", line
418, in backup_set_fs_globals
    dest_fsa = rpout.conn.fs_abilities.get_fsabilities_readwrite(
  File "/usr/lib64/python2.3/site-packages/rdiff_backup/fs_abilities.py",
line 408, in get_fsabilities_readwrite
    return FSAbilities(desc_string).init_readwrite(
  File "/usr/lib64/python2.3/site-packages/rdiff_backup/fs_abilities.py",
line 155, in init_readwrite
    if override_chars_to_quote is None: self.set_chars_to_quote(subdir)
  File "/usr/lib64/python2.3/site-packages/rdiff_backup/fs_abilities.py",
line 266, in set_chars_to_quote
    sanity_check()
  File "/usr/lib64/python2.3/site-packages/rdiff_backup/fs_abilities.py",
line 262, in sanity_check
    rp.touch()
  File "/usr/lib64/python2.3/site-packages/rdiff_backup/rpath.py", line
840, in touch
    self.conn.open(self.path, "w").close()
IOError: [Errno 2] No such file or directory:
'emory/cgunay/home_backup/rdiff-backup-data/rdiff-backup.tmp.3/5-_ a.'
---

My system info:
---
# uname -a
Linux laotse 2.4.21-291-smp #1 SMP Tue May 24 10:54:02 UTC 2005 x86_64
x86_64 x86_64 GNU/Linux
# rdiff-backup -V
rdiff-backup 1.0.3
# python -V
Python 2.3+
# cat /etc/SuSE-release
SuSE Linux 9.0 (x86-64)
VERSION = 9.0
---

Thanks in advance,
Cengiz Gunay
--
[hidden email]  [hidden email] (MSN) [hidden email]
Lab: +1-404-727-3565 Home: +1-337-935-0136 Cell: +1-337-255-3660
http://userwww.service.emory.edu/~cgunay/
IMs: ICQ# 21104923, cengique@{jabber.org,yahoo.com,Skype}
--


_______________________________________________
rdiff-backup-users mailing list at [hidden email]
http://lists.nongnu.org/mailman/listinfo/rdiff-backup-users
Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki
Reply | Threaded
Open this post in threaded view
|

Re: rdiff to SMB share?

Ben Escoto
>>>>> Cengiz Gunay <[hidden email]>

>>>>> wrote the following on Fri, 6 Jan 2006 17:43:57 -0500 (EST)
>
> I have been trying to use rdiff-backup on a Linux system to back things up
> to a SMB/CIFS share on some remote server. I have been getting the same
> errors regardsless of the rdiff-backup version I'm using. I also tried
> using CIFS versus the regular SMBFS. Searching the mailing list led me to
> find this thread. I'd appreciate any input. Here's a listing of the error
> with the latest stable version of rdiff-backup, followed by my system
> info:
...
>   File "/usr/lib64/python2.3/site-packages/rdiff_backup/fs_abilities.py",
> line 262, in sanity_check
>     rp.touch()
>   File "/usr/lib64/python2.3/site-packages/rdiff_backup/rpath.py", line
> 840, in touch
>     self.conn.open(self.path, "w").close()
> IOError: [Errno 2] No such file or directory:
> 'emory/cgunay/home_backup/rdiff-backup-data/rdiff-backup.tmp.3/5-_ a.'

rdiff-backup is failing during the "sanity check" phase, where it just
does some basic stuff to a directory and makes sure the results make
sense.  In this case it seems it is trying to touch the file mentioned
in the error message, and its failing with ENOENT.

Maybe after it fails you can try creating that file, and see if it
works.


--
Ben Escoto

_______________________________________________
rdiff-backup-users mailing list at [hidden email]
http://lists.nongnu.org/mailman/listinfo/rdiff-backup-users
Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki

attachment0 (196 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: rdiff to SMB share?

Cengiz Günay
On Mon, 9 Jan 2006, Ben Escoto wrote:

> ...
> > IOError: [Errno 2] No such file or directory:
> > 'emory/cgunay/home_backup/rdiff-backup-data/rdiff-backup.tmp.3/5-_ a.'
>
> rdiff-backup is failing during the "sanity check" phase, where it just
> does some basic stuff to a directory and makes sure the results make
> sense.  In this case it seems it is trying to touch the file mentioned
> in the error message, and its failing with ENOENT.
>
> Maybe after it fails you can try creating that file, and see if it
> works.

Yes, you're right. I tried creating the file manually with:

  $ touch '5-_ a.'
  touch: cannot touch `5-_ a.': No such file or directory

but, the same works with '5-_ a.a'. The filesystem is a remote Windows
share. It doesn't seem to accept files with empty space after a dot. If I
try to do this from a Windows machine, it simply ignores the trailing dot
and makes a file with no dots in it. This seems to be a general rule in
Windows, it doesn't matter where I create the file, it doesn't seem to
accept the dot to be the last character in a file name.

Would it be possible to add this condition to the sanity check in
rdiff-backup?

Thanks,
Cengiz Gunay
--
[hidden email]  [hidden email] (MSN) [hidden email]
Lab: +1-404-727-3565 Home/Cell: +1-678-559-8694
http://userwww.service.emory.edu/~cgunay/
IMs: ICQ# 21104923, cengique@{jabber.org,yahoo.com,Skype}
--


_______________________________________________
rdiff-backup-users mailing list at [hidden email]
http://lists.nongnu.org/mailman/listinfo/rdiff-backup-users
Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki
Reply | Threaded
Open this post in threaded view
|

Re: rdiff to SMB share?

Ben Escoto
>>>>> Cengiz Gunay <[hidden email]>

>>>>> wrote the following on Tue, 10 Jan 2006 14:24:33 -0500 (EST)
>
> Yes, you're right. I tried creating the file manually with:
>
>   $ touch '5-_ a.'
>   touch: cannot touch `5-_ a.': No such file or directory
>
> but, the same works with '5-_ a.a'. The filesystem is a remote
> Windows share. It doesn't seem to accept files with empty space
> after a dot. If I try to do this from a Windows machine, it simply
> ignores the trailing dot and makes a file with no dots in it. This
> seems to be a general rule in Windows, it doesn't matter where I
> create the file, it doesn't seem to accept the dot to be the last
> character in a file name.
>
> Would it be possible to add this condition to the sanity check in
> rdiff-backup?
Sure, the sanity case was just supposed to check whether basic
characters were allowed.  I just checked in this simple patch:

                def sanity_check():
                        """Make sure basic filenames writable"""
-                       for filename in ['5-_ a.']:
+                       for filename in ['5-_ a.snapshot.gz']:
                                rp = subdir.append(filename)
                                rp.touch()
                                assert rp.lstat()

which is a better test anyway.


--
Ben Escoto

_______________________________________________
rdiff-backup-users mailing list at [hidden email]
http://lists.nongnu.org/mailman/listinfo/rdiff-backup-users
Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki

attachment0 (196 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: rdiff to SMB share?

Cengiz Günay


On Tue, 10 Jan 2006, Ben Escoto wrote:

> > Would it be possible to add this condition to the sanity check in
> > rdiff-backup?
>
> Sure, the sanity case was just supposed to check whether basic
> characters were allowed.  I just checked in this simple patch: ...

Ok, I got the CVS version, and I upgraded to librsync 0.9.7. I got
rdiff-backup to work within my ext3 filesystem. When I target the SMB
filesystem, this time I got a different error. Any clues? The error
follows and I have some final comments at the bottom:

---
> PYTHONPATH=$RDB_CVS $RDB_CVS/rdiff-backup -v6 ~/bin ~/emory/cgunay/home_backup/
Unable to import module xattr.
Extended attributes not supported on filesystem at /Lab/cengiz/bin
Unable to import module posix1e from pylibacl package.
ACLs not supported on filesystem at /Lab/cengiz/bin
-----------------------------------------------------------------
Detected abilities for source (read only) file system:
  Access control lists                         Off
  Extended attributes                          Off
  Case sensitivity                             On
  Mac OS X style resource forks                Off
  Mac OS X Finder information                  Off
-----------------------------------------------------------------
Making directory
/Lab/cengiz/emory/cgunay/home_backup/rdiff-backup-data/rdiff-backup.tmp.2
Exception 'local variable 'ext_rp' referenced before assignment' raised of
class 'exceptions.UnboundLocalError':
  File "/usr/src/rdiff-backup/rdiff_backup/Main.py", line 295, in
error_check_Main
    try: Main(arglist)
  File "/usr/src/rdiff-backup/rdiff_backup/Main.py", line 315, in Main
    take_action(rps)
  File "/usr/src/rdiff-backup/rdiff_backup/Main.py", line 271, in
take_action
    elif action == "backup": Backup(rps[0], rps[1])
  File "/usr/src/rdiff-backup/rdiff_backup/Main.py", line 325, in Backup
    rpout.conn.fs_abilities.backup_set_globals(rpin)
  File "/usr/src/rdiff-backup/rdiff_backup/fs_abilities.py", line 602, in
backup_set_globals
    dest_fsa = FSAbilities('destination').init_readwrite(Globals.rbdir)
  File "/usr/src/rdiff-backup/rdiff_backup/fs_abilities.py", line 137, in
init_readwrite
    self.set_extended_filenames(subdir)
  File "/usr/src/rdiff-backup/rdiff_backup/fs_abilities.py", line 208, in
set_extended_filenames
    assert not ext_rp.lstat()

Traceback (most recent call last):
  File "/usr/src/rdiff-backup/rdiff-backup", line 23, in ?
    rdiff_backup.Main.error_check_Main(sys.argv[1:])
  File "/usr/src/rdiff-backup/rdiff_backup/Main.py", line 295, in
error_check_Main
    try: Main(arglist)
  File "/usr/src/rdiff-backup/rdiff_backup/Main.py", line 315, in Main
    take_action(rps)
  File "/usr/src/rdiff-backup/rdiff_backup/Main.py", line 271, in
take_action
    elif action == "backup": Backup(rps[0], rps[1])
  File "/usr/src/rdiff-backup/rdiff_backup/Main.py", line 325, in Backup
    rpout.conn.fs_abilities.backup_set_globals(rpin)
  File "/usr/src/rdiff-backup/rdiff_backup/fs_abilities.py", line 602, in
backup_set_globals
    dest_fsa = FSAbilities('destination').init_readwrite(Globals.rbdir)
  File "/usr/src/rdiff-backup/rdiff_backup/fs_abilities.py", line 137, in
init_readwrite
    self.set_extended_filenames(subdir)
  File "/usr/src/rdiff-backup/rdiff_backup/fs_abilities.py", line 208, in
set_extended_filenames
    assert not ext_rp.lstat()
UnboundLocalError: local variable 'ext_rp' referenced before assignment
---

Looking at the code, I see that within the try block, an error occurs
*before* ext_rp is assigned on line:

fs_abilities:205:  ext_rp = subdir.append(extended_filename)

but we cannot see this error, because in the else block ext_rp is used
before assignment. I'm not sure how to modify this code to get to the real
error.

Cengiz Gunay
--
[hidden email]  [hidden email] (MSN) [hidden email]
Lab: +1-404-727-3565 Home/Cell: +1-678-559-8694
http://userwww.service.emory.edu/~cgunay/
IMs: ICQ# 21104923, cengique@{jabber.org,yahoo.com,Skype}
--


_______________________________________________
rdiff-backup-users mailing list at [hidden email]
http://lists.nongnu.org/mailman/listinfo/rdiff-backup-users
Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki
Reply | Threaded
Open this post in threaded view
|

Re: rdiff to SMB share?

Ben Escoto
>>>>> Cengiz Gunay <[hidden email]>
>>>>> wrote the following on Wed, 11 Jan 2006 15:02:44 -0500 (EST)
>
> > Sure, the sanity case was just supposed to check whether basic
> > characters were allowed.  I just checked in this simple patch: ...
>
> Ok, I got the CVS version, and I upgraded to librsync 0.9.7. I got
> rdiff-backup to work within my ext3 filesystem. When I target the SMB
> filesystem, this time I got a different error. Any clues? The error
> follows and I have some final comments at the bottom:

Ahh, if you checked out CVS, you got the development version, and will
get entirely different errors. :-)  

> Exception 'local variable 'ext_rp' referenced before assignment' raised of
...
>     assert not ext_rp.lstat()

Looks like a silly mistake that just got added.  Update your CVS or
apply:

--- fs_abilities.py     10 Jan 2006 22:28:15 -0000      1.27
+++ fs_abilities.py     11 Jan 2006 21:22:25 -0000
@@ -201,11 +201,12 @@
 
                # Try a UTF-8 encoded character
                extended_filename = ':\\ ' + chr(225) + chr(132) + chr(137)
+               ext_rp = None
                try:
                        ext_rp = subdir.append(extended_filename)
                        ext_rp.touch()
                except (IOError, OSError):
-                       assert not ext_rp.lstat()
+                       if ext_rp: assert not ext_rp.lstat()
                        self.extended_filenames = 0
                else:
                        assert ext_rp.lstat()


--
Ben Escoto

_______________________________________________
rdiff-backup-users mailing list at [hidden email]
http://lists.nongnu.org/mailman/listinfo/rdiff-backup-users
Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki

attachment0 (196 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: rdiff to SMB share?

Cengiz Günay

On Wed, 11 Jan 2006, Ben Escoto wrote:

> Ahh, if you checked out CVS, you got the development version, and will
> get entirely different errors. :-)  

:) Keeping with the tradition of coming up with entirely different errors,
I have some more..

> Looks like a silly mistake that just got added.  Update your CVS or
> apply: ...

I updated from CVS and this time I get this error (note that it still
works fine for within-unix-fs operations):

---
> PYTHONPATH=$RDB_CVS $RDB_CVS/rdiff-backup -v6 ~/bin
~/emory/cgunay/home_backup/
Making directory /Lab/cengiz/emory/cgunay/home_backup/rdiff-backup-data
Unable to import module xattr.
Extended attributes not supported on filesystem at /Lab/cengiz/bin
Unable to import module posix1e from pylibacl package.
ACLs not supported on filesystem at /Lab/cengiz/bin
-----------------------------------------------------------------
Detected abilities for source (read only) file system:
  Access control lists                         Off
  Extended attributes                          Off
  Case sensitivity                             On
  Mac OS X style resource forks                Off
  Mac OS X Finder information                  Off
-----------------------------------------------------------------
Making directory
/Lab/cengiz/emory/cgunay/home_backup/rdiff-backup-data/rdiff-backup.tmp.0
Making directory
/Lab/cengiz/emory/cgunay/home_backup/rdiff-backup-data/rdiff-backup.tmp.0/hl
Hard linking
/Lab/cengiz/emory/cgunay/home_backup/rdiff-backup-data/rdiff-backup.tmp.0/hl/hardlinked_file2
to
/Lab/cengiz/emory/cgunay/home_backup/rdiff-backup-data/rdiff-backup.tmp.0/hardlinked_file1
Warning: hard linking not supported by filesystem at
/Lab/cengiz/emory/cgunay/home_backup/rdiff-backup-data
Directories on file system at
/Lab/cengiz/emory/cgunay/home_backup/rdiff-backup-data/rdiff-backup.tmp.0
are not fsyncable.
Assuming it's unnecessary.
Unable to import module xattr.
Extended attributes not supported on filesystem at
/Lab/cengiz/emory/cgunay/home_backup/rdiff-backup-data/rdiff-backup.tmp.0
Unable to import module posix1e from pylibacl package.
ACLs not supported on filesystem at
/Lab/cengiz/emory/cgunay/home_backup/rdiff-backup-data/rdiff-backup.tmp.0
Removing directory
/Lab/cengiz/emory/cgunay/home_backup/rdiff-backup-data/rdiff-backup.tmp.0
Exception '[Errno 16] Device or resource busy removing
/Lab/cengiz/emory/cgunay/home_backup/rdiff-backup-data/rdiff-backup.tmp.0'
raised of class 'exceptions.OSError':
  File "/usr/src/rdiff-backup/rdiff_backup/Main.py", line 295, in
error_check_Main
    try: Main(arglist)
  File "/usr/src/rdiff-backup/rdiff_backup/Main.py", line 315, in Main
    take_action(rps)
  File "/usr/src/rdiff-backup/rdiff_backup/Main.py", line 271, in
take_action
    elif action == "backup": Backup(rps[0], rps[1])
  File "/usr/src/rdiff-backup/rdiff_backup/Main.py", line 325, in Backup
    rpout.conn.fs_abilities.backup_set_globals(rpin)
  File "/usr/src/rdiff-backup/rdiff_backup/fs_abilities.py", line 603, in
backup_set_globals
    dest_fsa = FSAbilities('destination').init_readwrite(Globals.rbdir)
  File "/usr/src/rdiff-backup/rdiff_backup/fs_abilities.py", line 149, in
init_readwrite
    subdir.delete()
  File "/usr/src/rdiff-backup/rdiff_backup/rpath.py", line 945, in delete
    self.conn.shutil.rmtree(self.path)
  File "/usr/lib64/python2.3/shutil.py", line 140, in rmtree
    raise exc[0], (exc[1][0], exc[1][1] + ' removing '+arg)

Traceback (most recent call last):
  File "/usr/src/rdiff-backup/rdiff-backup", line 23, in ?
    rdiff_backup.Main.error_check_Main(sys.argv[1:])
  File "/usr/src/rdiff-backup/rdiff_backup/Main.py", line 295, in
error_check_Main
    try: Main(arglist)
  File "/usr/src/rdiff-backup/rdiff_backup/Main.py", line 315, in Main
    take_action(rps)
  File "/usr/src/rdiff-backup/rdiff_backup/Main.py", line 271, in
take_action
    elif action == "backup": Backup(rps[0], rps[1])
  File "/usr/src/rdiff-backup/rdiff_backup/Main.py", line 325, in Backup
    rpout.conn.fs_abilities.backup_set_globals(rpin)
  File "/usr/src/rdiff-backup/rdiff_backup/fs_abilities.py", line 603, in
backup_set_globals
    dest_fsa = FSAbilities('destination').init_readwrite(Globals.rbdir)
  File "/usr/src/rdiff-backup/rdiff_backup/fs_abilities.py", line 149, in
init_readwrite
    subdir.delete()
  File "/usr/src/rdiff-backup/rdiff_backup/rpath.py", line 945, in delete
    self.conn.shutil.rmtree(self.path)
  File "/usr/lib64/python2.3/shutil.py", line 140, in rmtree
    raise exc[0], (exc[1][0], exc[1][1] + ' removing '+arg)
OSError: [Errno 16] Device or resource busy removing
/Lab/cengiz/emory/cgunay/home_backup/rdiff-backup-data/rdiff-backup.tmp.0
---

As it said, rdiff-backup.tmp.0 directory was still there after it crashed.
But a simple "rm -r" was sufficient to remove it. So I'm not sure what it
means by saying "Device or resource busy removing"?

Cengiz Gunay
--
[hidden email]  [hidden email] (MSN) [hidden email]
Lab: +1-404-727-3565 Home/Cell: +1-678-559-8694
http://userwww.service.emory.edu/~cgunay/
IMs: ICQ# 21104923, cengique@{jabber.org,yahoo.com,Skype}
--


_______________________________________________
rdiff-backup-users mailing list at [hidden email]
http://lists.nongnu.org/mailman/listinfo/rdiff-backup-users
Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki
Reply | Threaded
Open this post in threaded view
|

Re: rdiff to SMB share?

Ben Escoto
>>>>> Cengiz Gunay <[hidden email]>
>>>>> wrote the following on Wed, 11 Jan 2006 16:37:30 -0500 (EST)
>
> I updated from CVS and this time I get this error (note that it still
> works fine for within-unix-fs operations):
...
> Exception '[Errno 16] Device or resource busy removing
> /Lab/cengiz/emory/cgunay/home_backup/rdiff-backup-data/rdiff-backup.tmp.0'
> raised of class 'exceptions.OSError':
...
>     subdir.delete()
>   File "/usr/src/rdiff-backup/rdiff_backup/rpath.py", line 945, in delete
>     self.conn.shutil.rmtree(self.path)
>   File "/usr/lib64/python2.3/shutil.py", line 140, in rmtree
>     raise exc[0], (exc[1][0], exc[1][1] + ' removing '+arg)

> As it said, rdiff-backup.tmp.0 directory was still there after it crashed.
> But a simple "rm -r" was sufficient to remove it. So I'm not sure what it
> means by saying "Device or resource busy removing"?

I'm not sure, why don't you add an endless loop, like

        while 1: pass

right before subdir.delete() in fs_abilities.py.  Remember to intent
it just like the subdir.delete() line (in python, whitespace is
significant).

Then run rdiff-backup again, and when it loops, find that
rdiff-backup.tmp.0 directory and try to remove it.  You should get
that same error.  Tell me which file(s) cause problems, and what's in
those files.


--
Ben Escoto

_______________________________________________
rdiff-backup-users mailing list at [hidden email]
http://lists.nongnu.org/mailman/listinfo/rdiff-backup-users
Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki

attachment0 (196 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: rdiff to SMB share?

Cengiz Günay
On Wed, 11 Jan 2006, Ben Escoto wrote:

> I'm not sure, why don't you add an endless loop, like ...
> right before subdir.delete() in fs_abilities.py.  Remember to intent
> it just like the subdir.delete() line (in python, whitespace is
> significant).
>
> Then run rdiff-backup again, and when it loops, find that
> rdiff-backup.tmp.0 directory and try to remove it.  You should get
> that same error.  Tell me which file(s) cause problems, and what's in
> those files.

Ok, I added the endless loop, and looking in the rdiff-backup-data, I see
this:

---
> ls -lR
.:
total 4
drwxr-xr-x    1 cengiz   users        4096 2006-01-11 18:45 rdiff-backup.tmp.0

./rdiff-backup.tmp.0:
total 4
-rwxr-xr-x    1 cengiz   users           0 2006-01-11 17:59 hardlinked_file1
drwxr-xr-x    1 cengiz   users        4096 2006-01-11 18:45 hl

./rdiff-backup.tmp.0/hl:
total 0
> rm -r rdiff-backup.tmp.0
rm: cannot remove directory `rdiff-backup.tmp.0': Device or resource busy
> ls -la rdiff-backup.tmp.0/
total 8
drwxr-xr-x    1 cengiz   users        4096 2006-01-11 18:46 .
drwxr-xr-x    1 cengiz   users        4096 2006-01-11 18:45 ..
---

As you can see, after I attempted deleting the .0 directory, the insides
were wiped out. But it still didn't want to remove the directory itself.  
Is it possible the program is currently sitting in that directory and
making it busy? Once I Ctrl-C rdiff-backup out of the loop, I am able to
delete the directory.

Cengiz Gunay
--
[hidden email]  [hidden email] (MSN) [hidden email]
Lab: +1-404-727-3565 Home/Cell: +1-678-559-8694
http://userwww.service.emory.edu/~cgunay/
IMs: ICQ# 21104923, cengique@{jabber.org,yahoo.com,Skype}
--


_______________________________________________
rdiff-backup-users mailing list at [hidden email]
http://lists.nongnu.org/mailman/listinfo/rdiff-backup-users
Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki
Reply | Threaded
Open this post in threaded view
|

Re: rdiff to SMB share?

Ben Escoto
>>>>> Cengiz Gunay <[hidden email]>
>>>>> wrote the following on Wed, 11 Jan 2006 18:07:38 -0500 (EST)
>
> Ok, I added the endless loop, and looking in the rdiff-backup-data, I see
> this:
...

> > rm -r rdiff-backup.tmp.0
> rm: cannot remove directory `rdiff-backup.tmp.0': Device or resource busy
> > ls -la rdiff-backup.tmp.0/
> total 8
> drwxr-xr-x    1 cengiz   users        4096 2006-01-11 18:46 .
> drwxr-xr-x    1 cengiz   users        4096 2006-01-11 18:45 ..
> ---
>
> As you can see, after I attempted deleting the .0 directory, the
> insides were wiped out. But it still didn't want to remove the
> directory itself.  Is it possible the program is currently sitting
> in that directory and making it busy? Once I Ctrl-C rdiff-backup out
> of the loop, I am able to delete the directory.
rdiff-backup never chdirs, so I don't think that is the problem.

Do if you just try to rmdir a directory that has files in it, do you
get that same "Device or resource busy" error?  There was a similar
bug only fixed recently, where rdiff-backup would open a file, and
then delete it.  On some systems (like Linux) you're allowed to then
remove the directory, but sometimes this causes an error.

If that's what's happening you can try using lsof (or maybe strace) to
see if rdiff-backup is leaving any files open when it tries to remove
the directory.  I looked through the relevant code for about 5 minutes
and it seems to me all the files get closed.

But if removing a non-empty directory gives you the more common
"Directory not empty" (ENOTEMPTY) error, leaving files open may not be
the problem.  Maybe someone else could tell you why trying to remove
an empty SMB directory would raise an EBUSY.


--
Ben Escoto

_______________________________________________
rdiff-backup-users mailing list at [hidden email]
http://lists.nongnu.org/mailman/listinfo/rdiff-backup-users
Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki

attachment0 (196 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: rdiff to SMB share?

Cengiz Günay

On Wed, 11 Jan 2006, Ben Escoto wrote:

> ...
> Do if you just try to rmdir a directory that has files in it, do you
> get that same "Device or resource busy" error?  ...

Nope, I get "directory not empty".

> If that's what's happening you can try using lsof (or maybe strace) to
> see if rdiff-backup is leaving any files open when it tries to remove
> the directory.  I looked through the relevant code for about 5 minutes
> and it seems to me all the files get closed.

You were right on this. I was looking at the /proc/.../fd for open files,
but lsof (I didn't know about this one!) is a better way to visualize it:

---
> lsof rdiff-backup.tmp.4
COMMAND   PID   USER   FD   TYPE DEVICE SIZE NODE NAME
python  24765 cengiz    3r   DIR    0,0 4096 4920 rdiff-backup.tmp.4
---

And that is the directory it is about to remove after the loop. I can give
you some clues on where it may be in the code by looking at the strace
output:

---
...
open("/Lab/cengiz/emory/cgunay/home_backup/rdiff-backup-data/rdiff-backup.tmp.4/
hardlinked_file1", O_WRONLY|O_CREAT|O_TRUNC, 0666) = 3
fstat(3, {st_mode=S_IFREG|0755, st_size=0, ...}) = 0
close(3)                                = 0
lstat("/Lab/cengiz/emory/cgunay/home_backup/rdiff-backup-data/rdiff-backup.tmp.4
/hardlinked_file1", {st_mode=S_IFREG|0755, st_size=0, ...}) = 0
link("/Lab/cengiz/emory/cgunay/home_backup/rdiff-backup-data/rdiff-backup.tmp.4/
hardlinked_file1",
"/Lab/cengiz/emory/cgunay/home_backup/rdiff-backup-data/rdiff
-backup.tmp.4/hl/hardlinked_file2") = -1 EPERM (Operation not permitted)
open("/Lab/cengiz/emory/cgunay/home_backup/rdiff-backup-data/rdiff-backup.tmp.4"
, O_RDONLY) = 3
fsync(3)                                = -1 EINVAL (Invalid argument)
...
---

So rdiff-backup opens a handle to the directory itself, right after doing
the hardlink test within the directory. This handle is never closed until
the point we're interested in.

> But if removing a non-empty directory gives you the more common
> "Directory not empty" (ENOTEMPTY) error, leaving files open may not be
> the problem.  Maybe someone else could tell you why trying to remove
> an empty SMB directory would raise an EBUSY.

If you can tell me where to put the close function, I can test this
easily.

Cengiz Gunay
--
[hidden email]  [hidden email] (MSN) [hidden email]
Lab: +1-404-727-3565 Home/Cell: +1-678-559-8694
http://userwww.service.emory.edu/~cgunay/
IMs: ICQ# 21104923, cengique@{jabber.org,yahoo.com,Skype}
--


_______________________________________________
rdiff-backup-users mailing list at [hidden email]
http://lists.nongnu.org/mailman/listinfo/rdiff-backup-users
Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki
Reply | Threaded
Open this post in threaded view
|

Re: rdiff to SMB share?

Ben Escoto
>>>>> Cengiz Gunay <[hidden email]>
>>>>> wrote the following on Thu, 12 Jan 2006 11:55:12 -0500 (EST)
>
> So rdiff-backup opens a handle to the directory itself, right after doing
> the hardlink test within the directory. This handle is never closed until
> the point we're interested in.

Great, I think this should work now:

--- rdiff_backup/rpath.py       16 Dec 2005 20:25:58 -0000      1.98
+++ rdiff_backup/rpath.py       13 Jan 2006 05:26:10 -0000
@@ -1148,6 +1148,7 @@
                        os.fsync(fd)
                        os.close(fd)
                except OSError, e:
+                       if locals().has_key('fd'): os.close(fd)
                        if e.errno != errno.EPERM or self.isdir(): raise
 
                        # Maybe the system doesn't like read-only fsyncing.


--
Ben Escoto

_______________________________________________
rdiff-backup-users mailing list at [hidden email]
http://lists.nongnu.org/mailman/listinfo/rdiff-backup-users
Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki

attachment0 (196 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: rdiff to SMB share?

Cengiz Günay
On Thu, 12 Jan 2006, Ben Escoto wrote:

> Great, I think this should work now:
> ...
> +                       if locals().has_key('fd'): os.close(fd)

Thanks! It seems like we've gone past the sanity-checking part. But
unfortunately I had one more error before finishing the backup. I happened
to have a symlink in the source directory, and it gives this error when
processing the file:

---
Regular copying ('subpixelview',) to
/Lab/cengiz/emory/cgunay/home_backup/rdiff-backup.tmp.4
Exception '[Errno 1] Operation not permitted' raised of class
'exceptions.OSError':
  File "/usr/src/rdiff-backup/rdiff_backup/robust.py", line 32, in
check_common_error
    try: return function(*args)
  File "/usr/src/rdiff-backup/rdiff_backup/rpath.py", line 103, in copy
    elif rpin.issym(): rpout.symlink(rpin.readlink())
  File "/usr/src/rdiff-backup/rdiff_backup/rpath.py", line 882, in symlink
    self.conn.os.symlink(linktext, self.path)

SpecialFileError subpixelview [Errno 1] Operation not permitted
UpdateError subpixelview Updated mirror temp file
/Lab/cengiz/emory/cgunay/home_backup/rdiff-backup.tmp.4 does not match
source
Writing mirror marker
/Lab/cengiz/emory/cgunay/home_backup/rdiff-backup-data/current_mirror.2006-01-13;08414;05833;05844-05;05800.data
Cleaning up
---

rdiff-backup is checking the destination directory properties. Isn't it
also checking if the target supports symlinking? I got this list at the
beginning:
-----------------------------------------------------------------
Detected abilities for destination (read/write) file system:
  Ownership changing                           Off
  Hard linking                                 Off
  fsync() directories                          Off
  Directory inc permissions                    Off
  High-bit permissions                         Off
  Extended filenames                           Off
  Access control lists                         Off
  Extended attributes                          Off
  Case sensitivity                             Off
  Mac OS X style resource forks                Off
  Mac OS X Finder information                  Off
-----------------------------------------------------------------

It says hard linking, but nothing about symlinking..

Cengiz Gunay
--
[hidden email]  [hidden email] (MSN) [hidden email]
Lab: +1-404-727-3565 Home/Cell: +1-678-559-8694
http://userwww.service.emory.edu/~cgunay/
IMs: ICQ# 21104923, cengique@{jabber.org,yahoo.com,Skype}
--


_______________________________________________
rdiff-backup-users mailing list at [hidden email]
http://lists.nongnu.org/mailman/listinfo/rdiff-backup-users
Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki
Reply | Threaded
Open this post in threaded view
|

Re: rdiff to SMB share?

Ben Escoto
>>>>> Cengiz Gunay <[hidden email]>
>>>>> wrote the following on Fri, 13 Jan 2006 19:24:52 -0500 (EST)
>
> Thanks! It seems like we've gone past the sanity-checking part. But
> unfortunately I had one more error before finishing the backup. I happened
> to have a symlink in the source directory, and it gives this error when
> processing the file:
...
> SpecialFileError subpixelview [Errno 1] Operation not permitted
> UpdateError subpixelview Updated mirror temp file
...
> rdiff-backup is checking the destination directory properties. Isn't it
> also checking if the target supports symlinking? I got this list at the
> beginning:
...
> It says hard linking, but nothing about symlinking..

It looks like your backup completed successfully.  For various
reasons, there is no test to see if various special files can be made.
Instead, failures to make special files like symlinks should raise
SpecialFileErrors.  See

http://rdiff-backup.nongnu.org/error_policy.html

for more information.  So getting that SpecialFileError is the
intended behavior.  The only problem is that you also get an
UpdateError, which is unnecessary, but doesn't seem like a very
serious flaw.

Does the symlink restore correctly?


--
Ben Escoto

_______________________________________________
rdiff-backup-users mailing list at [hidden email]
http://lists.nongnu.org/mailman/listinfo/rdiff-backup-users
Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki

attachment0 (196 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: rdiff to SMB share?

Cengiz Günay
On Fri, 13 Jan 2006, Ben Escoto wrote:

> for more information.  So getting that SpecialFileError is the
> intended behavior.  The only problem is that you also get an
> UpdateError, which is unnecessary, but doesn't seem like a very
> serious flaw.

Ok, that's fine.

> Does the symlink restore correctly?

No, but the other files are restored correctly. Thanks a lot for fixing
this! FYI, here's the log from the restore, it doesn't mention the
symlinked file:

---
> PYTHONPATH=$RDB_CVS $RDB_CVS/rdiff-backup -v6 -r 3m
~/emory/cgunay/home_backup/ tmp/
Using mirror root directory /Lab/cengiz/emory/cgunay/home_backup
Unable to import module xattr.
Extended attributes not supported on filesystem at
/Lab/cengiz/emory/cgunay/home_backup/rdiff-backup-data
Unable to import module posix1e from pylibacl package.
ACLs not supported on filesystem at
/Lab/cengiz/emory/cgunay/home_backup/rdiff-backup-data
-----------------------------------------------------------------
Detected abilities for rdiff-backup repository (read only) file system:
  Access control lists                         Off
  Extended attributes                          Off
  Case sensitivity                             Off
  Mac OS X style resource forks                Off
  Mac OS X Finder information                  Off
-----------------------------------------------------------------
Making directory tmp
Making directory tmp/rdiff-backup.tmp.0
Making directory tmp/rdiff-backup.tmp.0/hl
Hard linking tmp/rdiff-backup.tmp.0/hl/hardlinked_file2 to
tmp/rdiff-backup.tmp.0/hardlinked_file1
Unable to import module xattr.
Extended attributes not supported on filesystem at
tmp/rdiff-backup.tmp.0
Unable to import module posix1e from pylibacl package.
ACLs not supported on filesystem at tmp/rdiff-backup.tmp.0
Removing directory tmp/rdiff-backup.tmp.0
-----------------------------------------------------------------
Detected abilities for restore target (read/write) file system:
  Ownership changing                           Off
  Hard linking                                 On
  fsync() directories                          On
  Directory inc permissions                    On
  High-bit permissions                         On
  Extended filenames                           On
  Access control lists                         Off
  Extended attributes                          Off
  Case sensitivity                             On
  Mac OS X style resource forks                Off
  Mac OS X Finder information                  Off
-----------------------------------------------------------------
Starting restore of /Lab/cengiz/emory/cgunay/home_backup to tmp as it
was as of Sat Jan 14 13:25:20 2006.
Processing changed file .
Processing changed file makedvd
Regular copying ('makedvd',) to tmp/rdiff-backup.tmp.1
Hash cff864ef43e09329553bed5a42e632e8ac4d6fef of makedvd verified
Processing changed file mount_emory
Regular copying ('mount_emory',) to tmp/rdiff-backup.tmp.2
Hash 71c7c14c02eb8605336e1d30eed6535fcdb8e3e8 of mount_emory verified
Processing changed file mountwww
Regular copying ('mountwww',) to tmp/rdiff-backup.tmp.3
Hash f25a502bbf6f4c2774f91c86db33ce3eaa46b5ec of mountwww verified
Restore finished
Cleaning up
cengiz@laotse:~/tmp> ls -al tmp/
total 16
drwxr-xr-x    2 cengiz   users         128 2006-01-06 14:05 .
drwxr-xr-x    9 cengiz   users        4440 2006-01-14 13:28 ..
-rwxr-xr-x    1 cengiz   users         590 2005-12-13 18:25 makedvd
-rwxr-xr-x    1 cengiz   users          80 2006-01-06 14:05 mount_emory
-rwxr-xr-x    1 cengiz   users         148 2004-11-23 15:22 mountwww
---

Here's the listing of the original location:
---
> ls -la ~/bin
total 19
drwxr-xr-x    2 cengiz   users         160 2006-01-06 14:05 .
drwxr-xr-x   94 cengiz   users        7320 2006-01-14 13:29 ..
-rwxr-xr-x    1 cengiz   users         590 2005-12-13 18:25 makedvd
-rwxr-xr-x    1 cengiz   users          80 2006-01-06 14:05 mount_emory
-rwxr-xr-x    1 cengiz   users         148 2004-11-23 15:22 mountwww
lrwxrwxrwx    1 cengiz   users          42 2005-10-25 12:56
subpixelview -> /Lab/cengiz/work/subpixelJava/subpixelview
---

Cengiz Gunay
--
[hidden email]  [hidden email] (MSN) [hidden email]
Lab: +1-404-727-3565 Home/Cell: +1-678-559-8694
http://userwww.service.emory.edu/~cgunay/
IMs: ICQ# 21104923, cengique@{jabber.org,yahoo.com,Skype}
--


_______________________________________________
rdiff-backup-users mailing list at [hidden email]
http://lists.nongnu.org/mailman/listinfo/rdiff-backup-users
Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki
Reply | Threaded
Open this post in threaded view
|

Re: rdiff to SMB share?

Ben Escoto
>>>>> Cengiz Gunay <[hidden email]>
>>>>> wrote the following on Sat, 14 Jan 2006 13:32:41 -0500 (EST)
>
> > Does the symlink restore correctly?
>
> No, but the other files are restored correctly.

Ok, update CVS or try the patch at:

http://cvs.savannah.nongnu.org/viewcvs/rdiff-backup/rdiff_backup/backup.py?root=rdiff-backup&r1=1.39&r2=1.40

I hope that should get rid of the UpdateError.


--
Ben Escoto

_______________________________________________
rdiff-backup-users mailing list at [hidden email]
http://lists.nongnu.org/mailman/listinfo/rdiff-backup-users
Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki

attachment0 (196 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: rdiff to SMB share?

Cengiz Günay
On Sat, 14 Jan 2006, Ben Escoto wrote:

> I hope that should get rid of the UpdateError.

Yes, it did. I can backup and restore the symlink perfectly. The
UpdateError during backup is replaced with this error:
"SpecialFileError subpixelview [Errno 1] Operation not permitted"

Thanks again..
Cengiz Gunay
--
[hidden email]  [hidden email] (MSN) [hidden email]
Lab: +1-404-727-3565 Home/Cell: +1-678-559-8694
http://userwww.service.emory.edu/~cgunay/
IMs: ICQ# 21104923, cengique@{jabber.org,yahoo.com,Skype}
--


_______________________________________________
rdiff-backup-users mailing list at [hidden email]
http://lists.nongnu.org/mailman/listinfo/rdiff-backup-users
Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki