Errors during test phase of installation.....

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
28 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Errors during test phase of installation.....

duplicity-talk mailing list
I maintain the fink installer for duplicity and so far it seems that it installs and runs correctly.  However since duplicity has an extensive test suite when installing it runs that test suite in “maintainer mode” for those of us who maintain the packages.

I get a *lot* of errors when running the test suite with duplicity 0.7.11.  At least the test suite runs. :-)

One source may be permissions to run the tests and hardwired folders, but I don’t see that from the error file.  Or permissions to do these tests.  It seems to have an issue with some sort of “attempt failed” but not clear why or what it was attempting.

Here is the output from the testing.   If anything pops out as the issue let me know.  The software is built and tested in the directory /sw/src/fink.build/ which should have read/write privileges for the testing.

-Scott


/sw/bin/python2.7 setup.py test
running test
running egg_info
creating duplicity.egg-info
writing requirements to duplicity.egg-info/requires.txt
writing duplicity.egg-info/PKG-INFO
writing top-level names to duplicity.egg-info/top_level.txt
writing dependency_links to duplicity.egg-info/dependency_links.txt
writing manifest file 'duplicity.egg-info/SOURCES.txt'
reading manifest file 'duplicity.egg-info/SOURCES.txt'
writing manifest file 'duplicity.egg-info/SOURCES.txt'
running build_ext
copying build/lib.macosx-10.11-x86_64-2.7/duplicity/_librsync.so -> duplicity
Attempt 1 failed. Exception:
Attempt 1 failed. Exception:
Attempt 1 failed. Exception:
Attempt 1 failed. Exception:
Attempt 1 failed. Exception:
Attempt 2 failed. Exception:
Attempt 1 failed. Exception:
Attempt 2 failed. Exception:
Attempt 1 failed. Exception:
Attempt 2 failed. Exception:
Attempt 1 failed. Exception:
Attempt 2 failed. Exception:
Attempt 1 failed. Exception:
Warning, found the following remote orphaned signature file:
duplicity-new-signatures.2001-08-17T02:05:13-05:00.to.2002-08-17T05:05:14-05:00.sigtar.gpg
Warning, found signatures but no corresponding backup files
Warning, found incomplete backup sets, probably left from aborted session
Warning, found the following orphaned backup file:
[duplicity-inc.2000-08-17T16:17:01-07:00.to.2000-08-18T00:04:30-07:00.manifest.gpg, duplicity-inc.2000-08-17T16:17:01-07:00.to.2000-08-18T00:04:30-07:00.vol1.difftar.gpg]
Warning, found the following remote orphaned signature file:
duplicity-new-signatures.2001-08-17T02:05:13-05:00.to.2002-08-17T05:05:14-05:00.sigtar.gpg
Warning, found signatures but no corresponding backup files
Warning, found incomplete backup sets, probably left from aborted session
Warning, found the following orphaned backup file:
[duplicity-inc.2000-08-17T16:17:01-07:00.to.2000-08-18T00:04:30-07:00.manifest.gpg, duplicity-inc.2000-08-17T16:17:01-07:00.to.2000-08-18T00:04:30-07:00.vol1.difftar.gpg]
Warning: base testfiles/dir3 doesn't exist, continuing
Warning: foo has negative mtime, treating as 0.
Warning: base testfiles/various_file_types doesn't exist, continuing
(('largefile',) testfiles/output/sequence/largefile reg) not equal to (('largefile',) testfiles/dir1/largefile reg)
Warning: base testfiles/select doesn't exist, continuing
Warning: base testfiles/select doesn't exist, continuing
Warning: base testfiles/select2/1/1sub1 doesn't exist, continuing
### execution of /sw/bin/python2.7 failed, exit code 1
### execution of /tmp/fink.vX9FE failed, exit code 1


_______________________________________________
Duplicity-talk mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/duplicity-talk
Reply | Threaded
Open this post in threaded view
|

Re: Errors during test phase of installation.....

duplicity-talk mailing list
Those look like the normal errors we force for testing.  Unless the tests end with ERROR or FAIL, all is good.

...Thanks,
...Ken


On Sun, Jan 15, 2017 at 10:04 AM, Scott Hannahs via Duplicity-talk <[hidden email]> wrote:
I maintain the fink installer for duplicity and so far it seems that it installs and runs correctly.  However since duplicity has an extensive test suite when installing it runs that test suite in “maintainer mode” for those of us who maintain the packages.

I get a *lot* of errors when running the test suite with duplicity 0.7.11.  At least the test suite runs. :-)

One source may be permissions to run the tests and hardwired folders, but I don’t see that from the error file.  Or permissions to do these tests.  It seems to have an issue with some sort of “attempt failed” but not clear why or what it was attempting.

Here is the output from the testing.   If anything pops out as the issue let me know.  The software is built and tested in the directory /sw/src/fink.build/ which should have read/write privileges for the testing.

-Scott


/sw/bin/python2.7 setup.py test
running test
running egg_info
creating duplicity.egg-info
writing requirements to duplicity.egg-info/requires.txt
writing duplicity.egg-info/PKG-INFO
writing top-level names to duplicity.egg-info/top_level.txt
writing dependency_links to duplicity.egg-info/dependency_links.txt
writing manifest file 'duplicity.egg-info/SOURCES.txt'
reading manifest file 'duplicity.egg-info/SOURCES.txt'
writing manifest file 'duplicity.egg-info/SOURCES.txt'
running build_ext
copying build/lib.macosx-10.11-x86_64-2.7/duplicity/_librsync.so -> duplicity
Attempt 1 failed. Exception:
Attempt 1 failed. Exception:
Attempt 1 failed. Exception:
Attempt 1 failed. Exception:
Attempt 1 failed. Exception:
Attempt 2 failed. Exception:
Attempt 1 failed. Exception:
Attempt 2 failed. Exception:
Attempt 1 failed. Exception:
Attempt 2 failed. Exception:
Attempt 1 failed. Exception:
Attempt 2 failed. Exception:
Attempt 1 failed. Exception:
Warning, found the following remote orphaned signature file:
duplicity-new-signatures.2001-08-17T02:05:13-05:00.to.2002-08-17T05:05:14-05:00.sigtar.gpg
Warning, found signatures but no corresponding backup files
Warning, found incomplete backup sets, probably left from aborted session
Warning, found the following orphaned backup file:
[duplicity-inc.2000-08-17T16:17:01-07:00.to.2000-08-18T00:04:30-07:00.manifest.gpg, duplicity-inc.2000-08-17T16:17:01-07:00.to.2000-08-18T00:04:30-07:00.vol1.difftar.gpg]
Warning, found the following remote orphaned signature file:
duplicity-new-signatures.2001-08-17T02:05:13-05:00.to.2002-08-17T05:05:14-05:00.sigtar.gpg
Warning, found signatures but no corresponding backup files
Warning, found incomplete backup sets, probably left from aborted session
Warning, found the following orphaned backup file:
[duplicity-inc.2000-08-17T16:17:01-07:00.to.2000-08-18T00:04:30-07:00.manifest.gpg, duplicity-inc.2000-08-17T16:17:01-07:00.to.2000-08-18T00:04:30-07:00.vol1.difftar.gpg]
Warning: base testfiles/dir3 doesn't exist, continuing
Warning: foo has negative mtime, treating as 0.
Warning: base testfiles/various_file_types doesn't exist, continuing
(('largefile',) testfiles/output/sequence/largefile reg) not equal to (('largefile',) testfiles/dir1/largefile reg)
Warning: base testfiles/select doesn't exist, continuing
Warning: base testfiles/select doesn't exist, continuing
Warning: base testfiles/select2/1/1sub1 doesn't exist, continuing
### execution of /sw/bin/python2.7 failed, exit code 1
### execution of /tmp/fink.vX9FE failed, exit code 1


_______________________________________________
Duplicity-talk mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/duplicity-talk


_______________________________________________
Duplicity-talk mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/duplicity-talk
Reply | Threaded
Open this post in threaded view
|

Re: Errors during test phase of installation.....

duplicity-talk mailing list
Ok, but the standard error out has 581 k lines of output:
6700 lines end with “ERROR”
1000 lines end with “FAIL”
23K lines end with “ok”
37K lines start with “Traceback"

This seems disturbing?  Especially the traceback failures in python.  It is possible that I have not installed some necessary component.

But as I said, my backup script is working every night (with S3 backend).  It just may not be working for a variety of backends?

There are a lot of “directory not empty” failures (about 5K).  But the number of errors in the whole file is somewhat daunting to me at the moment.  But if it is normal, I will move on and publish this installer package.

The whole error log is 36 MB and took 14,000 seconds to run.
It can be downloaded at http://www.p-hall.net/files/duplicity-error.txt

for the record, here is the installed components (and of course their defined dependencies)

duplicity 0.7.11-3
boto-py27 2.36.0-1
gnupg-unified 1.4.21-1
lftp 4.6.5-1
librsync 0.9.7-1006
lockfile-py27 0.12.2-1
paramiko-py27 1.7.6-1
pycryptopp-py27 0.7.1-1
python27 1:2.7.13-1
MacOSX 10.11.6

Here are the first two Traceback reports:
======================================================================
ERROR: test_cleanup_after_partial (testing.functional.test_cleanup.CleanupTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/sw/src/fink.build/duplicity-0.7.11-1/duplicity-0.7.11/testing/functional/test_cleanup.py", line 38, in test_cleanup_after_partial
    good_files = self.backup("full", "testfiles/largefiles")
  File "/sw/src/fink.build/duplicity-0.7.11-1/duplicity-0.7.11/testing/functional/__init__.py", line 143, in backup
    result = self.run_duplicity(options=options, **kwargs)
  File "/sw/src/fink.build/duplicity-0.7.11-1/duplicity-0.7.11/testing/functional/__init__.py", line 130, in run_duplicity
    raise CmdError(return_val)
CmdError: 31

======================================================================
ERROR: test_remove_all_but_n (testing.functional.test_cleanup.CleanupTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/sw/src/fink.build/duplicity-0.7.11-1/duplicity-0.7.11/testing/functional/test_cleanup.py", line 56, in test_remove_all_but_n
    full1_files = self.backup("full", "testfiles/empty_dir")
  File "/sw/src/fink.build/duplicity-0.7.11-1/duplicity-0.7.11/testing/functional/__init__.py", line 143, in backup
    result = self.run_duplicity(options=options, **kwargs)
  File "/sw/src/fink.build/duplicity-0.7.11-1/duplicity-0.7.11/testing/functional/__init__.py", line 130, in run_duplicity
    raise CmdError(return_val)
CmdError: 31

======================================================================


-Scott


> On Jan 16, 2017, at 10:41 AM, Kenneth Loafman <[hidden email]> wrote:
>
> Those look like the normal errors we force for testing.  Unless the tests end with ERROR or FAIL, all is good.
>
> ...Thanks,
> ...Ken
>
>


_______________________________________________
Duplicity-talk mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/duplicity-talk
Reply | Threaded
Open this post in threaded view
|

Re: Errors during test phase of installation.....

duplicity-talk mailing list
Hmm, those errors should have been fixed in 0.7.11 in testing/test_selection.py.TestLockedFoldersNotError.  Mac Sierra made some funky changes that keeps you from deleting a directory with perms of 0o0000, thus causing test cleanup of those two tests to fail, and in general messing up the entire test cycle.

There should be two lines before each test def with:
    @unittest.skipUnless(platform.platform().startswith('Linux'),
                         'Skip on non-Linux systems')

Plus, if you ran the tests already, then the directory testing/testfiles may still be around to screw up the next test run.

...Ken


On Mon, Jan 16, 2017 at 10:30 AM, Scott Hannahs <[hidden email]> wrote:
Ok, but the standard error out has 581 k lines of output:
6700 lines end with “ERROR”
1000 lines end with “FAIL”
23K lines end with “ok”
37K lines start with “Traceback"

This seems disturbing?  Especially the traceback failures in python.  It is possible that I have not installed some necessary component.

But as I said, my backup script is working every night (with S3 backend).  It just may not be working for a variety of backends?

There are a lot of “directory not empty” failures (about 5K).  But the number of errors in the whole file is somewhat daunting to me at the moment.  But if it is normal, I will move on and publish this installer package.

The whole error log is 36 MB and took 14,000 seconds to run.
It can be downloaded at http://www.p-hall.net/files/duplicity-error.txt

for the record, here is the installed components (and of course their defined dependencies)

duplicity                       0.7.11-3
boto-py27               2.36.0-1
gnupg-unified           1.4.21-1
lftp                            4.6.5-1
librsync                        0.9.7-1006
lockfile-py27           0.12.2-1
paramiko-py27   1.7.6-1
pycryptopp-py27 0.7.1-1
python27                        1:2.7.13-1
MacOSX                  10.11.6

Here are the first two Traceback reports:
======================================================================
ERROR: test_cleanup_after_partial (testing.functional.test_cleanup.CleanupTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/sw/src/fink.build/duplicity-0.7.11-1/duplicity-0.7.11/testing/functional/test_cleanup.py", line 38, in test_cleanup_after_partial
    good_files = self.backup("full", "testfiles/largefiles")
  File "/sw/src/fink.build/duplicity-0.7.11-1/duplicity-0.7.11/testing/functional/__init__.py", line 143, in backup
    result = self.run_duplicity(options=options, **kwargs)
  File "/sw/src/fink.build/duplicity-0.7.11-1/duplicity-0.7.11/testing/functional/__init__.py", line 130, in run_duplicity
    raise CmdError(return_val)
CmdError: 31

======================================================================
ERROR: test_remove_all_but_n (testing.functional.test_cleanup.CleanupTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/sw/src/fink.build/duplicity-0.7.11-1/duplicity-0.7.11/testing/functional/test_cleanup.py", line 56, in test_remove_all_but_n
    full1_files = self.backup("full", "testfiles/empty_dir")
  File "/sw/src/fink.build/duplicity-0.7.11-1/duplicity-0.7.11/testing/functional/__init__.py", line 143, in backup
    result = self.run_duplicity(options=options, **kwargs)
  File "/sw/src/fink.build/duplicity-0.7.11-1/duplicity-0.7.11/testing/functional/__init__.py", line 130, in run_duplicity
    raise CmdError(return_val)
CmdError: 31

======================================================================


-Scott


> On Jan 16, 2017, at 10:41 AM, Kenneth Loafman <[hidden email]> wrote:
>
> Those look like the normal errors we force for testing.  Unless the tests end with ERROR or FAIL, all is good.
>
> ...Thanks,
> ...Ken
>
>



_______________________________________________
Duplicity-talk mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/duplicity-talk
Reply | Threaded
Open this post in threaded view
|

Re: Errors during test phase of installation.....

duplicity-talk mailing list
Ken,

The fink installation/testing system uses a sandbox directory and blows away the whole thing afterwards.  So there shouldn’t be a previous testing directory unless specifically requested to keep the build folders.  I think it creates those testing directories and files in the build directory structure.

This is on Mac OS X El Capitan (10.11).  I haven’t tried to move to Sierra (10.12) quite yet.  Though I think I can test that next week on another system if that is helpful.  First I was going to try to reduce these errors to a manageable amount on El Cap.

Those two lines to skip some tests are in the code as they should be.  I don’t quite see why the “TestLockedFoldersNotError” is related to the errors posted “test_cleanup_after_partial” and “test_remove_all_but_n” where duplicity crashed.

-Scott


On Jan 16, 2017, at 11:45 AM, Kenneth Loafman <[hidden email]> wrote:

Hmm, those errors should have been fixed in 0.7.11 in testing/test_selection.py.TestLockedFoldersNotError.  Mac Sierra made some funky changes that keeps you from deleting a directory with perms of 0o0000, thus causing test cleanup of those two tests to fail, and in general messing up the entire test cycle.

There should be two lines before each test def with:
    @unittest.skipUnless(platform.platform().startswith('Linux'),
                         'Skip on non-Linux systems')

Plus, if you ran the tests already, then the directory testing/testfiles may still be around to screw up the next test run.

...Ken


On Mon, Jan 16, 2017 at 10:30 AM, Scott Hannahs <[hidden email]> wrote:
Ok, but the standard error out has 581 k lines of output:
6700 lines end with “ERROR”
1000 lines end with “FAIL”
23K lines end with “ok”
37K lines start with “Traceback"

This seems disturbing?  Especially the traceback failures in python.  It is possible that I have not installed some necessary component.

But as I said, my backup script is working every night (with S3 backend).  It just may not be working for a variety of backends?

There are a lot of “directory not empty” failures (about 5K).  But the number of errors in the whole file is somewhat daunting to me at the moment.  But if it is normal, I will move on and publish this installer package.

The whole error log is 36 MB and took 14,000 seconds to run.
It can be downloaded at http://www.p-hall.net/files/duplicity-error.txt

for the record, here is the installed components (and of course their defined dependencies)

duplicity                       0.7.11-3
boto-py27               2.36.0-1
gnupg-unified           1.4.21-1
lftp                            4.6.5-1
librsync                        0.9.7-1006
lockfile-py27           0.12.2-1
paramiko-py27   1.7.6-1
pycryptopp-py27 0.7.1-1
python27                        1:2.7.13-1
MacOSX                  10.11.6

Here are the first two Traceback reports:
======================================================================
ERROR: test_cleanup_after_partial (testing.functional.test_cleanup.CleanupTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/sw/src/fink.build/duplicity-0.7.11-1/duplicity-0.7.11/testing/functional/test_cleanup.py", line 38, in test_cleanup_after_partial
    good_files = self.backup("full", "testfiles/largefiles")
  File "/sw/src/fink.build/duplicity-0.7.11-1/duplicity-0.7.11/testing/functional/__init__.py", line 143, in backup
    result = self.run_duplicity(options=options, **kwargs)
  File "/sw/src/fink.build/duplicity-0.7.11-1/duplicity-0.7.11/testing/functional/__init__.py", line 130, in run_duplicity
    raise CmdError(return_val)
CmdError: 31

======================================================================
ERROR: test_remove_all_but_n (testing.functional.test_cleanup.CleanupTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/sw/src/fink.build/duplicity-0.7.11-1/duplicity-0.7.11/testing/functional/test_cleanup.py", line 56, in test_remove_all_but_n
    full1_files = self.backup("full", "testfiles/empty_dir")
  File "/sw/src/fink.build/duplicity-0.7.11-1/duplicity-0.7.11/testing/functional/__init__.py", line 143, in backup
    result = self.run_duplicity(options=options, **kwargs)
  File "/sw/src/fink.build/duplicity-0.7.11-1/duplicity-0.7.11/testing/functional/__init__.py", line 130, in run_duplicity
    raise CmdError(return_val)
CmdError: 31

======================================================================


-Scott


> On Jan 16, 2017, at 10:41 AM, Kenneth Loafman <[hidden email]> wrote:
>
> Those look like the normal errors we force for testing.  Unless the tests end with ERROR or FAIL, all is good.
>
> ...Thanks,
> ...Ken
>
>




_______________________________________________
Duplicity-talk mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/duplicity-talk
Reply | Threaded
Open this post in threaded view
|

Re: Errors during test phase of installation.....

duplicity-talk mailing list
Any way to check to see if the testing/testfiles directory is being left around after a test?  Maybe run it manually?  If so, do a manual 'rm -r testing/testfiles' and see if El Capitan complains.  That will go a long way to find out what's going on.
 

On Mon, Jan 16, 2017 at 12:05 PM, Scott Hannahs <[hidden email]> wrote:
Ken,

The fink installation/testing system uses a sandbox directory and blows away the whole thing afterwards.  So there shouldn’t be a previous testing directory unless specifically requested to keep the build folders.  I think it creates those testing directories and files in the build directory structure.

This is on Mac OS X El Capitan (10.11).  I haven’t tried to move to Sierra (10.12) quite yet.  Though I think I can test that next week on another system if that is helpful.  First I was going to try to reduce these errors to a manageable amount on El Cap.

Those two lines to skip some tests are in the code as they should be.  I don’t quite see why the “TestLockedFoldersNotError” is related to the errors posted “test_cleanup_after_partial” and “test_remove_all_but_n” where duplicity crashed.

-Scott



On Jan 16, 2017, at 11:45 AM, Kenneth Loafman <[hidden email]> wrote:

Hmm, those errors should have been fixed in 0.7.11 in testing/test_selection.py.TestLockedFoldersNotError.  Mac Sierra made some funky changes that keeps you from deleting a directory with perms of 0o0000, thus causing test cleanup of those two tests to fail, and in general messing up the entire test cycle.

There should be two lines before each test def with:
    @unittest.skipUnless(platform.platform().startswith('Linux'),
                         'Skip on non-Linux systems')

Plus, if you ran the tests already, then the directory testing/testfiles may still be around to screw up the next test run.

...Ken


On Mon, Jan 16, 2017 at 10:30 AM, Scott Hannahs <[hidden email]> wrote:
Ok, but the standard error out has 581 k lines of output:
6700 lines end with “ERROR”
1000 lines end with “FAIL”
23K lines end with “ok”
37K lines start with “Traceback"

This seems disturbing?  Especially the traceback failures in python.  It is possible that I have not installed some necessary component.

But as I said, my backup script is working every night (with S3 backend).  It just may not be working for a variety of backends?

There are a lot of “directory not empty” failures (about 5K).  But the number of errors in the whole file is somewhat daunting to me at the moment.  But if it is normal, I will move on and publish this installer package.

The whole error log is 36 MB and took 14,000 seconds to run.
It can be downloaded at http://www.p-hall.net/files/duplicity-error.txt

for the record, here is the installed components (and of course their defined dependencies)

duplicity                       0.7.11-3
boto-py27               2.36.0-1
gnupg-unified           1.4.21-1
lftp                            4.6.5-1
librsync                        0.9.7-1006
lockfile-py27           0.12.2-1
paramiko-py27   1.7.6-1
pycryptopp-py27 0.7.1-1
python27                        1:2.7.13-1
MacOSX                  10.11.6

Here are the first two Traceback reports:
======================================================================
ERROR: test_cleanup_after_partial (testing.functional.test_cleanup.CleanupTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/sw/src/fink.build/duplicity-0.7.11-1/duplicity-0.7.11/testing/functional/test_cleanup.py", line 38, in test_cleanup_after_partial
    good_files = self.backup("full", "testfiles/largefiles")
  File "/sw/src/fink.build/duplicity-0.7.11-1/duplicity-0.7.11/testing/functional/__init__.py", line 143, in backup
    result = self.run_duplicity(options=options, **kwargs)
  File "/sw/src/fink.build/duplicity-0.7.11-1/duplicity-0.7.11/testing/functional/__init__.py", line 130, in run_duplicity
    raise CmdError(return_val)
CmdError: 31

======================================================================
ERROR: test_remove_all_but_n (testing.functional.test_cleanup.CleanupTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/sw/src/fink.build/duplicity-0.7.11-1/duplicity-0.7.11/testing/functional/test_cleanup.py", line 56, in test_remove_all_but_n
    full1_files = self.backup("full", "testfiles/empty_dir")
  File "/sw/src/fink.build/duplicity-0.7.11-1/duplicity-0.7.11/testing/functional/__init__.py", line 143, in backup
    result = self.run_duplicity(options=options, **kwargs)
  File "/sw/src/fink.build/duplicity-0.7.11-1/duplicity-0.7.11/testing/functional/__init__.py", line 130, in run_duplicity
    raise CmdError(return_val)
CmdError: 31

======================================================================


-Scott


> On Jan 16, 2017, at 10:41 AM, Kenneth Loafman <[hidden email]> wrote:
>
> Those look like the normal errors we force for testing.  Unless the tests end with ERROR or FAIL, all is good.
>
> ...Thanks,
> ...Ken
>
>





_______________________________________________
Duplicity-talk mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/duplicity-talk
Reply | Threaded
Open this post in threaded view
|

Re: Errors during test phase of installation.....

duplicity-talk mailing list
Thanks for the pointer.  I can’t remove the directory since it is owned by user/group 1000/1000.  Tar should have set the owner/group to 600/266 which are the default software build group that it executes all the building under.

Let me look into how those file user/group permissions are set.

-Scott

On Jan 16, 2017, at 1:28 PM, Kenneth Loafman <[hidden email]> wrote:

Any way to check to see if the testing/testfiles directory is being left around after a test?  Maybe run it manually?  If so, do a manual 'rm -r testing/testfiles' and see if El Capitan complains.  That will go a long way to find out what's going on.


_______________________________________________
Duplicity-talk mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/duplicity-talk
Reply | Threaded
Open this post in threaded view
|

Re: Errors during test phase of installation.....

duplicity-talk mailing list
In reply to this post by duplicity-talk mailing list
The files seem to have somewhat restrictive execution and r/w access.  But I build and test duplicity as a non privileged user.  It seems to install and remove the testfiles directory structure while running.  But I have 23 failures and 100 errors out of the 418 tests.

The summary is:
Ran 418 tests in 179.702s

FAILED (failures=23, errors=100, skipped=5)
Test failed: <unittest.runner.TextTestResult run=418 errors=100 failures=23>
error: Test failed: <unittest.runner.TextTestResult run=418 errors=100 failures=23>
phase test: warning

And the errors seem to be at the beginning of the tests.  Here is the first N lines of the error output:
test_missing_file (testing.functional.test_badupload.BadUploadTest) ... FAIL
test_cleanup_after_partial (testing.functional.test_cleanup.CleanupTest) ... ERROR
test_remove_all_but_n (testing.functional.test_cleanup.CleanupTest) ... ERROR
test_remove_all_inc_of_but_n (testing.functional.test_cleanup.CleanupTest) ... ERROR
test_asym_cycle (testing.functional.test_final.FinalTest)
Like test_basic_cycle but use asymmetric encryption and signing ... ERROR
test_asym_with_hidden_recipient_cycle (testing.functional.test_final.FinalTest)
Like test_basic_cycle but use asymmetric encryption (hiding key id) and signing ... ERROR
test_basic_cycle (testing.functional.test_final.FinalTest)
Run backup/restore test on basic directories ... ERROR
test_empty_backup (testing.functional.test_final.FinalTest)
Make sure backup works when no files change ... ERROR
test_empty_restore (testing.functional.test_final.FinalTest)
Make sure error raised when restore doesn't match anything ... ERROR
test_long_filenames (testing.functional.test_final.FinalTest)
Test backing up a directory with long filenames in it ... ERROR
test_piped_password (testing.functional.test_final.FinalTest)
Make sure that prompting for a password works ... ERROR
test_remove_older_than (testing.functional.test_final.FinalTest)
Test removing old backup chains ... ERROR
test_single_regfile (testing.functional.test_final.FinalTest)
Test backing and restoring up a single regular file ... ERROR
test_asym_cycle (testing.functional.test_final.OldFilenamesFinalTest)
Like test_basic_cycle but use asymmetric encryption and signing ... ERROR
test_asym_with_hidden_recipient_cycle (testing.functional.test_final.OldFilenamesFinalTest)
Like test_basic_cycle but use asymmetric encryption (hiding key id) and signing ... ERROR
test_basic_cycle (testing.functional.test_final.OldFilenamesFinalTest)
Run backup/restore test on basic directories ... ERROR
test_empty_backup (testing.functional.test_final.OldFilenamesFinalTest)
Make sure backup works when no files change ... ERROR
test_empty_restore (testing.functional.test_final.OldFilenamesFinalTest)
Make sure error raised when restore doesn't match anything ... ERROR
test_long_filenames (testing.functional.test_final.OldFilenamesFinalTest)
Test backing up a directory with long filenames in it ... ERROR
test_piped_password (testing.functional.test_final.OldFilenamesFinalTest)
Make sure that prompting for a password works ... ERROR

Then there are a bunch of tests that are “ok”

Then I get some python failures with traceback:
======================================================================
ERROR: test_cleanup_after_partial (testing.functional.test_cleanup.CleanupTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/sw/src/fink.build/duplicity-0.7.11-3/duplicity-0.7.11/testing/functional/test_cleanup.py", line 38, in test_cleanup_after_partial
    good_files = self.backup("full", "testfiles/largefiles")
  File "/sw/src/fink.build/duplicity-0.7.11-3/duplicity-0.7.11/testing/functional/__init__.py", line 143, in backup
    result = self.run_duplicity(options=options, **kwargs)
  File "/sw/src/fink.build/duplicity-0.7.11-3/duplicity-0.7.11/testing/functional/__init__.py", line 130, in run_duplicity
    raise CmdError(return_val)
CmdError: 37

======================================================================
ERROR: test_remove_all_but_n (testing.functional.test_cleanup.CleanupTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/sw/src/fink.build/duplicity-0.7.11-3/duplicity-0.7.11/testing/functional/test_cleanup.py", line 56, in test_remove_all_but_n
    full1_files = self.backup("full", "testfiles/empty_dir")
  File "/sw/src/fink.build/duplicity-0.7.11-3/duplicity-0.7.11/testing/functional/__init__.py", line 143, in backup
    result = self.run_duplicity(options=options, **kwargs)
  File "/sw/src/fink.build/duplicity-0.7.11-3/duplicity-0.7.11/testing/functional/__init__.py", line 130, in run_duplicity
    raise CmdError(return_val)
CmdError: 37



This just doesn’t seem good to me.

-Scott


On Jan 16, 2017, at 1:28 PM, Kenneth Loafman <[hidden email]> wrote:

Any way to check to see if the testing/testfiles directory is being left around after a test?  Maybe run it manually?  If so, do a manual 'rm -r testing/testfiles' and see if El Capitan complains.  That will go a long way to find out what's going on.
 

On Mon, Jan 16, 2017 at 12:05 PM, Scott Hannahs <[hidden email]> wrote:
Ken,

The fink installation/testing system uses a sandbox directory and blows away the whole thing afterwards.  So there shouldn’t be a previous testing directory unless specifically requested to keep the build folders.  I think it creates those testing directories and files in the build directory structure.

This is on Mac OS X El Capitan (10.11).  I haven’t tried to move to Sierra (10.12) quite yet.  Though I think I can test that next week on another system if that is helpful.  First I was going to try to reduce these errors to a manageable amount on El Cap.

Those two lines to skip some tests are in the code as they should be.  I don’t quite see why the “TestLockedFoldersNotError” is related to the errors posted “test_cleanup_after_partial” and “test_remove_all_but_n” where duplicity crashed.

-Scott



On Jan 16, 2017, at 11:45 AM, Kenneth Loafman <[hidden email]> wrote:

Hmm, those errors should have been fixed in 0.7.11 in testing/test_selection.py.TestLockedFoldersNotError.  Mac Sierra made some funky changes that keeps you from deleting a directory with perms of 0o0000, thus causing test cleanup of those two tests to fail, and in general messing up the entire test cycle.

There should be two lines before each test def with:
    @unittest.skipUnless(platform.platform().startswith('Linux'),
                         'Skip on non-Linux systems')

Plus, if you ran the tests already, then the directory testing/testfiles may still be around to screw up the next test run.

...Ken


On Mon, Jan 16, 2017 at 10:30 AM, Scott Hannahs <[hidden email]> wrote:
Ok, but the standard error out has 581 k lines of output:
6700 lines end with “ERROR”
1000 lines end with “FAIL”
23K lines end with “ok”
37K lines start with “Traceback"

This seems disturbing?  Especially the traceback failures in python.  It is possible that I have not installed some necessary component.

But as I said, my backup script is working every night (with S3 backend).  It just may not be working for a variety of backends?

There are a lot of “directory not empty” failures (about 5K).  But the number of errors in the whole file is somewhat daunting to me at the moment.  But if it is normal, I will move on and publish this installer package.

The whole error log is 36 MB and took 14,000 seconds to run.
It can be downloaded at http://www.p-hall.net/files/duplicity-error.txt

for the record, here is the installed components (and of course their defined dependencies)

duplicity                       0.7.11-3
boto-py27               2.36.0-1
gnupg-unified           1.4.21-1
lftp                            4.6.5-1
librsync                        0.9.7-1006
lockfile-py27           0.12.2-1
paramiko-py27   1.7.6-1
pycryptopp-py27 0.7.1-1
python27                        1:2.7.13-1
MacOSX                  10.11.6

Here are the first two Traceback reports:
======================================================================
ERROR: test_cleanup_after_partial (testing.functional.test_cleanup.CleanupTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/sw/src/fink.build/duplicity-0.7.11-1/duplicity-0.7.11/testing/functional/test_cleanup.py", line 38, in test_cleanup_after_partial
    good_files = self.backup("full", "testfiles/largefiles")
  File "/sw/src/fink.build/duplicity-0.7.11-1/duplicity-0.7.11/testing/functional/__init__.py", line 143, in backup
    result = self.run_duplicity(options=options, **kwargs)
  File "/sw/src/fink.build/duplicity-0.7.11-1/duplicity-0.7.11/testing/functional/__init__.py", line 130, in run_duplicity
    raise CmdError(return_val)
CmdError: 31

======================================================================
ERROR: test_remove_all_but_n (testing.functional.test_cleanup.CleanupTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/sw/src/fink.build/duplicity-0.7.11-1/duplicity-0.7.11/testing/functional/test_cleanup.py", line 56, in test_remove_all_but_n
    full1_files = self.backup("full", "testfiles/empty_dir")
  File "/sw/src/fink.build/duplicity-0.7.11-1/duplicity-0.7.11/testing/functional/__init__.py", line 143, in backup
    result = self.run_duplicity(options=options, **kwargs)
  File "/sw/src/fink.build/duplicity-0.7.11-1/duplicity-0.7.11/testing/functional/__init__.py", line 130, in run_duplicity
    raise CmdError(return_val)
CmdError: 31

======================================================================


-Scott


> On Jan 16, 2017, at 10:41 AM, Kenneth Loafman <[hidden email]> wrote:
>
> Those look like the normal errors we force for testing.  Unless the tests end with ERROR or FAIL, all is good.
>
> ...Thanks,
> ...Ken
>
>






_______________________________________________
Duplicity-talk mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/duplicity-talk
Reply | Threaded
Open this post in threaded view
|

Re: Errors during test phase of installation.....

duplicity-talk mailing list
No, this is not good, but what you describe is practically the same problem I got when testfiles was not being fully removed on Sierra, but from what I can tell from above, the offending tests are being skipped.  To run a single test, do this on the user console from within duplicity:

$ python -u setup.py test -s testing.functional.test_badupload.BadUploadTest

A weird thought... is it possible that tar is not available to the tests?

...Ken



On Tue, Jan 17, 2017 at 12:02 PM, Scott Hannahs via Duplicity-talk <[hidden email]> wrote:
The files seem to have somewhat restrictive execution and r/w access.  But I build and test duplicity as a non privileged user.  It seems to install and remove the testfiles directory structure while running.  But I have 23 failures and 100 errors out of the 418 tests.

The summary is:
Ran 418 tests in 179.702s

FAILED (failures=23, errors=100, skipped=5)
Test failed: <unittest.runner.TextTestResult run=418 errors=100 failures=23>
error: Test failed: <unittest.runner.TextTestResult run=418 errors=100 failures=23>
phase test: warning

And the errors seem to be at the beginning of the tests.  Here is the first N lines of the error output:
test_missing_file (testing.functional.test_badupload.BadUploadTest) ... FAIL
test_cleanup_after_partial (testing.functional.test_cleanup.CleanupTest) ... ERROR
test_remove_all_but_n (testing.functional.test_cleanup.CleanupTest) ... ERROR
test_remove_all_inc_of_but_n (testing.functional.test_cleanup.CleanupTest) ... ERROR
test_asym_cycle (testing.functional.test_final.FinalTest)
Like test_basic_cycle but use asymmetric encryption and signing ... ERROR
test_asym_with_hidden_recipient_cycle (testing.functional.test_final.FinalTest)
Like test_basic_cycle but use asymmetric encryption (hiding key id) and signing ... ERROR
test_basic_cycle (testing.functional.test_final.FinalTest)
Run backup/restore test on basic directories ... ERROR
test_empty_backup (testing.functional.test_final.FinalTest)
Make sure backup works when no files change ... ERROR
test_empty_restore (testing.functional.test_final.FinalTest)
Make sure error raised when restore doesn't match anything ... ERROR
test_long_filenames (testing.functional.test_final.FinalTest)
Test backing up a directory with long filenames in it ... ERROR
test_piped_password (testing.functional.test_final.FinalTest)
Make sure that prompting for a password works ... ERROR
test_remove_older_than (testing.functional.test_final.FinalTest)
Test removing old backup chains ... ERROR
test_single_regfile (testing.functional.test_final.FinalTest)
Test backing and restoring up a single regular file ... ERROR
test_asym_cycle (testing.functional.test_final.OldFilenamesFinalTest)
Like test_basic_cycle but use asymmetric encryption and signing ... ERROR
test_asym_with_hidden_recipient_cycle (testing.functional.test_final.OldFilenamesFinalTest)
Like test_basic_cycle but use asymmetric encryption (hiding key id) and signing ... ERROR
test_basic_cycle (testing.functional.test_final.OldFilenamesFinalTest)
Run backup/restore test on basic directories ... ERROR
test_empty_backup (testing.functional.test_final.OldFilenamesFinalTest)
Make sure backup works when no files change ... ERROR
test_empty_restore (testing.functional.test_final.OldFilenamesFinalTest)
Make sure error raised when restore doesn't match anything ... ERROR
test_long_filenames (testing.functional.test_final.OldFilenamesFinalTest)
Test backing up a directory with long filenames in it ... ERROR
test_piped_password (testing.functional.test_final.OldFilenamesFinalTest)
Make sure that prompting for a password works ... ERROR

Then there are a bunch of tests that are “ok”

Then I get some python failures with traceback:
======================================================================
ERROR: test_cleanup_after_partial (testing.functional.test_cleanup.CleanupTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/sw/src/fink.build/duplicity-0.7.11-3/duplicity-0.7.11/testing/functional/test_cleanup.py", line 38, in test_cleanup_after_partial
    good_files = self.backup("full", "testfiles/largefiles")
  File "/sw/src/fink.build/duplicity-0.7.11-3/duplicity-0.7.11/testing/functional/__init__.py", line 143, in backup
    result = self.run_duplicity(options=options, **kwargs)
  File "/sw/src/fink.build/duplicity-0.7.11-3/duplicity-0.7.11/testing/functional/__init__.py", line 130, in run_duplicity
    raise CmdError(return_val)
CmdError: 37

======================================================================
ERROR: test_remove_all_but_n (testing.functional.test_cleanup.CleanupTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/sw/src/fink.build/duplicity-0.7.11-3/duplicity-0.7.11/testing/functional/test_cleanup.py", line 56, in test_remove_all_but_n
    full1_files = self.backup("full", "testfiles/empty_dir")
  File "/sw/src/fink.build/duplicity-0.7.11-3/duplicity-0.7.11/testing/functional/__init__.py", line 143, in backup
    result = self.run_duplicity(options=options, **kwargs)
  File "/sw/src/fink.build/duplicity-0.7.11-3/duplicity-0.7.11/testing/functional/__init__.py", line 130, in run_duplicity
    raise CmdError(return_val)
CmdError: 37



This just doesn’t seem good to me.

-Scott


On Jan 16, 2017, at 1:28 PM, Kenneth Loafman <[hidden email]> wrote:

Any way to check to see if the testing/testfiles directory is being left around after a test?  Maybe run it manually?  If so, do a manual 'rm -r testing/testfiles' and see if El Capitan complains.  That will go a long way to find out what's going on.
 

On Mon, Jan 16, 2017 at 12:05 PM, Scott Hannahs <[hidden email]> wrote:
Ken,

The fink installation/testing system uses a sandbox directory and blows away the whole thing afterwards.  So there shouldn’t be a previous testing directory unless specifically requested to keep the build folders.  I think it creates those testing directories and files in the build directory structure.

This is on Mac OS X El Capitan (10.11).  I haven’t tried to move to Sierra (10.12) quite yet.  Though I think I can test that next week on another system if that is helpful.  First I was going to try to reduce these errors to a manageable amount on El Cap.

Those two lines to skip some tests are in the code as they should be.  I don’t quite see why the “TestLockedFoldersNotError” is related to the errors posted “test_cleanup_after_partial” and “test_remove_all_but_n” where duplicity crashed.

-Scott



On Jan 16, 2017, at 11:45 AM, Kenneth Loafman <[hidden email]> wrote:

Hmm, those errors should have been fixed in 0.7.11 in testing/test_selection.py.TestLockedFoldersNotError.  Mac Sierra made some funky changes that keeps you from deleting a directory with perms of 0o0000, thus causing test cleanup of those two tests to fail, and in general messing up the entire test cycle.

There should be two lines before each test def with:
    @unittest.skipUnless(platform.platform().startswith('Linux'),
                         'Skip on non-Linux systems')

Plus, if you ran the tests already, then the directory testing/testfiles may still be around to screw up the next test run.

...Ken


On Mon, Jan 16, 2017 at 10:30 AM, Scott Hannahs <[hidden email]> wrote:
Ok, but the standard error out has 581 k lines of output:
6700 lines end with “ERROR”
1000 lines end with “FAIL”
23K lines end with “ok”
37K lines start with “Traceback"

This seems disturbing?  Especially the traceback failures in python.  It is possible that I have not installed some necessary component.

But as I said, my backup script is working every night (with S3 backend).  It just may not be working for a variety of backends?

There are a lot of “directory not empty” failures (about 5K).  But the number of errors in the whole file is somewhat daunting to me at the moment.  But if it is normal, I will move on and publish this installer package.

The whole error log is 36 MB and took 14,000 seconds to run.
It can be downloaded at http://www.p-hall.net/files/duplicity-error.txt

for the record, here is the installed components (and of course their defined dependencies)

duplicity                       0.7.11-3
boto-py27               2.36.0-1
gnupg-unified           1.4.21-1
lftp                            4.6.5-1
librsync                        0.9.7-1006
lockfile-py27           0.12.2-1
paramiko-py27   1.7.6-1
pycryptopp-py27 0.7.1-1
python27                        1:2.7.13-1
MacOSX                  10.11.6

Here are the first two Traceback reports:
======================================================================
ERROR: test_cleanup_after_partial (testing.functional.test_cleanup.CleanupTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/sw/src/fink.build/duplicity-0.7.11-1/duplicity-0.7.11/testing/functional/test_cleanup.py", line 38, in test_cleanup_after_partial
    good_files = self.backup("full", "testfiles/largefiles")
  File "/sw/src/fink.build/duplicity-0.7.11-1/duplicity-0.7.11/testing/functional/__init__.py", line 143, in backup
    result = self.run_duplicity(options=options, **kwargs)
  File "/sw/src/fink.build/duplicity-0.7.11-1/duplicity-0.7.11/testing/functional/__init__.py", line 130, in run_duplicity
    raise CmdError(return_val)
CmdError: 31

======================================================================
ERROR: test_remove_all_but_n (testing.functional.test_cleanup.CleanupTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/sw/src/fink.build/duplicity-0.7.11-1/duplicity-0.7.11/testing/functional/test_cleanup.py", line 56, in test_remove_all_but_n
    full1_files = self.backup("full", "testfiles/empty_dir")
  File "/sw/src/fink.build/duplicity-0.7.11-1/duplicity-0.7.11/testing/functional/__init__.py", line 143, in backup
    result = self.run_duplicity(options=options, **kwargs)
  File "/sw/src/fink.build/duplicity-0.7.11-1/duplicity-0.7.11/testing/functional/__init__.py", line 130, in run_duplicity
    raise CmdError(return_val)
CmdError: 31

======================================================================


-Scott


> On Jan 16, 2017, at 10:41 AM, Kenneth Loafman <[hidden email]> wrote:
>
> Those look like the normal errors we force for testing.  Unless the tests end with ERROR or FAIL, all is good.
>
> ...Thanks,
> ...Ken
>
>






_______________________________________________
Duplicity-talk mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/duplicity-talk



_______________________________________________
Duplicity-talk mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/duplicity-talk
Reply | Threaded
Open this post in threaded view
|

Re: Errors during test phase of installation.....

duplicity-talk mailing list
Any progress on this?

Is there a way I can get hold of a previous version of MacOS for testing.  VirtualBox should be able to run it.

...Ken


On Wed, Jan 18, 2017 at 8:38 AM, Kenneth Loafman <[hidden email]> wrote:
No, this is not good, but what you describe is practically the same problem I got when testfiles was not being fully removed on Sierra, but from what I can tell from above, the offending tests are being skipped.  To run a single test, do this on the user console from within duplicity:

$ python -u setup.py test -s testing.functional.test_badupload.BadUploadTest

A weird thought... is it possible that tar is not available to the tests?

...Ken



On Tue, Jan 17, 2017 at 12:02 PM, Scott Hannahs via Duplicity-talk <[hidden email]> wrote:
The files seem to have somewhat restrictive execution and r/w access.  But I build and test duplicity as a non privileged user.  It seems to install and remove the testfiles directory structure while running.  But I have 23 failures and 100 errors out of the 418 tests.

The summary is:
Ran 418 tests in 179.702s

FAILED (failures=23, errors=100, skipped=5)
Test failed: <unittest.runner.TextTestResult run=418 errors=100 failures=23>
error: Test failed: <unittest.runner.TextTestResult run=418 errors=100 failures=23>
phase test: warning

And the errors seem to be at the beginning of the tests.  Here is the first N lines of the error output:
test_missing_file (testing.functional.test_badupload.BadUploadTest) ... FAIL
test_cleanup_after_partial (testing.functional.test_cleanup.CleanupTest) ... ERROR
test_remove_all_but_n (testing.functional.test_cleanup.CleanupTest) ... ERROR
test_remove_all_inc_of_but_n (testing.functional.test_cleanup.CleanupTest) ... ERROR
test_asym_cycle (testing.functional.test_final.FinalTest)
Like test_basic_cycle but use asymmetric encryption and signing ... ERROR
test_asym_with_hidden_recipient_cycle (testing.functional.test_final.FinalTest)
Like test_basic_cycle but use asymmetric encryption (hiding key id) and signing ... ERROR
test_basic_cycle (testing.functional.test_final.FinalTest)
Run backup/restore test on basic directories ... ERROR
test_empty_backup (testing.functional.test_final.FinalTest)
Make sure backup works when no files change ... ERROR
test_empty_restore (testing.functional.test_final.FinalTest)
Make sure error raised when restore doesn't match anything ... ERROR
test_long_filenames (testing.functional.test_final.FinalTest)
Test backing up a directory with long filenames in it ... ERROR
test_piped_password (testing.functional.test_final.FinalTest)
Make sure that prompting for a password works ... ERROR
test_remove_older_than (testing.functional.test_final.FinalTest)
Test removing old backup chains ... ERROR
test_single_regfile (testing.functional.test_final.FinalTest)
Test backing and restoring up a single regular file ... ERROR
test_asym_cycle (testing.functional.test_final.OldFilenamesFinalTest)
Like test_basic_cycle but use asymmetric encryption and signing ... ERROR
test_asym_with_hidden_recipient_cycle (testing.functional.test_final.OldFilenamesFinalTest)
Like test_basic_cycle but use asymmetric encryption (hiding key id) and signing ... ERROR
test_basic_cycle (testing.functional.test_final.OldFilenamesFinalTest)
Run backup/restore test on basic directories ... ERROR
test_empty_backup (testing.functional.test_final.OldFilenamesFinalTest)
Make sure backup works when no files change ... ERROR
test_empty_restore (testing.functional.test_final.OldFilenamesFinalTest)
Make sure error raised when restore doesn't match anything ... ERROR
test_long_filenames (testing.functional.test_final.OldFilenamesFinalTest)
Test backing up a directory with long filenames in it ... ERROR
test_piped_password (testing.functional.test_final.OldFilenamesFinalTest)
Make sure that prompting for a password works ... ERROR

Then there are a bunch of tests that are “ok”

Then I get some python failures with traceback:
======================================================================
ERROR: test_cleanup_after_partial (testing.functional.test_cleanup.CleanupTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/sw/src/fink.build/duplicity-0.7.11-3/duplicity-0.7.11/testing/functional/test_cleanup.py", line 38, in test_cleanup_after_partial
    good_files = self.backup("full", "testfiles/largefiles")
  File "/sw/src/fink.build/duplicity-0.7.11-3/duplicity-0.7.11/testing/functional/__init__.py", line 143, in backup
    result = self.run_duplicity(options=options, **kwargs)
  File "/sw/src/fink.build/duplicity-0.7.11-3/duplicity-0.7.11/testing/functional/__init__.py", line 130, in run_duplicity
    raise CmdError(return_val)
CmdError: 37

======================================================================
ERROR: test_remove_all_but_n (testing.functional.test_cleanup.CleanupTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/sw/src/fink.build/duplicity-0.7.11-3/duplicity-0.7.11/testing/functional/test_cleanup.py", line 56, in test_remove_all_but_n
    full1_files = self.backup("full", "testfiles/empty_dir")
  File "/sw/src/fink.build/duplicity-0.7.11-3/duplicity-0.7.11/testing/functional/__init__.py", line 143, in backup
    result = self.run_duplicity(options=options, **kwargs)
  File "/sw/src/fink.build/duplicity-0.7.11-3/duplicity-0.7.11/testing/functional/__init__.py", line 130, in run_duplicity
    raise CmdError(return_val)
CmdError: 37



This just doesn’t seem good to me.

-Scott


On Jan 16, 2017, at 1:28 PM, Kenneth Loafman <[hidden email]> wrote:

Any way to check to see if the testing/testfiles directory is being left around after a test?  Maybe run it manually?  If so, do a manual 'rm -r testing/testfiles' and see if El Capitan complains.  That will go a long way to find out what's going on.
 

On Mon, Jan 16, 2017 at 12:05 PM, Scott Hannahs <[hidden email]> wrote:
Ken,

The fink installation/testing system uses a sandbox directory and blows away the whole thing afterwards.  So there shouldn’t be a previous testing directory unless specifically requested to keep the build folders.  I think it creates those testing directories and files in the build directory structure.

This is on Mac OS X El Capitan (10.11).  I haven’t tried to move to Sierra (10.12) quite yet.  Though I think I can test that next week on another system if that is helpful.  First I was going to try to reduce these errors to a manageable amount on El Cap.

Those two lines to skip some tests are in the code as they should be.  I don’t quite see why the “TestLockedFoldersNotError” is related to the errors posted “test_cleanup_after_partial” and “test_remove_all_but_n” where duplicity crashed.

-Scott



On Jan 16, 2017, at 11:45 AM, Kenneth Loafman <[hidden email]> wrote:

Hmm, those errors should have been fixed in 0.7.11 in testing/test_selection.py.TestLockedFoldersNotError.  Mac Sierra made some funky changes that keeps you from deleting a directory with perms of 0o0000, thus causing test cleanup of those two tests to fail, and in general messing up the entire test cycle.

There should be two lines before each test def with:
    @unittest.skipUnless(platform.platform().startswith('Linux'),
                         'Skip on non-Linux systems')

Plus, if you ran the tests already, then the directory testing/testfiles may still be around to screw up the next test run.

...Ken


On Mon, Jan 16, 2017 at 10:30 AM, Scott Hannahs <[hidden email]> wrote:
Ok, but the standard error out has 581 k lines of output:
6700 lines end with “ERROR”
1000 lines end with “FAIL”
23K lines end with “ok”
37K lines start with “Traceback"

This seems disturbing?  Especially the traceback failures in python.  It is possible that I have not installed some necessary component.

But as I said, my backup script is working every night (with S3 backend).  It just may not be working for a variety of backends?

There are a lot of “directory not empty” failures (about 5K).  But the number of errors in the whole file is somewhat daunting to me at the moment.  But if it is normal, I will move on and publish this installer package.

The whole error log is 36 MB and took 14,000 seconds to run.
It can be downloaded at http://www.p-hall.net/files/duplicity-error.txt

for the record, here is the installed components (and of course their defined dependencies)

duplicity                       0.7.11-3
boto-py27               2.36.0-1
gnupg-unified           1.4.21-1
lftp                            4.6.5-1
librsync                        0.9.7-1006
lockfile-py27           0.12.2-1
paramiko-py27   1.7.6-1
pycryptopp-py27 0.7.1-1
python27                        1:2.7.13-1
MacOSX                  10.11.6

Here are the first two Traceback reports:
======================================================================
ERROR: test_cleanup_after_partial (testing.functional.test_cleanup.CleanupTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/sw/src/fink.build/duplicity-0.7.11-1/duplicity-0.7.11/testing/functional/test_cleanup.py", line 38, in test_cleanup_after_partial
    good_files = self.backup("full", "testfiles/largefiles")
  File "/sw/src/fink.build/duplicity-0.7.11-1/duplicity-0.7.11/testing/functional/__init__.py", line 143, in backup
    result = self.run_duplicity(options=options, **kwargs)
  File "/sw/src/fink.build/duplicity-0.7.11-1/duplicity-0.7.11/testing/functional/__init__.py", line 130, in run_duplicity
    raise CmdError(return_val)
CmdError: 31

======================================================================
ERROR: test_remove_all_but_n (testing.functional.test_cleanup.CleanupTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/sw/src/fink.build/duplicity-0.7.11-1/duplicity-0.7.11/testing/functional/test_cleanup.py", line 56, in test_remove_all_but_n
    full1_files = self.backup("full", "testfiles/empty_dir")
  File "/sw/src/fink.build/duplicity-0.7.11-1/duplicity-0.7.11/testing/functional/__init__.py", line 143, in backup
    result = self.run_duplicity(options=options, **kwargs)
  File "/sw/src/fink.build/duplicity-0.7.11-1/duplicity-0.7.11/testing/functional/__init__.py", line 130, in run_duplicity
    raise CmdError(return_val)
CmdError: 31

======================================================================


-Scott


> On Jan 16, 2017, at 10:41 AM, Kenneth Loafman <[hidden email]> wrote:
>
> Those look like the normal errors we force for testing.  Unless the tests end with ERROR or FAIL, all is good.
>
> ...Thanks,
> ...Ken
>
>






_______________________________________________
Duplicity-talk mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/duplicity-talk




_______________________________________________
Duplicity-talk mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/duplicity-talk
Reply | Threaded
Open this post in threaded view
|

Re: Errors during test phase of installation.....

duplicity-talk mailing list
I hope to play with it sometime this weekend and hopefully some progress to report.  What version of Mac OS are you looking for?

-Scott

On Jan 27, 2017, at 11:20 AM, Kenneth Loafman <[hidden email]> wrote:

Any progress on this?

Is there a way I can get hold of a previous version of MacOS for testing.  VirtualBox should be able to run it.

...Ken


On Wed, Jan 18, 2017 at 8:38 AM, Kenneth Loafman <[hidden email]> wrote:
No, this is not good, but what you describe is practically the same problem I got when testfiles was not being fully removed on Sierra, but from what I can tell from above, the offending tests are being skipped.  To run a single test, do this on the user console from within duplicity:

$ python -u setup.py test -s testing.functional.test_badupload.BadUploadTest

A weird thought... is it possible that tar is not available to the tests?

...Ken



On Tue, Jan 17, 2017 at 12:02 PM, Scott Hannahs via Duplicity-talk <[hidden email]> wrote:
The files seem to have somewhat restrictive execution and r/w access.  But I build and test duplicity as a non privileged user.  It seems to install and remove the testfiles directory structure while running.  But I have 23 failures and 100 errors out of the 418 tests.

The summary is:
Ran 418 tests in 179.702s

FAILED (failures=23, errors=100, skipped=5)
Test failed: <unittest.runner.TextTestResult run=418 errors=100 failures=23>
error: Test failed: <unittest.runner.TextTestResult run=418 errors=100 failures=23>
phase test: warning

And the errors seem to be at the beginning of the tests.  Here is the first N lines of the error output:
test_missing_file (testing.functional.test_badupload.BadUploadTest) ... FAIL
test_cleanup_after_partial (testing.functional.test_cleanup.CleanupTest) ... ERROR
test_remove_all_but_n (testing.functional.test_cleanup.CleanupTest) ... ERROR
test_remove_all_inc_of_but_n (testing.functional.test_cleanup.CleanupTest) ... ERROR
test_asym_cycle (testing.functional.test_final.FinalTest)
Like test_basic_cycle but use asymmetric encryption and signing ... ERROR
test_asym_with_hidden_recipient_cycle (testing.functional.test_final.FinalTest)
Like test_basic_cycle but use asymmetric encryption (hiding key id) and signing ... ERROR
test_basic_cycle (testing.functional.test_final.FinalTest)
Run backup/restore test on basic directories ... ERROR
test_empty_backup (testing.functional.test_final.FinalTest)
Make sure backup works when no files change ... ERROR
test_empty_restore (testing.functional.test_final.FinalTest)
Make sure error raised when restore doesn't match anything ... ERROR
test_long_filenames (testing.functional.test_final.FinalTest)
Test backing up a directory with long filenames in it ... ERROR
test_piped_password (testing.functional.test_final.FinalTest)
Make sure that prompting for a password works ... ERROR
test_remove_older_than (testing.functional.test_final.FinalTest)
Test removing old backup chains ... ERROR
test_single_regfile (testing.functional.test_final.FinalTest)
Test backing and restoring up a single regular file ... ERROR
test_asym_cycle (testing.functional.test_final.OldFilenamesFinalTest)
Like test_basic_cycle but use asymmetric encryption and signing ... ERROR
test_asym_with_hidden_recipient_cycle (testing.functional.test_final.OldFilenamesFinalTest)
Like test_basic_cycle but use asymmetric encryption (hiding key id) and signing ... ERROR
test_basic_cycle (testing.functional.test_final.OldFilenamesFinalTest)
Run backup/restore test on basic directories ... ERROR
test_empty_backup (testing.functional.test_final.OldFilenamesFinalTest)
Make sure backup works when no files change ... ERROR
test_empty_restore (testing.functional.test_final.OldFilenamesFinalTest)
Make sure error raised when restore doesn't match anything ... ERROR
test_long_filenames (testing.functional.test_final.OldFilenamesFinalTest)
Test backing up a directory with long filenames in it ... ERROR
test_piped_password (testing.functional.test_final.OldFilenamesFinalTest)
Make sure that prompting for a password works ... ERROR

Then there are a bunch of tests that are “ok”

Then I get some python failures with traceback:
======================================================================
ERROR: test_cleanup_after_partial (testing.functional.test_cleanup.CleanupTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/sw/src/fink.build/duplicity-0.7.11-3/duplicity-0.7.11/testing/functional/test_cleanup.py", line 38, in test_cleanup_after_partial
    good_files = self.backup("full", "testfiles/largefiles")
  File "/sw/src/fink.build/duplicity-0.7.11-3/duplicity-0.7.11/testing/functional/__init__.py", line 143, in backup
    result = self.run_duplicity(options=options, **kwargs)
  File "/sw/src/fink.build/duplicity-0.7.11-3/duplicity-0.7.11/testing/functional/__init__.py", line 130, in run_duplicity
    raise CmdError(return_val)
CmdError: 37

======================================================================
ERROR: test_remove_all_but_n (testing.functional.test_cleanup.CleanupTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/sw/src/fink.build/duplicity-0.7.11-3/duplicity-0.7.11/testing/functional/test_cleanup.py", line 56, in test_remove_all_but_n
    full1_files = self.backup("full", "testfiles/empty_dir")
  File "/sw/src/fink.build/duplicity-0.7.11-3/duplicity-0.7.11/testing/functional/__init__.py", line 143, in backup
    result = self.run_duplicity(options=options, **kwargs)
  File "/sw/src/fink.build/duplicity-0.7.11-3/duplicity-0.7.11/testing/functional/__init__.py", line 130, in run_duplicity
    raise CmdError(return_val)
CmdError: 37



This just doesn’t seem good to me.

-Scott


On Jan 16, 2017, at 1:28 PM, Kenneth Loafman <[hidden email]> wrote:

Any way to check to see if the testing/testfiles directory is being left around after a test?  Maybe run it manually?  If so, do a manual 'rm -r testing/testfiles' and see if El Capitan complains.  That will go a long way to find out what's going on.
 

On Mon, Jan 16, 2017 at 12:05 PM, Scott Hannahs <[hidden email]> wrote:
Ken,

The fink installation/testing system uses a sandbox directory and blows away the whole thing afterwards.  So there shouldn’t be a previous testing directory unless specifically requested to keep the build folders.  I think it creates those testing directories and files in the build directory structure.

This is on Mac OS X El Capitan (10.11).  I haven’t tried to move to Sierra (10.12) quite yet.  Though I think I can test that next week on another system if that is helpful.  First I was going to try to reduce these errors to a manageable amount on El Cap.

Those two lines to skip some tests are in the code as they should be.  I don’t quite see why the “TestLockedFoldersNotError” is related to the errors posted “test_cleanup_after_partial” and “test_remove_all_but_n” where duplicity crashed.

-Scott



On Jan 16, 2017, at 11:45 AM, Kenneth Loafman <[hidden email]> wrote:

Hmm, those errors should have been fixed in 0.7.11 in testing/test_selection.py.TestLockedFoldersNotError.  Mac Sierra made some funky changes that keeps you from deleting a directory with perms of 0o0000, thus causing test cleanup of those two tests to fail, and in general messing up the entire test cycle.

There should be two lines before each test def with:
    @unittest.skipUnless(platform.platform().startswith('Linux'),
                         'Skip on non-Linux systems')

Plus, if you ran the tests already, then the directory testing/testfiles may still be around to screw up the next test run.

...Ken


On Mon, Jan 16, 2017 at 10:30 AM, Scott Hannahs <[hidden email]> wrote:
Ok, but the standard error out has 581 k lines of output:
6700 lines end with “ERROR”
1000 lines end with “FAIL”
23K lines end with “ok”
37K lines start with “Traceback"

This seems disturbing?  Especially the traceback failures in python.  It is possible that I have not installed some necessary component.

But as I said, my backup script is working every night (with S3 backend).  It just may not be working for a variety of backends?

There are a lot of “directory not empty” failures (about 5K).  But the number of errors in the whole file is somewhat daunting to me at the moment.  But if it is normal, I will move on and publish this installer package.

The whole error log is 36 MB and took 14,000 seconds to run.
It can be downloaded at http://www.p-hall.net/files/duplicity-error.txt

for the record, here is the installed components (and of course their defined dependencies)

duplicity                       0.7.11-3
boto-py27               2.36.0-1
gnupg-unified           1.4.21-1
lftp                            4.6.5-1
librsync                        0.9.7-1006
lockfile-py27           0.12.2-1
paramiko-py27   1.7.6-1
pycryptopp-py27 0.7.1-1
python27                        1:2.7.13-1
MacOSX                  10.11.6

Here are the first two Traceback reports:
======================================================================
ERROR: test_cleanup_after_partial (testing.functional.test_cleanup.CleanupTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/sw/src/fink.build/duplicity-0.7.11-1/duplicity-0.7.11/testing/functional/test_cleanup.py", line 38, in test_cleanup_after_partial
    good_files = self.backup("full", "testfiles/largefiles")
  File "/sw/src/fink.build/duplicity-0.7.11-1/duplicity-0.7.11/testing/functional/__init__.py", line 143, in backup
    result = self.run_duplicity(options=options, **kwargs)
  File "/sw/src/fink.build/duplicity-0.7.11-1/duplicity-0.7.11/testing/functional/__init__.py", line 130, in run_duplicity
    raise CmdError(return_val)
CmdError: 31

======================================================================
ERROR: test_remove_all_but_n (testing.functional.test_cleanup.CleanupTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/sw/src/fink.build/duplicity-0.7.11-1/duplicity-0.7.11/testing/functional/test_cleanup.py", line 56, in test_remove_all_but_n
    full1_files = self.backup("full", "testfiles/empty_dir")
  File "/sw/src/fink.build/duplicity-0.7.11-1/duplicity-0.7.11/testing/functional/__init__.py", line 143, in backup
    result = self.run_duplicity(options=options, **kwargs)
  File "/sw/src/fink.build/duplicity-0.7.11-1/duplicity-0.7.11/testing/functional/__init__.py", line 130, in run_duplicity
    raise CmdError(return_val)
CmdError: 31

======================================================================


-Scott


> On Jan 16, 2017, at 10:41 AM, Kenneth Loafman <[hidden email]> wrote:
>
> Those look like the normal errors we force for testing.  Unless the tests end with ERROR or FAIL, all is good.
>
> ...Thanks,
> ...Ken
>
>






_______________________________________________
Duplicity-talk mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/duplicity-talk





_______________________________________________
Duplicity-talk mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/duplicity-talk
Reply | Threaded
Open this post in threaded view
|

Re: Errors during test phase of installation.....

duplicity-talk mailing list
El Capitan, like you're running.  I upgraded to Sierra when it came out and forgot to keep a copy to play with.


On Fri, Jan 27, 2017 at 10:30 AM, Scott Hannahs <[hidden email]> wrote:
I hope to play with it sometime this weekend and hopefully some progress to report.  What version of Mac OS are you looking for?

-Scott

On Jan 27, 2017, at 11:20 AM, Kenneth Loafman <[hidden email]> wrote:

Any progress on this?

Is there a way I can get hold of a previous version of MacOS for testing.  VirtualBox should be able to run it.

...Ken


On Wed, Jan 18, 2017 at 8:38 AM, Kenneth Loafman <[hidden email]> wrote:
No, this is not good, but what you describe is practically the same problem I got when testfiles was not being fully removed on Sierra, but from what I can tell from above, the offending tests are being skipped.  To run a single test, do this on the user console from within duplicity:

$ python -u setup.py test -s testing.functional.test_badupload.BadUploadTest

A weird thought... is it possible that tar is not available to the tests?

...Ken



On Tue, Jan 17, 2017 at 12:02 PM, Scott Hannahs via Duplicity-talk <[hidden email]> wrote:
The files seem to have somewhat restrictive execution and r/w access.  But I build and test duplicity as a non privileged user.  It seems to install and remove the testfiles directory structure while running.  But I have 23 failures and 100 errors out of the 418 tests.

The summary is:
Ran 418 tests in 179.702s

FAILED (failures=23, errors=100, skipped=5)
Test failed: <unittest.runner.TextTestResult run=418 errors=100 failures=23>
error: Test failed: <unittest.runner.TextTestResult run=418 errors=100 failures=23>
phase test: warning

And the errors seem to be at the beginning of the tests.  Here is the first N lines of the error output:
test_missing_file (testing.functional.test_badupload.BadUploadTest) ... FAIL
test_cleanup_after_partial (testing.functional.test_cleanup.CleanupTest) ... ERROR
test_remove_all_but_n (testing.functional.test_cleanup.CleanupTest) ... ERROR
test_remove_all_inc_of_but_n (testing.functional.test_cleanup.CleanupTest) ... ERROR
test_asym_cycle (testing.functional.test_final.FinalTest)
Like test_basic_cycle but use asymmetric encryption and signing ... ERROR
test_asym_with_hidden_recipient_cycle (testing.functional.test_final.FinalTest)
Like test_basic_cycle but use asymmetric encryption (hiding key id) and signing ... ERROR
test_basic_cycle (testing.functional.test_final.FinalTest)
Run backup/restore test on basic directories ... ERROR
test_empty_backup (testing.functional.test_final.FinalTest)
Make sure backup works when no files change ... ERROR
test_empty_restore (testing.functional.test_final.FinalTest)
Make sure error raised when restore doesn't match anything ... ERROR
test_long_filenames (testing.functional.test_final.FinalTest)
Test backing up a directory with long filenames in it ... ERROR
test_piped_password (testing.functional.test_final.FinalTest)
Make sure that prompting for a password works ... ERROR
test_remove_older_than (testing.functional.test_final.FinalTest)
Test removing old backup chains ... ERROR
test_single_regfile (testing.functional.test_final.FinalTest)
Test backing and restoring up a single regular file ... ERROR
test_asym_cycle (testing.functional.test_final.OldFilenamesFinalTest)
Like test_basic_cycle but use asymmetric encryption and signing ... ERROR
test_asym_with_hidden_recipient_cycle (testing.functional.test_final.OldFilenamesFinalTest)
Like test_basic_cycle but use asymmetric encryption (hiding key id) and signing ... ERROR
test_basic_cycle (testing.functional.test_final.OldFilenamesFinalTest)
Run backup/restore test on basic directories ... ERROR
test_empty_backup (testing.functional.test_final.OldFilenamesFinalTest)
Make sure backup works when no files change ... ERROR
test_empty_restore (testing.functional.test_final.OldFilenamesFinalTest)
Make sure error raised when restore doesn't match anything ... ERROR
test_long_filenames (testing.functional.test_final.OldFilenamesFinalTest)
Test backing up a directory with long filenames in it ... ERROR
test_piped_password (testing.functional.test_final.OldFilenamesFinalTest)
Make sure that prompting for a password works ... ERROR

Then there are a bunch of tests that are “ok”

Then I get some python failures with traceback:
======================================================================
ERROR: test_cleanup_after_partial (testing.functional.test_cleanup.CleanupTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/sw/src/fink.build/duplicity-0.7.11-3/duplicity-0.7.11/testing/functional/test_cleanup.py", line 38, in test_cleanup_after_partial
    good_files = self.backup("full", "testfiles/largefiles")
  File "/sw/src/fink.build/duplicity-0.7.11-3/duplicity-0.7.11/testing/functional/__init__.py", line 143, in backup
    result = self.run_duplicity(options=options, **kwargs)
  File "/sw/src/fink.build/duplicity-0.7.11-3/duplicity-0.7.11/testing/functional/__init__.py", line 130, in run_duplicity
    raise CmdError(return_val)
CmdError: 37

======================================================================
ERROR: test_remove_all_but_n (testing.functional.test_cleanup.CleanupTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/sw/src/fink.build/duplicity-0.7.11-3/duplicity-0.7.11/testing/functional/test_cleanup.py", line 56, in test_remove_all_but_n
    full1_files = self.backup("full", "testfiles/empty_dir")
  File "/sw/src/fink.build/duplicity-0.7.11-3/duplicity-0.7.11/testing/functional/__init__.py", line 143, in backup
    result = self.run_duplicity(options=options, **kwargs)
  File "/sw/src/fink.build/duplicity-0.7.11-3/duplicity-0.7.11/testing/functional/__init__.py", line 130, in run_duplicity
    raise CmdError(return_val)
CmdError: 37



This just doesn’t seem good to me.

-Scott


On Jan 16, 2017, at 1:28 PM, Kenneth Loafman <[hidden email]> wrote:

Any way to check to see if the testing/testfiles directory is being left around after a test?  Maybe run it manually?  If so, do a manual 'rm -r testing/testfiles' and see if El Capitan complains.  That will go a long way to find out what's going on.
 

On Mon, Jan 16, 2017 at 12:05 PM, Scott Hannahs <[hidden email]> wrote:
Ken,

The fink installation/testing system uses a sandbox directory and blows away the whole thing afterwards.  So there shouldn’t be a previous testing directory unless specifically requested to keep the build folders.  I think it creates those testing directories and files in the build directory structure.

This is on Mac OS X El Capitan (10.11).  I haven’t tried to move to Sierra (10.12) quite yet.  Though I think I can test that next week on another system if that is helpful.  First I was going to try to reduce these errors to a manageable amount on El Cap.

Those two lines to skip some tests are in the code as they should be.  I don’t quite see why the “TestLockedFoldersNotError” is related to the errors posted “test_cleanup_after_partial” and “test_remove_all_but_n” where duplicity crashed.

-Scott



On Jan 16, 2017, at 11:45 AM, Kenneth Loafman <[hidden email]> wrote:

Hmm, those errors should have been fixed in 0.7.11 in testing/test_selection.py.TestLockedFoldersNotError.  Mac Sierra made some funky changes that keeps you from deleting a directory with perms of 0o0000, thus causing test cleanup of those two tests to fail, and in general messing up the entire test cycle.

There should be two lines before each test def with:
    @unittest.skipUnless(platform.platform().startswith('Linux'),
                         'Skip on non-Linux systems')

Plus, if you ran the tests already, then the directory testing/testfiles may still be around to screw up the next test run.

...Ken


On Mon, Jan 16, 2017 at 10:30 AM, Scott Hannahs <[hidden email]> wrote:
Ok, but the standard error out has 581 k lines of output:
6700 lines end with “ERROR”
1000 lines end with “FAIL”
23K lines end with “ok”
37K lines start with “Traceback"

This seems disturbing?  Especially the traceback failures in python.  It is possible that I have not installed some necessary component.

But as I said, my backup script is working every night (with S3 backend).  It just may not be working for a variety of backends?

There are a lot of “directory not empty” failures (about 5K).  But the number of errors in the whole file is somewhat daunting to me at the moment.  But if it is normal, I will move on and publish this installer package.

The whole error log is 36 MB and took 14,000 seconds to run.
It can be downloaded at http://www.p-hall.net/files/duplicity-error.txt

for the record, here is the installed components (and of course their defined dependencies)

duplicity                       0.7.11-3
boto-py27               2.36.0-1
gnupg-unified           1.4.21-1
lftp                            4.6.5-1
librsync                        0.9.7-1006
lockfile-py27           0.12.2-1
paramiko-py27   1.7.6-1
pycryptopp-py27 0.7.1-1
python27                        1:2.7.13-1
MacOSX                  10.11.6

Here are the first two Traceback reports:
======================================================================
ERROR: test_cleanup_after_partial (testing.functional.test_cleanup.CleanupTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/sw/src/fink.build/duplicity-0.7.11-1/duplicity-0.7.11/testing/functional/test_cleanup.py", line 38, in test_cleanup_after_partial
    good_files = self.backup("full", "testfiles/largefiles")
  File "/sw/src/fink.build/duplicity-0.7.11-1/duplicity-0.7.11/testing/functional/__init__.py", line 143, in backup
    result = self.run_duplicity(options=options, **kwargs)
  File "/sw/src/fink.build/duplicity-0.7.11-1/duplicity-0.7.11/testing/functional/__init__.py", line 130, in run_duplicity
    raise CmdError(return_val)
CmdError: 31

======================================================================
ERROR: test_remove_all_but_n (testing.functional.test_cleanup.CleanupTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/sw/src/fink.build/duplicity-0.7.11-1/duplicity-0.7.11/testing/functional/test_cleanup.py", line 56, in test_remove_all_but_n
    full1_files = self.backup("full", "testfiles/empty_dir")
  File "/sw/src/fink.build/duplicity-0.7.11-1/duplicity-0.7.11/testing/functional/__init__.py", line 143, in backup
    result = self.run_duplicity(options=options, **kwargs)
  File "/sw/src/fink.build/duplicity-0.7.11-1/duplicity-0.7.11/testing/functional/__init__.py", line 130, in run_duplicity
    raise CmdError(return_val)
CmdError: 31

======================================================================


-Scott


> On Jan 16, 2017, at 10:41 AM, Kenneth Loafman <[hidden email]> wrote:
>
> Those look like the normal errors we force for testing.  Unless the tests end with ERROR or FAIL, all is good.
>
> ...Thanks,
> ...Ken
>
>






_______________________________________________
Duplicity-talk mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/duplicity-talk






_______________________________________________
Duplicity-talk mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/duplicity-talk
Reply | Threaded
Open this post in threaded view
|

Re: Errors during test phase of installation.....

duplicity-talk mailing list
In reply to this post by duplicity-talk mailing list
Ken,

tar seems to be available to the tests.  I ran just the first test (it fails) as a stand alone as you suggest.
The command used was:
sudo python2.7 -u setup.py test -s testing.functional.test_badupload.BadUploadTest > ~/Desktop/badupload.out.txt 2> ~/Desktop/badupload.err.txt

This is the “test missing file” in the "bad upload” test.

The standard out seems normal, but I am unclear as to why it is copying the librsync library which is already installed.  Maybe it is not where the testing routines expect it?

running test
running egg_info
writing requirements to duplicity.egg-info/requires.txt
writing duplicity.egg-info/PKG-INFO
writing top-level names to duplicity.egg-info/top_level.txt
writing dependency_links to duplicity.egg-info/dependency_links.txt
reading manifest file 'duplicity.egg-info/SOURCES.txt'
writing manifest file 'duplicity.egg-info/SOURCES.txt'
running build_ext
copying build/lib.macosx-10.11-x86_64-2.7/duplicity/_librsync.so -> duplicity
—————————————————————————————

The error output shows an assertion error.  As far as I can tell it is getting a return value of 37 and expected a return value of 44????  From grepping the sources this means that there are not enough file descriptors?  I think the Mac OS X default is a paltry 256.

test_missing_file (testing.functional.test_badupload.BadUploadTest) ... FAIL

======================================================================
FAIL: test_missing_file (testing.functional.test_badupload.BadUploadTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/sw/src/fink.build/duplicity-0.7.11-3A/duplicity-0.7.11/testing/functional/test_badupload.py", line 40, in test_missing_file
    self.assertEqual(e.exit_status, 44, str(e))
AssertionError: 37

----------------------------------------------------------------------
Ran 1 test in 1.584s

FAILED (failures=1)
Test failed: <unittest.runner.TextTestResult run=1 errors=0 failures=1>
error: Test failed: <unittest.runner.TextTestResult run=1 errors=0 failures=1>
—————————————————————————————

-Scott



On Jan 18, 2017, at 9:38 AM, Kenneth Loafman <[hidden email]> wrote:

No, this is not good, but what you describe is practically the same problem I got when testfiles was not being fully removed on Sierra, but from what I can tell from above, the offending tests are being skipped.  To run a single test, do this on the user console from within duplicity:


_______________________________________________
Duplicity-talk mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/duplicity-talk
Reply | Threaded
Open this post in threaded view
|

Re: Errors during test phase of installation.....

duplicity-talk mailing list
When you run 'setup.py' test command, it has to compile the _librsyncmodule.c module to make the interface _librsync.so module, which is the interface to the system installed librsync.so module and then copy it.  Just part of Python's setup process.

I'm still unable to reproduce your problem.  Are there restrictive permissions in the /sw/src/fink.build directories?


On Sun, Jan 29, 2017 at 3:38 PM, Scott Hannahs <[hidden email]> wrote:
Ken,

tar seems to be available to the tests.  I ran just the first test (it fails) as a stand alone as you suggest.
The command used was:
sudo python2.7 -u setup.py test -s testing.functional.test_badupload.BadUploadTest > ~/Desktop/badupload.out.txt 2> ~/Desktop/badupload.err.txt

This is the “test missing file” in the "bad upload” test.

The standard out seems normal, but I am unclear as to why it is copying the librsync library which is already installed.  Maybe it is not where the testing routines expect it?

running test
running egg_info
writing requirements to duplicity.egg-info/requires.txt
writing duplicity.egg-info/PKG-INFO
writing top-level names to duplicity.egg-info/top_level.txt
writing dependency_links to duplicity.egg-info/dependency_links.txt
reading manifest file 'duplicity.egg-info/SOURCES.txt'
writing manifest file 'duplicity.egg-info/SOURCES.txt'
running build_ext
copying build/lib.macosx-10.11-x86_64-2.7/duplicity/_librsync.so -> duplicity
—————————————————————————————

The error output shows an assertion error.  As far as I can tell it is getting a return value of 37 and expected a return value of 44????  From grepping the sources this means that there are not enough file descriptors?  I think the Mac OS X default is a paltry 256.

test_missing_file (testing.functional.test_badupload.BadUploadTest) ... FAIL

======================================================================
FAIL: test_missing_file (testing.functional.test_badupload.BadUploadTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/sw/src/fink.build/duplicity-0.7.11-3A/duplicity-0.7.11/testing/functional/test_badupload.py", line 40, in test_missing_file
    self.assertEqual(e.exit_status, 44, str(e))
AssertionError: 37

----------------------------------------------------------------------
Ran 1 test in 1.584s

FAILED (failures=1)
Test failed: <unittest.runner.TextTestResult run=1 errors=0 failures=1>
error: Test failed: <unittest.runner.TextTestResult run=1 errors=0 failures=1>
—————————————————————————————

-Scott



On Jan 18, 2017, at 9:38 AM, Kenneth Loafman <[hidden email]> wrote:

No, this is not good, but what you describe is practically the same problem I got when testfiles was not being fully removed on Sierra, but from what I can tell from above, the offending tests are being skipped.  To run a single test, do this on the user console from within duplicity:



_______________________________________________
Duplicity-talk mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/duplicity-talk
Reply | Threaded
Open this post in threaded view
|

Re: Errors during test phase of installation.....

duplicity-talk mailing list
Ken,

I am sorry, I did not get back to you on this….   I thought I had written the email.

I have resolved almost all the test issues.  The main problem was that in setting up and running the test, there are a huge number of open files simultaneously and this runs up against the Mac OS X default setting for number of open files per process.  That defaults to 256 files.  It turns out that the duplicity test needs more than that simultaneous open files and if I set that limit to 8192 most of the tests pass.  I think all the remaining test issues are due to the missing paramiko version 2.X backend.

The clues was the “AssertionError 37” which had insufficient file system resources or something like that when I looked at the code.  So when I give it a lot more file resources it works.  This is only during the testing phase since I think I have been running it on big directories with the default values.

The other big error was that the un-tar of the test files would set file permissions and owners so that the test user could not delete them.   I set a note that the testing phase has to run as root.  It shouldn’t but that seems to be an issue.  If the no-owner no-group flags were set on the untar command for the test directories it might work.  But that may mess up some of the tests.

The librsyncmodule is compiled correctly.  As I said, I have installed it and run duplicity for a long time using the rsync and now S3 backend.

-Scott

On Feb 2, 2017, at 12:16 PM, Kenneth Loafman <[hidden email]> wrote:

When you run 'setup.py' test command, it has to compile the _librsyncmodule.c module to make the interface _librsync.so module, which is the interface to the system installed librsync.so module and then copy it.  Just part of Python's setup process.

I'm still unable to reproduce your problem.  Are there restrictive permissions in the /sw/src/fink.build directories?


On Sun, Jan 29, 2017 at 3:38 PM, Scott Hannahs <[hidden email]> wrote:
Ken,

tar seems to be available to the tests.  I ran just the first test (it fails) as a stand alone as you suggest.
The command used was:
sudo python2.7 -u setup.py test -s testing.functional.test_badupload.BadUploadTest > ~/Desktop/badupload.out.txt 2> ~/Desktop/badupload.err.txt

This is the “test missing file” in the "bad upload” test.

The standard out seems normal, but I am unclear as to why it is copying the librsync library which is already installed.  Maybe it is not where the testing routines expect it?

running test
running egg_info
writing requirements to duplicity.egg-info/requires.txt
writing duplicity.egg-info/PKG-INFO
writing top-level names to duplicity.egg-info/top_level.txt
writing dependency_links to duplicity.egg-info/dependency_links.txt
reading manifest file 'duplicity.egg-info/SOURCES.txt'
writing manifest file 'duplicity.egg-info/SOURCES.txt'
running build_ext
copying build/lib.macosx-10.11-x86_64-2.7/duplicity/_librsync.so -> duplicity
—————————————————————————————

The error output shows an assertion error.  As far as I can tell it is getting a return value of 37 and expected a return value of 44????  From grepping the sources this means that there are not enough file descriptors?  I think the Mac OS X default is a paltry 256.

test_missing_file (testing.functional.test_badupload.BadUploadTest) ... FAIL

======================================================================
FAIL: test_missing_file (testing.functional.test_badupload.BadUploadTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/sw/src/fink.build/duplicity-0.7.11-3A/duplicity-0.7.11/testing/functional/test_badupload.py", line 40, in test_missing_file
    self.assertEqual(e.exit_status, 44, str(e))
AssertionError: 37

----------------------------------------------------------------------
Ran 1 test in 1.584s

FAILED (failures=1)
Test failed: <unittest.runner.TextTestResult run=1 errors=0 failures=1>
error: Test failed: <unittest.runner.TextTestResult run=1 errors=0 failures=1>
—————————————————————————————

-Scott



On Jan 18, 2017, at 9:38 AM, Kenneth Loafman <[hidden email]> wrote:

No, this is not good, but what you describe is practically the same problem I got when testfiles was not being fully removed on Sierra, but from what I can tell from above, the offending tests are being skipped.  To run a single test, do this on the user console from within duplicity:




_______________________________________________
Duplicity-talk mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/duplicity-talk
Reply | Threaded
Open this post in threaded view
|

Re: Errors during test phase of installation.....

duplicity-talk mailing list
I think you are right in your analysis.  The issue with permissions is something I encountered.  It works fine on Linux, but not on MacOS.  Running as root is one workaround.  The other is to just not run the tests on MacOS since it seems to be an issue with it's permissions (you should not be able to create something you can't delete).


On Thu, Feb 2, 2017 at 11:27 AM, Scott Hannahs <[hidden email]> wrote:
Ken,

I am sorry, I did not get back to you on this….   I thought I had written the email.

I have resolved almost all the test issues.  The main problem was that in setting up and running the test, there are a huge number of open files simultaneously and this runs up against the Mac OS X default setting for number of open files per process.  That defaults to 256 files.  It turns out that the duplicity test needs more than that simultaneous open files and if I set that limit to 8192 most of the tests pass.  I think all the remaining test issues are due to the missing paramiko version 2.X backend.

The clues was the “AssertionError 37” which had insufficient file system resources or something like that when I looked at the code.  So when I give it a lot more file resources it works.  This is only during the testing phase since I think I have been running it on big directories with the default values.

The other big error was that the un-tar of the test files would set file permissions and owners so that the test user could not delete them.   I set a note that the testing phase has to run as root.  It shouldn’t but that seems to be an issue.  If the no-owner no-group flags were set on the untar command for the test directories it might work.  But that may mess up some of the tests.

The librsyncmodule is compiled correctly.  As I said, I have installed it and run duplicity for a long time using the rsync and now S3 backend.

-Scott

On Feb 2, 2017, at 12:16 PM, Kenneth Loafman <[hidden email]> wrote:

When you run 'setup.py' test command, it has to compile the _librsyncmodule.c module to make the interface _librsync.so module, which is the interface to the system installed librsync.so module and then copy it.  Just part of Python's setup process.

I'm still unable to reproduce your problem.  Are there restrictive permissions in the /sw/src/fink.build directories?


On Sun, Jan 29, 2017 at 3:38 PM, Scott Hannahs <[hidden email]> wrote:
Ken,

tar seems to be available to the tests.  I ran just the first test (it fails) as a stand alone as you suggest.
The command used was:
sudo python2.7 -u setup.py test -s testing.functional.test_badupload.BadUploadTest > ~/Desktop/badupload.out.txt 2> ~/Desktop/badupload.err.txt

This is the “test missing file” in the "bad upload” test.

The standard out seems normal, but I am unclear as to why it is copying the librsync library which is already installed.  Maybe it is not where the testing routines expect it?

running test
running egg_info
writing requirements to duplicity.egg-info/requires.txt
writing duplicity.egg-info/PKG-INFO
writing top-level names to duplicity.egg-info/top_level.txt
writing dependency_links to duplicity.egg-info/dependency_links.txt
reading manifest file 'duplicity.egg-info/SOURCES.txt'
writing manifest file 'duplicity.egg-info/SOURCES.txt'
running build_ext
copying build/lib.macosx-10.11-x86_64-2.7/duplicity/_librsync.so -> duplicity
—————————————————————————————

The error output shows an assertion error.  As far as I can tell it is getting a return value of 37 and expected a return value of 44????  From grepping the sources this means that there are not enough file descriptors?  I think the Mac OS X default is a paltry 256.

test_missing_file (testing.functional.test_badupload.BadUploadTest) ... FAIL

======================================================================
FAIL: test_missing_file (testing.functional.test_badupload.BadUploadTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/sw/src/fink.build/duplicity-0.7.11-3A/duplicity-0.7.11/testing/functional/test_badupload.py", line 40, in test_missing_file
    self.assertEqual(e.exit_status, 44, str(e))
AssertionError: 37

----------------------------------------------------------------------
Ran 1 test in 1.584s

FAILED (failures=1)
Test failed: <unittest.runner.TextTestResult run=1 errors=0 failures=1>
error: Test failed: <unittest.runner.TextTestResult run=1 errors=0 failures=1>
—————————————————————————————

-Scott



On Jan 18, 2017, at 9:38 AM, Kenneth Loafman <[hidden email]> wrote:

No, this is not good, but what you describe is practically the same problem I got when testfiles was not being fully removed on Sierra, but from what I can tell from above, the offending tests are being skipped.  To run a single test, do this on the user console from within duplicity:





_______________________________________________
Duplicity-talk mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/duplicity-talk
Reply | Threaded
Open this post in threaded view
|

Re: Errors during test phase of installation.....

duplicity-talk mailing list
Agree.  The fact that I can run most of the tests as root is fine for those who want to run the tests, I have put in some developer notes on the packaging.  I think you can delete those files and a simple test running as the non-priveleged account it is not the tar command that is creating the files that can’t be deleted.  Something else is weird.

Maybe when I get around to working on the next package version I will get that fully debugged.

-Scott

On Feb 3, 2017, at 3:19 PM, Kenneth Loafman <[hidden email]> wrote:

I think you are right in your analysis.  The issue with permissions is something I encountered.  It works fine on Linux, but not on MacOS.  Running as root is one workaround.  The other is to just not run the tests on MacOS since it seems to be an issue with it's permissions (you should not be able to create something you can't delete).


On Thu, Feb 2, 2017 at 11:27 AM, Scott Hannahs <[hidden email]> wrote:
Ken,

I am sorry, I did not get back to you on this….   I thought I had written the email.

I have resolved almost all the test issues.  The main problem was that in setting up and running the test, there are a huge number of open files simultaneously and this runs up against the Mac OS X default setting for number of open files per process.  That defaults to 256 files.  It turns out that the duplicity test needs more than that simultaneous open files and if I set that limit to 8192 most of the tests pass.  I think all the remaining test issues are due to the missing paramiko version 2.X backend.

The clues was the “AssertionError 37” which had insufficient file system resources or something like that when I looked at the code.  So when I give it a lot more file resources it works.  This is only during the testing phase since I think I have been running it on big directories with the default values.

The other big error was that the un-tar of the test files would set file permissions and owners so that the test user could not delete them.   I set a note that the testing phase has to run as root.  It shouldn’t but that seems to be an issue.  If the no-owner no-group flags were set on the untar command for the test directories it might work.  But that may mess up some of the tests.

The librsyncmodule is compiled correctly.  As I said, I have installed it and run duplicity for a long time using the rsync and now S3 backend.

-Scott

On Feb 2, 2017, at 12:16 PM, Kenneth Loafman <[hidden email]> wrote:

When you run 'setup.py' test command, it has to compile the _librsyncmodule.c module to make the interface _librsync.so module, which is the interface to the system installed librsync.so module and then copy it.  Just part of Python's setup process.

I'm still unable to reproduce your problem.  Are there restrictive permissions in the /sw/src/fink.build directories?


On Sun, Jan 29, 2017 at 3:38 PM, Scott Hannahs <[hidden email]> wrote:
Ken,

tar seems to be available to the tests.  I ran just the first test (it fails) as a stand alone as you suggest.
The command used was:
sudo python2.7 -u setup.py test -s testing.functional.test_badupload.BadUploadTest > ~/Desktop/badupload.out.txt 2> ~/Desktop/badupload.err.txt

This is the “test missing file” in the "bad upload” test.

The standard out seems normal, but I am unclear as to why it is copying the librsync library which is already installed.  Maybe it is not where the testing routines expect it?

running test
running egg_info
writing requirements to duplicity.egg-info/requires.txt
writing duplicity.egg-info/PKG-INFO
writing top-level names to duplicity.egg-info/top_level.txt
writing dependency_links to duplicity.egg-info/dependency_links.txt
reading manifest file 'duplicity.egg-info/SOURCES.txt'
writing manifest file 'duplicity.egg-info/SOURCES.txt'
running build_ext
copying build/lib.macosx-10.11-x86_64-2.7/duplicity/_librsync.so -> duplicity
—————————————————————————————

The error output shows an assertion error.  As far as I can tell it is getting a return value of 37 and expected a return value of 44????  From grepping the sources this means that there are not enough file descriptors?  I think the Mac OS X default is a paltry 256.

test_missing_file (testing.functional.test_badupload.BadUploadTest) ... FAIL

======================================================================
FAIL: test_missing_file (testing.functional.test_badupload.BadUploadTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/sw/src/fink.build/duplicity-0.7.11-3A/duplicity-0.7.11/testing/functional/test_badupload.py", line 40, in test_missing_file
    self.assertEqual(e.exit_status, 44, str(e))
AssertionError: 37

----------------------------------------------------------------------
Ran 1 test in 1.584s

FAILED (failures=1)
Test failed: <unittest.runner.TextTestResult run=1 errors=0 failures=1>
error: Test failed: <unittest.runner.TextTestResult run=1 errors=0 failures=1>
—————————————————————————————

-Scott



On Jan 18, 2017, at 9:38 AM, Kenneth Loafman <[hidden email]> wrote:

No, this is not good, but what you describe is practically the same problem I got when testfiles was not being fully removed on Sierra, but from what I can tell from above, the offending tests are being skipped.  To run a single test, do this on the user console from within duplicity:






_______________________________________________
Duplicity-talk mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/duplicity-talk
Reply | Threaded
Open this post in threaded view
|

Re: Errors during test phase of installation.....

duplicity-talk mailing list
On 04.02.2017 00:23, Scott Hannahs via Duplicity-talk wrote:
> The other big error was that the un-tar of the test files would set file permissions and owners so that the test user could not delete them.   I set a note that the testing phase has to run as root.  It shouldn’t but that seems to be an issue.  If the no-owner no-group flags were set on the untar command for the test directories it might work.  But that may mess up some of the tests.

Scott,

can you explain this a little bit more detailed? do you mean that untar-ing the duplicity tarball creates files/folders with improper permissions?

what do those improper persmissions/owner values look like?

..ede/duply.net

_______________________________________________
Duplicity-talk mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/duplicity-talk
Reply | Threaded
Open this post in threaded view
|

Re: Errors during test phase of installation.....

duplicity-talk mailing list
I'll let Scott answer for El Capitan, but for Sierra, here's the reproduction.

On MacOS Sierra:

ken@dione:~$ mkdir -p foo/bar
ken@dione:~$ rm -r foo
ken@dione:~$ mkdir -p foo/bar
ken@dione:~$ chmod 0000 foo/bar
ken@dione:~$ rm -r foo
rm: foo/bar: Permission denied
rm: foo: Directory not empty
ken@dione:~$ rm -rf foo
rm: foo/bar: Permission denied
rm: foo: Directory not empty

On Linux:

ken@yakety:~$ mkdir -p foo/bar
ken@yakety:~$ rm -r foo
ken@yakety:~$ mkdir -p foo/bar
ken@yakety:~$ chmod 0000 foo/bar
ken@yakety:~$ rm -r foo
rm: descend into write-protected directory 'foo/bar'? y
rm: remove write-protected directory 'foo/bar'? y
ken@yakety:~$ rm -rf foo

It's a fairly major difference if you expect 'rm -rf' to remove everything you own.  This happens in two tests in testing/functional/test_selection.py, TestLockedFoldersNoError, lines 982 and 999, at the statements 986 and 1005, thus the check for platform before those tests.

...Ken


On Sun, Feb 5, 2017 at 7:28 AM, edgar.soldin--- via Duplicity-talk <[hidden email]> wrote:
On 04.02.2017 00:23, Scott Hannahs via Duplicity-talk wrote:
> The other big error was that the un-tar of the test files would set file permissions and owners so that the test user could not delete them.   I set a note that the testing phase has to run as root.  It shouldn’t but that seems to be an issue.  If the no-owner no-group flags were set on the untar command for the test directories it might work.  But that may mess up some of the tests.

Scott,

can you explain this a little bit more detailed? do you mean that untar-ing the duplicity tarball creates files/folders with improper permissions?

what do those improper persmissions/owner values look like?

..ede/duply.net

_______________________________________________
Duplicity-talk mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/duplicity-talk


_______________________________________________
Duplicity-talk mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/duplicity-talk
Reply | Threaded
Open this post in threaded view
|

Re: Errors during test phase of installation.....

duplicity-talk mailing list
Ken,

i just wonder if
- setting a proper umask
- switching to the proper user
beforehand would mitigate the extraction issue, if it is the initial extraction at all.

also there is this tar switch, from the manpage https://www.gnu.org/software/tar/manual/tar.html
"
`--preserve-permissions'
`--same-permissions'
`-p'

    When tar is extracting an archive, it normally subtracts the users' umask from the permissions specified in the archive and uses that number as the permissions to create the destination file. Specifying this option instructs tar that it should use the permissions directly from the archive. See section Setting Access Permissions.
"

i see your issue with the 'rm -rf' below, but i guess what rm is doing there as root on linux is probably not posix at all and chmod-ing your target beforehand would be expected.

..ede

On 05.02.2017 17:08, Kenneth Loafman wrote:

> I'll let Scott answer for El Capitan, but for Sierra, here's the
> reproduction.
>
> On MacOS Sierra:
>
> ken@dione:~$ mkdir -p foo/bar
> ken@dione:~$ rm -r foo
> ken@dione:~$ mkdir -p foo/bar
> ken@dione:~$ chmod 0000 foo/bar
> ken@dione:~$ rm -r foo
> rm: foo/bar: Permission denied
> rm: foo: Directory not empty
> ken@dione:~$ rm -rf foo
> rm: foo/bar: Permission denied
> rm: foo: Directory not empty
>
> On Linux:
>
> ken@yakety:~$ mkdir -p foo/bar
> ken@yakety:~$ rm -r foo
> ken@yakety:~$ mkdir -p foo/bar
> ken@yakety:~$ chmod 0000 foo/bar
> ken@yakety:~$ rm -r foo
> rm: descend into write-protected directory 'foo/bar'? y
> rm: remove write-protected directory 'foo/bar'? y
> ken@yakety:~$ rm -rf foo
>
> It's a fairly major difference if you expect 'rm -rf' to remove everything
> you own.  This happens in two tests in
> testing/functional/test_selection.py, TestLockedFoldersNoError, lines 982
> and 999, at the statements 986 and 1005, thus the check for platform before
> those tests.
>
> ...Ken
>
>
> On Sun, Feb 5, 2017 at 7:28 AM, edgar.soldin--- via Duplicity-talk <
> [hidden email]> wrote:
>
>> On 04.02.2017 00:23, Scott Hannahs via Duplicity-talk wrote:
>>> The other big error was that the un-tar of the test files would set file
>> permissions and owners so that the test user could not delete them.   I set
>> a note that the testing phase has to run as root.  It shouldn’t but that
>> seems to be an issue.  If the no-owner no-group flags were set on the untar
>> command for the test directories it might work.  But that may mess up some
>> of the tests.
>>
>> Scott,
>>
>> can you explain this a little bit more detailed? do you mean that
>> untar-ing the duplicity tarball creates files/folders with improper
>> permissions?
>>
>> what do those improper persmissions/owner values look like?
>>
>> ..ede/duply.net
>>
>> _______________________________________________
>> Duplicity-talk mailing list
>> [hidden email]
>> https://lists.nongnu.org/mailman/listinfo/duplicity-talk
>>
>

_______________________________________________
Duplicity-talk mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/duplicity-talk
12