Errors during test phase with duplicity

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|

Errors during test phase with duplicity

duplicity-talk mailing list
When I build duplicity and try to run the tests, I get several errors.   One question that might resolve some of them.

Are the par2 libraries/commands necessary for duplicity?  Should successful testing depend on those libraries or should the test sequence skip over those if the par2 libraries/commands are not installed?

This is on Mac OS X

-Scott



_______________________________________________
Duplicity-talk mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/duplicity-talk
Reply | Threaded
Open this post in threaded view
|

Re: Errors during test phase with duplicity

duplicity-talk mailing list
Scott,

par2 binaries are only needed for the par2 backend. so unless the par2 backend is tested, they are not needed.

..ede/duply.net

On 9/3/2017 20:04, Scott Hannahs via Duplicity-talk wrote:

> When I build duplicity and try to run the tests, I get several errors.   One question that might resolve some of them.
>
> Are the par2 libraries/commands necessary for duplicity?  Should successful testing depend on those libraries or should the test sequence skip over those if the par2 libraries/commands are not installed?
>
> This is on Mac OS X
>
> -Scott
>
>
>
> _______________________________________________
> Duplicity-talk mailing list
> [hidden email]
> https://lists.nongnu.org/mailman/listinfo/duplicity-talk
>


_______________________________________________
Duplicity-talk mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/duplicity-talk
Reply | Threaded
Open this post in threaded view
|

Re: Errors during test phase with duplicity

duplicity-talk mailing list
Is there a flag to disable certain backend tests?  Or should they detect if that backend is not installed and skip over it?

-Scott

> On Sep 3, 2017, at 3:39 PM, edgar.soldin--- via Duplicity-talk <[hidden email]> wrote:
>
> Scott,
>
> par2 binaries are only needed for the par2 backend. so unless the par2 backend is tested, they are not needed.
>
> ..ede/duply.net
>
> On 9/3/2017 20:04, Scott Hannahs via Duplicity-talk wrote:
>> When I build duplicity and try to run the tests, I get several errors.   One question that might resolve some of them.
>>
>> Are the par2 libraries/commands necessary for duplicity?  Should successful testing depend on those libraries or should the test sequence skip over those if the par2 libraries/commands are not installed?
>>
>> This is on Mac OS X
>>
>> -Scott
>>
>>
>>
>> _______________________________________________
>> Duplicity-talk mailing list
>> [hidden email]
>> https://lists.nongnu.org/mailman/listinfo/duplicity-talk
>>
>
>
> _______________________________________________
> Duplicity-talk mailing list
> [hidden email]
> https://lists.nongnu.org/mailman/listinfo/duplicity-talk


_______________________________________________
Duplicity-talk mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/duplicity-talk
Reply | Threaded
Open this post in threaded view
|

Re: Errors during test phase with duplicity

duplicity-talk mailing list
The tests should detect the absence and skip over it, so please submit a bug report.

I would go ahead and install the options needed if you want to test, like so:

    # apt-get install -y \
            build-essential \
            bzr \
            intltool \
            lftp \
            librsync-dev \
            libffi-dev \
            libssl-dev \
            openssl \
            par2 \
            python-dev \
            rdiff \
            tzdata

I've attached a requirement.txt file for testing as well.  Use it like so:

    # pip install --requirement requirements.txt

Hope that helps.

...Ken


On Sun, Sep 3, 2017 at 4:14 PM, Scott Hannahs via Duplicity-talk <[hidden email]> wrote:
Is there a flag to disable certain backend tests?  Or should they detect if that backend is not installed and skip over it?

-Scott

> On Sep 3, 2017, at 3:39 PM, edgar.soldin--- via Duplicity-talk <[hidden email]> wrote:
>
> Scott,
>
> par2 binaries are only needed for the par2 backend. so unless the par2 backend is tested, they are not needed.
>
> ..ede/duply.net
>
> On 9/3/2017 20:04, Scott Hannahs via Duplicity-talk wrote:
>> When I build duplicity and try to run the tests, I get several errors.   One question that might resolve some of them.
>>
>> Are the par2 libraries/commands necessary for duplicity?  Should successful testing depend on those libraries or should the test sequence skip over those if the par2 libraries/commands are not installed?
>>
>> This is on Mac OS X
>>
>> -Scott
>>
>>
>>
>> _______________________________________________
>> Duplicity-talk mailing list
>> [hidden email]
>> https://lists.nongnu.org/mailman/listinfo/duplicity-talk
>>
>
>
> _______________________________________________
> Duplicity-talk mailing list
> [hidden email]
> https://lists.nongnu.org/mailman/listinfo/duplicity-talk


_______________________________________________
Duplicity-talk mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/duplicity-talk


_______________________________________________
Duplicity-talk mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/duplicity-talk

requirements.txt (164 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Errors during test phase with duplicity

duplicity-talk mailing list
Ken,

I think it helps.  Of course it leads to more questions… :-)
I think this bug is already reported.  Bug #1399843 "par2 is required to run duplicity ./setup.py test”

Actually I am trying to set up the Mac OS X fink package still and I am getting some test failures.  For running duplicity, I have the following which is *close* to what you have.  Some such as libffi are already installed as part of python27.  This should cover most of the backends that are popular?

Depends: <<
        python27,
        librsync-shlibs (>= 0.9.7-1002),
        gnupg-unified,
        fasteners-py27,
        intltool40,
        paramiko-py27,
        pycrypto-py27,
python-pycryptopp,
        pexpect-py27,
        lftp,
        ncftp,
        par2,
        boto-py27 (>= 2.1.1),
        requests-oauthlib-py27
<<

For testing, I have the additional items, but not pycodestyle, pydrive, pylint, python-cloudfiles or pyrax.  I assume that python-cloudfiles and pyrax are for the cloud files (cf) backend which I haven’t implemented yet.

Are the others necessary only when RUN_CODE_TESTS=1??

However one of the tests that fails is rdiff that I have no build for since it seems to be fairly obsolete and not a maintained.  I only see it in one test for a large file.  I can live with that one failure rather than install yet another utility.
./testing/unit/test_diffdir.py:        assert not os.system("rdiff patch testfiles/dir2/largefile "

    TestDepends: <<
        mock-py27,
        pluggy-py27,
        py-py27,
        tox-py27
<<

There are 12 tests that give an error.  Some of these are entering passwords, but I thought that pexpect would handle those but there is an empty string error.

The rest are Restart errors and those give a CmdError=30 which I have no idea what that is but seems to be a generic error?

ERROR: test_piped_password (testing.functional.test_final.FinalTest)
ERROR: test_piped_password (testing.functional.test_final.OldFilenamesFinalTest)
ERROR: test_piped_password (testing.functional.test_final.ShortFilenamesFinalTest)

ERROR: test_new_file (testing.functional.test_restart.RestartTest)
ERROR: test_split_after_large (testing.functional.test_restart.RestartTest)
ERROR: test_split_after_small (testing.functional.test_restart.RestartTest)
ERROR: test_split_inside_large (testing.functional.test_restart.RestartTest)
ERROR: test_ignore_double_snapshot (testing.functional.test_restart.RestartTestWithoutEncryption)
ERROR: test_new_file (testing.functional.test_restart.RestartTestWithoutEncryption)
ERROR: test_split_after_large (testing.functional.test_restart.RestartTestWithoutEncryption)
ERROR: test_split_after_small (testing.functional.test_restart.RestartTestWithoutEncryption)
ERROR: test_split_inside_large (testing.functional.test_restart.RestartTestWithoutEncryption)

-Scott


On Sep 4, 2017, at 6:58 AM, Kenneth Loafman <[hidden email]> wrote:

The tests should detect the absence and skip over it, so please submit a bug report.

I would go ahead and install the options needed if you want to test, like so:

    # apt-get install -y \
            build-essential \
            bzr \
            intltool \
            lftp \
            librsync-dev \
            libffi-dev \
            libssl-dev \
            openssl \
            par2 \
            python-dev \
            rdiff \
            tzdata

I've attached a requirement.txt file for testing as well.  Use it like so:

    # pip install --requirement requirements.txt

Hope that helps.

...Ken


On Sun, Sep 3, 2017 at 4:14 PM, Scott Hannahs via Duplicity-talk <[hidden email]> wrote:
Is there a flag to disable certain backend tests?  Or should they detect if that backend is not installed and skip over it?

-Scott

> On Sep 3, 2017, at 3:39 PM, edgar.soldin--- via Duplicity-talk <[hidden email]> wrote:
>
> Scott,
>
> par2 binaries are only needed for the par2 backend. so unless the par2 backend is tested, they are not needed.
>
> ..ede/duply.net
>
> On 9/3/2017 20:04, Scott Hannahs via Duplicity-talk wrote:
>> When I build duplicity and try to run the tests, I get several errors.   One question that might resolve some of them.
>>
>> Are the par2 libraries/commands necessary for duplicity?  Should successful testing depend on those libraries or should the test sequence skip over those if the par2 libraries/commands are not installed?
>>
>> This is on Mac OS X
>>
>> -Scott
>>
>>
>>
>> _______________________________________________
>> Duplicity-talk mailing list
>> [hidden email]
>> https://lists.nongnu.org/mailman/listinfo/duplicity-talk
>>
>
>
> _______________________________________________
> Duplicity-talk mailing list
> [hidden email]
> https://lists.nongnu.org/mailman/listinfo/duplicity-talk


_______________________________________________
Duplicity-talk mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/duplicity-talk

<requirements.txt>


_______________________________________________
Duplicity-talk mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/duplicity-talk
Reply | Threaded
Open this post in threaded view
|

Re: Errors during test phase with duplicity

duplicity-talk mailing list
Yes, the extras are there for the code tests and the cloudfiles tests, so you don't need them unless you want them.

We normally use tox for testing.  It sets up a clean environment and then runs ./setup.py test.  When I run '$ tox -e py27', I get:

Ran 419 tests in 476.537s

OK (skipped=2)

so that tells me that something is setup differently between us.

Please do:

    $ pip2 freeze > pip-freeze.txt
    $ ./setup.py test &> setup-test.log

and attach the files to your reply.

...Thanks,
...Ken


On Mon, Sep 4, 2017 at 3:40 PM, Scott Hannahs <[hidden email]> wrote:
Ken,

I think it helps.  Of course it leads to more questions… :-)
I think this bug is already reported.  Bug #1399843 "par2 is required to run duplicity ./setup.py test”

Actually I am trying to set up the Mac OS X fink package still and I am getting some test failures.  For running duplicity, I have the following which is *close* to what you have.  Some such as libffi are already installed as part of python27.  This should cover most of the backends that are popular?

Depends: <<
        python27,
        librsync-shlibs (>= 0.9.7-1002),
        gnupg-unified,
        fasteners-py27,
        intltool40,
        paramiko-py27,
        pycrypto-py27,
python-pycryptopp,
        pexpect-py27,
        lftp,
        ncftp,
        par2,
        boto-py27 (>= 2.1.1),
        requests-oauthlib-py27
<<

For testing, I have the additional items, but not pycodestyle, pydrive, pylint, python-cloudfiles or pyrax.  I assume that python-cloudfiles and pyrax are for the cloud files (cf) backend which I haven’t implemented yet.

Are the others necessary only when RUN_CODE_TESTS=1??

However one of the tests that fails is rdiff that I have no build for since it seems to be fairly obsolete and not a maintained.  I only see it in one test for a large file.  I can live with that one failure rather than install yet another utility.
./testing/unit/test_diffdir.py:        assert not os.system("rdiff patch testfiles/dir2/largefile "

    TestDepends: <<
        mock-py27,
        pluggy-py27,
        py-py27,
        tox-py27
<<

There are 12 tests that give an error.  Some of these are entering passwords, but I thought that pexpect would handle those but there is an empty string error.

The rest are Restart errors and those give a CmdError=30 which I have no idea what that is but seems to be a generic error?

ERROR: test_piped_password (testing.functional.test_final.FinalTest)
ERROR: test_piped_password (testing.functional.test_final.OldFilenamesFinalTest)
ERROR: test_piped_password (testing.functional.test_final.ShortFilenamesFinalTest)

ERROR: test_new_file (testing.functional.test_restart.RestartTest)
ERROR: test_split_after_large (testing.functional.test_restart.RestartTest)
ERROR: test_split_after_small (testing.functional.test_restart.RestartTest)
ERROR: test_split_inside_large (testing.functional.test_restart.RestartTest)
ERROR: test_ignore_double_snapshot (testing.functional.test_restart.RestartTestWithoutEncryption)
ERROR: test_new_file (testing.functional.test_restart.RestartTestWithoutEncryption)
ERROR: test_split_after_large (testing.functional.test_restart.RestartTestWithoutEncryption)
ERROR: test_split_after_small (testing.functional.test_restart.RestartTestWithoutEncryption)
ERROR: test_split_inside_large (testing.functional.test_restart.RestartTestWithoutEncryption)

-Scott


On Sep 4, 2017, at 6:58 AM, Kenneth Loafman <[hidden email]> wrote:

The tests should detect the absence and skip over it, so please submit a bug report.

I would go ahead and install the options needed if you want to test, like so:

    # apt-get install -y \
            build-essential \
            bzr \
            intltool \
            lftp \
            librsync-dev \
            libffi-dev \
            libssl-dev \
            openssl \
            par2 \
            python-dev \
            rdiff \
            tzdata

I've attached a requirement.txt file for testing as well.  Use it like so:

    # pip install --requirement requirements.txt

Hope that helps.

...Ken


On Sun, Sep 3, 2017 at 4:14 PM, Scott Hannahs via Duplicity-talk <[hidden email]> wrote:
Is there a flag to disable certain backend tests?  Or should they detect if that backend is not installed and skip over it?

-Scott

> On Sep 3, 2017, at 3:39 PM, edgar.soldin--- via Duplicity-talk <[hidden email]> wrote:
>
> Scott,
>
> par2 binaries are only needed for the par2 backend. so unless the par2 backend is tested, they are not needed.
>
> ..ede/duply.net
>
> On 9/3/2017 20:04, Scott Hannahs via Duplicity-talk wrote:
>> When I build duplicity and try to run the tests, I get several errors.   One question that might resolve some of them.
>>
>> Are the par2 libraries/commands necessary for duplicity?  Should successful testing depend on those libraries or should the test sequence skip over those if the par2 libraries/commands are not installed?
>>
>> This is on Mac OS X
>>
>> -Scott
>>
>>
>>
>> _______________________________________________
>> Duplicity-talk mailing list
>> [hidden email]
>> https://lists.nongnu.org/mailman/listinfo/duplicity-talk
>>
>
>
> _______________________________________________
> Duplicity-talk mailing list
> [hidden email]
> https://lists.nongnu.org/mailman/listinfo/duplicity-talk


_______________________________________________
Duplicity-talk mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/duplicity-talk

<requirements.txt>



_______________________________________________
Duplicity-talk mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/duplicity-talk
Reply | Threaded
Open this post in threaded view
|

Re: Errors during test phase with duplicity

duplicity-talk mailing list
Ken,

So, my setup is a bit different, for example I included pycryptopp which isn’t in your list.


I ran
$ pip2-py2.7 freeze > Desktop/pip-freeze.txt
and here is the list of installed stuff:


And 
$ sudo -u fink-bld  ./setup.py test &> ~/Desktop/setup-test.log
Gives the following, the user fink-bld is the owner of the files as they are built.


I can build this as root and I think I get the same errors.  I will check that again.  The testing should not require root access?

Scott

On Sep 5, 2017, at 10:57 AM, Kenneth Loafman <[hidden email]> wrote:

Yes, the extras are there for the code tests and the cloudfiles tests, so you don't need them unless you want them.

We normally use tox for testing.  It sets up a clean environment and then runs ./setup.py test.  When I run '$ tox -e py27', I get:

Ran 419 tests in 476.537s

OK (skipped=2)

so that tells me that something is setup differently between us.

Please do:

    $ pip2 freeze > pip-freeze.txt
    $ ./setup.py test &> setup-test.log

and attach the files to your reply.

...Thanks,
...Ken


On Mon, Sep 4, 2017 at 3:40 PM, Scott Hannahs <[hidden email]> wrote:
Ken,

I think it helps.  Of course it leads to more questions… :-)
I think this bug is already reported.  Bug #1399843 "par2 is required to run duplicity ./setup.py test”

Actually I am trying to set up the Mac OS X fink package still and I am getting some test failures.  For running duplicity, I have the following which is *close* to what you have.  Some such as libffi are already installed as part of python27.  This should cover most of the backends that are popular?

Depends: <<
        python27,
        librsync-shlibs (>= 0.9.7-1002),
        gnupg-unified,
        fasteners-py27,
        intltool40,
        paramiko-py27,
        pycrypto-py27,
python-pycryptopp,
        pexpect-py27,
        lftp,
        ncftp,
        par2,
        boto-py27 (>= 2.1.1),
        requests-oauthlib-py27
<<

For testing, I have the additional items, but not pycodestyle, pydrive, pylint, python-cloudfiles or pyrax.  I assume that python-cloudfiles and pyrax are for the cloud files (cf) backend which I haven’t implemented yet.

Are the others necessary only when RUN_CODE_TESTS=1??

However one of the tests that fails is rdiff that I have no build for since it seems to be fairly obsolete and not a maintained.  I only see it in one test for a large file.  I can live with that one failure rather than install yet another utility.
./testing/unit/test_diffdir.py:        assert not os.system("rdiff patch testfiles/dir2/largefile "

    TestDepends: <<
        mock-py27,
        pluggy-py27,
        py-py27,
        tox-py27
<<

There are 12 tests that give an error.  Some of these are entering passwords, but I thought that pexpect would handle those but there is an empty string error.

The rest are Restart errors and those give a CmdError=30 which I have no idea what that is but seems to be a generic error?

ERROR: test_piped_password (testing.functional.test_final.FinalTest)
ERROR: test_piped_password (testing.functional.test_final.OldFilenamesFinalTest)
ERROR: test_piped_password (testing.functional.test_final.ShortFilenamesFinalTest)

ERROR: test_new_file (testing.functional.test_restart.RestartTest)
ERROR: test_split_after_large (testing.functional.test_restart.RestartTest)
ERROR: test_split_after_small (testing.functional.test_restart.RestartTest)
ERROR: test_split_inside_large (testing.functional.test_restart.RestartTest)
ERROR: test_ignore_double_snapshot (testing.functional.test_restart.RestartTestWithoutEncryption)
ERROR: test_new_file (testing.functional.test_restart.RestartTestWithoutEncryption)
ERROR: test_split_after_large (testing.functional.test_restart.RestartTestWithoutEncryption)
ERROR: test_split_after_small (testing.functional.test_restart.RestartTestWithoutEncryption)
ERROR: test_split_inside_large (testing.functional.test_restart.RestartTestWithoutEncryption)

-Scott


On Sep 4, 2017, at 6:58 AM, Kenneth Loafman <[hidden email]> wrote:

The tests should detect the absence and skip over it, so please submit a bug report.

I would go ahead and install the options needed if you want to test, like so:

    # apt-get install -y \
            build-essential \
            bzr \
            intltool \
            lftp \
            librsync-dev \
            libffi-dev \
            libssl-dev \
            openssl \
            par2 \
            python-dev \
            rdiff \
            tzdata

I've attached a requirement.txt file for testing as well.  Use it like so:

    # pip install --requirement requirements.txt

Hope that helps.

...Ken


On Sun, Sep 3, 2017 at 4:14 PM, Scott Hannahs via Duplicity-talk <[hidden email]> wrote:
Is there a flag to disable certain backend tests?  Or should they detect if that backend is not installed and skip over it?

-Scott

> On Sep 3, 2017, at 3:39 PM, edgar.soldin--- via Duplicity-talk <[hidden email]> wrote:
>
> Scott,
>
> par2 binaries are only needed for the par2 backend. so unless the par2 backend is tested, they are not needed.
>
> ..ede/duply.net
>
> On 9/3/2017 20:04, Scott Hannahs via Duplicity-talk wrote:
>> When I build duplicity and try to run the tests, I get several errors.   One question that might resolve some of them.
>>
>> Are the par2 libraries/commands necessary for duplicity?  Should successful testing depend on those libraries or should the test sequence skip over those if the par2 libraries/commands are not installed?
>>
>> This is on Mac OS X
>>
>> -Scott
>>
>>
>>
>> _______________________________________________
>> Duplicity-talk mailing list
>> [hidden email]
>> https://lists.nongnu.org/mailman/listinfo/duplicity-talk
>>
>
>
> _______________________________________________
> Duplicity-talk mailing list
> [hidden email]
> https://lists.nongnu.org/mailman/listinfo/duplicity-talk


_______________________________________________
Duplicity-talk mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/duplicity-talk

<requirements.txt>




_______________________________________________
Duplicity-talk mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/duplicity-talk

pip-freeze.txt (865 bytes) Download Attachment
setup-test.log (108K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Errors during test phase with duplicity

duplicity-talk mailing list
In reply to this post by duplicity-talk mailing list
Ken,

I can build this as root and I think I get the same errors.  I will check that again.  The testing should not require root access?

I take that back.  When the testing is run as root I get less errors (3 instead of 12) :
% grep FAILED duplicity.*.err.txt
duplicity.nonroot.err.txt:FAILED (failures=1, errors=12, skipped=5)
duplicity.root.err.txt:FAILED (failures=2, errors=3, skipped=5)

As root, just the "piped password” tests fail.  The other “restart” test errors seem to work.  But restarting shouldn’t necessitate root permissions?  As root, there is a second and different failure in the “patching” sequence??

test_patch_seq2ropath (testing.unit.test_patchdir.TestInnerFuncs)
Test patching sequence ... FAIL

======================================================================
FAIL: test_patch_seq2ropath (testing.unit.test_patchdir.TestInnerFuncs)
Test patching sequence
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/sw/src/fink.build/duplicity-0.7.14-4/duplicity-0.7.14/testing/unit/test_patchdir.py", line 290, in test_patch_seq2ropath
    testseq([self.snapshot()], ("%s 600" % ids), "hello, world!")
  File "/sw/src/fink.build/duplicity-0.7.14-4/duplicity-0.7.14/testing/unit/test_patchdir.py", line 282, in testseq
    assert result.getperms() == perms, (result.getperms(), perms)
AssertionError: ('0:1000 600', '0:0 600')


Scott

On Sep 5, 2017, at 10:57 AM, Kenneth Loafman <[hidden email]> wrote:

Yes, the extras are there for the code tests and the cloudfiles tests, so you don't need them unless you want them.

We normally use tox for testing.  It sets up a clean environment and then runs ./setup.py test.  When I run '$ tox -e py27', I get:

Ran 419 tests in 476.537s

OK (skipped=2)

so that tells me that something is setup differently between us.

Please do:

    $ pip2 freeze > pip-freeze.txt
    $ ./setup.py test &> setup-test.log

and attach the files to your reply.

...Thanks,
...Ken


On Mon, Sep 4, 2017 at 3:40 PM, Scott Hannahs <[hidden email]> wrote:
Ken,

I think it helps.  Of course it leads to more questions… :-)
I think this bug is already reported.  Bug #1399843 "par2 is required to run duplicity ./setup.py test”

Actually I am trying to set up the Mac OS X fink package still and I am getting some test failures.  For running duplicity, I have the following which is *close* to what you have.  Some such as libffi are already installed as part of python27.  This should cover most of the backends that are popular?

Depends: <<
        python27,
        librsync-shlibs (>= 0.9.7-1002),
        gnupg-unified,
        fasteners-py27,
        intltool40,
        paramiko-py27,
        pycrypto-py27,
python-pycryptopp,
        pexpect-py27,
        lftp,
        ncftp,
        par2,
        boto-py27 (>= 2.1.1),
        requests-oauthlib-py27
<<

For testing, I have the additional items, but not pycodestyle, pydrive, pylint, python-cloudfiles or pyrax.  I assume that python-cloudfiles and pyrax are for the cloud files (cf) backend which I haven’t implemented yet.

Are the others necessary only when RUN_CODE_TESTS=1??

However one of the tests that fails is rdiff that I have no build for since it seems to be fairly obsolete and not a maintained.  I only see it in one test for a large file.  I can live with that one failure rather than install yet another utility.
./testing/unit/test_diffdir.py:        assert not os.system("rdiff patch testfiles/dir2/largefile "

    TestDepends: <<
        mock-py27,
        pluggy-py27,
        py-py27,
        tox-py27
<<

There are 12 tests that give an error.  Some of these are entering passwords, but I thought that pexpect would handle those but there is an empty string error.

The rest are Restart errors and those give a CmdError=30 which I have no idea what that is but seems to be a generic error?

ERROR: test_piped_password (testing.functional.test_final.FinalTest)
ERROR: test_piped_password (testing.functional.test_final.OldFilenamesFinalTest)
ERROR: test_piped_password (testing.functional.test_final.ShortFilenamesFinalTest)

ERROR: test_new_file (testing.functional.test_restart.RestartTest)
ERROR: test_split_after_large (testing.functional.test_restart.RestartTest)
ERROR: test_split_after_small (testing.functional.test_restart.RestartTest)
ERROR: test_split_inside_large (testing.functional.test_restart.RestartTest)
ERROR: test_ignore_double_snapshot (testing.functional.test_restart.RestartTestWithoutEncryption)
ERROR: test_new_file (testing.functional.test_restart.RestartTestWithoutEncryption)
ERROR: test_split_after_large (testing.functional.test_restart.RestartTestWithoutEncryption)
ERROR: test_split_after_small (testing.functional.test_restart.RestartTestWithoutEncryption)
ERROR: test_split_inside_large (testing.functional.test_restart.RestartTestWithoutEncryption)

-Scott


On Sep 4, 2017, at 6:58 AM, Kenneth Loafman <[hidden email]> wrote:

The tests should detect the absence and skip over it, so please submit a bug report.

I would go ahead and install the options needed if you want to test, like so:

    # apt-get install -y \
            build-essential \
            bzr \
            intltool \
            lftp \
            librsync-dev \
            libffi-dev \
            libssl-dev \
            openssl \
            par2 \
            python-dev \
            rdiff \
            tzdata

I've attached a requirement.txt file for testing as well.  Use it like so:

    # pip install --requirement requirements.txt

Hope that helps.

...Ken


On Sun, Sep 3, 2017 at 4:14 PM, Scott Hannahs via Duplicity-talk <[hidden email]> wrote:
Is there a flag to disable certain backend tests?  Or should they detect if that backend is not installed and skip over it?

-Scott

> On Sep 3, 2017, at 3:39 PM, edgar.soldin--- via Duplicity-talk <[hidden email]> wrote:
>
> Scott,
>
> par2 binaries are only needed for the par2 backend. so unless the par2 backend is tested, they are not needed.
>
> ..ede/duply.net
>
> On 9/3/2017 20:04, Scott Hannahs via Duplicity-talk wrote:
>> When I build duplicity and try to run the tests, I get several errors.   One question that might resolve some of them.
>>
>> Are the par2 libraries/commands necessary for duplicity?  Should successful testing depend on those libraries or should the test sequence skip over those if the par2 libraries/commands are not installed?
>>
>> This is on Mac OS X
>>
>> -Scott
>>
>>
>>
>> _______________________________________________
>> Duplicity-talk mailing list
>> [hidden email]
>> https://lists.nongnu.org/mailman/listinfo/duplicity-talk
>>
>
>
> _______________________________________________
> Duplicity-talk mailing list
> [hidden email]
> https://lists.nongnu.org/mailman/listinfo/duplicity-talk


_______________________________________________
Duplicity-talk mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/duplicity-talk

<requirements.txt>



_______________________________________________
Duplicity-talk mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/duplicity-talk

_______________________________________________
Duplicity-talk mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/duplicity-talk
Reply | Threaded
Open this post in threaded view
|

Re: Errors during test phase with duplicity

duplicity-talk mailing list
Scott,

You should not need root to build or test duplicity.  Is it possible that you've run pip with sudo at some point?  That could be a problem.

No, there is no problem in the patching tests.  There's something else going on there.

From the setup-test.log file, I'm seeing a lot of permission denied errors that I don't get, mostly from chown calls.  

Try with -i 
$ sudo -i -u fink-bld  ./setup.py test &> ~/Desktop/setup-test.log

or just as yourself
$ ./setup.py test &> ~/Desktop/setup-test.log

...Ken


On Tue, Sep 5, 2017 at 9:50 PM, Scott Hannahs <[hidden email]> wrote:
Ken,

I can build this as root and I think I get the same errors.  I will check that again.  The testing should not require root access?

I take that back.  When the testing is run as root I get less errors (3 instead of 12) :
% grep FAILED duplicity.*.err.txt
duplicity.nonroot.err.txt:FAILED (failures=1, errors=12, skipped=5)
duplicity.root.err.txt:FAILED (failures=2, errors=3, skipped=5)

As root, just the "piped password” tests fail.  The other “restart” test errors seem to work.  But restarting shouldn’t necessitate root permissions?  As root, there is a second and different failure in the “patching” sequence??

test_patch_seq2ropath (testing.unit.test_patchdir.TestInnerFuncs)
Test patching sequence ... FAIL

======================================================================
FAIL: test_patch_seq2ropath (testing.unit.test_patchdir.TestInnerFuncs)
Test patching sequence
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/sw/src/fink.build/duplicity-0.7.14-4/duplicity-0.7.14/testing/unit/test_patchdir.py", line 290, in test_patch_seq2ropath
    testseq([self.snapshot()], ("%s 600" % ids), "hello, world!")
  File "/sw/src/fink.build/duplicity-0.7.14-4/duplicity-0.7.14/testing/unit/test_patchdir.py", line 282, in testseq
    assert result.getperms() == perms, (result.getperms(), perms)
AssertionError: ('0:1000 600', '0:0 600')


Scott

On Sep 5, 2017, at 10:57 AM, Kenneth Loafman <[hidden email]> wrote:

Yes, the extras are there for the code tests and the cloudfiles tests, so you don't need them unless you want them.

We normally use tox for testing.  It sets up a clean environment and then runs ./setup.py test.  When I run '$ tox -e py27', I get:

Ran 419 tests in 476.537s

OK (skipped=2)

so that tells me that something is setup differently between us.

Please do:

    $ pip2 freeze > pip-freeze.txt
    $ ./setup.py test &> setup-test.log

and attach the files to your reply.

...Thanks,
...Ken


On Mon, Sep 4, 2017 at 3:40 PM, Scott Hannahs <[hidden email]> wrote:
Ken,

I think it helps.  Of course it leads to more questions… :-)
I think this bug is already reported.  Bug #1399843 "par2 is required to run duplicity ./setup.py test”

Actually I am trying to set up the Mac OS X fink package still and I am getting some test failures.  For running duplicity, I have the following which is *close* to what you have.  Some such as libffi are already installed as part of python27.  This should cover most of the backends that are popular?

Depends: <<
        python27,
        librsync-shlibs (>= 0.9.7-1002),
        gnupg-unified,
        fasteners-py27,
        intltool40,
        paramiko-py27,
        pycrypto-py27,
python-pycryptopp,
        pexpect-py27,
        lftp,
        ncftp,
        par2,
        boto-py27 (>= 2.1.1),
        requests-oauthlib-py27
<<

For testing, I have the additional items, but not pycodestyle, pydrive, pylint, python-cloudfiles or pyrax.  I assume that python-cloudfiles and pyrax are for the cloud files (cf) backend which I haven’t implemented yet.

Are the others necessary only when RUN_CODE_TESTS=1??

However one of the tests that fails is rdiff that I have no build for since it seems to be fairly obsolete and not a maintained.  I only see it in one test for a large file.  I can live with that one failure rather than install yet another utility.
./testing/unit/test_diffdir.py:        assert not os.system("rdiff patch testfiles/dir2/largefile "

    TestDepends: <<
        mock-py27,
        pluggy-py27,
        py-py27,
        tox-py27
<<

There are 12 tests that give an error.  Some of these are entering passwords, but I thought that pexpect would handle those but there is an empty string error.

The rest are Restart errors and those give a CmdError=30 which I have no idea what that is but seems to be a generic error?

ERROR: test_piped_password (testing.functional.test_final.FinalTest)
ERROR: test_piped_password (testing.functional.test_final.OldFilenamesFinalTest)
ERROR: test_piped_password (testing.functional.test_final.ShortFilenamesFinalTest)

ERROR: test_new_file (testing.functional.test_restart.RestartTest)
ERROR: test_split_after_large (testing.functional.test_restart.RestartTest)
ERROR: test_split_after_small (testing.functional.test_restart.RestartTest)
ERROR: test_split_inside_large (testing.functional.test_restart.RestartTest)
ERROR: test_ignore_double_snapshot (testing.functional.test_restart.RestartTestWithoutEncryption)
ERROR: test_new_file (testing.functional.test_restart.RestartTestWithoutEncryption)
ERROR: test_split_after_large (testing.functional.test_restart.RestartTestWithoutEncryption)
ERROR: test_split_after_small (testing.functional.test_restart.RestartTestWithoutEncryption)
ERROR: test_split_inside_large (testing.functional.test_restart.RestartTestWithoutEncryption)

-Scott


On Sep 4, 2017, at 6:58 AM, Kenneth Loafman <[hidden email]> wrote:

The tests should detect the absence and skip over it, so please submit a bug report.

I would go ahead and install the options needed if you want to test, like so:

    # apt-get install -y \
            build-essential \
            bzr \
            intltool \
            lftp \
            librsync-dev \
            libffi-dev \
            libssl-dev \
            openssl \
            par2 \
            python-dev \
            rdiff \
            tzdata

I've attached a requirement.txt file for testing as well.  Use it like so:

    # pip install --requirement requirements.txt

Hope that helps.

...Ken


On Sun, Sep 3, 2017 at 4:14 PM, Scott Hannahs via Duplicity-talk <[hidden email]> wrote:
Is there a flag to disable certain backend tests?  Or should they detect if that backend is not installed and skip over it?

-Scott

> On Sep 3, 2017, at 3:39 PM, edgar.soldin--- via Duplicity-talk <[hidden email]> wrote:
>
> Scott,
>
> par2 binaries are only needed for the par2 backend. so unless the par2 backend is tested, they are not needed.
>
> ..ede/duply.net
>
> On 9/3/2017 20:04, Scott Hannahs via Duplicity-talk wrote:
>> When I build duplicity and try to run the tests, I get several errors.   One question that might resolve some of them.
>>
>> Are the par2 libraries/commands necessary for duplicity?  Should successful testing depend on those libraries or should the test sequence skip over those if the par2 libraries/commands are not installed?
>>
>> This is on Mac OS X
>>
>> -Scott
>>
>>
>>
>> _______________________________________________
>> Duplicity-talk mailing list
>> [hidden email]
>> https://lists.nongnu.org/mailman/listinfo/duplicity-talk
>>
>
>
> _______________________________________________
> Duplicity-talk mailing list
> [hidden email]
> https://lists.nongnu.org/mailman/listinfo/duplicity-talk


_______________________________________________
Duplicity-talk mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/duplicity-talk

<requirements.txt>



_______________________________________________
Duplicity-talk mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/duplicity-talk


_______________________________________________
Duplicity-talk mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/duplicity-talk
Reply | Threaded
Open this post in threaded view
|

Re: Errors during test phase with duplicity

duplicity-talk mailing list
Ken,

I get the same errors in that test.  I did not use the -i with sudo since there isn’t a login directory with that user.
ERROR: test_piped_password (testing.functional.test_final.FinalTest)
ERROR: test_piped_password (testing.functional.test_final.OldFilenamesFinalTest)
ERROR: test_piped_password (testing.functional.test_final.ShortFilenamesFinalTest)
ERROR: test_new_file (testing.functional.test_restart.RestartTest)
ERROR: test_split_after_large (testing.functional.test_restart.RestartTest)
ERROR: test_split_after_small (testing.functional.test_restart.RestartTest)
ERROR: test_split_inside_large (testing.functional.test_restart.RestartTest)
ERROR: test_ignore_double_snapshot (testing.functional.test_restart.RestartTestWithoutEncryption)
ERROR: test_new_file (testing.functional.test_restart.RestartTestWithoutEncryption)
ERROR: test_split_after_large (testing.functional.test_restart.RestartTestWithoutEncryption)
ERROR: test_split_after_small (testing.functional.test_restart.RestartTestWithoutEncryption)
ERROR: test_split_inside_large (testing.functional.test_restart.RestartTestWithoutEncryption)

However, they seem to be a chown error?  That is certainly a root only operation.  Is that what the call to os.chown does?
  File "/sw/src/fink.build/duplicity-0.7.14-2/duplicity-0.7.14/duplicity/path.py", line 469, in <lambda>
    util.maybe_ignore_errors(lambda: os.chown(other.name, self.stat.st_uid, self.stat.st_gid))
OSError: [Errno 1] Operation not permitted: 'testfiles/cache/8545a8b798d7684a61ae2472e32d92e2/duplicity-full-signatures.20170907T222605Z.sigtar.gz’

Is that function trying to chown the destination file during a restore?  If the restore is not being done as root then it should skip that and leave the file with the different owner of the restoring account.

Here is the full test log from 
$ sudo -u fink-bld  ./setup.py test &> ~/Desktop/setup-test1.log



The good news is despite these test errors duplicity seems to be executing a nightly run on my files to an s3 backend.

-Scott



On Sep 6, 2017, at 7:55 AM, Kenneth Loafman via Duplicity-talk <[hidden email]> wrote:

Scott,

You should not need root to build or test duplicity.  Is it possible that you've run pip with sudo at some point?  That could be a problem.

No, there is no problem in the patching tests.  There's something else going on there.

From the setup-test.log file, I'm seeing a lot of permission denied errors that I don't get, mostly from chown calls.  

Try with -i 
$ sudo -i -u fink-bld  ./setup.py test &> ~/Desktop/setup-test.log

or just as yourself
$ ./setup.py test &> ~/Desktop/setup-test.log

...Ken



_______________________________________________
Duplicity-talk mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/duplicity-talk

setup-test1.log (112K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Errors during test phase with duplicity

duplicity-talk mailing list
Yes, that is what the os.chown() call does, it changes the ownership, so yes, root is needed.  Gonna have to look at Mac's permissions again.  I appear to be running with more access than I thought.  Thanks for the questions!

...Ken


On Thu, Sep 7, 2017 at 6:13 PM, Scott Hannahs via Duplicity-talk <[hidden email]> wrote:
Ken,

I get the same errors in that test.  I did not use the -i with sudo since there isn’t a login directory with that user.
ERROR: test_piped_password (testing.functional.test_final.FinalTest)
ERROR: test_piped_password (testing.functional.test_final.OldFilenamesFinalTest)
ERROR: test_piped_password (testing.functional.test_final.ShortFilenamesFinalTest)
ERROR: test_new_file (testing.functional.test_restart.RestartTest)
ERROR: test_split_after_large (testing.functional.test_restart.RestartTest)
ERROR: test_split_after_small (testing.functional.test_restart.RestartTest)
ERROR: test_split_inside_large (testing.functional.test_restart.RestartTest)
ERROR: test_ignore_double_snapshot (testing.functional.test_restart.RestartTestWithoutEncryption)
ERROR: test_new_file (testing.functional.test_restart.RestartTestWithoutEncryption)
ERROR: test_split_after_large (testing.functional.test_restart.RestartTestWithoutEncryption)
ERROR: test_split_after_small (testing.functional.test_restart.RestartTestWithoutEncryption)
ERROR: test_split_inside_large (testing.functional.test_restart.RestartTestWithoutEncryption)

However, they seem to be a chown error?  That is certainly a root only operation.  Is that what the call to os.chown does?
  File "/sw/src/fink.build/duplicity-0.7.14-2/duplicity-0.7.14/duplicity/path.py", line 469, in <lambda>
    util.maybe_ignore_errors(lambda: os.chown(other.name, self.stat.st_uid, self.stat.st_gid))
OSError: [Errno 1] Operation not permitted: 'testfiles/cache/8545a8b798d7684a61ae2472e32d92e2/duplicity-full-signatures.20170907T222605Z.sigtar.gz’

Is that function trying to chown the destination file during a restore?  If the restore is not being done as root then it should skip that and leave the file with the different owner of the restoring account.

Here is the full test log from 
$ sudo -u fink-bld  ./setup.py test &> ~/Desktop/setup-test1.log



The good news is despite these test errors duplicity seems to be executing a nightly run on my files to an s3 backend.

-Scott



On Sep 6, 2017, at 7:55 AM, Kenneth Loafman via Duplicity-talk <[hidden email]> wrote:

Scott,

You should not need root to build or test duplicity.  Is it possible that you've run pip with sudo at some point?  That could be a problem.

No, there is no problem in the patching tests.  There's something else going on there.

From the setup-test.log file, I'm seeing a lot of permission denied errors that I don't get, mostly from chown calls.  

Try with -i 
$ sudo -i -u fink-bld  ./setup.py test &> ~/Desktop/setup-test.log

or just as yourself
$ ./setup.py test &> ~/Desktop/setup-test.log

...Ken



_______________________________________________
Duplicity-talk mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/duplicity-talk



_______________________________________________
Duplicity-talk mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/duplicity-talk