Dropping Python 2 support

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

Dropping Python 2 support

duplicity-talk mailing list
Hello all,

Shall we move to 0.9 and drop Python 2 support now? In the first
instance I would just suggest changing the readme, tests etc to remove
Python 2 and then we can gradually remove Python 2 code (and all the
compatibility junk we had to add for the dual-version support) during
the 0.9 series.

Kind regards,

Aaron

_______________________________________________
Duplicity-talk mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/duplicity-talk
Reply | Threaded
Open this post in threaded view
|

Re: Dropping Python 2 support

duplicity-talk mailing list
On 08.06.2020 10:07, Aaron via Duplicity-talk wrote:
> Hello all,
>
> Shall we move to 0.9 and drop Python 2 support now? In the first instance I would just suggest changing the readme, tests etc to remove Python 2 and then we can gradually remove Python 2 code (and all the compatibility junk we had to add for the dual-version support) during the 0.9 series.
>

is there any urgent need to remove the compatibility code? does it prohibit further development or make it specifically more difficult?

my impression is that 0.8 just got kinda stable, so i'd rather have it stabilized some more before branching a new major version that all development efforts go into.

also, while understanding that python2 is eol, cutting of all those on old machines unwilling/able to update to python3 will surely be needed at some point, but how about giving some more time, let's say end of this year. announcing the intend to switch long beforehand in the changelog would be nice too!

..ede/duply.net

_______________________________________________
Duplicity-talk mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/duplicity-talk
Reply | Threaded
Open this post in threaded view
|

Re: Dropping Python 2 support

duplicity-talk mailing list
OK.  We are mostly in agreement.

Python 2 is not needed if we support back to Xenial, 16.04, since it supports Python 3.5.  If we want to support Py27 for earlier distros, we're going to need to do a lot of work.  Ubuntu, for some odd reason, is still supporting Precise, 12.04.  I would not suggest trying to go back into the dark ages.

Pure Python 3 code will not be that difficult.  Better testing would help a lot.

Yes, 0.8 is stable, once we got a handle on handling unicode in exceptions.  Yet again, more testing is needed.

Other than getting setup.py to handle dual versions during install, I see no problem with supporting 0.8 for a while longer.  We will have to bite the bullet at some point, but we're not there yet..  We might even be able to support Precise if Python 2.7 is available there.

...Ken


On Mon, Jun 8, 2020 at 5:39 AM edgar.soldin--- via Duplicity-talk <[hidden email]> wrote:
On 08.06.2020 10:07, Aaron via Duplicity-talk wrote:
> Hello all,
>
> Shall we move to 0.9 and drop Python 2 support now? In the first instance I would just suggest changing the readme, tests etc to remove Python 2 and then we can gradually remove Python 2 code (and all the compatibility junk we had to add for the dual-version support) during the 0.9 series.
>

is there any urgent need to remove the compatibility code? does it prohibit further development or make it specifically more difficult?

my impression is that 0.8 just got kinda stable, so i'd rather have it stabilized some more before branching a new major version that all development efforts go into.

also, while understanding that python2 is eol, cutting of all those on old machines unwilling/able to update to python3 will surely be needed at some point, but how about giving some more time, let's say end of this year. announcing the intend to switch long beforehand in the changelog would be nice too!

..ede/duply.net

_______________________________________________
Duplicity-talk mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/duplicity-talk

_______________________________________________
Duplicity-talk mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/duplicity-talk
Reply | Threaded
Open this post in threaded view
|

Re: Dropping Python 2 support

duplicity-talk mailing list
hey Aaron,

On 08.06.2020 22:04, Aaron Whitehouse wrote:

> Thanks Ede, Kenneth,
>
> On 2020-06-08 11:38, edgar.soldin--- via Duplicity-talk wrote:
>> is there any urgent need to remove the compatibility code?
>
> Nope.
>
>> does it
>> prohibit further development or make it specifically more difficult?
>
> Yes. As one specific example, our CI tests for Python 2.7 currently fail:
> https://gitlab.com/duplicity/duplicity/-/pipelines/153729163
> The cause appears to be to do with the future compatibility imports. Somebody needs to do the work to get that test job green.

not necessarily. if we know we plan to remove python2 in the future and generally duplicity is working this effort would be wasted. not an elegant solution but hey, worx ;)

> As another example, in our testing we update pip before we pull in our requirements. We do that because libraries we depend on have started dropping Python 2 support and the new pip will only upgrade you to the latest version that has Python 2 support instead of the latest version. That is great, but means we have increasing divergence between our dependencies on Python 2 and Python 3 (and potentially unfixed issues in the Python 2 dependencies). From memory some dependencies are different packages between the two Python versions.

good point. as soon as we run into a python3 only dependency you guys may be forced to switch (or write exhaustive compat glue code, but who wants to do that).

> Thirdly, some areas of Python 2 are just not as good, for example Unicode handling. We have custom code, e.g.:
> https://gitlab.com/duplicity/duplicity/-/blob/master/duplicity/util.py
> that uses the Python 3 version in Python 3, but falls back to an encoding conversion that has gaps.

same as above. not elegant, but worked in the past. so no urgency because of that.

> So yes, there is a cost to maintaining both. My question is therefore whether there is enough need to make that worthwhile. We also have the Snap versions, that let you run code from a modern, predictable and tested environment on old platforms (a huge benefit with the variety of dependencies we have).

just know your maintaining it. does that run on any old distro? is there a mini howto somewhere?

..ede/duply.net

ps. :) https://dilbert.com/strip/2020-06-06

> On 2020-06-08 15:05, Kenneth Loafman via Duplicity-talk wrote:
>> Python 2 is not needed if we support back to Xenial, 16.04, since it
>> supports Python 3.5.  If we want to support Py27 for earlier distros, we're
>> going to need to do a lot of work.
>
> Agreed.  Python 3.5 was released in 2015. Python 3 was released in *2008*. I am struggling to see a situation where the user is running something that could not use the Python 3 version (with a cutting-edge duplicity).
>
>> Pure Python 3 code will not be that difficult.  Better testing would help a
>> lot.
>
> Yes. To me the big gap in our testing now is the backends. With secret environment variables in Gitlab I think there is something interesting we can do there, but I have not yet had a play.
>
>> Other than getting setup.py to handle dual versions during install, I see
>> no problem with supporting 0.8 for a while longer.  We will have to bite
>> the bullet at some point, but we're not there yet..
>
> I do not really have a problem with this (and it is more your problem to deal with than mine, Ken), I just wanted to ask whether there was a good reason to maintain it. I am still not convinced there is a substantial user base running Python 2-only systems with cutting-edge duplicity versions. We could add a warning to duplicity output to say that Python 2 support will be dropped at the end of this year and see if anybody complains.
>
> Kind regards,
>
> Aaron


_______________________________________________
Duplicity-talk mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/duplicity-talk
Reply | Threaded
Open this post in threaded view
|

Re: Dropping Python 2 support

duplicity-talk mailing list
On Mon, Jun 8, 2020 at 3:44 PM edgar.soldin--- via Duplicity-talk <[hidden email]> wrote:
hey Aaron,

On 08.06.2020 22:04, Aaron Whitehouse wrote:
> Thanks Ede, Kenneth,
>
> On 2020-06-08 11:38, edgar.soldin--- via Duplicity-talk wrote:
>> is there any urgent need to remove the compatibility code?
>
> Nope.
>
>> does it
>> prohibit further development or make it specifically more difficult?
>
> Yes. As one specific example, our CI tests for Python 2.7 currently fail:
> https://gitlab.com/duplicity/duplicity/-/pipelines/153729163
> The cause appears to be to do with the future compatibility imports. Somebody needs to do the work to get that test job green.

not necessarily. if we know we plan to remove python2 in the future and generally duplicity is working this effort would be wasted. not an elegant solution but hey, worx ;)

Actually, it's something in the way we're handling the testing/overrides/bin directory.  Remember, I changed all the shebangs to Python3?  This was offset by explicitly using the TOXPYTHON interpreter defined in tox.ini.  For some reason the tox.ini environment is not passed to spawned duplicity code and we're running the bin scripts directly as their shebang specifies.  Because of this, we try to test using a mix of python 2 & 3 code.  The new CI control only loads requirements for python 2, so the future import problem happens now.  I'm working on fixing both problems so we have a pure Python 2 test going on..

> As another example, in our testing we update pip before we pull in our requirements. We do that because libraries we depend on have started dropping Python 2 support and the new pip will only upgrade you to the latest version that has Python 2 support instead of the latest version. That is great, but means we have increasing divergence between our dependencies on Python 2 and Python 3 (and potentially unfixed issues in the Python 2 dependencies). From memory some dependencies are different packages between the two Python versions.

good point. as soon as we run into a python3 only dependency you guys may be forced to switch (or write exhaustive compat glue code, but who wants to do that).

We can solve that by specifying version limits in requirements.txt.

> Thirdly, some areas of Python 2 are just not as good, for example Unicode handling. We have custom code, e.g.:
> https://gitlab.com/duplicity/duplicity/-/blob/master/duplicity/util.py
> that uses the Python 3 version in Python 3, but falls back to an encoding conversion that has gaps.

same as above. not elegant, but worked in the past. so no urgency because of that.

> So yes, there is a cost to maintaining both. My question is therefore whether there is enough need to make that worthwhile. We also have the Snap versions, that let you run code from a modern, predictable and tested environment on old platforms (a huge benefit with the variety of dependencies we have).

just know your maintaining it. does that run on any old distro? is there a mini howto somewhere?

..ede/duply.net

ps. :) https://dilbert.com/strip/2020-06-06

Good catch! 

> On 2020-06-08 15:05, Kenneth Loafman via Duplicity-talk wrote:
>> Python 2 is not needed if we support back to Xenial, 16.04, since it
>> supports Python 3.5.  If we want to support Py27 for earlier distros, we're
>> going to need to do a lot of work.
>
> Agreed.  Python 3.5 was released in 2015. Python 3 was released in *2008*. I am struggling to see a situation where the user is running something that could not use the Python 3 version (with a cutting-edge duplicity).
>
>> Pure Python 3 code will not be that difficult.  Better testing would help a
>> lot.
>
> Yes. To me the big gap in our testing now is the backends. With secret environment variables in Gitlab I think there is something interesting we can do there, but I have not yet had a play.

Backends are a real problem since we need to set up accounts, share passwords, etc.. The main problems we've had are less in the code itself, but the exception handling.  I know of no  good mechanism to test exceptions.  I think it would require inserted code in a debug environment to cause the try: suite to raise an exception, but it would be complicated to test without the real answer back from the site itself.

>> Other than getting setup.py to handle dual versions during install, I see
>> no problem with supporting 0.8 for a while longer.  We will have to bite
>> the bullet at some point, but we're not there yet..
>
> I do not really have a problem with this (and it is more your problem to deal with than mine, Ken), I just wanted to ask whether there was a good reason to maintain it. I am still not convinced there is a substantial user base running Python 2-only systems with cutting-edge duplicity versions. We could add a warning to duplicity output to say that Python 2 support will be dropped at the end of this year and see if anybody complains.
>
> Kind regards,
>
> Aaron

...Thanks,
...Ken
 

_______________________________________________
Duplicity-talk mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/duplicity-talk