Removing glupy from release 5.7

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
36 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Removing glupy from release 5.7

Hari Gowtham
Hi,

Due to the recent changes we made. we have a build issue because of glupy.
As glupy is already removed from master, we are thinking of removing
it in 5.7 as well rather than fixing the issue.

The release of 5.7 will be delayed as we have send a patch to fix this issue.
And if anyone has any concerns, do let us know.

--
Regards,
Hari Gowtham.
_______________________________________________

Community Meeting Calendar:

APAC Schedule -
Every 2nd and 4th Tuesday at 11:30 AM IST
Bridge: https://bluejeans.com/836554017

NA/EMEA Schedule -
Every 1st and 3rd Tuesday at 01:00 PM EDT
Bridge: https://bluejeans.com/486278655

Gluster-devel mailing list
[hidden email]
https://lists.gluster.org/mailman/listinfo/gluster-devel

Reply | Threaded
Open this post in threaded view
|

Re: Removing glupy from release 5.7

Niels de Vos-5
On Wed, Jun 12, 2019 at 02:44:04PM +0530, Hari Gowtham wrote:
> Hi,
>
> Due to the recent changes we made. we have a build issue because of glupy.
> As glupy is already removed from master, we are thinking of removing
> it in 5.7 as well rather than fixing the issue.
>
> The release of 5.7 will be delayed as we have send a patch to fix this issue.
> And if anyone has any concerns, do let us know.

Could you link to the BZ with the build error and patches that attempt
fixing it?

We normally do not remove features with minor updates. Fixing the build
error would be the preferred approach.

Thanks,
Niels
_______________________________________________

Community Meeting Calendar:

APAC Schedule -
Every 2nd and 4th Tuesday at 11:30 AM IST
Bridge: https://bluejeans.com/836554017

NA/EMEA Schedule -
Every 1st and 3rd Tuesday at 01:00 PM EDT
Bridge: https://bluejeans.com/486278655

Gluster-devel mailing list
[hidden email]
https://lists.gluster.org/mailman/listinfo/gluster-devel

Reply | Threaded
Open this post in threaded view
|

Re: Removing glupy from release 5.7

Hari Gowtham
We haven't sent any patch to fix it.
Waiting for the decision to be made.
The bz: https://bugzilla.redhat.com/show_bug.cgi?id=1719778
The link to the build log:
https://build.gluster.org/job/strfmt_errors/18888/artifact/RPMS/el6/i686/build.log

The last few messages in the log:

config.status: creating xlators/features/changelog/lib/src/Makefile
config.status: creating xlators/features/changetimerecorder/Makefile
config.status: creating xlators/features/changetimerecorder/src/Makefile
BUILDSTDERR: config.status: error: cannot find input file:
xlators/features/glupy/Makefile.in
RPM build errors:
BUILDSTDERR: error: Bad exit status from /var/tmp/rpm-tmp.kGZI5V (%build)
BUILDSTDERR:     Bad exit status from /var/tmp/rpm-tmp.kGZI5V (%build)
Child return code was: 1
EXCEPTION: [Error()]
Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/mockbuild/trace_decorator.py",
line 96, in trace
    result = func(*args, **kw)
  File "/usr/lib/python3.6/site-packages/mockbuild/util.py", line 736,
in do_with_status
    raise exception.Error("Command failed: \n # %s\n%s" % (command,
output), child.returncode)
mockbuild.exception.Error: Command failed:
 # bash --login -c /usr/bin/rpmbuild -bb --target i686 --nodeps
/builddir/build/SPECS/glusterfs.spec

On Wed, Jun 12, 2019 at 7:04 PM Niels de Vos <[hidden email]> wrote:

>
> On Wed, Jun 12, 2019 at 02:44:04PM +0530, Hari Gowtham wrote:
> > Hi,
> >
> > Due to the recent changes we made. we have a build issue because of glupy.
> > As glupy is already removed from master, we are thinking of removing
> > it in 5.7 as well rather than fixing the issue.
> >
> > The release of 5.7 will be delayed as we have send a patch to fix this issue.
> > And if anyone has any concerns, do let us know.
>
> Could you link to the BZ with the build error and patches that attempt
> fixing it?
>
> We normally do not remove features with minor updates. Fixing the build
> error would be the preferred approach.
>
> Thanks,
> Niels



--
Regards,
Hari Gowtham.
_______________________________________________

Community Meeting Calendar:

APAC Schedule -
Every 2nd and 4th Tuesday at 11:30 AM IST
Bridge: https://bluejeans.com/836554017

NA/EMEA Schedule -
Every 1st and 3rd Tuesday at 01:00 PM EDT
Bridge: https://bluejeans.com/486278655

Gluster-devel mailing list
[hidden email]
https://lists.gluster.org/mailman/listinfo/gluster-devel

Reply | Threaded
Open this post in threaded view
|

Re: Removing glupy from release 5.7

Niels de Vos-5
On Wed, Jun 12, 2019 at 07:54:17PM +0530, Hari Gowtham wrote:

> We haven't sent any patch to fix it.
> Waiting for the decision to be made.
> The bz: https://bugzilla.redhat.com/show_bug.cgi?id=1719778
> The link to the build log:
> https://build.gluster.org/job/strfmt_errors/18888/artifact/RPMS/el6/i686/build.log
>
> The last few messages in the log:
>
> config.status: creating xlators/features/changelog/lib/src/Makefile
> config.status: creating xlators/features/changetimerecorder/Makefile
> config.status: creating xlators/features/changetimerecorder/src/Makefile
> BUILDSTDERR: config.status: error: cannot find input file:
> xlators/features/glupy/Makefile.in
> RPM build errors:
> BUILDSTDERR: error: Bad exit status from /var/tmp/rpm-tmp.kGZI5V (%build)
> BUILDSTDERR:     Bad exit status from /var/tmp/rpm-tmp.kGZI5V (%build)
> Child return code was: 1
> EXCEPTION: [Error()]
> Traceback (most recent call last):
>   File "/usr/lib/python3.6/site-packages/mockbuild/trace_decorator.py",
> line 96, in trace
>     result = func(*args, **kw)
>   File "/usr/lib/python3.6/site-packages/mockbuild/util.py", line 736,
> in do_with_status
>     raise exception.Error("Command failed: \n # %s\n%s" % (command,
> output), child.returncode)
> mockbuild.exception.Error: Command failed:
>  # bash --login -c /usr/bin/rpmbuild -bb --target i686 --nodeps
> /builddir/build/SPECS/glusterfs.spec

Those messages are caused by missing files. The 'make dist' that
generates the tarball in the previous step did not included the glupy
files.

https://build.gluster.org/job/strfmt_errors/18888/console contains the
following message:

    configure: WARNING:
            ---------------------------------------------------------------------------------
            cannot build glupy. python 3.6 and python-devel/python-dev package are required.
            ---------------------------------------------------------------------------------

I am not sure if there have been any recent backports to release-5 that
introduced this behaviour. Maybe it is related to the builder where the
tarball is generated. The job seems to detect python-3.6.8, which is not
included in CentOS-7 for all I know?

Maybe someone else understands how this can happen?

HTH,
Niels


>
> On Wed, Jun 12, 2019 at 7:04 PM Niels de Vos <[hidden email]> wrote:
> >
> > On Wed, Jun 12, 2019 at 02:44:04PM +0530, Hari Gowtham wrote:
> > > Hi,
> > >
> > > Due to the recent changes we made. we have a build issue because of glupy.
> > > As glupy is already removed from master, we are thinking of removing
> > > it in 5.7 as well rather than fixing the issue.
> > >
> > > The release of 5.7 will be delayed as we have send a patch to fix this issue.
> > > And if anyone has any concerns, do let us know.
> >
> > Could you link to the BZ with the build error and patches that attempt
> > fixing it?
> >
> > We normally do not remove features with minor updates. Fixing the build
> > error would be the preferred approach.
> >
> > Thanks,
> > Niels
>
>
>
> --
> Regards,
> Hari Gowtham.
_______________________________________________

Community Meeting Calendar:

APAC Schedule -
Every 2nd and 4th Tuesday at 11:30 AM IST
Bridge: https://bluejeans.com/836554017

NA/EMEA Schedule -
Every 1st and 3rd Tuesday at 01:00 PM EDT
Bridge: https://bluejeans.com/486278655

Gluster-devel mailing list
[hidden email]
https://lists.gluster.org/mailman/listinfo/gluster-devel

Reply | Threaded
Open this post in threaded view
|

Re: Removing glupy from release 5.7

Amar Tumballi Suryanarayan

On Wed, Jun 12, 2019 at 8:42 PM Niels de Vos <[hidden email]> wrote:
On Wed, Jun 12, 2019 at 07:54:17PM +0530, Hari Gowtham wrote:
> We haven't sent any patch to fix it.
> Waiting for the decision to be made.
> The bz: https://bugzilla.redhat.com/show_bug.cgi?id=1719778
> The link to the build log:
> https://build.gluster.org/job/strfmt_errors/18888/artifact/RPMS/el6/i686/build.log
>
> The last few messages in the log:
>
> config.status: creating xlators/features/changelog/lib/src/Makefile
> config.status: creating xlators/features/changetimerecorder/Makefile
> config.status: creating xlators/features/changetimerecorder/src/Makefile
> BUILDSTDERR: config.status: error: cannot find input file:
> xlators/features/glupy/Makefile.in
> RPM build errors:
> BUILDSTDERR: error: Bad exit status from /var/tmp/rpm-tmp.kGZI5V (%build)
> BUILDSTDERR:     Bad exit status from /var/tmp/rpm-tmp.kGZI5V (%build)
> Child return code was: 1
> EXCEPTION: [Error()]
> Traceback (most recent call last):
>   File "/usr/lib/python3.6/site-packages/mockbuild/trace_decorator.py",
> line 96, in trace
>     result = func(*args, **kw)
>   File "/usr/lib/python3.6/site-packages/mockbuild/util.py", line 736,
> in do_with_status
>     raise exception.Error("Command failed: \n # %s\n%s" % (command,
> output), child.returncode)
> mockbuild.exception.Error: Command failed:
>  # bash --login -c /usr/bin/rpmbuild -bb --target i686 --nodeps
> /builddir/build/SPECS/glusterfs.spec

Those messages are caused by missing files. The 'make dist' that
generates the tarball in the previous step did not included the glupy
files.

https://build.gluster.org/job/strfmt_errors/18888/console contains the
following message:

    configure: WARNING:
            ---------------------------------------------------------------------------------
            cannot build glupy. python 3.6 and python-devel/python-dev package are required.
            ---------------------------------------------------------------------------------

I am not sure if there have been any recent backports to release-5 that
introduced this behaviour. Maybe it is related to the builder where the
tarball is generated. The job seems to detect python-3.6.8, which is not
included in CentOS-7 for all I know?


We recently noticed that in one of the package update on builder (ie, centos7.x machines), python3.6 got installed as a dependency. So, yes, it is possible to have python3 in centos7 now.

-Amar
 
Maybe someone else understands how this can happen?

HTH,
Niels


>
> On Wed, Jun 12, 2019 at 7:04 PM Niels de Vos <[hidden email]> wrote:
> >
> > On Wed, Jun 12, 2019 at 02:44:04PM +0530, Hari Gowtham wrote:
> > > Hi,
> > >
> > > Due to the recent changes we made. we have a build issue because of glupy.
> > > As glupy is already removed from master, we are thinking of removing
> > > it in 5.7 as well rather than fixing the issue.
> > >
> > > The release of 5.7 will be delayed as we have send a patch to fix this issue.
> > > And if anyone has any concerns, do let us know.
> >
> > Could you link to the BZ with the build error and patches that attempt
> > fixing it?
> >
> > We normally do not remove features with minor updates. Fixing the build
> > error would be the preferred approach.
> >
> > Thanks,
> > Niels
>
>
>
> --
> Regards,
> Hari Gowtham.
_______________________________________________

Community Meeting Calendar:

APAC Schedule -
Every 2nd and 4th Tuesday at 11:30 AM IST
Bridge: https://bluejeans.com/836554017

NA/EMEA Schedule -
Every 1st and 3rd Tuesday at 01:00 PM EDT
Bridge: https://bluejeans.com/486278655

Gluster-devel mailing list
[hidden email]
https://lists.gluster.org/mailman/listinfo/gluster-devel




--
Amar Tumballi (amarts)

_______________________________________________

Community Meeting Calendar:

APAC Schedule -
Every 2nd and 4th Tuesday at 11:30 AM IST
Bridge: https://bluejeans.com/836554017

NA/EMEA Schedule -
Every 1st and 3rd Tuesday at 01:00 PM EDT
Bridge: https://bluejeans.com/486278655

Gluster-devel mailing list
[hidden email]
https://lists.gluster.org/mailman/listinfo/gluster-devel

Reply | Threaded
Open this post in threaded view
|

Re: Removing glupy from release 5.7

Kaleb Keithley

On Wed, Jun 12, 2019 at 10:43 AM Amar Tumballi Suryanarayan <[hidden email]> wrote:

We recently noticed that in one of the package update on builder (ie, centos7.x machines), python3.6 got installed as a dependency. So, yes, it is possible to have python3 in centos7 now.

EPEL updated from python34 to python36 recently, but C7 doesn't have python3 in the base. I don't think we've ever used EPEL packages for building.

And GlusterFS-5 isn't python3 ready.

--

Kaleb

 

_______________________________________________

Community Meeting Calendar:

APAC Schedule -
Every 2nd and 4th Tuesday at 11:30 AM IST
Bridge: https://bluejeans.com/836554017

NA/EMEA Schedule -
Every 1st and 3rd Tuesday at 01:00 PM EDT
Bridge: https://bluejeans.com/486278655

Gluster-devel mailing list
[hidden email]
https://lists.gluster.org/mailman/listinfo/gluster-devel

Reply | Threaded
Open this post in threaded view
|

Re: Removing glupy from release 5.7

Kaleb Keithley


On Wed, Jun 12, 2019 at 11:36 AM Kaleb Keithley <[hidden email]> wrote:

On Wed, Jun 12, 2019 at 10:43 AM Amar Tumballi Suryanarayan <[hidden email]> wrote:

We recently noticed that in one of the package update on builder (ie, centos7.x machines), python3.6 got installed as a dependency. So, yes, it is possible to have python3 in centos7 now.

EPEL updated from python34 to python36 recently, but C7 doesn't have python3 in the base. I don't think we've ever used EPEL packages for building.

And GlusterFS-5 isn't python3 ready.

Correction: GlusterFS-5 is mostly or completely python3 ready.  FWIW, python33 is available on both RHEL7 and CentOS7 from the Software Collection Library (SCL), and python34 and now python36 are available from EPEL.

But packages built for the CentOS Storage SIG have never used the SCL or EPEL (EPEL not allowed) and the shebangs in the .py files are converted from /usr/bin/python3 to /usr/bin/python2 during the rpmbuild %prep stage. All the python dependencies for the packages remain the python2 flavors. AFAIK the centos-regression machines ought to be building the same way.
 
--

Kaleb

_______________________________________________

Community Meeting Calendar:

APAC Schedule -
Every 2nd and 4th Tuesday at 11:30 AM IST
Bridge: https://bluejeans.com/836554017

NA/EMEA Schedule -
Every 1st and 3rd Tuesday at 01:00 PM EDT
Bridge: https://bluejeans.com/486278655

Gluster-devel mailing list
[hidden email]
https://lists.gluster.org/mailman/listinfo/gluster-devel

Reply | Threaded
Open this post in threaded view
|

Re: Removing glupy from release 5.7

Deepshikha Khandelwal

On Thu, Jun 13, 2019 at 4:41 AM Kaleb Keithley <[hidden email]> wrote:


On Wed, Jun 12, 2019 at 11:36 AM Kaleb Keithley <[hidden email]> wrote:

On Wed, Jun 12, 2019 at 10:43 AM Amar Tumballi Suryanarayan <[hidden email]> wrote:

We recently noticed that in one of the package update on builder (ie, centos7.x machines), python3.6 got installed as a dependency. So, yes, it is possible to have python3 in centos7 now.

EPEL updated from python34 to python36 recently, but C7 doesn't have python3 in the base. I don't think we've ever used EPEL packages for building.

And GlusterFS-5 isn't python3 ready.

Correction: GlusterFS-5 is mostly or completely python3 ready.  FWIW, python33 is available on both RHEL7 and CentOS7 from the Software Collection Library (SCL), and python34 and now python36 are available from EPEL.

But packages built for the CentOS Storage SIG have never used the SCL or EPEL (EPEL not allowed) and the shebangs in the .py files are converted from /usr/bin/python3 to /usr/bin/python2 during the rpmbuild %prep stage. All the python dependencies for the packages remain the python2 flavors. AFAIK the centos-regression machines ought to be building the same way.

centos-regression machines have 'CentOS Linux release 7.6.1810 (Core)' and using python3.6. Looking at the tracebacks when compiling we confirmed that it is picking up python3.6 somehow. 

To resolve this issue either we can remove glupy from the release(which is dead anyways) or install glupy on the instances. 
 
--

Kaleb
_______________________________________________

Community Meeting Calendar:

APAC Schedule -
Every 2nd and 4th Tuesday at 11:30 AM IST
Bridge: https://bluejeans.com/836554017

NA/EMEA Schedule -
Every 1st and 3rd Tuesday at 01:00 PM EDT
Bridge: https://bluejeans.com/486278655

Gluster-devel mailing list
[hidden email]
https://lists.gluster.org/mailman/listinfo/gluster-devel


_______________________________________________

Community Meeting Calendar:

APAC Schedule -
Every 2nd and 4th Tuesday at 11:30 AM IST
Bridge: https://bluejeans.com/836554017

NA/EMEA Schedule -
Every 1st and 3rd Tuesday at 01:00 PM EDT
Bridge: https://bluejeans.com/486278655

Gluster-devel mailing list
[hidden email]
https://lists.gluster.org/mailman/listinfo/gluster-devel

Reply | Threaded
Open this post in threaded view
|

Re: Removing glupy from release 5.7

Kaleb Keithley


On Wed, Jun 12, 2019 at 8:13 PM Deepshikha Khandelwal <[hidden email]> wrote:

On Thu, Jun 13, 2019 at 4:41 AM Kaleb Keithley <[hidden email]> wrote:


On Wed, Jun 12, 2019 at 11:36 AM Kaleb Keithley <[hidden email]> wrote:

On Wed, Jun 12, 2019 at 10:43 AM Amar Tumballi Suryanarayan <[hidden email]> wrote:

We recently noticed that in one of the package update on builder (ie, centos7.x machines), python3.6 got installed as a dependency. So, yes, it is possible to have python3 in centos7 now.

EPEL updated from python34 to python36 recently, but C7 doesn't have python3 in the base. I don't think we've ever used EPEL packages for building.

And GlusterFS-5 isn't python3 ready.

Correction: GlusterFS-5 is mostly or completely python3 ready.  FWIW, python33 is available on both RHEL7 and CentOS7 from the Software Collection Library (SCL), and python34 and now python36 are available from EPEL.

But packages built for the CentOS Storage SIG have never used the SCL or EPEL (EPEL not allowed) and the shebangs in the .py files are converted from /usr/bin/python3 to /usr/bin/python2 during the rpmbuild %prep stage. All the python dependencies for the packages remain the python2 flavors. AFAIK the centos-regression machines ought to be building the same way.

centos-regression machines have 'CentOS Linux release 7.6.1810 (Core)' and using python3.6. Looking at the tracebacks when compiling we confirmed that it is picking up python3.6 somehow. 

We need to figure out why? BTW, my CentOS 7 box is up to date and does not have any version of python3. I would have to use the SCL or EPEL to get it.


6309 was the last centos-regression with python2.7.  6310 and all subsequent centos-regressions have been built with python3.6.

Somebody added EPEL!  Do we not have a record of the changes made and who made them?

And BTW, this affects more than just glusterfs-5, it's affecting all versions: glusterfs-4.1, glusterfs-5, glusterfs-6, and master.
 
To resolve this issue either we can remove glupy from the release(which is dead anyways) or install glupy on the instances.

Or you can resolve where python36 came from and undo the change that introduced it.

At the risk of being repetitious – reiterating what Niels said – it's highly unusual to remove features in a bug fix update.

It's also unusual to have switched to python3 on rhel7 like this. Was there any discussion of such a change? If there was I seem to have missed it.

I suggest figuring out where python3.6 on rhel7 came from. Fix that first.  Removing glupy is a bandaid over a unrelated problem. Once the real problem is fixed then there can be a separate discussion about removing the glupy feature in glusterfs-5.

--

Kaleb

_______________________________________________

Community Meeting Calendar:

APAC Schedule -
Every 2nd and 4th Tuesday at 11:30 AM IST
Bridge: https://bluejeans.com/836554017

NA/EMEA Schedule -
Every 1st and 3rd Tuesday at 01:00 PM EDT
Bridge: https://bluejeans.com/486278655

Gluster-devel mailing list
[hidden email]
https://lists.gluster.org/mailman/listinfo/gluster-devel

Reply | Threaded
Open this post in threaded view
|

Re: Removing glupy from release 5.7

Niels de Vos-5
In reply to this post by Kaleb Keithley
On Wed, Jun 12, 2019 at 04:09:55PM -0700, Kaleb Keithley wrote:

> On Wed, Jun 12, 2019 at 11:36 AM Kaleb Keithley <[hidden email]> wrote:
>
> >
> > On Wed, Jun 12, 2019 at 10:43 AM Amar Tumballi Suryanarayan <
> > [hidden email]> wrote:
> >
> >>
> >> We recently noticed that in one of the package update on builder (ie,
> >> centos7.x machines), python3.6 got installed as a dependency. So, yes, it
> >> is possible to have python3 in centos7 now.
> >>
> >
> > EPEL updated from python34 to python36 recently, but C7 doesn't have
> > python3 in the base. I don't think we've ever used EPEL packages for
> > building.
> >
> > And GlusterFS-5 isn't python3 ready.
> >
>
> Correction: GlusterFS-5 is mostly or completely python3 ready.  FWIW,
> python33 is available on both RHEL7 and CentOS7 from the Software
> Collection Library (SCL), and python34 and now python36 are available from
> EPEL.
>
> But packages built for the CentOS Storage SIG have never used the SCL or
> EPEL (EPEL not allowed) and the shebangs in the .py files are converted
> from /usr/bin/python3 to /usr/bin/python2 during the rpmbuild %prep stage.
> All the python dependencies for the packages remain the python2 flavors.
> AFAIK the centos-regression machines ought to be building the same way.

Indeed, there should not be a requirement on having EPEL enabled on the
CentOS-7 builders. At least not for the building of the glusterfs
tarball. We still need to do releases of glusterfs-4.1 and glusterfs-5,
until then it is expected to have python2 as the (only?) version for the
system. Is it possible to remove python3 from the CentOS-7 builders and
run the jobs that require python3 on the Fedora builders instead?

I guess we could force the release-4.1 and release-5 branches to use
python2 only. This might be done by exporting PYTHON=/usr/bin/python2 in
the environment where './configure' is run. That would likely require
changes to multiple Jenkins jobs...

Niels
_______________________________________________

Community Meeting Calendar:

APAC Schedule -
Every 2nd and 4th Tuesday at 11:30 AM IST
Bridge: https://bluejeans.com/836554017

NA/EMEA Schedule -
Every 1st and 3rd Tuesday at 01:00 PM EDT
Bridge: https://bluejeans.com/486278655

Gluster-devel mailing list
[hidden email]
https://lists.gluster.org/mailman/listinfo/gluster-devel

Reply | Threaded
Open this post in threaded view
|

Re: Removing glupy from release 5.7

Deepshikha Khandelwal
In reply to this post by Kaleb Keithley


On Thu, Jun 13, 2019 at 10:06 AM Kaleb Keithley <[hidden email]> wrote:


On Wed, Jun 12, 2019 at 8:13 PM Deepshikha Khandelwal <[hidden email]> wrote:

On Thu, Jun 13, 2019 at 4:41 AM Kaleb Keithley <[hidden email]> wrote:


On Wed, Jun 12, 2019 at 11:36 AM Kaleb Keithley <[hidden email]> wrote:

On Wed, Jun 12, 2019 at 10:43 AM Amar Tumballi Suryanarayan <[hidden email]> wrote:

We recently noticed that in one of the package update on builder (ie, centos7.x machines), python3.6 got installed as a dependency. So, yes, it is possible to have python3 in centos7 now.

EPEL updated from python34 to python36 recently, but C7 doesn't have python3 in the base. I don't think we've ever used EPEL packages for building.

And GlusterFS-5 isn't python3 ready.

Correction: GlusterFS-5 is mostly or completely python3 ready.  FWIW, python33 is available on both RHEL7 and CentOS7 from the Software Collection Library (SCL), and python34 and now python36 are available from EPEL.

But packages built for the CentOS Storage SIG have never used the SCL or EPEL (EPEL not allowed) and the shebangs in the .py files are converted from /usr/bin/python3 to /usr/bin/python2 during the rpmbuild %prep stage. All the python dependencies for the packages remain the python2 flavors. AFAIK the centos-regression machines ought to be building the same way.

centos-regression machines have 'CentOS Linux release 7.6.1810 (Core)' and using python3.6. Looking at the tracebacks when compiling we confirmed that it is picking up python3.6 somehow. 

We need to figure out why? BTW, my CentOS 7 box is up to date and does not have any version of python3. I would have to use the SCL or EPEL to get it.

Yes, you are right. OS got upgrade to newer version on 5th June. It has EPEL repo enabled for various other things.

Can we make changes in configure.ac file (python version specific to the branches) rather than falling back to python2 for other branches too?    

6309 was the last centos-regression with python2.7.  6310 and all subsequent centos-regressions have been built with python3.6.

Somebody added EPEL!  Do we not have a record of the changes made and who made them?

And BTW, this affects more than just glusterfs-5, it's affecting all versions: glusterfs-4.1, glusterfs-5, glusterfs-6, and master.
 
To resolve this issue either we can remove glupy from the release(which is dead anyways) or install glupy on the instances.

Or you can resolve where python36 came from and undo the change that introduced it.

At the risk of being repetitious – reiterating what Niels said – it's highly unusual to remove features in a bug fix update.

It's also unusual to have switched to python3 on rhel7 like this. Was there any discussion of such a change? If there was I seem to have missed it.

I suggest figuring out where python3.6 on rhel7 came from. Fix that first.  Removing glupy is a bandaid over a unrelated problem. Once the real problem is fixed then there can be a separate discussion about removing the glupy feature in glusterfs-5.

--

Kaleb

_______________________________________________

Community Meeting Calendar:

APAC Schedule -
Every 2nd and 4th Tuesday at 11:30 AM IST
Bridge: https://bluejeans.com/836554017

NA/EMEA Schedule -
Every 1st and 3rd Tuesday at 01:00 PM EDT
Bridge: https://bluejeans.com/486278655

Gluster-devel mailing list
[hidden email]
https://lists.gluster.org/mailman/listinfo/gluster-devel

Reply | Threaded
Open this post in threaded view
|

Re: Removing glupy from release 5.7

Niels de Vos-5
In reply to this post by Niels de Vos-5
On Thu, Jun 13, 2019 at 11:08:25AM +0200, Niels de Vos wrote:

> On Wed, Jun 12, 2019 at 04:09:55PM -0700, Kaleb Keithley wrote:
> > On Wed, Jun 12, 2019 at 11:36 AM Kaleb Keithley <[hidden email]> wrote:
> >
> > >
> > > On Wed, Jun 12, 2019 at 10:43 AM Amar Tumballi Suryanarayan <
> > > [hidden email]> wrote:
> > >
> > >>
> > >> We recently noticed that in one of the package update on builder (ie,
> > >> centos7.x machines), python3.6 got installed as a dependency. So, yes, it
> > >> is possible to have python3 in centos7 now.
> > >>
> > >
> > > EPEL updated from python34 to python36 recently, but C7 doesn't have
> > > python3 in the base. I don't think we've ever used EPEL packages for
> > > building.
> > >
> > > And GlusterFS-5 isn't python3 ready.
> > >
> >
> > Correction: GlusterFS-5 is mostly or completely python3 ready.  FWIW,
> > python33 is available on both RHEL7 and CentOS7 from the Software
> > Collection Library (SCL), and python34 and now python36 are available from
> > EPEL.
> >
> > But packages built for the CentOS Storage SIG have never used the SCL or
> > EPEL (EPEL not allowed) and the shebangs in the .py files are converted
> > from /usr/bin/python3 to /usr/bin/python2 during the rpmbuild %prep stage.
> > All the python dependencies for the packages remain the python2 flavors.
> > AFAIK the centos-regression machines ought to be building the same way.
>
> Indeed, there should not be a requirement on having EPEL enabled on the
> CentOS-7 builders. At least not for the building of the glusterfs
> tarball. We still need to do releases of glusterfs-4.1 and glusterfs-5,
> until then it is expected to have python2 as the (only?) version for the
> system. Is it possible to remove python3 from the CentOS-7 builders and
> run the jobs that require python3 on the Fedora builders instead?

Actually, if the python-devel package for python3 is installed on the
CentOS-7 builders, things may work too. It still feels like some sort of
Frankenstein deployment, and we don't expect to this see in production
environments. But maybe this is a workaround in case something really,
really, REALLY depends on python3 on the builders.

Niels
_______________________________________________

Community Meeting Calendar:

APAC Schedule -
Every 2nd and 4th Tuesday at 11:30 AM IST
Bridge: https://bluejeans.com/836554017

NA/EMEA Schedule -
Every 1st and 3rd Tuesday at 01:00 PM EDT
Bridge: https://bluejeans.com/486278655

Gluster-devel mailing list
[hidden email]
https://lists.gluster.org/mailman/listinfo/gluster-devel

Reply | Threaded
Open this post in threaded view
|

Re: Removing glupy from release 5.7

Kaleb Keithley
In reply to this post by Deepshikha Khandelwal
On Thu, Jun 13, 2019 at 2:22 AM Deepshikha Khandelwal <[hidden email]> wrote:
On Thu, Jun 13, 2019 at 10:06 AM Kaleb Keithley <[hidden email]> wrote:
On Wed, Jun 12, 2019 at 8:13 PM Deepshikha Khandelwal <[hidden email]> wrote:
On Thu, Jun 13, 2019 at 4:41 AM Kaleb Keithley <[hidden email]> wrote:
On Wed, Jun 12, 2019 at 11:36 AM Kaleb Keithley <[hidden email]> wrote:



We need to figure out why? BTW, my CentOS 7 box is up to date and does not have any version of python3. I would have to use the SCL or EPEL to get it.

Yes, you are right. OS got upgrade to newer version on 5th June. It has EPEL repo enabled for various other things.

What other things?  Are there not python2 versions of these things? That work just as well as the pyton3 versions?

Adding EPEL and installing python3 on the centos boxes seems like a mistake to me, if only because it has broken the builds there. Was there any discussion of adding EPEL and python3 ? I don't recall seeing any.

But since EPEL was added, one possible work-around would be to do the build in mock.

--

Kaleb



_______________________________________________

Community Meeting Calendar:

APAC Schedule -
Every 2nd and 4th Tuesday at 11:30 AM IST
Bridge: https://bluejeans.com/836554017

NA/EMEA Schedule -
Every 1st and 3rd Tuesday at 01:00 PM EDT
Bridge: https://bluejeans.com/486278655

Gluster-devel mailing list
[hidden email]
https://lists.gluster.org/mailman/listinfo/gluster-devel

Reply | Threaded
Open this post in threaded view
|

Re: Removing glupy from release 5.7

Michael Scherer
Le jeudi 13 juin 2019 à 05:55 -0700, Kaleb Keithley a écrit :

> On Thu, Jun 13, 2019 at 2:22 AM Deepshikha Khandelwal <
> [hidden email]>
> wrote:
>
> > On Thu, Jun 13, 2019 at 10:06 AM Kaleb Keithley <
> > [hidden email]>
> > wrote:
> >
> > > On Wed, Jun 12, 2019 at 8:13 PM Deepshikha Khandelwal <
> > > [hidden email]> wrote:
> > >
> > > > On Thu, Jun 13, 2019 at 4:41 AM Kaleb Keithley <
> > > > [hidden email]>
> > > > wrote:
> > > >
> > > > > On Wed, Jun 12, 2019 at 11:36 AM Kaleb Keithley <
> > > > > [hidden email]>
> > > > > wrote:
> > > > >
> > > > > >
> > > > > >
> > > > > >
> > >
> > > We need to figure out why? BTW, my CentOS 7 box is up to date and
> > > does
> > > not have any version of python3. I would have to use the SCL or
> > > EPEL to get
> > > it.
> > >
> > > What changed on June 5th?  Between
> > > https://build.gluster.org/job/centos7-regression/6309/consoleFull
> > >  and
> > > https://build.gluster.org/job/centos7-regression/6310/consoleFull
> > > ?
> > >
> >
> > Yes, you are right. OS got upgrade to newer version on 5th June. It
> > has
> > EPEL repo enabled for various other things.
> >
>
> What other things?  Are there not python2 versions of these things?
> That work just as well as the pyton3 versions?
Mock pull python3:

[root@builder11 ~]# LC_ALL=C rpm -e --test python36-rpm
error: Failed dependencies:
        python36-rpm is needed by (installed) mock-1.4.16-1.el7.noarch

And I think it would be ill advised to backport a EOL version of mock
with python 2 on the builders.

> Adding EPEL and installing python3 on the centos boxes seems like a
> mistake to me, if only because it has broken the builds there. Was
> there any discussion of adding EPEL and python3 ? I don't recall
> seeing any.

We have EPEL for:
- munin
- nagios
- golang
- clang, cppcheck
- mock
- nginx

nginx could be removed now (that's kinda legacy). The rest look like
very much stuff we use, so I think EPEL is here to stay.


> But since EPEL was added, one possible work-around would be to do the
> build
> in mock.

The detection logic is hitting a corner case (granted, that's one that
changed under people feet). There is a 2 step approach:
- detect the most recent version of python
- verify that there is headers for that python version

But having python 3 do not mean we want to use that one for building.

So I think the right autodetection would be to
- list all version of python
- take the most recent one with -devel installed

(eg, 1 loop that check 2 things, instead of 1 loop for version, and a
check after).

Or, as a work around, we should be explicit on the python version with
a configure switch, so we can be sure we test and build the right one,
since the autodetection hit a corner case.
--
Michael Scherer
Sysadmin, Community Infrastructure




_______________________________________________

Community Meeting Calendar:

APAC Schedule -
Every 2nd and 4th Tuesday at 11:30 AM IST
Bridge: https://bluejeans.com/836554017

NA/EMEA Schedule -
Every 1st and 3rd Tuesday at 01:00 PM EDT
Bridge: https://bluejeans.com/486278655

Gluster-devel mailing list
[hidden email]
https://lists.gluster.org/mailman/listinfo/gluster-devel


signature.asc (853 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Removing glupy from release 5.7

Michael Scherer
In reply to this post by Niels de Vos-5
Le jeudi 13 juin 2019 à 14:28 +0200, Niels de Vos a écrit :

> On Thu, Jun 13, 2019 at 11:08:25AM +0200, Niels de Vos wrote:
> > On Wed, Jun 12, 2019 at 04:09:55PM -0700, Kaleb Keithley wrote:
> > > On Wed, Jun 12, 2019 at 11:36 AM Kaleb Keithley <
> > > [hidden email]> wrote:
> > >
> > > >
> > > > On Wed, Jun 12, 2019 at 10:43 AM Amar Tumballi Suryanarayan <
> > > > [hidden email]> wrote:
> > > >
> > > > >
> > > > > We recently noticed that in one of the package update on
> > > > > builder (ie,
> > > > > centos7.x machines), python3.6 got installed as a dependency.
> > > > > So, yes, it
> > > > > is possible to have python3 in centos7 now.
> > > > >
> > > >
> > > > EPEL updated from python34 to python36 recently, but C7 doesn't
> > > > have
> > > > python3 in the base. I don't think we've ever used EPEL
> > > > packages for
> > > > building.
> > > >
> > > > And GlusterFS-5 isn't python3 ready.
> > > >
> > >
> > > Correction: GlusterFS-5 is mostly or completely python3
> > > ready.  FWIW,
> > > python33 is available on both RHEL7 and CentOS7 from the Software
> > > Collection Library (SCL), and python34 and now python36 are
> > > available from
> > > EPEL.
> > >
> > > But packages built for the CentOS Storage SIG have never used the
> > > SCL or
> > > EPEL (EPEL not allowed) and the shebangs in the .py files are
> > > converted
> > > from /usr/bin/python3 to /usr/bin/python2 during the rpmbuild
> > > %prep stage.
> > > All the python dependencies for the packages remain the python2
> > > flavors.
> > > AFAIK the centos-regression machines ought to be building the
> > > same way.
> >
> > Indeed, there should not be a requirement on having EPEL enabled on
> > the
> > CentOS-7 builders. At least not for the building of the glusterfs
> > tarball. We still need to do releases of glusterfs-4.1 and
> > glusterfs-5,
> > until then it is expected to have python2 as the (only?) version
> > for the
> > system. Is it possible to remove python3 from the CentOS-7 builders
> > and
> > run the jobs that require python3 on the Fedora builders instead?
>
> Actually, if the python-devel package for python3 is installed on the
> CentOS-7 builders, things may work too. It still feels like some sort
> of
> Frankenstein deployment, and we don't expect to this see in
> production
> environments. But maybe this is a workaround in case something
> really,
> really, REALLY depends on python3 on the builders.
To be honest, people would be surprised what happen in production
around (sysadmins tend to discuss around, we all have horrors stories,
stuff that were supposed to be cleaned and wasn't, etc)

After all, "frankenstein deployment now" is better than "perfect
later", especially since lots of IT departements are under constant
pressure (so that's more "perfect never").

I can understand that we want clean and simple code (who doesn't), but
real life is much messier than we want to admit, so we need something
robust.

--
Michael Scherer
Sysadmin, Community Infrastructure




_______________________________________________

Community Meeting Calendar:

APAC Schedule -
Every 2nd and 4th Tuesday at 11:30 AM IST
Bridge: https://bluejeans.com/836554017

NA/EMEA Schedule -
Every 1st and 3rd Tuesday at 01:00 PM EDT
Bridge: https://bluejeans.com/486278655

Gluster-devel mailing list
[hidden email]
https://lists.gluster.org/mailman/listinfo/gluster-devel


signature.asc (853 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Removing glupy from release 5.7

Amar Tumballi Suryanarayan

Considering python3 is anyways the future, I vote for taking the patch we did in master for fixing regression tests with python3 into the release-6 and release-5 branch and getting over this deadlock.

Patch in discussion here is https://review.gluster.org/#/c/glusterfs/+/22829/ and if anyone notices, it changes only the files inside 'tests/' directory, which is not packaged in a release anyways.

Hari, can we get the backport of this patch to both the release branches?

Regards,
Amar

On Thu, Jun 13, 2019 at 7:26 PM Michael Scherer <[hidden email]> wrote:
Le jeudi 13 juin 2019 à 14:28 +0200, Niels de Vos a écrit :
> On Thu, Jun 13, 2019 at 11:08:25AM +0200, Niels de Vos wrote:
> > On Wed, Jun 12, 2019 at 04:09:55PM -0700, Kaleb Keithley wrote:
> > > On Wed, Jun 12, 2019 at 11:36 AM Kaleb Keithley <
> > > [hidden email]> wrote:
> > >
> > > >
> > > > On Wed, Jun 12, 2019 at 10:43 AM Amar Tumballi Suryanarayan <
> > > > [hidden email]> wrote:
> > > >
> > > > >
> > > > > We recently noticed that in one of the package update on
> > > > > builder (ie,
> > > > > centos7.x machines), python3.6 got installed as a dependency.
> > > > > So, yes, it
> > > > > is possible to have python3 in centos7 now.
> > > > >
> > > >
> > > > EPEL updated from python34 to python36 recently, but C7 doesn't
> > > > have
> > > > python3 in the base. I don't think we've ever used EPEL
> > > > packages for
> > > > building.
> > > >
> > > > And GlusterFS-5 isn't python3 ready.
> > > >
> > >
> > > Correction: GlusterFS-5 is mostly or completely python3
> > > ready.  FWIW,
> > > python33 is available on both RHEL7 and CentOS7 from the Software
> > > Collection Library (SCL), and python34 and now python36 are
> > > available from
> > > EPEL.
> > >
> > > But packages built for the CentOS Storage SIG have never used the
> > > SCL or
> > > EPEL (EPEL not allowed) and the shebangs in the .py files are
> > > converted
> > > from /usr/bin/python3 to /usr/bin/python2 during the rpmbuild
> > > %prep stage.
> > > All the python dependencies for the packages remain the python2
> > > flavors.
> > > AFAIK the centos-regression machines ought to be building the
> > > same way.
> >
> > Indeed, there should not be a requirement on having EPEL enabled on
> > the
> > CentOS-7 builders. At least not for the building of the glusterfs
> > tarball. We still need to do releases of glusterfs-4.1 and
> > glusterfs-5,
> > until then it is expected to have python2 as the (only?) version
> > for the
> > system. Is it possible to remove python3 from the CentOS-7 builders
> > and
> > run the jobs that require python3 on the Fedora builders instead?
>
> Actually, if the python-devel package for python3 is installed on the
> CentOS-7 builders, things may work too. It still feels like some sort
> of
> Frankenstein deployment, and we don't expect to this see in
> production
> environments. But maybe this is a workaround in case something
> really,
> really, REALLY depends on python3 on the builders.

To be honest, people would be surprised what happen in production
around (sysadmins tend to discuss around, we all have horrors stories,
stuff that were supposed to be cleaned and wasn't, etc)

After all, "frankenstein deployment now" is better than "perfect
later", especially since lots of IT departements are under constant
pressure (so that's more "perfect never").

I can understand that we want clean and simple code (who doesn't), but
real life is much messier than we want to admit, so we need something
robust.

--
Michael Scherer
Sysadmin, Community Infrastructure



_______________________________________________

Community Meeting Calendar:

APAC Schedule -
Every 2nd and 4th Tuesday at 11:30 AM IST
Bridge: https://bluejeans.com/836554017

NA/EMEA Schedule -
Every 1st and 3rd Tuesday at 01:00 PM EDT
Bridge: https://bluejeans.com/486278655

Gluster-devel mailing list
[hidden email]
https://lists.gluster.org/mailman/listinfo/gluster-devel



--
Amar Tumballi (amarts)

_______________________________________________

Community Meeting Calendar:

APAC Schedule -
Every 2nd and 4th Tuesday at 11:30 AM IST
Bridge: https://bluejeans.com/836554017

NA/EMEA Schedule -
Every 1st and 3rd Tuesday at 01:00 PM EDT
Bridge: https://bluejeans.com/486278655

Gluster-devel mailing list
[hidden email]
https://lists.gluster.org/mailman/listinfo/gluster-devel

Reply | Threaded
Open this post in threaded view
|

Re: Removing glupy from release 5.7

Hari Gowtham
Hi Amar,

I have done the above request earlier with release 5 and still it fails.

Patch: https://review.gluster.org/#/c/glusterfs/+/22855/
build log for failure:
https://build.gluster.org/job/strfmt_errors/18889/artifact/RPMS/el6/i686/build.log

The failure is related to building. So we need to fix the python 3
compatibility issues with release 5 as well.
The build uses python3 and gluster relies on python2.
I'm not sure if the patches to make gluster python3 compatible have
made its way to release 5 and 6.
If not then we have to work on that and make the build changes
necessary to start consuming python3 for release branches.
Or we have to make the build script smarter to use python 2 for
release branches and python 3 for master.

On Thu, Jun 20, 2019 at 11:38 AM Amar Tumballi Suryanarayan
<[hidden email]> wrote:

>
>
> Considering python3 is anyways the future, I vote for taking the patch we did in master for fixing regression tests with python3 into the release-6 and release-5 branch and getting over this deadlock.
>
> Patch in discussion here is https://review.gluster.org/#/c/glusterfs/+/22829/ and if anyone notices, it changes only the files inside 'tests/' directory, which is not packaged in a release anyways.
>
> Hari, can we get the backport of this patch to both the release branches?
>
> Regards,
> Amar
>
> On Thu, Jun 13, 2019 at 7:26 PM Michael Scherer <[hidden email]> wrote:
>>
>> Le jeudi 13 juin 2019 à 14:28 +0200, Niels de Vos a écrit :
>> > On Thu, Jun 13, 2019 at 11:08:25AM +0200, Niels de Vos wrote:
>> > > On Wed, Jun 12, 2019 at 04:09:55PM -0700, Kaleb Keithley wrote:
>> > > > On Wed, Jun 12, 2019 at 11:36 AM Kaleb Keithley <
>> > > > [hidden email]> wrote:
>> > > >
>> > > > >
>> > > > > On Wed, Jun 12, 2019 at 10:43 AM Amar Tumballi Suryanarayan <
>> > > > > [hidden email]> wrote:
>> > > > >
>> > > > > >
>> > > > > > We recently noticed that in one of the package update on
>> > > > > > builder (ie,
>> > > > > > centos7.x machines), python3.6 got installed as a dependency.
>> > > > > > So, yes, it
>> > > > > > is possible to have python3 in centos7 now.
>> > > > > >
>> > > > >
>> > > > > EPEL updated from python34 to python36 recently, but C7 doesn't
>> > > > > have
>> > > > > python3 in the base. I don't think we've ever used EPEL
>> > > > > packages for
>> > > > > building.
>> > > > >
>> > > > > And GlusterFS-5 isn't python3 ready.
>> > > > >
>> > > >
>> > > > Correction: GlusterFS-5 is mostly or completely python3
>> > > > ready.  FWIW,
>> > > > python33 is available on both RHEL7 and CentOS7 from the Software
>> > > > Collection Library (SCL), and python34 and now python36 are
>> > > > available from
>> > > > EPEL.
>> > > >
>> > > > But packages built for the CentOS Storage SIG have never used the
>> > > > SCL or
>> > > > EPEL (EPEL not allowed) and the shebangs in the .py files are
>> > > > converted
>> > > > from /usr/bin/python3 to /usr/bin/python2 during the rpmbuild
>> > > > %prep stage.
>> > > > All the python dependencies for the packages remain the python2
>> > > > flavors.
>> > > > AFAIK the centos-regression machines ought to be building the
>> > > > same way.
>> > >
>> > > Indeed, there should not be a requirement on having EPEL enabled on
>> > > the
>> > > CentOS-7 builders. At least not for the building of the glusterfs
>> > > tarball. We still need to do releases of glusterfs-4.1 and
>> > > glusterfs-5,
>> > > until then it is expected to have python2 as the (only?) version
>> > > for the
>> > > system. Is it possible to remove python3 from the CentOS-7 builders
>> > > and
>> > > run the jobs that require python3 on the Fedora builders instead?
>> >
>> > Actually, if the python-devel package for python3 is installed on the
>> > CentOS-7 builders, things may work too. It still feels like some sort
>> > of
>> > Frankenstein deployment, and we don't expect to this see in
>> > production
>> > environments. But maybe this is a workaround in case something
>> > really,
>> > really, REALLY depends on python3 on the builders.
>>
>> To be honest, people would be surprised what happen in production
>> around (sysadmins tend to discuss around, we all have horrors stories,
>> stuff that were supposed to be cleaned and wasn't, etc)
>>
>> After all, "frankenstein deployment now" is better than "perfect
>> later", especially since lots of IT departements are under constant
>> pressure (so that's more "perfect never").
>>
>> I can understand that we want clean and simple code (who doesn't), but
>> real life is much messier than we want to admit, so we need something
>> robust.
>>
>> --
>> Michael Scherer
>> Sysadmin, Community Infrastructure
>>
>>
>>
>> _______________________________________________
>>
>> Community Meeting Calendar:
>>
>> APAC Schedule -
>> Every 2nd and 4th Tuesday at 11:30 AM IST
>> Bridge: https://bluejeans.com/836554017
>>
>> NA/EMEA Schedule -
>> Every 1st and 3rd Tuesday at 01:00 PM EDT
>> Bridge: https://bluejeans.com/486278655
>>
>> Gluster-devel mailing list
>> [hidden email]
>> https://lists.gluster.org/mailman/listinfo/gluster-devel
>>
>
>
> --
> Amar Tumballi (amarts)
> _______________________________________________
>
> Community Meeting Calendar:
>
> APAC Schedule -
> Every 2nd and 4th Tuesday at 11:30 AM IST
> Bridge: https://bluejeans.com/836554017
>
> NA/EMEA Schedule -
> Every 1st and 3rd Tuesday at 01:00 PM EDT
> Bridge: https://bluejeans.com/486278655
>
> Gluster-devel mailing list
> [hidden email]
> https://lists.gluster.org/mailman/listinfo/gluster-devel
>


--
Regards,
Hari Gowtham.
_______________________________________________

Community Meeting Calendar:

APAC Schedule -
Every 2nd and 4th Tuesday at 11:30 AM IST
Bridge: https://bluejeans.com/836554017

NA/EMEA Schedule -
Every 1st and 3rd Tuesday at 01:00 PM EDT
Bridge: https://bluejeans.com/486278655

Gluster-devel mailing list
[hidden email]
https://lists.gluster.org/mailman/listinfo/gluster-devel

Reply | Threaded
Open this post in threaded view
|

Re: Removing glupy from release 5.7

Niels de Vos-5
In reply to this post by Amar Tumballi Suryanarayan
On Thu, Jun 20, 2019 at 11:36:46AM +0530, Amar Tumballi Suryanarayan wrote:

> Considering python3 is anyways the future, I vote for taking the patch we
> did in master for fixing regression tests with python3 into the release-6
> and release-5 branch and getting over this deadlock.
>
> Patch in discussion here is
> https://review.gluster.org/#/c/glusterfs/+/22829/ and if anyone notices, it
> changes only the files inside 'tests/' directory, which is not packaged in
> a release anyways.
>
> Hari, can we get the backport of this patch to both the release branches?

When going this route, you still need to make sure that the
python3-devel package is available on the CentOS-7 builders. And I
don't know if installing that package is already sufficient, maybe the
backport is not even needed in that case.

Niels


>
> Regards,
> Amar
>
> On Thu, Jun 13, 2019 at 7:26 PM Michael Scherer <[hidden email]> wrote:
>
> > Le jeudi 13 juin 2019 à 14:28 +0200, Niels de Vos a écrit :
> > > On Thu, Jun 13, 2019 at 11:08:25AM +0200, Niels de Vos wrote:
> > > > On Wed, Jun 12, 2019 at 04:09:55PM -0700, Kaleb Keithley wrote:
> > > > > On Wed, Jun 12, 2019 at 11:36 AM Kaleb Keithley <
> > > > > [hidden email]> wrote:
> > > > >
> > > > > >
> > > > > > On Wed, Jun 12, 2019 at 10:43 AM Amar Tumballi Suryanarayan <
> > > > > > [hidden email]> wrote:
> > > > > >
> > > > > > >
> > > > > > > We recently noticed that in one of the package update on
> > > > > > > builder (ie,
> > > > > > > centos7.x machines), python3.6 got installed as a dependency.
> > > > > > > So, yes, it
> > > > > > > is possible to have python3 in centos7 now.
> > > > > > >
> > > > > >
> > > > > > EPEL updated from python34 to python36 recently, but C7 doesn't
> > > > > > have
> > > > > > python3 in the base. I don't think we've ever used EPEL
> > > > > > packages for
> > > > > > building.
> > > > > >
> > > > > > And GlusterFS-5 isn't python3 ready.
> > > > > >
> > > > >
> > > > > Correction: GlusterFS-5 is mostly or completely python3
> > > > > ready.  FWIW,
> > > > > python33 is available on both RHEL7 and CentOS7 from the Software
> > > > > Collection Library (SCL), and python34 and now python36 are
> > > > > available from
> > > > > EPEL.
> > > > >
> > > > > But packages built for the CentOS Storage SIG have never used the
> > > > > SCL or
> > > > > EPEL (EPEL not allowed) and the shebangs in the .py files are
> > > > > converted
> > > > > from /usr/bin/python3 to /usr/bin/python2 during the rpmbuild
> > > > > %prep stage.
> > > > > All the python dependencies for the packages remain the python2
> > > > > flavors.
> > > > > AFAIK the centos-regression machines ought to be building the
> > > > > same way.
> > > >
> > > > Indeed, there should not be a requirement on having EPEL enabled on
> > > > the
> > > > CentOS-7 builders. At least not for the building of the glusterfs
> > > > tarball. We still need to do releases of glusterfs-4.1 and
> > > > glusterfs-5,
> > > > until then it is expected to have python2 as the (only?) version
> > > > for the
> > > > system. Is it possible to remove python3 from the CentOS-7 builders
> > > > and
> > > > run the jobs that require python3 on the Fedora builders instead?
> > >
> > > Actually, if the python-devel package for python3 is installed on the
> > > CentOS-7 builders, things may work too. It still feels like some sort
> > > of
> > > Frankenstein deployment, and we don't expect to this see in
> > > production
> > > environments. But maybe this is a workaround in case something
> > > really,
> > > really, REALLY depends on python3 on the builders.
> >
> > To be honest, people would be surprised what happen in production
> > around (sysadmins tend to discuss around, we all have horrors stories,
> > stuff that were supposed to be cleaned and wasn't, etc)
> >
> > After all, "frankenstein deployment now" is better than "perfect
> > later", especially since lots of IT departements are under constant
> > pressure (so that's more "perfect never").
> >
> > I can understand that we want clean and simple code (who doesn't), but
> > real life is much messier than we want to admit, so we need something
> > robust.
> >
> > --
> > Michael Scherer
> > Sysadmin, Community Infrastructure
> >
> >
> >
> > _______________________________________________
> >
> > Community Meeting Calendar:
> >
> > APAC Schedule -
> > Every 2nd and 4th Tuesday at 11:30 AM IST
> > Bridge: https://bluejeans.com/836554017
> >
> > NA/EMEA Schedule -
> > Every 1st and 3rd Tuesday at 01:00 PM EDT
> > Bridge: https://bluejeans.com/486278655
> >
> > Gluster-devel mailing list
> > [hidden email]
> > https://lists.gluster.org/mailman/listinfo/gluster-devel
> >
> >
>
> --
> Amar Tumballi (amarts)
_______________________________________________

Community Meeting Calendar:

APAC Schedule -
Every 2nd and 4th Tuesday at 11:30 AM IST
Bridge: https://bluejeans.com/836554017

NA/EMEA Schedule -
Every 1st and 3rd Tuesday at 01:00 PM EDT
Bridge: https://bluejeans.com/486278655

Gluster-devel mailing list
[hidden email]
https://lists.gluster.org/mailman/listinfo/gluster-devel

Reply | Threaded
Open this post in threaded view
|

Re: Removing glupy from release 5.7

Amar Tumballi Suryanarayan


On Thu, Jun 20, 2019 at 1:13 PM Niels de Vos <[hidden email]> wrote:
On Thu, Jun 20, 2019 at 11:36:46AM +0530, Amar Tumballi Suryanarayan wrote:
> Considering python3 is anyways the future, I vote for taking the patch we
> did in master for fixing regression tests with python3 into the release-6
> and release-5 branch and getting over this deadlock.
>
> Patch in discussion here is
> https://review.gluster.org/#/c/glusterfs/+/22829/ and if anyone notices, it
> changes only the files inside 'tests/' directory, which is not packaged in
> a release anyways.
>
> Hari, can we get the backport of this patch to both the release branches?

When going this route, you still need to make sure that the
python3-devel package is available on the CentOS-7 builders. And I
don't know if installing that package is already sufficient, maybe the
backport is not even needed in that case.


I was thinking, having this patch makes it compatible with both python2 and python3, so technically, it allows us to move to Fedora30 if we need to run regression there. (and CentOS7 with only python2).

The above patch made it compatible, not mandatory to have python3. So, treating it as a bug fix.
 
Niels


>
> Regards,
> Amar
>
> On Thu, Jun 13, 2019 at 7:26 PM Michael Scherer <[hidden email]> wrote:
>
> > Le jeudi 13 juin 2019 à 14:28 +0200, Niels de Vos a écrit :
> > > On Thu, Jun 13, 2019 at 11:08:25AM +0200, Niels de Vos wrote:
> > > > On Wed, Jun 12, 2019 at 04:09:55PM -0700, Kaleb Keithley wrote:
> > > > > On Wed, Jun 12, 2019 at 11:36 AM Kaleb Keithley <
> > > > > [hidden email]> wrote:
> > > > >
> > > > > >
> > > > > > On Wed, Jun 12, 2019 at 10:43 AM Amar Tumballi Suryanarayan <
> > > > > > [hidden email]> wrote:
> > > > > >
> > > > > > >
> > > > > > > We recently noticed that in one of the package update on
> > > > > > > builder (ie,
> > > > > > > centos7.x machines), python3.6 got installed as a dependency.
> > > > > > > So, yes, it
> > > > > > > is possible to have python3 in centos7 now.
> > > > > > >
> > > > > >
> > > > > > EPEL updated from python34 to python36 recently, but C7 doesn't
> > > > > > have
> > > > > > python3 in the base. I don't think we've ever used EPEL
> > > > > > packages for
> > > > > > building.
> > > > > >
> > > > > > And GlusterFS-5 isn't python3 ready.
> > > > > >
> > > > >
> > > > > Correction: GlusterFS-5 is mostly or completely python3
> > > > > ready.  FWIW,
> > > > > python33 is available on both RHEL7 and CentOS7 from the Software
> > > > > Collection Library (SCL), and python34 and now python36 are
> > > > > available from
> > > > > EPEL.
> > > > >
> > > > > But packages built for the CentOS Storage SIG have never used the
> > > > > SCL or
> > > > > EPEL (EPEL not allowed) and the shebangs in the .py files are
> > > > > converted
> > > > > from /usr/bin/python3 to /usr/bin/python2 during the rpmbuild
> > > > > %prep stage.
> > > > > All the python dependencies for the packages remain the python2
> > > > > flavors.
> > > > > AFAIK the centos-regression machines ought to be building the
> > > > > same way.
> > > >
> > > > Indeed, there should not be a requirement on having EPEL enabled on
> > > > the
> > > > CentOS-7 builders. At least not for the building of the glusterfs
> > > > tarball. We still need to do releases of glusterfs-4.1 and
> > > > glusterfs-5,
> > > > until then it is expected to have python2 as the (only?) version
> > > > for the
> > > > system. Is it possible to remove python3 from the CentOS-7 builders
> > > > and
> > > > run the jobs that require python3 on the Fedora builders instead?
> > >
> > > Actually, if the python-devel package for python3 is installed on the
> > > CentOS-7 builders, things may work too. It still feels like some sort
> > > of
> > > Frankenstein deployment, and we don't expect to this see in
> > > production
> > > environments. But maybe this is a workaround in case something
> > > really,
> > > really, REALLY depends on python3 on the builders.
> >
> > To be honest, people would be surprised what happen in production
> > around (sysadmins tend to discuss around, we all have horrors stories,
> > stuff that were supposed to be cleaned and wasn't, etc)
> >
> > After all, "frankenstein deployment now" is better than "perfect
> > later", especially since lots of IT departements are under constant
> > pressure (so that's more "perfect never").
> >
> > I can understand that we want clean and simple code (who doesn't), but
> > real life is much messier than we want to admit, so we need something
> > robust.
> >
> > --
> > Michael Scherer
> > Sysadmin, Community Infrastructure
> >
> >
> >
> > _______________________________________________
> >
> > Community Meeting Calendar:
> >
> > APAC Schedule -
> > Every 2nd and 4th Tuesday at 11:30 AM IST
> > Bridge: https://bluejeans.com/836554017
> >
> > NA/EMEA Schedule -
> > Every 1st and 3rd Tuesday at 01:00 PM EDT
> > Bridge: https://bluejeans.com/486278655
> >
> > Gluster-devel mailing list
> > [hidden email]
> > https://lists.gluster.org/mailman/listinfo/gluster-devel
> >
> >
>
> --
> Amar Tumballi (amarts)


--
Amar Tumballi (amarts)

_______________________________________________

Community Meeting Calendar:

APAC Schedule -
Every 2nd and 4th Tuesday at 11:30 AM IST
Bridge: https://bluejeans.com/836554017

NA/EMEA Schedule -
Every 1st and 3rd Tuesday at 01:00 PM EDT
Bridge: https://bluejeans.com/486278655

Gluster-devel mailing list
[hidden email]
https://lists.gluster.org/mailman/listinfo/gluster-devel

Reply | Threaded
Open this post in threaded view
|

Re: Removing glupy from release 5.7

Niels de Vos-5
On Thu, Jun 20, 2019 at 02:11:21PM +0530, Amar Tumballi Suryanarayan wrote:

> On Thu, Jun 20, 2019 at 1:13 PM Niels de Vos <[hidden email]> wrote:
>
> > On Thu, Jun 20, 2019 at 11:36:46AM +0530, Amar Tumballi Suryanarayan wrote:
> > > Considering python3 is anyways the future, I vote for taking the patch we
> > > did in master for fixing regression tests with python3 into the release-6
> > > and release-5 branch and getting over this deadlock.
> > >
> > > Patch in discussion here is
> > > https://review.gluster.org/#/c/glusterfs/+/22829/ and if anyone
> > notices, it
> > > changes only the files inside 'tests/' directory, which is not packaged
> > in
> > > a release anyways.
> > >
> > > Hari, can we get the backport of this patch to both the release branches?
> >
> > When going this route, you still need to make sure that the
> > python3-devel package is available on the CentOS-7 builders. And I
> > don't know if installing that package is already sufficient, maybe the
> > backport is not even needed in that case.
> >
> >
> I was thinking, having this patch makes it compatible with both python2 and
> python3, so technically, it allows us to move to Fedora30 if we need to run
> regression there. (and CentOS7 with only python2).
>
> The above patch made it compatible, not mandatory to have python3. So,
> treating it as a bug fix.

Well, whatever Python is detected (python3 has preference over python2),
needs to have the -devel package available too. Detection is done by
probing the python<X> executable. The Matching header files from -devel
need to be present in order to be able to build glupy (and others?).

I do not think compatibility for python3/2 is the problem while
building the tarball. The backport might become relevant while running
tests on environments where there is no python2.

Niels


>
>
> > Niels
> >
> >
> > >
> > > Regards,
> > > Amar
> > >
> > > On Thu, Jun 13, 2019 at 7:26 PM Michael Scherer <[hidden email]>
> > wrote:
> > >
> > > > Le jeudi 13 juin 2019 à 14:28 +0200, Niels de Vos a écrit :
> > > > > On Thu, Jun 13, 2019 at 11:08:25AM +0200, Niels de Vos wrote:
> > > > > > On Wed, Jun 12, 2019 at 04:09:55PM -0700, Kaleb Keithley wrote:
> > > > > > > On Wed, Jun 12, 2019 at 11:36 AM Kaleb Keithley <
> > > > > > > [hidden email]> wrote:
> > > > > > >
> > > > > > > >
> > > > > > > > On Wed, Jun 12, 2019 at 10:43 AM Amar Tumballi Suryanarayan <
> > > > > > > > [hidden email]> wrote:
> > > > > > > >
> > > > > > > > >
> > > > > > > > > We recently noticed that in one of the package update on
> > > > > > > > > builder (ie,
> > > > > > > > > centos7.x machines), python3.6 got installed as a dependency.
> > > > > > > > > So, yes, it
> > > > > > > > > is possible to have python3 in centos7 now.
> > > > > > > > >
> > > > > > > >
> > > > > > > > EPEL updated from python34 to python36 recently, but C7 doesn't
> > > > > > > > have
> > > > > > > > python3 in the base. I don't think we've ever used EPEL
> > > > > > > > packages for
> > > > > > > > building.
> > > > > > > >
> > > > > > > > And GlusterFS-5 isn't python3 ready.
> > > > > > > >
> > > > > > >
> > > > > > > Correction: GlusterFS-5 is mostly or completely python3
> > > > > > > ready.  FWIW,
> > > > > > > python33 is available on both RHEL7 and CentOS7 from the Software
> > > > > > > Collection Library (SCL), and python34 and now python36 are
> > > > > > > available from
> > > > > > > EPEL.
> > > > > > >
> > > > > > > But packages built for the CentOS Storage SIG have never used the
> > > > > > > SCL or
> > > > > > > EPEL (EPEL not allowed) and the shebangs in the .py files are
> > > > > > > converted
> > > > > > > from /usr/bin/python3 to /usr/bin/python2 during the rpmbuild
> > > > > > > %prep stage.
> > > > > > > All the python dependencies for the packages remain the python2
> > > > > > > flavors.
> > > > > > > AFAIK the centos-regression machines ought to be building the
> > > > > > > same way.
> > > > > >
> > > > > > Indeed, there should not be a requirement on having EPEL enabled on
> > > > > > the
> > > > > > CentOS-7 builders. At least not for the building of the glusterfs
> > > > > > tarball. We still need to do releases of glusterfs-4.1 and
> > > > > > glusterfs-5,
> > > > > > until then it is expected to have python2 as the (only?) version
> > > > > > for the
> > > > > > system. Is it possible to remove python3 from the CentOS-7 builders
> > > > > > and
> > > > > > run the jobs that require python3 on the Fedora builders instead?
> > > > >
> > > > > Actually, if the python-devel package for python3 is installed on the
> > > > > CentOS-7 builders, things may work too. It still feels like some sort
> > > > > of
> > > > > Frankenstein deployment, and we don't expect to this see in
> > > > > production
> > > > > environments. But maybe this is a workaround in case something
> > > > > really,
> > > > > really, REALLY depends on python3 on the builders.
> > > >
> > > > To be honest, people would be surprised what happen in production
> > > > around (sysadmins tend to discuss around, we all have horrors stories,
> > > > stuff that were supposed to be cleaned and wasn't, etc)
> > > >
> > > > After all, "frankenstein deployment now" is better than "perfect
> > > > later", especially since lots of IT departements are under constant
> > > > pressure (so that's more "perfect never").
> > > >
> > > > I can understand that we want clean and simple code (who doesn't), but
> > > > real life is much messier than we want to admit, so we need something
> > > > robust.
> > > >
> > > > --
> > > > Michael Scherer
> > > > Sysadmin, Community Infrastructure
> > > >
> > > >
> > > >
> > > > _______________________________________________
> > > >
> > > > Community Meeting Calendar:
> > > >
> > > > APAC Schedule -
> > > > Every 2nd and 4th Tuesday at 11:30 AM IST
> > > > Bridge: https://bluejeans.com/836554017
> > > >
> > > > NA/EMEA Schedule -
> > > > Every 1st and 3rd Tuesday at 01:00 PM EDT
> > > > Bridge: https://bluejeans.com/486278655
> > > >
> > > > Gluster-devel mailing list
> > > > [hidden email]
> > > > https://lists.gluster.org/mailman/listinfo/gluster-devel
> > > >
> > > >
> > >
> > > --
> > > Amar Tumballi (amarts)
> >
>
>
> --
> Amar Tumballi (amarts)
_______________________________________________

Community Meeting Calendar:

APAC Schedule -
Every 2nd and 4th Tuesday at 11:30 AM IST
Bridge: https://bluejeans.com/836554017

NA/EMEA Schedule -
Every 1st and 3rd Tuesday at 01:00 PM EDT
Bridge: https://bluejeans.com/486278655

Gluster-devel mailing list
[hidden email]
https://lists.gluster.org/mailman/listinfo/gluster-devel

12