Proposal to change gNFS status

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
15 messages Options
Reply | Threaded
Open this post in threaded view
|

Proposal to change gNFS status

Amar Tumballi-2
Hi All,

As per the discussion on https://review.gluster.org/23645, recently we changed the status of gNFS (gluster's native NFSv3 support) feature to 'Depricated / Orphan' state. (ref: https://github.com/gluster/glusterfs/blob/master/MAINTAINERS#L185..L189). With this email, I am proposing to change the status again to 'Odd Fixes' (ref: https://github.com/gluster/glusterfs/blob/master/MAINTAINERS#L22

TL;DR;

I understand the current maintainers are not able to focus on maintaining it as the focus of the project, as earlier described, is keeping NFS-Ganesha based integration with glusterfs. But, I am volunteering along with Xie Changlong (currently working at Chinamobile), to keep the feature running as it used to in previous versions. Hence the status of 'Odd Fixes'. 

Before sending the patch to make these changes, I am proposing it here now, as gNFS is not even shipped with latest glusterfs-7.0 releases. I have heard from some users that it was working great for them with earlier releases, as all they wanted was NFS v3 support, and not much of features from gNFS. Also note that, even though the packages are not built, none of the regression tests using gNFS are stopped with latest master, so it is working same from at least last 2 years. 

I request the package maintainers to please add '--with gnfs' (or --enable-gnfs) back to their release script through this email, so those users wanting to use gNFS happily can continue to use it. Also points to users/admins is that, the status is 'Odd Fixes', so don't expect any 'enhancements' on the features provided by gNFS.

Happy to hear feedback, if any.

Regards,
Amar


_______________________________________________

Community Meeting Calendar:

APAC Schedule -
Every 2nd and 4th Tuesday at 11:30 AM IST
Bridge: https://bluejeans.com/441850968


NA/EMEA Schedule -
Every 1st and 3rd Tuesday at 01:00 PM EDT
Bridge: https://bluejeans.com/441850968

Gluster-devel mailing list
[hidden email]
https://lists.gluster.org/mailman/listinfo/gluster-devel

Reply | Threaded
Open this post in threaded view
|

Re: Proposal to change gNFS status

Yaniv Kaul


On Thu, Nov 21, 2019 at 12:31 PM Amar Tumballi <[hidden email]> wrote:
Hi All,

As per the discussion on https://review.gluster.org/23645, recently we changed the status of gNFS (gluster's native NFSv3 support) feature to 'Depricated / Orphan' state. (ref: https://github.com/gluster/glusterfs/blob/master/MAINTAINERS#L185..L189). With this email, I am proposing to change the status again to 'Odd Fixes' (ref: https://github.com/gluster/glusterfs/blob/master/MAINTAINERS#L22

TL;DR;

I understand the current maintainers are not able to focus on maintaining it as the focus of the project, as earlier described, is keeping NFS-Ganesha based integration with glusterfs. But, I am volunteering along with Xie Changlong (currently working at Chinamobile), to keep the feature running as it used to in previous versions. Hence the status of 'Odd Fixes'. 

As long as there are volunteers to keep maintaining it, I think it's a great idea to have it.
I personally do not see the value, as I believe it's somewhat of a redundant effort in several ways (it's not helping stabilizing having more than one implementation, it doesn't help in improving it from features perspective - pNFS, NFS v4.2, IPv6, performance... and it 'costs' more in CI and testing), but indeed if there's interest from both the users and the development community - then sure!


Before sending the patch to make these changes, I am proposing it here now, as gNFS is not even shipped with latest glusterfs-7.0 releases. I have heard from some users that it was working great for them with earlier releases, as all they wanted was NFS v3 support, and not much of features from gNFS. Also note that, even though the packages are not built, none of the regression tests using gNFS are stopped with latest master, so it is working same from at least last 2 years. 

I request the package maintainers to please add '--with gnfs' (or --enable-gnfs) back to their release script through this email, so those users wanting to use gNFS happily can continue to use it. Also points to users/admins is that, the status is 'Odd Fixes', so don't expect any 'enhancements' on the features provided by gNFS.

'Odd fixes' sounds odd to me. Better come with a better terminology. ;-)
Y.

Happy to hear feedback, if any.

Regards,
Amar

_______________________________________________

Community Meeting Calendar:

APAC Schedule -
Every 2nd and 4th Tuesday at 11:30 AM IST
Bridge: https://bluejeans.com/441850968


NA/EMEA Schedule -
Every 1st and 3rd Tuesday at 01:00 PM EDT
Bridge: https://bluejeans.com/441850968

Gluster-devel mailing list
[hidden email]
https://lists.gluster.org/mailman/listinfo/gluster-devel


_______________________________________________

Community Meeting Calendar:

APAC Schedule -
Every 2nd and 4th Tuesday at 11:30 AM IST
Bridge: https://bluejeans.com/441850968


NA/EMEA Schedule -
Every 1st and 3rd Tuesday at 01:00 PM EDT
Bridge: https://bluejeans.com/441850968

Gluster-devel mailing list
[hidden email]
https://lists.gluster.org/mailman/listinfo/gluster-devel

Reply | Threaded
Open this post in threaded view
|

Fwd: Proposal to change gNFS status

Amar Tumballi-2
In reply to this post by Amar Tumballi-2
Hi Markus,

Looks like your email got bounced as you were not member of the list. I got the email as I had you in Cc earlier (through the github communications).

Forwarding it to the list so everyone gets your email.


Also, Yaniv, about the 'name' Odd Fixes, I am not a big fan either, but out of the existing options in MAINTAINERS file, that was best suited. If we have to change the name, 'Maintained, but no enhancements'  would give very clear messaging IMO. Happy to hear what people think.

-Amar 

---------- Forwarded message ---------
From: Markus Seywald <[hidden email]>
Date: Thu, Nov 21, 2019 at 4:48 PM
Subject: RE: Proposal to change gNFS status

Hi Everyone,

 

Just an Feedback from 1 User End.

 

We used GNFS with NFS3 now since  some years.  The 3.12.15 have some Bugs,… but it’s running most stable in comparisons to older Versions.

 

NFS-Ganesha we where not able to get it stable since Years. NFS-Ganesha is very instable once its getting used with VMware (6.0/6.7). Instable in such Way making it unusable. Even 2.8.x. The Idea of using NFS4.1 with Vmware was an great Mind,……. But never worked so far. Vmware just support “Session Trunking” and not pnfs in general. They don’t support using non Vsan, binded to open-source community. I’am not sure how other Hypervisor works with,……

 

Also I think there is a big Complication once it comes to Issues once 2 Open-Source Products are in use. Once an Issue appears and not clear what ever End is the root Cause,…………… could be an extensive Process to get a Fix for.

 

So looking forward, on my End I really would appreciate if GNFS would be as a Default Package part of Gluster. As it’s the most simple Way providing Peoples an easy NFS Solution.  Glusterfs as a Client is not supported or available for each OS. It would be good providing “Fixes” once there are Bugs with newer Glusterfs Versions with GNFS in use (like #764,#765).

 

Also I would be more then happy if Devs would also deep Focus support Matrix with Hypervisors and to make it stable for most common ones like Vmware. May Hyper-V/KVM as well. (nfs-ganesha, gluster,..)

 

Reason why I’am thinking this Way from User-End is following:

 

Markets are moving very very fast. Everything need to get more stable, more efficient and flexible with the tiniest Effort possible. This means as well based on Costs > Reason why Open-Source getting more and more interesting in Total. Changing Products causing opposite Situation. Biggest Kickback is once Products getting changed. Leading to Risks Use Cases not working anymore or if lots Bugs/Issues arise.  That can happen with Fully Payed Products as well > but there is then a big Support Channel behind.

 

Top Leader in Virtualization Sector also focusing Kubernetes and combining Stacks for easier usages.

 

So I think it would be really really good if gluster and NFS-ganesha  getting also fully adopted for the Virtualization Solutions.  As this World is changing more and more into this Layers. And more Activities into this Direction we see in the Future.

 

I understand this is may just some Words from 1 user,…. But may some others thinking Similar.

 

Appreciated and Thx for Reading.

 

Max

 

From: Amar Tumballi <[hidden email]>
Sent: Donnerstag, 21. November 2019 11:31
To: GlusterFS Maintainers <[hidden email]>
Cc: Gluster Devel <[hidden email]>; Gluster users maillist <[hidden email]>; xiechanglong.d <[hidden email]>
Subject: Proposal to change gNFS status

 

Hi All,

 

As per the discussion on https://review.gluster.org/23645, recently we changed the status of gNFS (gluster's native NFSv3 support) feature to 'Depricated / Orphan' state. (ref: https://github.com/gluster/glusterfs/blob/master/MAINTAINERS#L185..L189). With this email, I am proposing to change the status again to 'Odd Fixes' (ref: https://github.com/gluster/glusterfs/blob/master/MAINTAINERS#L22

 

TL;DR;

 

I understand the current maintainers are not able to focus on maintaining it as the focus of the project, as earlier described, is keeping NFS-Ganesha based integration with glusterfs. But, I am volunteering along with Xie Changlong (currently working at Chinamobile), to keep the feature running as it used to in previous versions. Hence the status of 'Odd Fixes'. 

 

Before sending the patch to make these changes, I am proposing it here now, as gNFS is not even shipped with latest glusterfs-7.0 releases. I have heard from some users that it was working great for them with earlier releases, as all they wanted was NFS v3 support, and not much of features from gNFS. Also note that, even though the packages are not built, none of the regression tests using gNFS are stopped with latest master, so it is working same from at least last 2 years. 

 

I request the package maintainers to please add '--with gnfs' (or --enable-gnfs) back to their release script through this email, so those users wanting to use gNFS happily can continue to use it. Also points to users/admins is that, the status is 'Odd Fixes', so don't expect any 'enhancements' on the features provided by gNFS.

 

Happy to hear feedback, if any.

 

Regards,

Amar

 

Privacy Notice: If you are receiving this email it is because you have given us your email address or we have a legitimate business
interest with your organisation. Beeks Financial Cloud will not retain any of your personal information without lawful reason to do so.
You have the right to request that we remove your email address and other personal details at any time. 
Read our full Data Protection Policy and Privacy Notice
Beeks Financial Cloud Group plc is incorporated in Scotland with registration number SC521839 and registered office at Lumina Building, 40 Ainslie Road, Ground Floor, Hillington Park, Glasgow, G52 4RU, United Kingdom.


_______________________________________________

Community Meeting Calendar:

APAC Schedule -
Every 2nd and 4th Tuesday at 11:30 AM IST
Bridge: https://bluejeans.com/441850968


NA/EMEA Schedule -
Every 1st and 3rd Tuesday at 01:00 PM EDT
Bridge: https://bluejeans.com/441850968

Gluster-devel mailing list
[hidden email]
https://lists.gluster.org/mailman/listinfo/gluster-devel

Reply | Threaded
Open this post in threaded view
|

Re: [Gluster-Maintainers] Proposal to change gNFS status

Kaleb Keithley
In reply to this post by Amar Tumballi-2
I personally wouldn't call three years ago — when we started to deprecate it, in glusterfs-3.9 — a recent change.

As a community the decision was made to move to NFS-Ganesha as the preferred NFS solution, but it was agreed to keep the old code in the tree for those who wanted it. There have been plans to drop it from the community packages for most of those three years, but we didn't follow through across the board until fairly recently. Perhaps the most telling piece of data is that it's been gone from the packages in the CentOS Storage SIG in glusterfs-4.0, -4.1, -5, -6, and -7 with no complaints ever, that I can recall.

Ganesha is a preferable solution because it supports NFSv4, NFSv4.1, NFSv4.2, and pNFS, in addition to legacy NFSv3. More importantly, it is actively developed, maintained, and supported, both in the community and commercially. There are several vendors selling it, or support for it; and there are community packages for it for all the same distributions that Gluster packages are available for.

Out in the world, the default these days is NFSv4. Specifically v4.2 or v4.1 depending on how recent your linux kernel is. In the linux kernel, client mounts start negotiating for v4.2 and work down to v4.1, v4.0, and only as a last resort v3. NFSv3 client support in the linux kernel largely exists at this point only because of the large number of legacy servers still running that can't do anything higher than v3. The linux NFS developers would drop the v3 support in a heartbeat if they could.

IMO, providing it, and calling it maintained, only encourages people to keep using a dead end solution. Anyone in favor of bringing back NFSv2, SSHv1, or X10R4? No? I didn't think so.

The recent issue[1] where someone built gnfs in glusterfs-7.0 on CentOS7 strongly suggests to me that gnfs is not actually working well. Three years of no maintenance seems to have taken its toll.

Other people are more than welcome to build their own packages from the src.rpms and/or tarballs that are available from gluster — and support them. It's still in the source and there are no plans to remove it. (Unlike most of the other deprecated features which were recently removed in glusterfs-7.)




On Thu, Nov 21, 2019 at 5:31 AM Amar Tumballi <[hidden email]> wrote:
Hi All,

As per the discussion on https://review.gluster.org/23645, recently we changed the status of gNFS (gluster's native NFSv3 support) feature to 'Depricated / Orphan' state. (ref: https://github.com/gluster/glusterfs/blob/master/MAINTAINERS#L185..L189). With this email, I am proposing to change the status again to 'Odd Fixes' (ref: https://github.com/gluster/glusterfs/blob/master/MAINTAINERS#L22

TL;DR;

I understand the current maintainers are not able to focus on maintaining it as the focus of the project, as earlier described, is keeping NFS-Ganesha based integration with glusterfs. But, I am volunteering along with Xie Changlong (currently working at Chinamobile), to keep the feature running as it used to in previous versions. Hence the status of 'Odd Fixes'. 

Before sending the patch to make these changes, I am proposing it here now, as gNFS is not even shipped with latest glusterfs-7.0 releases. I have heard from some users that it was working great for them with earlier releases, as all they wanted was NFS v3 support, and not much of features from gNFS. Also note that, even though the packages are not built, none of the regression tests using gNFS are stopped with latest master, so it is working same from at least last 2 years. 

I request the package maintainers to please add '--with gnfs' (or --enable-gnfs) back to their release script through this email, so those users wanting to use gNFS happily can continue to use it. Also points to users/admins is that, the status is 'Odd Fixes', so don't expect any 'enhancements' on the features provided by gNFS.

Happy to hear feedback, if any.

Regards,
Amar

_______________________________________________
maintainers mailing list
[hidden email]
https://lists.gluster.org/mailman/listinfo/maintainers

_______________________________________________

Community Meeting Calendar:

APAC Schedule -
Every 2nd and 4th Tuesday at 11:30 AM IST
Bridge: https://bluejeans.com/441850968


NA/EMEA Schedule -
Every 1st and 3rd Tuesday at 01:00 PM EDT
Bridge: https://bluejeans.com/441850968

Gluster-devel mailing list
[hidden email]
https://lists.gluster.org/mailman/listinfo/gluster-devel

Reply | Threaded
Open this post in threaded view
|

Re: [Gluster-Maintainers] Proposal to change gNFS status

Kaleb Keithley

Independent of anything else—

Maintain it. Send patches to gerrit. Get the requisite +2 reviews on the patches. Amar still has commit privs AFAIK; he can merge anything that gets two votes.

It's open source meritocracy.

If there's real support for it then it makes a stronger case for adding it back to the community packages.

On Thu, Nov 21, 2019 at 4:14 PM Kaleb Keithley <[hidden email]> wrote:
I personally wouldn't call three years ago — when we started to deprecate it, in glusterfs-3.9 — a recent change.

As a community the decision was made to move to NFS-Ganesha as the preferred NFS solution, but it was agreed to keep the old code in the tree for those who wanted it. There have been plans to drop it from the community packages for most of those three years, but we didn't follow through across the board until fairly recently. Perhaps the most telling piece of data is that it's been gone from the packages in the CentOS Storage SIG in glusterfs-4.0, -4.1, -5, -6, and -7 with no complaints ever, that I can recall.

Ganesha is a preferable solution because it supports NFSv4, NFSv4.1, NFSv4.2, and pNFS, in addition to legacy NFSv3. More importantly, it is actively developed, maintained, and supported, both in the community and commercially. There are several vendors selling it, or support for it; and there are community packages for it for all the same distributions that Gluster packages are available for.

Out in the world, the default these days is NFSv4. Specifically v4.2 or v4.1 depending on how recent your linux kernel is. In the linux kernel, client mounts start negotiating for v4.2 and work down to v4.1, v4.0, and only as a last resort v3. NFSv3 client support in the linux kernel largely exists at this point only because of the large number of legacy servers still running that can't do anything higher than v3. The linux NFS developers would drop the v3 support in a heartbeat if they could.

IMO, providing it, and calling it maintained, only encourages people to keep using a dead end solution. Anyone in favor of bringing back NFSv2, SSHv1, or X10R4? No? I didn't think so.

The recent issue[1] where someone built gnfs in glusterfs-7.0 on CentOS7 strongly suggests to me that gnfs is not actually working well. Three years of no maintenance seems to have taken its toll.

Other people are more than welcome to build their own packages from the src.rpms and/or tarballs that are available from gluster — and support them. It's still in the source and there are no plans to remove it. (Unlike most of the other deprecated features which were recently removed in glusterfs-7.)




On Thu, Nov 21, 2019 at 5:31 AM Amar Tumballi <[hidden email]> wrote:
Hi All,

As per the discussion on https://review.gluster.org/23645, recently we changed the status of gNFS (gluster's native NFSv3 support) feature to 'Depricated / Orphan' state. (ref: https://github.com/gluster/glusterfs/blob/master/MAINTAINERS#L185..L189). With this email, I am proposing to change the status again to 'Odd Fixes' (ref: https://github.com/gluster/glusterfs/blob/master/MAINTAINERS#L22

TL;DR;

I understand the current maintainers are not able to focus on maintaining it as the focus of the project, as earlier described, is keeping NFS-Ganesha based integration with glusterfs. But, I am volunteering along with Xie Changlong (currently working at Chinamobile), to keep the feature running as it used to in previous versions. Hence the status of 'Odd Fixes'. 

Before sending the patch to make these changes, I am proposing it here now, as gNFS is not even shipped with latest glusterfs-7.0 releases. I have heard from some users that it was working great for them with earlier releases, as all they wanted was NFS v3 support, and not much of features from gNFS. Also note that, even though the packages are not built, none of the regression tests using gNFS are stopped with latest master, so it is working same from at least last 2 years. 

I request the package maintainers to please add '--with gnfs' (or --enable-gnfs) back to their release script through this email, so those users wanting to use gNFS happily can continue to use it. Also points to users/admins is that, the status is 'Odd Fixes', so don't expect any 'enhancements' on the features provided by gNFS.

Happy to hear feedback, if any.

Regards,
Amar

_______________________________________________
maintainers mailing list
[hidden email]
https://lists.gluster.org/mailman/listinfo/maintainers

_______________________________________________

Community Meeting Calendar:

APAC Schedule -
Every 2nd and 4th Tuesday at 11:30 AM IST
Bridge: https://bluejeans.com/441850968


NA/EMEA Schedule -
Every 1st and 3rd Tuesday at 01:00 PM EDT
Bridge: https://bluejeans.com/441850968

Gluster-devel mailing list
[hidden email]
https://lists.gluster.org/mailman/listinfo/gluster-devel

Reply | Threaded
Open this post in threaded view
|

Re: [Gluster-Maintainers] Proposal to change gNFSstatus

Xie Changlong-2
In reply to this post by Kaleb Keithley


在 2019/11/22 5:14, Kaleb Keithley 写道:
I personally wouldn't call three years ago — when we started to deprecate it, in glusterfs-3.9 — a recent change.

As a community the decision was made to move to NFS-Ganesha as the preferred NFS solution, but it was agreed to keep the old code in the tree for those who wanted it. There have been plans to drop it from the community packages for most of those three years, but we didn't follow through across the board until fairly recently. Perhaps the most telling piece of data is that it's been gone from the packages in the CentOS Storage SIG in glusterfs-4.0, -4.1, -5, -6, and -7 with no complaints ever, that I can recall.

Ganesha is a preferable solution because it supports NFSv4, NFSv4.1, NFSv4.2, and pNFS, in addition to legacy NFSv3. More importantly, it is actively developed, maintained, and supported, both in the community and commercially. There are several vendors selling it, or support for it; and there are community packages for it for all the same distributions that Gluster packages are available for.

Out in the world, the default these days is NFSv4. Specifically v4.2 or v4.1 depending on how recent your linux kernel is. In the linux kernel, client mounts start negotiating for v4.2 and work down to v4.1, v4.0, and only as a last resort v3. NFSv3 client support in the linux kernel largely exists at this point only because of the large number of legacy servers still running that can't do anything higher than v3. The linux NFS developers would drop the v3 support in a heartbeat if they could.

IMO, providing it, and calling it maintained, only encourages people to keep using a dead end solution. Anyone in favor of bringing back NFSv2, SSHv1, or X10R4? No? I didn't think so.

The recent issue[1] where someone built gnfs in glusterfs-7.0 on CentOS7 strongly suggests to me that gnfs is not actually working well. Three years of no maintenance seems to have taken its toll.

Other people are more than welcome to build their own packages from the src.rpms and/or tarballs that are available from gluster — and support them. It's still in the source and there are no plans to remove it. (Unlike most of the other deprecated features which were recently removed in glusterfs-7.)





It seems https://bugzilla.redhat.com/show_bug.cgi?id=1727248 has resolved this issue.

Here i'll talk about something from commerical company view.  For security reasons most government procurement projects only allow universal storage protocol(nfs, cifs etc) what means fuse will be excluded.  Consindering performance requirements, the only option is nfs. 

Nfsv4 is stateful protocol, but i see on performance improvement. Trust me, nfs-ganesha(v3, v4) shows  ~30% performance degradation versus gnfs  for either small or big files r/w in practice.  Further, many customers prefer nfs client than cifs in windows, because the poor cifs performance, AFAIK nfs-ganesha is not going well with windows nfs client.

Gnfs is stable enough, we have about ~1000 servers, 4~24 servers for a gluster cluster, about ~2000 nfs clients, all works fine till the last two years expect some memleak issue.

Thanks

    -Xie

On Thu, Nov 21, 2019 at 5:31 AM Amar Tumballi <[hidden email]> wrote:
Hi All,

As per the discussion on https://review.gluster.org/23645, recently we changed the status of gNFS (gluster's native NFSv3 support) feature to 'Depricated / Orphan' state. (ref: https://github.com/gluster/glusterfs/blob/master/MAINTAINERS#L185..L189). With this email, I am proposing to change the status again to 'Odd Fixes' (ref: https://github.com/gluster/glusterfs/blob/master/MAINTAINERS#L22

TL;DR;

I understand the current maintainers are not able to focus on maintaining it as the focus of the project, as earlier described, is keeping NFS-Ganesha based integration with glusterfs. But, I am volunteering along with Xie Changlong (currently working at Chinamobile), to keep the feature running as it used to in previous versions. Hence the status of 'Odd Fixes'. 

Before sending the patch to make these changes, I am proposing it here now, as gNFS is not even shipped with latest glusterfs-7.0 releases. I have heard from some users that it was working great for them with earlier releases, as all they wanted was NFS v3 support, and not much of features from gNFS. Also note that, even though the packages are not built, none of the regression tests using gNFS are stopped with latest master, so it is working same from at least last 2 years. 

I request the package maintainers to please add '--with gnfs' (or --enable-gnfs) back to their release script through this email, so those users wanting to use gNFS happily can continue to use it. Also points to users/admins is that, the status is 'Odd Fixes', so don't expect any 'enhancements' on the features provided by gNFS.

Happy to hear feedback, if any.

Regards,
Amar

_______________________________________________
maintainers mailing list
[hidden email]
https://lists.gluster.org/mailman/listinfo/maintainers

_______________________________________________

Community Meeting Calendar:

APAC Schedule -
Every 2nd and 4th Tuesday at 11:30 AM IST
Bridge: https://bluejeans.com/441850968


NA/EMEA Schedule -
Every 1st and 3rd Tuesday at 01:00 PM EDT
Bridge: https://bluejeans.com/441850968

Gluster-devel mailing list
[hidden email]
https://lists.gluster.org/mailman/listinfo/gluster-devel


_______________________________________________

Community Meeting Calendar:

APAC Schedule -
Every 2nd and 4th Tuesday at 11:30 AM IST
Bridge: https://bluejeans.com/441850968


NA/EMEA Schedule -
Every 1st and 3rd Tuesday at 01:00 PM EDT
Bridge: https://bluejeans.com/441850968

Gluster-devel mailing list
[hidden email]
https://lists.gluster.org/mailman/listinfo/gluster-devel

Reply | Threaded
Open this post in threaded view
|

Re: [Gluster-Maintainers] Proposal to change gNFSstatus

Aravinda Vishwanathapura Krishna Murthy


On Fri, Nov 22, 2019 at 8:33 AM Xie Changlong <[hidden email]> wrote:


在 2019/11/22 5:14, Kaleb Keithley 写道:
I personally wouldn't call three years ago — when we started to deprecate it, in glusterfs-3.9 — a recent change.

As a community the decision was made to move to NFS-Ganesha as the preferred NFS solution, but it was agreed to keep the old code in the tree for those who wanted it. There have been plans to drop it from the community packages for most of those three years, but we didn't follow through across the board until fairly recently. Perhaps the most telling piece of data is that it's been gone from the packages in the CentOS Storage SIG in glusterfs-4.0, -4.1, -5, -6, and -7 with no complaints ever, that I can recall.

Ganesha is a preferable solution because it supports NFSv4, NFSv4.1, NFSv4.2, and pNFS, in addition to legacy NFSv3. More importantly, it is actively developed, maintained, and supported, both in the community and commercially. There are several vendors selling it, or support for it; and there are community packages for it for all the same distributions that Gluster packages are available for.

Out in the world, the default these days is NFSv4. Specifically v4.2 or v4.1 depending on how recent your linux kernel is. In the linux kernel, client mounts start negotiating for v4.2 and work down to v4.1, v4.0, and only as a last resort v3. NFSv3 client support in the linux kernel largely exists at this point only because of the large number of legacy servers still running that can't do anything higher than v3. The linux NFS developers would drop the v3 support in a heartbeat if they could.

IMO, providing it, and calling it maintained, only encourages people to keep using a dead end solution. Anyone in favor of bringing back NFSv2, SSHv1, or X10R4? No? I didn't think so.

The recent issue[1] where someone built gnfs in glusterfs-7.0 on CentOS7 strongly suggests to me that gnfs is not actually working well. Three years of no maintenance seems to have taken its toll.

Other people are more than welcome to build their own packages from the src.rpms and/or tarballs that are available from gluster — and support them. It's still in the source and there are no plans to remove it. (Unlike most of the other deprecated features which were recently removed in glusterfs-7.)





It seems https://bugzilla.redhat.com/show_bug.cgi?id=1727248 has resolved this issue.

Here i'll talk about something from commerical company view.  For security reasons most government procurement projects only allow universal storage protocol(nfs, cifs etc) what means fuse will be excluded.  Consindering performance requirements, the only option is nfs. 

Nfsv4 is stateful protocol, but i see on performance improvement. Trust me, nfs-ganesha(v3, v4) shows  ~30% performance degradation versus gnfs  for either small or big files r/w in practice.  Further, many customers prefer nfs client than cifs in windows, because the poor cifs performance, AFAIK nfs-ganesha is not going well with windows nfs client.

Gnfs is stable enough, we have about ~1000 servers, 4~24 servers for a gluster cluster, about ~2000 nfs clients, all works fine till the last two years expect some memleak issue.


This is awesome. +1 for bringing gnfs back.

Thanks

    -Xie

On Thu, Nov 21, 2019 at 5:31 AM Amar Tumballi <[hidden email]> wrote:
Hi All,

As per the discussion on https://review.gluster.org/23645, recently we changed the status of gNFS (gluster's native NFSv3 support) feature to 'Depricated / Orphan' state. (ref: https://github.com/gluster/glusterfs/blob/master/MAINTAINERS#L185..L189). With this email, I am proposing to change the status again to 'Odd Fixes' (ref: https://github.com/gluster/glusterfs/blob/master/MAINTAINERS#L22

TL;DR;

I understand the current maintainers are not able to focus on maintaining it as the focus of the project, as earlier described, is keeping NFS-Ganesha based integration with glusterfs. But, I am volunteering along with Xie Changlong (currently working at Chinamobile), to keep the feature running as it used to in previous versions. Hence the status of 'Odd Fixes'. 

Before sending the patch to make these changes, I am proposing it here now, as gNFS is not even shipped with latest glusterfs-7.0 releases. I have heard from some users that it was working great for them with earlier releases, as all they wanted was NFS v3 support, and not much of features from gNFS. Also note that, even though the packages are not built, none of the regression tests using gNFS are stopped with latest master, so it is working same from at least last 2 years. 

I request the package maintainers to please add '--with gnfs' (or --enable-gnfs) back to their release script through this email, so those users wanting to use gNFS happily can continue to use it. Also points to users/admins is that, the status is 'Odd Fixes', so don't expect any 'enhancements' on the features provided by gNFS.

Happy to hear feedback, if any.

Regards,
Amar

_______________________________________________
maintainers mailing list
[hidden email]
https://lists.gluster.org/mailman/listinfo/maintainers

_______________________________________________

Community Meeting Calendar:

APAC Schedule -
Every 2nd and 4th Tuesday at 11:30 AM IST
Bridge: https://bluejeans.com/441850968


NA/EMEA Schedule -
Every 1st and 3rd Tuesday at 01:00 PM EDT
Bridge: https://bluejeans.com/441850968

Gluster-devel mailing list
[hidden email]
https://lists.gluster.org/mailman/listinfo/gluster-devel

_______________________________________________

Community Meeting Calendar:

APAC Schedule -
Every 2nd and 4th Tuesday at 11:30 AM IST
Bridge: https://bluejeans.com/441850968


NA/EMEA Schedule -
Every 1st and 3rd Tuesday at 01:00 PM EDT
Bridge: https://bluejeans.com/441850968

Gluster-devel mailing list
[hidden email]
https://lists.gluster.org/mailman/listinfo/gluster-devel



--
regards
Aravinda VK

_______________________________________________

Community Meeting Calendar:

APAC Schedule -
Every 2nd and 4th Tuesday at 11:30 AM IST
Bridge: https://bluejeans.com/441850968


NA/EMEA Schedule -
Every 1st and 3rd Tuesday at 01:00 PM EDT
Bridge: https://bluejeans.com/441850968

Gluster-devel mailing list
[hidden email]
https://lists.gluster.org/mailman/listinfo/gluster-devel

Reply | Threaded
Open this post in threaded view
|

Re: [Gluster-Maintainers] Proposal to change gNFSstatus

Yaniv Kaul
In reply to this post by Xie Changlong-2


On Fri, 22 Nov 2019, 5:03 Xie Changlong <[hidden email]> wrote:


在 2019/11/22 5:14, Kaleb Keithley 写道:
I personally wouldn't call three years ago — when we started to deprecate it, in glusterfs-3.9 — a recent change.

As a community the decision was made to move to NFS-Ganesha as the preferred NFS solution, but it was agreed to keep the old code in the tree for those who wanted it. There have been plans to drop it from the community packages for most of those three years, but we didn't follow through across the board until fairly recently. Perhaps the most telling piece of data is that it's been gone from the packages in the CentOS Storage SIG in glusterfs-4.0, -4.1, -5, -6, and -7 with no complaints ever, that I can recall.

Ganesha is a preferable solution because it supports NFSv4, NFSv4.1, NFSv4.2, and pNFS, in addition to legacy NFSv3. More importantly, it is actively developed, maintained, and supported, both in the community and commercially. There are several vendors selling it, or support for it; and there are community packages for it for all the same distributions that Gluster packages are available for.

Out in the world, the default these days is NFSv4. Specifically v4.2 or v4.1 depending on how recent your linux kernel is. In the linux kernel, client mounts start negotiating for v4.2 and work down to v4.1, v4.0, and only as a last resort v3. NFSv3 client support in the linux kernel largely exists at this point only because of the large number of legacy servers still running that can't do anything higher than v3. The linux NFS developers would drop the v3 support in a heartbeat if they could.

IMO, providing it, and calling it maintained, only encourages people to keep using a dead end solution. Anyone in favor of bringing back NFSv2, SSHv1, or X10R4? No? I didn't think so.

The recent issue[1] where someone built gnfs in glusterfs-7.0 on CentOS7 strongly suggests to me that gnfs is not actually working well. Three years of no maintenance seems to have taken its toll.

Other people are more than welcome to build their own packages from the src.rpms and/or tarballs that are available from gluster — and support them. It's still in the source and there are no plans to remove it. (Unlike most of the other deprecated features which were recently removed in glusterfs-7.)





It seems https://bugzilla.redhat.com/show_bug.cgi?id=1727248 has resolved this issue.

Here i'll talk about something from commerical company view.  For security reasons most government procurement projects only allow universal storage protocol(nfs, cifs etc) what means fuse will be excluded.  Consindering performance requirements, the only option is nfs. 


I don't see how NFSv3 is more secure than newer NFS versions. 

Nfsv4 is stateful protocol, but i see on performance improvement. Trust me, nfs-ganesha(v3, v4) shows  ~30% performance degradation versus gnfs  for either small or big files r/w in practice.  Further, many customers prefer nfs client than cifs in windows, because the poor cifs performance, AFAIK nfs-ganesha is not going well with windows nfs client.


Interesting - we've seen far better performance with Ganesha v4.1 vs. gnfs. 
Would be great if you could share the details. 
Same for NFS Ganesha and Windows support.

It's difficult to counterpart without referring to specific issues. It's eveb to harder to fix them ;-) 


Gnfs is stable enough, we have about ~1000 servers, 4~24 servers for a gluster cluster, about ~2000 nfs clients, all works fine till the last two years expect some memleak issue.


Nice! Would be great for the Gluster community to learn more about the use case! 
Y. 

Thanks

    -Xie

On Thu, Nov 21, 2019 at 5:31 AM Amar Tumballi <[hidden email]> wrote:
Hi All,

As per the discussion on https://review.gluster.org/23645, recently we changed the status of gNFS (gluster's native NFSv3 support) feature to 'Depricated / Orphan' state. (ref: https://github.com/gluster/glusterfs/blob/master/MAINTAINERS#L185..L189). With this email, I am proposing to change the status again to 'Odd Fixes' (ref: https://github.com/gluster/glusterfs/blob/master/MAINTAINERS#L22

TL;DR;

I understand the current maintainers are not able to focus on maintaining it as the focus of the project, as earlier described, is keeping NFS-Ganesha based integration with glusterfs. But, I am volunteering along with Xie Changlong (currently working at Chinamobile), to keep the feature running as it used to in previous versions. Hence the status of 'Odd Fixes'. 

Before sending the patch to make these changes, I am proposing it here now, as gNFS is not even shipped with latest glusterfs-7.0 releases. I have heard from some users that it was working great for them with earlier releases, as all they wanted was NFS v3 support, and not much of features from gNFS. Also note that, even though the packages are not built, none of the regression tests using gNFS are stopped with latest master, so it is working same from at least last 2 years. 

I request the package maintainers to please add '--with gnfs' (or --enable-gnfs) back to their release script through this email, so those users wanting to use gNFS happily can continue to use it. Also points to users/admins is that, the status is 'Odd Fixes', so don't expect any 'enhancements' on the features provided by gNFS.

Happy to hear feedback, if any.

Regards,
Amar

_______________________________________________
maintainers mailing list
[hidden email]
https://lists.gluster.org/mailman/listinfo/maintainers

_______________________________________________

Community Meeting Calendar:

APAC Schedule -
Every 2nd and 4th Tuesday at 11:30 AM IST
Bridge: https://bluejeans.com/441850968


NA/EMEA Schedule -
Every 1st and 3rd Tuesday at 01:00 PM EDT
Bridge: https://bluejeans.com/441850968

Gluster-devel mailing list
[hidden email]
https://lists.gluster.org/mailman/listinfo/gluster-devel

_______________________________________________

Community Meeting Calendar:

APAC Schedule -
Every 2nd and 4th Tuesday at 11:30 AM IST
Bridge: https://bluejeans.com/441850968


NA/EMEA Schedule -
Every 1st and 3rd Tuesday at 01:00 PM EDT
Bridge: https://bluejeans.com/441850968

Gluster-devel mailing list
[hidden email]
https://lists.gluster.org/mailman/listinfo/gluster-devel


_______________________________________________

Community Meeting Calendar:

APAC Schedule -
Every 2nd and 4th Tuesday at 11:30 AM IST
Bridge: https://bluejeans.com/441850968


NA/EMEA Schedule -
Every 1st and 3rd Tuesday at 01:00 PM EDT
Bridge: https://bluejeans.com/441850968

Gluster-devel mailing list
[hidden email]
https://lists.gluster.org/mailman/listinfo/gluster-devel

Reply | Threaded
Open this post in threaded view
|

Re: [Gluster-Maintainers] Proposal to change gNFSstatus

Xie Changlong-2


在 2019/11/22 13:39, Yaniv Kaul 写道:


On Fri, 22 Nov 2019, 5:03 Xie Changlong <[hidden email]> wrote:


在 2019/11/22 5:14, Kaleb Keithley 写道:
I personally wouldn't call three years ago — when we started to deprecate it, in glusterfs-3.9 — a recent change.

As a community the decision was made to move to NFS-Ganesha as the preferred NFS solution, but it was agreed to keep the old code in the tree for those who wanted it. There have been plans to drop it from the community packages for most of those three years, but we didn't follow through across the board until fairly recently. Perhaps the most telling piece of data is that it's been gone from the packages in the CentOS Storage SIG in glusterfs-4.0, -4.1, -5, -6, and -7 with no complaints ever, that I can recall.

Ganesha is a preferable solution because it supports NFSv4, NFSv4.1, NFSv4.2, and pNFS, in addition to legacy NFSv3. More importantly, it is actively developed, maintained, and supported, both in the community and commercially. There are several vendors selling it, or support for it; and there are community packages for it for all the same distributions that Gluster packages are available for.

Out in the world, the default these days is NFSv4. Specifically v4.2 or v4.1 depending on how recent your linux kernel is. In the linux kernel, client mounts start negotiating for v4.2 and work down to v4.1, v4.0, and only as a last resort v3. NFSv3 client support in the linux kernel largely exists at this point only because of the large number of legacy servers still running that can't do anything higher than v3. The linux NFS developers would drop the v3 support in a heartbeat if they could.

IMO, providing it, and calling it maintained, only encourages people to keep using a dead end solution. Anyone in favor of bringing back NFSv2, SSHv1, or X10R4? No? I didn't think so.

The recent issue[1] where someone built gnfs in glusterfs-7.0 on CentOS7 strongly suggests to me that gnfs is not actually working well. Three years of no maintenance seems to have taken its toll.

Other people are more than welcome to build their own packages from the src.rpms and/or tarballs that are available from gluster — and support them. It's still in the source and there are no plans to remove it. (Unlike most of the other deprecated features which were recently removed in glusterfs-7.)





It seems https://bugzilla.redhat.com/show_bug.cgi?id=1727248 has resolved this issue.

Here i'll talk about something from commerical company view.  For security reasons most government procurement projects only allow universal storage protocol(nfs, cifs etc) what means fuse will be excluded.  Consindering performance requirements, the only option is nfs. 


I don't see how NFSv3 is more secure than newer NFS versions. 


Here i mean fuse versus nfs.  Don't expect to install fuse client on customer's computer.


Nfsv4 is stateful protocol, but i see on performance improvement. Trust me, nfs-ganesha(v3, v4) shows  ~30% performance degradation versus gnfs  for either small or big files r/w in practice.  Further, many customers prefer nfs client than cifs in windows, because the poor cifs performance, AFAIK nfs-ganesha is not going well with windows nfs client.


Interesting - we've seen far better performance with Ganesha v4.1 vs. gnfs.
Would be great if you could share the details.

vdbench 6/4  random read/write


Same for NFS Ganesha and Windows support.

ganesha 2.5.5,  glusterfs 3.12.2, windows server 2003. Use windows nfsv3 mount nfs-ganesha and test read/write with vdbench50406. Following is crash bt

Btw, the environment has been redeployed, so i can't share more.


It's difficult to counterpart without referring to specific issues. It's eveb to harder to fix them ;-)

Gnfs is stable enough, we have about ~1000 servers, 4~24 servers for a gluster cluster, about ~2000 nfs clients, all works fine till the last two years expect some memleak issue.


Nice! Would be great for the Gluster community to learn more about the use case!


It's my pleasure.


Y. 

Thanks

    -Xie

On Thu, Nov 21, 2019 at 5:31 AM Amar Tumballi <[hidden email]> wrote:
Hi All,

As per the discussion on https://review.gluster.org/23645, recently we changed the status of gNFS (gluster's native NFSv3 support) feature to 'Depricated / Orphan' state. (ref: https://github.com/gluster/glusterfs/blob/master/MAINTAINERS#L185..L189). With this email, I am proposing to change the status again to 'Odd Fixes' (ref: https://github.com/gluster/glusterfs/blob/master/MAINTAINERS#L22

TL;DR;

I understand the current maintainers are not able to focus on maintaining it as the focus of the project, as earlier described, is keeping NFS-Ganesha based integration with glusterfs. But, I am volunteering along with Xie Changlong (currently working at Chinamobile), to keep the feature running as it used to in previous versions. Hence the status of 'Odd Fixes'. 

Before sending the patch to make these changes, I am proposing it here now, as gNFS is not even shipped with latest glusterfs-7.0 releases. I have heard from some users that it was working great for them with earlier releases, as all they wanted was NFS v3 support, and not much of features from gNFS. Also note that, even though the packages are not built, none of the regression tests using gNFS are stopped with latest master, so it is working same from at least last 2 years. 

I request the package maintainers to please add '--with gnfs' (or --enable-gnfs) back to their release script through this email, so those users wanting to use gNFS happily can continue to use it. Also points to users/admins is that, the status is 'Odd Fixes', so don't expect any 'enhancements' on the features provided by gNFS.

Happy to hear feedback, if any.

Regards,
Amar

_______________________________________________
maintainers mailing list
[hidden email]
https://lists.gluster.org/mailman/listinfo/maintainers

_______________________________________________

Community Meeting Calendar:

APAC Schedule -
Every 2nd and 4th Tuesday at 11:30 AM IST
Bridge: https://bluejeans.com/441850968


NA/EMEA Schedule -
Every 1st and 3rd Tuesday at 01:00 PM EDT
Bridge: https://bluejeans.com/441850968

Gluster-devel mailing list
[hidden email]
https://lists.gluster.org/mailman/listinfo/gluster-devel

_______________________________________________

Community Meeting Calendar:

APAC Schedule -
Every 2nd and 4th Tuesday at 11:30 AM IST
Bridge: https://bluejeans.com/441850968


NA/EMEA Schedule -
Every 1st and 3rd Tuesday at 01:00 PM EDT
Bridge: https://bluejeans.com/441850968

Gluster-devel mailing list
[hidden email]
https://lists.gluster.org/mailman/listinfo/gluster-devel


_______________________________________________

Community Meeting Calendar:

APAC Schedule -
Every 2nd and 4th Tuesday at 11:30 AM IST
Bridge: https://bluejeans.com/441850968


NA/EMEA Schedule -
Every 1st and 3rd Tuesday at 01:00 PM EDT
Bridge: https://bluejeans.com/441850968

Gluster-devel mailing list
[hidden email]
https://lists.gluster.org/mailman/listinfo/gluster-devel

Reply | Threaded
Open this post in threaded view
|

gNFS vs NFS Ganesha performance (was: Re: [Gluster-Maintainers] Proposal to change gNFSstatus

Yaniv Kaul
Spinning off the conversation, comments within

On Fri, Nov 22, 2019 at 8:28 AM Xie Changlong <[hidden email]> wrote:

<snip>

Interesting - we've seen far better performance with Ganesha v4.1 vs. gnfs.
Would be great if you could share the details.

vdbench 6/4  random read/write



You are comparing old versions of Gluster (assuming it's downstream, Red Hat's version - it was released January 2018 - almost 2 years ago) and certainly an old version of NFS Ganesha.
We believe newer releases are substantially better.
Just sharing the internal number of improvement we are seeing:
glusterfs-6.0-17 vs glusterfs-3.12.2-47 (rhel 7.7)
33.86%
1075.49%
232.88%
761.36%
91.15%
138.01%
-30.07%  <-- there's a bug about it.
28.59%
12.36%
-1.44%

(The RHEL release isn't up-to-date either)

Y.

Same for NFS Ganesha and Windows support.

ganesha 2.5.5,  glusterfs 3.12.2, windows server 2003. Use windows nfsv3 mount nfs-ganesha and test read/write with vdbench50406. Following is crash bt

Btw, the environment has been redeployed, so i can't share more.


It's difficult to counterpart without referring to specific issues. It's eveb to harder to fix them ;-)

Gnfs is stable enough, we have about ~1000 servers, 4~24 servers for a gluster cluster, about ~2000 nfs clients, all works fine till the last two years expect some memleak issue.


Nice! Would be great for the Gluster community to learn more about the use case!


It's my pleasure.


Y. 

Thanks

    -Xie

On Thu, Nov 21, 2019 at 5:31 AM Amar Tumballi <[hidden email]> wrote:
Hi All,

As per the discussion on https://review.gluster.org/23645, recently we changed the status of gNFS (gluster's native NFSv3 support) feature to 'Depricated / Orphan' state. (ref: https://github.com/gluster/glusterfs/blob/master/MAINTAINERS#L185..L189). With this email, I am proposing to change the status again to 'Odd Fixes' (ref: https://github.com/gluster/glusterfs/blob/master/MAINTAINERS#L22

TL;DR;

I understand the current maintainers are not able to focus on maintaining it as the focus of the project, as earlier described, is keeping NFS-Ganesha based integration with glusterfs. But, I am volunteering along with Xie Changlong (currently working at Chinamobile), to keep the feature running as it used to in previous versions. Hence the status of 'Odd Fixes'. 

Before sending the patch to make these changes, I am proposing it here now, as gNFS is not even shipped with latest glusterfs-7.0 releases. I have heard from some users that it was working great for them with earlier releases, as all they wanted was NFS v3 support, and not much of features from gNFS. Also note that, even though the packages are not built, none of the regression tests using gNFS are stopped with latest master, so it is working same from at least last 2 years. 

I request the package maintainers to please add '--with gnfs' (or --enable-gnfs) back to their release script through this email, so those users wanting to use gNFS happily can continue to use it. Also points to users/admins is that, the status is 'Odd Fixes', so don't expect any 'enhancements' on the features provided by gNFS.

Happy to hear feedback, if any.

Regards,
Amar

_______________________________________________
maintainers mailing list
[hidden email]
https://lists.gluster.org/mailman/listinfo/maintainers

_______________________________________________

Community Meeting Calendar:

APAC Schedule -
Every 2nd and 4th Tuesday at 11:30 AM IST
Bridge: https://bluejeans.com/441850968


NA/EMEA Schedule -
Every 1st and 3rd Tuesday at 01:00 PM EDT
Bridge: https://bluejeans.com/441850968

Gluster-devel mailing list
[hidden email]
https://lists.gluster.org/mailman/listinfo/gluster-devel

_______________________________________________

Community Meeting Calendar:

APAC Schedule -
Every 2nd and 4th Tuesday at 11:30 AM IST
Bridge: https://bluejeans.com/441850968


NA/EMEA Schedule -
Every 1st and 3rd Tuesday at 01:00 PM EDT
Bridge: https://bluejeans.com/441850968

Gluster-devel mailing list
[hidden email]
https://lists.gluster.org/mailman/listinfo/gluster-devel


_______________________________________________

Community Meeting Calendar:

APAC Schedule -
Every 2nd and 4th Tuesday at 11:30 AM IST
Bridge: https://bluejeans.com/441850968


NA/EMEA Schedule -
Every 1st and 3rd Tuesday at 01:00 PM EDT
Bridge: https://bluejeans.com/441850968

Gluster-devel mailing list
[hidden email]
https://lists.gluster.org/mailman/listinfo/gluster-devel

Reply | Threaded
Open this post in threaded view
|

Re: [Gluster-Maintainers] Proposal to change gNFS status

Niels de Vos-5
In reply to this post by Amar Tumballi-2
On Thu, Nov 21, 2019 at 04:01:23PM +0530, Amar Tumballi wrote:
> Hi All,
>
> As per the discussion on https://review.gluster.org/23645, recently we
> changed the status of gNFS (gluster's native NFSv3 support) feature to
> 'Depricated / Orphan' state. (ref:
> https://github.com/gluster/glusterfs/blob/master/MAINTAINERS#L185..L189).
> With this email, I am proposing to change the status again to 'Odd Fixes'
> (ref: https://github.com/gluster/glusterfs/blob/master/MAINTAINERS#L22)

I'd recommend against re-surrecting gNFS. The server is not very
extensible and adding new features is pretty tricky without breaking
other (mostly undocumented) use-cases. Eventhough NFSv3 is stateless,
the actual usage of NFSv3, mounting and locking is definitely not. The
server keeps track of which clients have an export mounted, and which
clients received grants for locks. These things are currently not very
reliable in combination with high-availability. And there is also the by
default disabled duplicate-reply-cache (DRC) that has always been very
buggy (and neither cluster-aware).

If we enable gNFS by default again, we're sending out an incorrect
message to our users. gNFS works fine for certain workloads and
environments, but it should not be advertised as 'clustered NFS'.

Instead of going the gNFS route, I suggest to make it easier to deploy
NFS-Ganesha as that is a more featured, well maintained and can be
configured for much more reliable high-availability than gNFS.

If someone really wants to maintain gNFS, I won't object much, but they
should know that previous maintainers have had many difficulties just
keeping it working well while other components evolved. Addressing some
of the bugs/limitations will be extremely difficult and may require
large rewrites of parts of gNFS.

Until now, I have not read convincing arguments in this thread that gNFS
is stable enough to be consumed by anyone in the community. Users should
be aware of its limitations and be careful what workloads to run on it.

HTH,
Niels

_______________________________________________

Community Meeting Calendar:

APAC Schedule -
Every 2nd and 4th Tuesday at 11:30 AM IST
Bridge: https://bluejeans.com/441850968


NA/EMEA Schedule -
Every 1st and 3rd Tuesday at 01:00 PM EDT
Bridge: https://bluejeans.com/441850968

Gluster-devel mailing list
[hidden email]
https://lists.gluster.org/mailman/listinfo/gluster-devel

Reply | Threaded
Open this post in threaded view
|

Re: [Gluster-Maintainers] gNFS vs NFS Ganesha performance (was: Re: Proposal to change gNFSstatus

Sankarshan Mukhopadhyay
In reply to this post by Yaniv Kaul
It would be good to compare and contrast performance numbers based on additional detail about the test workbench as well as configuration for the workloads. Else, it is just a lot of graphs along with lists of end-results and drawing inferences are difficult.

On Fri, Nov 22, 2019 at 4:21 PM Yaniv Kaul <[hidden email]> wrote:
Spinning off the conversation, comments within

On Fri, Nov 22, 2019 at 8:28 AM Xie Changlong <[hidden email]> wrote:

<snip>

Interesting - we've seen far better performance with Ganesha v4.1 vs. gnfs.
Would be great if you could share the details.

vdbench 6/4  random read/write



You are comparing old versions of Gluster (assuming it's downstream, Red Hat's version - it was released January 2018 - almost 2 years ago) and certainly an old version of NFS Ganesha.
We believe newer releases are substantially better.
Just sharing the internal number of improvement we are seeing:
glusterfs-6.0-17 vs glusterfs-3.12.2-47 (rhel 7.7)
33.86%
1075.49%
232.88%
761.36%
91.15%
138.01%
-30.07%  <-- there's a bug about it.
28.59%
12.36%
-1.44%

(The RHEL release isn't up-to-date either)

Y.

Same for NFS Ganesha and Windows support.

ganesha 2.5.5,  glusterfs 3.12.2, windows server 2003. Use windows nfsv3 mount nfs-ganesha and test read/write with vdbench50406. Following is crash bt

Btw, the environment has been redeployed, so i can't share more.


It's difficult to counterpart without referring to specific issues. It's eveb to harder to fix them ;-)

Gnfs is stable enough, we have about ~1000 servers, 4~24 servers for a gluster cluster, about ~2000 nfs clients, all works fine till the last two years expect some memleak issue.


Nice! Would be great for the Gluster community to learn more about the use case!


It's my pleasure.


Y. 

Thanks

    -Xie

On Thu, Nov 21, 2019 at 5:31 AM Amar Tumballi <[hidden email]> wrote:
Hi All,

As per the discussion on https://review.gluster.org/23645, recently we changed the status of gNFS (gluster's native NFSv3 support) feature to 'Depricated / Orphan' state. (ref: https://github.com/gluster/glusterfs/blob/master/MAINTAINERS#L185..L189). With this email, I am proposing to change the status again to 'Odd Fixes' (ref: https://github.com/gluster/glusterfs/blob/master/MAINTAINERS#L22

TL;DR;

I understand the current maintainers are not able to focus on maintaining it as the focus of the project, as earlier described, is keeping NFS-Ganesha based integration with glusterfs. But, I am volunteering along with Xie Changlong (currently working at Chinamobile), to keep the feature running as it used to in previous versions. Hence the status of 'Odd Fixes'. 

Before sending the patch to make these changes, I am proposing it here now, as gNFS is not even shipped with latest glusterfs-7.0 releases. I have heard from some users that it was working great for them with earlier releases, as all they wanted was NFS v3 support, and not much of features from gNFS. Also note that, even though the packages are not built, none of the regression tests using gNFS are stopped with latest master, so it is working same from at least last 2 years. 

I request the package maintainers to please add '--with gnfs' (or --enable-gnfs) back to their release script through this email, so those users wanting to use gNFS happily can continue to use it. Also points to users/admins is that, the status is 'Odd Fixes', so don't expect any 'enhancements' on the features provided by gNFS.

Happy to hear feedback, if any.

Regards,
Amar



_______________________________________________

Community Meeting Calendar:

APAC Schedule -
Every 2nd and 4th Tuesday at 11:30 AM IST
Bridge: https://bluejeans.com/441850968


NA/EMEA Schedule -
Every 1st and 3rd Tuesday at 01:00 PM EDT
Bridge: https://bluejeans.com/441850968

Gluster-devel mailing list
[hidden email]
https://lists.gluster.org/mailman/listinfo/gluster-devel

Reply | Threaded
Open this post in threaded view
|

Re: gNFS vs NFS Ganesha performance

Xie Changlong-2
In reply to this post by Yaniv Kaul


在 2019/11/22 17:45, Yaniv Kaul 写道:
Spinning off the conversation, comments within

On Fri, Nov 22, 2019 at 8:28 AM Xie Changlong <[hidden email]> wrote:

<snip>

Interesting - we've seen far better performance with Ganesha v4.1 vs. gnfs.
Would be great if you could share the details.

vdbench 6/4  random read/write



You are comparing old versions of Gluster (assuming it's downstream, Red Hat's version - it was released January 2018 - almost 2 years ago) and certainly an old version of NFS Ganesha.
We believe newer releases are substantially better.
Just sharing the internal number of improvement we are seeing:
glusterfs-6.0-17 vs glusterfs-3.12.2-47 (rhel 7.7)
33.86%
1075.49%
232.88%
761.36%
91.15%
138.01%
-30.07%  <-- there's a bug about it.
28.59%
12.36%
-1.44%

Amazing! But it's not fair,  the results is old glusterfs vs new glusterfs but not nfs-ganesha vs gnfs.

Anyway i'll test the last nfs-ganesha versus gnfs, and get back later.

Thanks

    -Xie

(The RHEL release isn't up-to-date either)

Y.

Same for NFS Ganesha and Windows support.

ganesha 2.5.5,  glusterfs 3.12.2, windows server 2003. Use windows nfsv3 mount nfs-ganesha and test read/write with vdbench50406. Following is crash bt

Btw, the environment has been redeployed, so i can't share more.


It's difficult to counterpart without referring to specific issues. It's eveb to harder to fix them ;-)

Gnfs is stable enough, we have about ~1000 servers, 4~24 servers for a gluster cluster, about ~2000 nfs clients, all works fine till the last two years expect some memleak issue.


Nice! Would be great for the Gluster community to learn more about the use case!


It's my pleasure.


Y. 

Thanks

    -Xie

On Thu, Nov 21, 2019 at 5:31 AM Amar Tumballi <[hidden email]> wrote:
Hi All,

As per the discussion on https://review.gluster.org/23645, recently we changed the status of gNFS (gluster's native NFSv3 support) feature to 'Depricated / Orphan' state. (ref: https://github.com/gluster/glusterfs/blob/master/MAINTAINERS#L185..L189). With this email, I am proposing to change the status again to 'Odd Fixes' (ref: https://github.com/gluster/glusterfs/blob/master/MAINTAINERS#L22

TL;DR;

I understand the current maintainers are not able to focus on maintaining it as the focus of the project, as earlier described, is keeping NFS-Ganesha based integration with glusterfs. But, I am volunteering along with Xie Changlong (currently working at Chinamobile), to keep the feature running as it used to in previous versions. Hence the status of 'Odd Fixes'. 

Before sending the patch to make these changes, I am proposing it here now, as gNFS is not even shipped with latest glusterfs-7.0 releases. I have heard from some users that it was working great for them with earlier releases, as all they wanted was NFS v3 support, and not much of features from gNFS. Also note that, even though the packages are not built, none of the regression tests using gNFS are stopped with latest master, so it is working same from at least last 2 years. 

I request the package maintainers to please add '--with gnfs' (or --enable-gnfs) back to their release script through this email, so those users wanting to use gNFS happily can continue to use it. Also points to users/admins is that, the status is 'Odd Fixes', so don't expect any 'enhancements' on the features provided by gNFS.

Happy to hear feedback, if any.

Regards,
Amar

_______________________________________________
maintainers mailing list
[hidden email]
https://lists.gluster.org/mailman/listinfo/maintainers

_______________________________________________

Community Meeting Calendar:

APAC Schedule -
Every 2nd and 4th Tuesday at 11:30 AM IST
Bridge: https://bluejeans.com/441850968


NA/EMEA Schedule -
Every 1st and 3rd Tuesday at 01:00 PM EDT
Bridge: https://bluejeans.com/441850968

Gluster-devel mailing list
[hidden email]
https://lists.gluster.org/mailman/listinfo/gluster-devel

_______________________________________________

Community Meeting Calendar:

APAC Schedule -
Every 2nd and 4th Tuesday at 11:30 AM IST
Bridge: https://bluejeans.com/441850968


NA/EMEA Schedule -
Every 1st and 3rd Tuesday at 01:00 PM EDT
Bridge: https://bluejeans.com/441850968

Gluster-devel mailing list
[hidden email]
https://lists.gluster.org/mailman/listinfo/gluster-devel


_______________________________________________

Community Meeting Calendar:

APAC Schedule -
Every 2nd and 4th Tuesday at 11:30 AM IST
Bridge: https://bluejeans.com/441850968


NA/EMEA Schedule -
Every 1st and 3rd Tuesday at 01:00 PM EDT
Bridge: https://bluejeans.com/441850968

Gluster-devel mailing list
[hidden email]
https://lists.gluster.org/mailman/listinfo/gluster-devel

Reply | Threaded
Open this post in threaded view
|

Re: [Gluster-users] [Gluster-Maintainers] Proposal to change gNFSstatus

Xie Changlong-2
In reply to this post by Kaleb Keithley

~~Get in a word~~

Hi,  Kaleb Keithley. Could you give some comments on below url ? We encounter it in practice.

https://review.gerrithub.io/c/ffilz/nfs-ganesha/+/465149

Thanks

在 2019/11/22 5:14, Kaleb Keithley 写道:
I personally wouldn't call three years ago — when we started to deprecate it, in glusterfs-3.9 — a recent change.

As a community the decision was made to move to NFS-Ganesha as the preferred NFS solution, but it was agreed to keep the old code in the tree for those who wanted it. There have been plans to drop it from the community packages for most of those three years, but we didn't follow through across the board until fairly recently. Perhaps the most telling piece of data is that it's been gone from the packages in the CentOS Storage SIG in glusterfs-4.0, -4.1, -5, -6, and -7 with no complaints ever, that I can recall.

Ganesha is a preferable solution because it supports NFSv4, NFSv4.1, NFSv4.2, and pNFS, in addition to legacy NFSv3. More importantly, it is actively developed, maintained, and supported, both in the community and commercially. There are several vendors selling it, or support for it; and there are community packages for it for all the same distributions that Gluster packages are available for.

Out in the world, the default these days is NFSv4. Specifically v4.2 or v4.1 depending on how recent your linux kernel is. In the linux kernel, client mounts start negotiating for v4.2 and work down to v4.1, v4.0, and only as a last resort v3. NFSv3 client support in the linux kernel largely exists at this point only because of the large number of legacy servers still running that can't do anything higher than v3. The linux NFS developers would drop the v3 support in a heartbeat if they could.

IMO, providing it, and calling it maintained, only encourages people to keep using a dead end solution. Anyone in favor of bringing back NFSv2, SSHv1, or X10R4? No? I didn't think so.

The recent issue[1] where someone built gnfs in glusterfs-7.0 on CentOS7 strongly suggests to me that gnfs is not actually working well. Three years of no maintenance seems to have taken its toll.

Other people are more than welcome to build their own packages from the src.rpms and/or tarballs that are available from gluster — and support them. It's still in the source and there are no plans to remove it. (Unlike most of the other deprecated features which were recently removed in glusterfs-7.)




On Thu, Nov 21, 2019 at 5:31 AM Amar Tumballi <[hidden email]> wrote:
Hi All,

As per the discussion on https://review.gluster.org/23645, recently we changed the status of gNFS (gluster's native NFSv3 support) feature to 'Depricated / Orphan' state. (ref: https://github.com/gluster/glusterfs/blob/master/MAINTAINERS#L185..L189). With this email, I am proposing to change the status again to 'Odd Fixes' (ref: https://github.com/gluster/glusterfs/blob/master/MAINTAINERS#L22

TL;DR;

I understand the current maintainers are not able to focus on maintaining it as the focus of the project, as earlier described, is keeping NFS-Ganesha based integration with glusterfs. But, I am volunteering along with Xie Changlong (currently working at Chinamobile), to keep the feature running as it used to in previous versions. Hence the status of 'Odd Fixes'. 

Before sending the patch to make these changes, I am proposing it here now, as gNFS is not even shipped with latest glusterfs-7.0 releases. I have heard from some users that it was working great for them with earlier releases, as all they wanted was NFS v3 support, and not much of features from gNFS. Also note that, even though the packages are not built, none of the regression tests using gNFS are stopped with latest master, so it is working same from at least last 2 years. 

I request the package maintainers to please add '--with gnfs' (or --enable-gnfs) back to their release script through this email, so those users wanting to use gNFS happily can continue to use it. Also points to users/admins is that, the status is 'Odd Fixes', so don't expect any 'enhancements' on the features provided by gNFS.

Happy to hear feedback, if any.

Regards,
Amar

_______________________________________________
maintainers mailing list
[hidden email]
https://lists.gluster.org/mailman/listinfo/maintainers

________

Community Meeting Calendar:

APAC Schedule -
Every 2nd and 4th Tuesday at 11:30 AM IST
Bridge: https://bluejeans.com/441850968

NA/EMEA Schedule -
Every 1st and 3rd Tuesday at 01:00 PM EDT
Bridge: https://bluejeans.com/441850968

Gluster-users mailing list
[hidden email]
https://lists.gluster.org/mailman/listinfo/gluster-users

_______________________________________________

Community Meeting Calendar:

APAC Schedule -
Every 2nd and 4th Tuesday at 11:30 AM IST
Bridge: https://bluejeans.com/441850968


NA/EMEA Schedule -
Every 1st and 3rd Tuesday at 01:00 PM EDT
Bridge: https://bluejeans.com/441850968

Gluster-devel mailing list
[hidden email]
https://lists.gluster.org/mailman/listinfo/gluster-devel

Reply | Threaded
Open this post in threaded view
|

Re: [Gluster-Maintainers] Proposal to change gNFS status

Amar Tumballi-2
In reply to this post by Niels de Vos-5
Responses inline.

On Fri, Nov 22, 2019 at 6:04 PM Niels de Vos <[hidden email]> wrote:
On Thu, Nov 21, 2019 at 04:01:23PM +0530, Amar Tumballi wrote:
> Hi All,
>
> As per the discussion on https://review.gluster.org/23645, recently we
> changed the status of gNFS (gluster's native NFSv3 support) feature to
> 'Depricated / Orphan' state. (ref:
> https://github.com/gluster/glusterfs/blob/master/MAINTAINERS#L185..L189).
> With this email, I am proposing to change the status again to 'Odd Fixes'
> (ref: https://github.com/gluster/glusterfs/blob/master/MAINTAINERS#L22)

I'd recommend against re-surrecting gNFS. The server is not very
extensible and adding new features is pretty tricky without breaking
other (mostly undocumented) use-cases.

I too am against adding the features/enhancements to gNFS. It doesn't make sense. We are removing features from glusterfs itself, adding features to gNFS after 3 years wouldn't even be feasible.

I guess you missed the intention of my proposal. It was not about 'resurrecting' gNFS to 'Maintained' or 'Supported' status. It was about taking it out of 'Orphan' status, because there are still users who are 'happy' with it. Hence I picked the status as 'Odd Fixes' (as per MAINTAINERS file, there was nothing else which would give meaning of 'This feature is still shipped, but we are not adding any features or not actively maintaining it'. 

 
Eventhough NFSv3 is stateless,
the actual usage of NFSv3, mounting and locking is definitely not. The
server keeps track of which clients have an export mounted, and which
clients received grants for locks. These things are currently not very
reliable in combination with high-availability. And there is also the by
default disabled duplicate-reply-cache (DRC) that has always been very
buggy (and neither cluster-aware).

If we enable gNFS by default again, we're sending out an incorrect
message to our users. gNFS works fine for certain workloads and
environments, but it should not be advertised as 'clustered NFS'.


I didn't talk or was intending to go this route. I am not even talking about making gNFS 'default' enable. That would take away our focus on glusterfs, and different things we can solve with Gluster alone. Not sure why my email was taken as there would be focus on gNFS.
 
Instead of going the gNFS route, I suggest to make it easier to deploy
NFS-Ganesha as that is a more featured, well maintained and can be
configured for much more reliable high-availability than gNFS.


I believe this is critical, and we surely need to work on it. But doesn't come in the way of doing 1-2 bug fixes in gNFS (if any) in a release. 
 
If someone really wants to maintain gNFS, I won't object much, but they
should know that previous maintainers have had many difficulties just
keeping it working well while other components evolved. Addressing some
of the bugs/limitations will be extremely difficult and may require
large rewrites of parts of gNFS.

Yes, that awareness is critical, and it should exist.


Until now, I have not read convincing arguments in this thread that gNFS
is stable enough to be consumed by anyone in the community. Users should
be aware of its limitations and be careful what workloads to run on it.

In this thread, Xie mentioned that he is managing gNFS on 1000+ servers with 2000+ clients (more than 24 gluster cluster overall) for more than 2 years now. If that doesn't sound as 'stability', not sure what sounds as.

I agree that the users should be careful about the proper usecase to use gNFS. I am even open to say we should add a warning or console log in gluster CLI when 'gluster volume set <VOL> nfs.disable false' is performed, saying it is advised to move to NFS-Ganesha based approach, and give a URL link in that message. But the whole point is, when we make a release, we should still ship gNFS as there are some users, very happy with gNFS, and their usecases are properly handled by gNFS in its current form itself. Why make them unhappy, or shift to other projects? 

End of the day, as developers it is our duty to make sure we suggest the best technologies to users, but the intentions should always be to make sure we solve problems. If there are already solved problems, why resurface them in the name of better technology?

So, again, my proposal is, to keep gNFS in the codebase (not as Orphan), and continue to make releases with gNFS binary shipped when we make release, not to make the focus of project to start working on enhancements of gNFS.

Happy to answer if anyone has further queries.

I have sent a patch https://review.gluster.org/23738 for the same, and I see people commenting already on that. I agree that Xie's contributions to Gluster may need to increase (specifically in gNFS component) to be called as MAINTAINER. Happy to introduce him as 'Peer' and change the title later when it is time. Jiffin, thanks for volunteering to have a look on patches when you have time till glusterfs-8.

Regards,
Amar
 


HTH,
Niels


_______________________________________________

Community Meeting Calendar:

APAC Schedule -
Every 2nd and 4th Tuesday at 11:30 AM IST
Bridge: https://bluejeans.com/441850968


NA/EMEA Schedule -
Every 1st and 3rd Tuesday at 01:00 PM EDT
Bridge: https://bluejeans.com/441850968

Gluster-devel mailing list
[hidden email]
https://lists.gluster.org/mailman/listinfo/gluster-devel