GlusterFS API to manipulate open file descriptor - glfs_fcntl?

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

GlusterFS API to manipulate open file descriptor - glfs_fcntl?

Anoop C S-5
Hi all,

This is to check and confirm whether we have an API(or an internal
implementation which can be exposed as API) to perform operations on an
open file descriptor as a wrapper around existing fcntl() system call.
We do have specific APIs for locks(glfs_posix_lock) and file descriptor
duplication(glfs_dup) which are important among those operations listed
as per man fcntl(2).

At present we have a requirement(very recent) from Samba to set file
descriptor flags through its VFS layer which would need a corresponding
mechanism inside GlusterFS. Due to its absence, VFS module for
GlusterFS inside Samba will have to workaround with the hack of
creating fake local file descriptors outside GlusterFS.

Thoughts and suggestions are welcome.

Anoop C S.

_______________________________________________

Community Meeting Calendar:

APAC Schedule -
Every 2nd and 4th Tuesday at 11:30 AM IST
Bridge: https://bluejeans.com/118564314

NA/EMEA Schedule -
Every 1st and 3rd Tuesday at 01:00 PM EDT
Bridge: https://bluejeans.com/118564314

Gluster-devel mailing list
[hidden email]
https://lists.gluster.org/mailman/listinfo/gluster-devel

Reply | Threaded
Open this post in threaded view
|

Re: GlusterFS API to manipulate open file descriptor - glfs_fcntl?

Amar Tumballi-2
It would be great if you can point at the requirements, or new additions you are talking about.


On Tue, Oct 15, 2019 at 12:26 PM Anoop C S <[hidden email]> wrote:
Hi all,

This is to check and confirm whether we have an API(or an internal
implementation which can be exposed as API) to perform operations on an
open file descriptor as a wrapper around existing fcntl() system call.
We do have specific APIs for locks(glfs_posix_lock) and file descriptor
duplication(glfs_dup) which are important among those operations listed
as per man fcntl(2). 

At present we have a requirement(very recent) from Samba to set file
descriptor flags through its VFS layer which would need a corresponding
mechanism inside GlusterFS. Due to its absence, VFS module for
GlusterFS inside Samba will have to workaround with the hack of
creating fake local file descriptors outside GlusterFS.

Thoughts and suggestions are welcome.


If there is a need have a feature, it makes sense to extend fd_t structure and provide it inside. If my memory serve right, we didn't support fcntl() behavior in gluster as there was no fcntl() through fuse when we started.. 

Would be good to understand what is needed, and then start working on design discussions. (if it makes sense).

-Amar
 
Anoop C S.

_______________________________________________

Community Meeting Calendar:

APAC Schedule -
Every 2nd and 4th Tuesday at 11:30 AM IST
Bridge: https://bluejeans.com/118564314

NA/EMEA Schedule -
Every 1st and 3rd Tuesday at 01:00 PM EDT
Bridge: https://bluejeans.com/118564314

Gluster-devel mailing list
[hidden email]
https://lists.gluster.org/mailman/listinfo/gluster-devel


_______________________________________________

Community Meeting Calendar:

APAC Schedule -
Every 2nd and 4th Tuesday at 11:30 AM IST
Bridge: https://bluejeans.com/118564314

NA/EMEA Schedule -
Every 1st and 3rd Tuesday at 01:00 PM EDT
Bridge: https://bluejeans.com/118564314

Gluster-devel mailing list
[hidden email]
https://lists.gluster.org/mailman/listinfo/gluster-devel

Reply | Threaded
Open this post in threaded view
|

Re: GlusterFS API to manipulate open file descriptor - glfs_fcntl?

Anoop C S-5
On Tue, 2019-10-15 at 12:40 +0530, Amar Tumballi wrote:
> It would be great if you can point at the requirements, or new
> additions you are talking about.

Explained below..

> On Tue, Oct 15, 2019 at 12:26 PM Anoop C S <[hidden email]>
> wrote:
> > Hi all,
> >
> > This is to check and confirm whether we have an API(or an internal
> > implementation which can be exposed as API) to perform operations
> > on an
> > open file descriptor as a wrapper around existing fcntl() system
> > call.
> > We do have specific APIs for locks(glfs_posix_lock) and file
> > descriptor
> > duplication(glfs_dup) which are important among those operations
> > listed
> > as per man fcntl(2).
> > At present we have a requirement(very recent) from Samba to set
> > file
> > descriptor flags through its VFS layer which would need a
> > corresponding
> > mechanism inside GlusterFS. Due to its absence, VFS module for
> > GlusterFS inside Samba will have to workaround with the hack of
> > creating fake local file descriptors outside GlusterFS.
> >
> > Thoughts and suggestions are welcome.
> >
>
> If there is a need have a feature, it makes sense to extend fd_t
> structure and provide it inside. If my memory serve right, we didn't
> support fcntl() behavior in gluster as there was no fcntl() through
> fuse when we started..
>
> Would be good to understand what is needed,

During open/create code path Samba tries to set blocking
flag(O_NONBLOCK on Linux) on open file descriptor through its VFS
layer. Comparing to fcntl() call command would be F_SETFL with required
flag(O_NONBLOCK). It is an expectation that backend file system(in this
case GlusterFS) executes this operation and return the result. VFS
module for GlusterFS inside Samba can extract required file descriptor
but would require a way to set the flag on it.

> and then start working on design discussions. (if it makes sense).
>
> -Amar
>  
> > Anoop C S.
> >
> > _______________________________________________
> >
> > Community Meeting Calendar:
> >
> > APAC Schedule -
> > Every 2nd and 4th Tuesday at 11:30 AM IST
> > Bridge: https://bluejeans.com/118564314
> >
> > NA/EMEA Schedule -
> > Every 1st and 3rd Tuesday at 01:00 PM EDT
> > Bridge: https://bluejeans.com/118564314
> >
> > Gluster-devel mailing list
> > [hidden email]
> > https://lists.gluster.org/mailman/listinfo/gluster-devel
> >

_______________________________________________

Community Meeting Calendar:

APAC Schedule -
Every 2nd and 4th Tuesday at 11:30 AM IST
Bridge: https://bluejeans.com/118564314

NA/EMEA Schedule -
Every 1st and 3rd Tuesday at 01:00 PM EDT
Bridge: https://bluejeans.com/118564314

Gluster-devel mailing list
[hidden email]
https://lists.gluster.org/mailman/listinfo/gluster-devel

Reply | Threaded
Open this post in threaded view
|

Re: GlusterFS API to manipulate open file descriptor - glfs_fcntl?

Niels de Vos-5
In reply to this post by Anoop C S-5
On Tue, Oct 15, 2019 at 12:20:54PM +0530, Anoop C S wrote:

> Hi all,
>
> This is to check and confirm whether we have an API(or an internal
> implementation which can be exposed as API) to perform operations on an
> open file descriptor as a wrapper around existing fcntl() system call.
> We do have specific APIs for locks(glfs_posix_lock) and file descriptor
> duplication(glfs_dup) which are important among those operations listed
> as per man fcntl(2).
>
> At present we have a requirement(very recent) from Samba to set file
> descriptor flags through its VFS layer which would need a corresponding
> mechanism inside GlusterFS. Due to its absence, VFS module for
> GlusterFS inside Samba will have to workaround with the hack of
> creating fake local file descriptors outside GlusterFS.
>
> Thoughts and suggestions are welcome.

The fcntl() operations are split when FUSE is used. There in direct
fcntl() call that FUSE passes on, instead it calls lock() and similar
interfaces. I think you refer to F_GETFD and F_SETFD commands for
fcntl(). For all I can see, these do not exist in FUSE, and have not
been added to gfapi either. Not sure if the single supported flag
FD_CLOEXEC can have a benefit on Gluster, as glfs_fini() is expected to
cleanup everything that gfapi allocates.

Can you explain your use-case a little more?

Also adding [hidden email] so that other projects interested
in gfapi can follow and comment on the discussion.

Niels
_______________________________________________

Community Meeting Calendar:

APAC Schedule -
Every 2nd and 4th Tuesday at 11:30 AM IST
Bridge: https://bluejeans.com/118564314

NA/EMEA Schedule -
Every 1st and 3rd Tuesday at 01:00 PM EDT
Bridge: https://bluejeans.com/118564314

Gluster-devel mailing list
[hidden email]
https://lists.gluster.org/mailman/listinfo/gluster-devel

Reply | Threaded
Open this post in threaded view
|

Re: GlusterFS API to manipulate open file descriptor - glfs_fcntl?

Anoop C S-5
On Tue, 2019-10-15 at 12:36 +0200, Niels de Vos wrote:

> On Tue, Oct 15, 2019 at 12:20:54PM +0530, Anoop C S wrote:
> > Hi all,
> >
> > This is to check and confirm whether we have an API(or an internal
> > implementation which can be exposed as API) to perform operations
> > on an
> > open file descriptor as a wrapper around existing fcntl() system
> > call.
> > We do have specific APIs for locks(glfs_posix_lock) and file
> > descriptor
> > duplication(glfs_dup) which are important among those operations
> > listed
> > as per man fcntl(2).
> >
> > At present we have a requirement(very recent) from Samba to set
> > file
> > descriptor flags through its VFS layer which would need a
> > corresponding
> > mechanism inside GlusterFS. Due to its absence, VFS module for
> > GlusterFS inside Samba will have to workaround with the hack of
> > creating fake local file descriptors outside GlusterFS.
> >
> > Thoughts and suggestions are welcome.
>
> The fcntl() operations are split when FUSE is used. There in direct
> fcntl() call that FUSE passes on, instead it calls lock() and similar
> interfaces. I think you refer to F_GETFD and F_SETFD commands for
> fcntl(). For all I can see, these do not exist in FUSE, and have not
> been added to gfapi either. Not sure if the single supported flag
> FD_CLOEXEC can have a benefit on Gluster, as glfs_fini() is expected
> to
> cleanup everything that gfapi allocates.
>
> Can you explain your use-case a little more?

Quoting the response from my other reply..

> During open/create code path Samba tries to set blocking
> flag(O_NONBLOCK on Linux) on open file descriptor through its VFS
> layer. Comparing to fcntl() call command would be F_SETFL with
> required
> flag(O_NONBLOCK). It is an expectation that backend file system(in
> this
> case GlusterFS) executes this operation and return the result. VFS
> module for GlusterFS inside Samba can extract required file
> descriptor
> but would require a way to set the flag on it.

So to be specific, it is F_GETFL and F_SETFL commands that we are
looking for to be consumed.

_______________________________________________

Community Meeting Calendar:

APAC Schedule -
Every 2nd and 4th Tuesday at 11:30 AM IST
Bridge: https://bluejeans.com/118564314

NA/EMEA Schedule -
Every 1st and 3rd Tuesday at 01:00 PM EDT
Bridge: https://bluejeans.com/118564314

Gluster-devel mailing list
[hidden email]
https://lists.gluster.org/mailman/listinfo/gluster-devel

Reply | Threaded
Open this post in threaded view
|

Re: GlusterFS API to manipulate open file descriptor - glfs_fcntl?

Kaleb Keithley
In reply to this post by Niels de Vos-5


On Tue, Oct 15, 2019 at 6:37 AM Niels de Vos <[hidden email]> wrote:

The fcntl() operations are split when FUSE is used. There in direct
fcntl() call that FUSE passes on, instead it calls lock() and similar
interfaces.

Sorry, I can't parse this. I think you mean "There is no (direct) fcntl(2) fop in FUSE."
 
I think you refer to F_GETFD and F_SETFD commands for

F_GETFL and F_SETFL !
 
fcntl(). For all I can see, these do not exist in FUSE, and have not
been added to gfapi either.

Am I reading socket_connect() in rpc/rpc-transport/socket/src/socket.c correctly? It looks like we already always set O_NONBLOCK on sockets.
 
Not sure if the single supported flag
FD_CLOEXEC can have a benefit on Gluster, as glfs_fini() is expected to
cleanup everything that gfapi allocates.

That presumes that the implementation always calls glfs_fini() before a call to exec(2). I guess it might be bug if it didn't.  AFAIK ganesha doesn't ever call exec(2).  Samba's client model is different. And it would be wrong to pass it over the wire to the server.

--

Kaleb
 

_______________________________________________

Community Meeting Calendar:

APAC Schedule -
Every 2nd and 4th Tuesday at 11:30 AM IST
Bridge: https://bluejeans.com/118564314

NA/EMEA Schedule -
Every 1st and 3rd Tuesday at 01:00 PM EDT
Bridge: https://bluejeans.com/118564314

Gluster-devel mailing list
[hidden email]
https://lists.gluster.org/mailman/listinfo/gluster-devel