Re: DazukoFS

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

Re: DazukoFS

John Ogness-2
Tvrtko Ursulin wrote:
> [another resend to you John privately - any idea why my messages are not
> appearing on the mailing list while I receive ML traffic OK?]

Your email address is listed as subscribed to dazuko-devel. Do you get a
bounce message? If yes, could you send it to me?


> I've grabbed your birthday tarball and played with it a bit (little bit). Hope
> it's OK if I ask a couple of questions?

Of course. We love questions! ;)


> Without going deeper into how it works, the first that struck me was why not
> provide file names?

This is an early preview release. Later we will provide file names
through a new Dazuko function, probably dazukoGetAccessFilename().


> After all, data is available in the dazukofs_open call,
> no? So I quickly started experimenting and cooked a patch (attached) which
> tries to do exactly that. It looks like it works, you tell me if something is
> wrong with that approach.

Well, yes, it does work. However, it uses the d_path() function. One of
the advantages of DazukoFS is that we can get away from the d_path()
function. The problem with this function is:

1. it is quite expensive

2. it is useless for chroot environments

3. we need to grab the root node to notice if we are in a chroot environment

4. __d_path() would allow us to get around #2, but it is not re-entrant
safe and it is not exported to modules

By using its own pseudo names, Dazuko can allow an application to access
file content immediately without expensive filename lookups. For
applications such as on-access scanners, this can have significant
performance improvements. (After all, on-access scanners rarely care how
the file is actually named.)

As mentioned above, for applications that _do_ want the filename, an
extra function will be made available. This function will be a bit more
expensive. As a quick and dirty solution, we could base this new
function on d_path(). However, I would prefer that Dazuko uses its own
routine (and caching techniques) to assemble the path. This helps to
prevent Dazuko from getting too tangled with the kernel and avoids the
problems listed above. Since Dazuko works on the filesystem level, it
will be fairly simple to implement this.


> Another thing which is puzzling me is what is that linux_dazuko_vfs_lookup
> bussines? What it is supposed to do, how and why?

In order to understand the usefulness of using vfs_lookup(), you need to
forget about d_path(). We don't want to assemble full filenames with
each access just to decide if we should pass the information to
registered applications or not. This has the problems listed above.
Instead, we want to use the information (and caching) readily available
in the kernel and DazukoFS.

vfs_lookup() is used by Dazuko to intercept file access for registered
applications. It needs to be done in vfs_lookup() because registered
applications work with pseudo filenames that do not actually exist on
the file system.

vfs_open() is used by Dazuko to intercept file access for non-registered
applications. This can't be used for intercepting file access for
registered applications because inside this function we do not want to
do expensive (and unreliable) filename lookups. So we need to use pseudo
names (which can only be interecepted with vfs_lookup()).

Here is the basic overview of what happens with a _single_ file access:

NA = non-registered application (typical system process)
RA = registered application (on-access scanner)

NA: calls fopen()
NA: vfs_lookup() is called, returns dentry to for file
    - no interception, this is NA
NA: vfs_open() is called, Dazuko intercepts
    - Dazuko stores dentry (for use by RA)
    - pseudo name is created
    - RA notified (with pseudo name)
    - (note: NA is still in fopen() and vfs_open())

RA: receives pseudo name
RA: opens pseudo name for scanning, calls fopen()
RA: vfs_lookup() is called, Dazuko intercepts
    - pseudo name recognized
    - stored dentry is returned
RA: vfs_open() is called, returns file information
    - no interception, this is RA
RA: fopen() finished, scan finished, RA tells Dazuko "deny=0"

NA: vfs_open() continues, returns file information
NA: fopen() finished, NA can access file

If any of this is unclear, feel free to continue asking questions.

John Ogness

--
Dazuko Maintainer


_______________________________________________
Dazuko-devel mailing list
[hidden email]
http://lists.nongnu.org/mailman/listinfo/dazuko-devel
Reply | Threaded
Open this post in threaded view
|

Re: DazukoFS

Tvrtko Ursulin-2
John Ogness <[hidden email]> wrote on 23/02/2007 16:27:29:

> Tvrtko Ursulin wrote:
> > [another resend to you John privately - any idea why my messages are
not
> > appearing on the mailing list while I receive ML traffic OK?]
>
> Your email address is listed as subscribed to dazuko-devel. Do you get a
> bounce message? If yes, could you send it to me?

Didn't get it. And this only came directly and hasn't showed up in the
mailing list. Maybe the server is down, I don't know how reliable savannah
is except that I was unable to access the CVS server a few weeks ago.
 
> > After all, data is available in the dazukofs_open call,
> > no? So I quickly started experimenting and cooked a patch (attached)
which

> > tries to do exactly that. It looks like it works, you tell me if
> something is
> > wrong with that approach.
>
> Well, yes, it does work. However, it uses the d_path() function. One of
> the advantages of DazukoFS is that we can get away from the d_path()
> function. The problem with this function is:
>
> 1. it is quite expensive
>
> 2. it is useless for chroot environments
>
> 3. we need to grab the root node to notice if we are in a chroot
environment

I agree with all the above points. And not to mention private
namespaces...
 
> 4. __d_path() would allow us to get around #2, but it is not re-entrant
> safe and it is not exported to modules

What exactly do you mean with re-entrant safe?
 
> By using its own pseudo names, Dazuko can allow an application to access
> file content immediately without expensive filename lookups. For
> applications such as on-access scanners, this can have significant
> performance improvements. (After all, on-access scanners rarely care how
> the file is actually named.)

Interesting approach, thanks for the explanation. Stuff below deleted -
everything does make sense so my curiousity is satisfied. :)

The only remaining problem will be hot to handle removable devices and new
mounts in general. I don't see a completely secure solution for that with
stackable filesystem only approach, right?

Tvrtko


Sophos Plc, The Pentagon, Abingdon Science Park, Abingdon,
OX14 3YP, United Kingdom.

Company Reg No 2096520. VAT Reg No GB 348 3873 20.



_______________________________________________
Dazuko-devel mailing list
[hidden email]
http://lists.nongnu.org/mailman/listinfo/dazuko-devel
Reply | Threaded
Open this post in threaded view
|

Re: DazukoFS

John Ogness-2
[hidden email] wrote:
>>Well, yes, it does work. However, it uses the d_path() function. One of
>>the advantages of DazukoFS is that we can get away from the d_path()
>>function. The problem with this function is:
>>
>>4. __d_path() would allow us to get around #2, but it is not re-entrant
>>safe and it is not exported to modules
>
> What exactly do you mean with re-entrant safe?

d_path() is a wrapper for __d_path() that does two things: It uses the
root mount of the current process and it calls __d_path() while holding
the necessary lock "dcache_lock".

Unfortunately "dcache_lock" is a static variable within fs/dcache.c,
which means no other kernel code outside of fs/dcache.c can access this
lock. This means that a kernel module cannot safely call __d_path() (on
SMP machines) and must use the "restricted" d_path() function.

Note: I just noticed that 2.6.20 now exports dcache_lock. Interesting...


> The only remaining problem will be hot to handle removable devices and new
> mounts in general. I don't see a completely secure solution for that with
> stackable filesystem only approach, right?

If by "secure" you mean "automatic", you are correct. Right now we are
not aware of a way to always auto-mount DazukoFS on top of any newly
mounted filesystem. However, I am not experienced in automounters and
what types of hooks they might support.

I also believe we have a fairly good chance of getting DazukoFS into the
mainline kernel (once it is ready). This would certainly help improve
efforts for automount support.

John Ogness

--
Dazuko Maintainer


_______________________________________________
Dazuko-devel mailing list
[hidden email]
http://lists.nongnu.org/mailman/listinfo/dazuko-devel
Reply | Threaded
Open this post in threaded view
|

Re: DazukoFS

Tvrtko Ursulin-2
John Ogness <[hidden email]> wrote on 23/02/2007 21:28:03:

> [hidden email] wrote:
> >>Well, yes, it does work. However, it uses the d_path() function. One
of
> >>the advantages of DazukoFS is that we can get away from the d_path()
> >>function. The problem with this function is:
> >>
> >>4. __d_path() would allow us to get around #2, but it is not
re-entrant

> >>safe and it is not exported to modules
> >
> > What exactly do you mean with re-entrant safe?
>
> d_path() is a wrapper for __d_path() that does two things: It uses the
> root mount of the current process and it calls __d_path() while holding
> the necessary lock "dcache_lock".
>
> Unfortunately "dcache_lock" is a static variable within fs/dcache.c,
> which means no other kernel code outside of fs/dcache.c can access this
> lock. This means that a kernel module cannot safely call __d_path() (on
> SMP machines) and must use the "restricted" d_path() function.
>
> Note: I just noticed that 2.6.20 now exports dcache_lock. Interesting...

Yes, and for earlier version you could still get it from the System.map
file which would be equivalent to syscall_table stuff. Then any
re-implementation of __d_path can be safe.
 
Btw, I just tried it on FreeBSD 6.2 and I seem to be missing
/usr/src/sbin/mount_nullfs directory. Any ideas on where can I get that?

Tvrtko


Sophos Plc, The Pentagon, Abingdon Science Park, Abingdon,
OX14 3YP, United Kingdom.

Company Reg No 2096520. VAT Reg No GB 348 3873 20.



_______________________________________________
Dazuko-devel mailing list
[hidden email]
http://lists.nongnu.org/mailman/listinfo/dazuko-devel
Reply | Threaded
Open this post in threaded view
|

Re: DazukoFS

John Ogness-2
[hidden email] wrote:
> Btw, I just tried it on FreeBSD 6.2 and I seem to be missing
> /usr/src/sbin/mount_nullfs directory. Any ideas on where can I get that?

# sysinstall

Configure -> Distributions -> src -> sbin

John Ogness

--
Dazuko Maintainer


_______________________________________________
Dazuko-devel mailing list
[hidden email]
http://lists.nongnu.org/mailman/listinfo/dazuko-devel