linux-kernel - Re: [RFC] Union mount readdir support in glibc

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <47D9F6CC.6010009@redhat.com>
Date:	Thu, 13 Mar 2008 20:53:48 -0700
From:	Ulrich Drepper <drepper@...hat.com>
To:	bharata@...ux.vnet.ibm.com
CC:	libc-alpha@...rceware.org, Jan Blunck <jblunck@...e.de>,
	Erez Zadok <ezk@...sunysb.edu>, linux-kernel@...r.kernel.org,
	linux-fsdevel@...r.kernel.org, viro@...iv.linux.org.uk,
	Christoph Hellwig <hch@....de>, Mingming Cao <cmm@...ibm.com>,
	Dave Hansen <haveblue@...ibm.com>
Subject: Re: [RFC] Union mount readdir support in glibc

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Bharata B Rao wrote:
> readdir in glibc
> ----------------
> glibc readdir would maintain a cache of dirents returned by readdir/getdents
> system call to perform duplicate elimination and whiteout supression.

As an optimization I have no problem with adding the code to glibc if it
does not impact non-union directories.  But requiring lockstep update of
kernel and glibc to use a feature like this which is not under control
of the application is a huge problem.  I don't think we ever had to
resort to this.  I consider this absolutely only the last resort.

Current readdir requires:

  opendir:   open
             fstat  (this can probably even be removed)
             malloc small buffer

  readdir:   if buffer is empty
                  getdents call (read multiple entries)
             return next entry

There is very little overhead.  Since we copy using getdents multiple
records it is more efficient than implementing readdir in the kernel.
This is how efficient normal directory operations must remain.  The only
slight inefficiency is that we have to copy the entries after getdents()
because the d_type field is not in the place we expect it at userlevel.
 For this a new interface could help.

To handle union FS at userlevel somewhere in that code sequence (perhaps
in the fstat call) we'd have to recognize such mounts.  Before any
agreement on userlevel sorting can be made you'll have to answer a
question Roland already asked:

- - How does this work with NFS?

Regarding questions you have: if a directory currently is read and file
are added or removed, all bets are off.

re seeking: you have to support seeking.  There is no way around it.
Once again, if any file has been added/removed, all bets are off.  So,
why not provide a cookie similar to what is done today?  I think it is
not acceptable to require caching the entire directory content at
userlevel.  It's bad enough if we have to store the file names for
duplicate elimination.

- --
➧ Ulrich Drepper ➧ Red Hat, Inc. ➧ 444 Castro St ➧ Mountain View, CA ❖
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.8 (GNU/Linux)
Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org

iEYEARECAAYFAkfZ9GgACgkQ2ijCOnn/RHS/nACgx/NQqWB0kIbXBkwuSZIr1alX
78EAn2PECJ/9Ax3RzyayatE61ZM9I42W
=vFJP
-----END PGP SIGNATURE-----
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/