linux-kernel - Re: [RFC] readdirplus implementations: xgetdents vs dirreadahead syscalls

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20140726003859.GF20518@dastard>
Date:	Sat, 26 Jul 2014 10:38:59 +1000
From:	Dave Chinner <david@...morbit.com>
To:	Zach Brown <zab@...hat.com>
Cc:	Abhijith Das <adas@...hat.com>, linux-kernel@...r.kernel.org,
	linux-fsdevel <linux-fsdevel@...r.kernel.org>,
	cluster-devel <cluster-devel@...hat.com>
Subject: Re: [RFC] readdirplus implementations: xgetdents vs dirreadahead
 syscalls

On Fri, Jul 25, 2014 at 10:52:57AM -0700, Zach Brown wrote:
> On Fri, Jul 25, 2014 at 01:37:19PM -0400, Abhijith Das wrote:
> > Hi all,
> > 
> > The topic of a readdirplus-like syscall had come up for discussion at last year's
> > LSF/MM collab summit. I wrote a couple of syscalls with their GFS2 implementations
> > to get at a directory's entries as well as stat() info on the individual inodes.
> > I'm presenting these patches and some early test results on a single-node GFS2
> > filesystem.
> > 
> > 1. dirreadahead() - This patchset is very simple compared to the xgetdents() system
> > call below and scales very well for large directories in GFS2. dirreadahead() is
> > designed to be called prior to getdents+stat operations.
> 
> Hmm.  Have you tried plumbing these read-ahead calls in under the normal
> getdents() syscalls?

The issue is not directory block readahead (which some filesystems
like XFS already have), but issuing inode readahead during the
getdents() syscall.

It's the semi-random, interleaved inode IO that is being optimised
here (i.e. queued, ordered, issued, cached), not the directory
blocks themselves. As such, why does this need to be done in the
kernel?  This can all be done in userspace, and even hidden within
the readdir() or ftw/ntfw() implementations themselves so it's OS,
kernel and filesystem independent......

Cheers,

Dave.
-- 
Dave Chinner
david@...morbit.com
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/