linux-ext4 - Re: [EXT] how to disable readahead

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20180918090258.GF10257@quack2.suse.cz>
Date:   Tue, 18 Sep 2018 11:02:58 +0200
From:   Jan Kara <jack@...e.cz>
To:     "Bean Huo (beanhuo)" <beanhuo@...ron.com>
Cc:     Jan Kara <jack@...e.cz>, Andreas Dilger <adilger@...ger.ca>,
        "jeffm@...e.com" <jeffm@...e.com>,
        "Theodore Y. Ts'o" <tytso@....edu>,
        "linux-ext4@...r.kernel.org" <linux-ext4@...r.kernel.org>
Subject: Re: [EXT] how to disable readahead

On Sat 15-09-18 12:42:15, Bean Huo (beanhuo) wrote:
> >Subject: Re: [EXT] how to disable readahead
> >
> >On Wed 12-09-18 12:29:50, Andreas Dilger wrote:
> >> On Sep 12, 2018, at 9:13 AM, Jan Kara <jack@...e.cz> wrote:
> >> >
> >> > On Thu 02-08-18 12:58:04, Theodore Y. Ts'o wrote:
> >> >> On Thu, Aug 02, 2018 at 01:56:41PM +0000, Bean Huo (beanhuo) wrote:
> >> >>>
> >> >>> I am newbie on ext4, I tried the above method to disable
> >> >>> readahead, echo 0 > /sys/block/<dev>/queue/read_ahead_kb Then I
> >> >>> read by 128kB chunk size, ext4 will read the file by 4KB chunk
> >> >>> size each time. that means ext4 splits 128KB into 32 4KB to read.
> >> >>> That's not my expectation.  Do you know how to still keep and let
> >> >>> ext4 read by 128KB in case of disable readahead?
> >> >>
> >> >> Hmm... that's not my expectation as well, but I've replicated your
> >> >> results.  More interestingly, I tried the same experiment using
> >> >> XFS, and it does the same thing.  I used as my test workload:
> >> >>
> >> >> dd if=/mnt/test bs=128k count=32 | sum
> >> >>
> >> >> Used strace to verify that dd was in fact issuing 128k reads:
> >> >>
> >> >> read(0,
> >> >>
> >"\377\253a)\307\10\230\6\360,,:\226Rq\204\343\2522&44\307\341\372\2
> >> >> 71\271/\224#?\346"..., 131072) = 131072 write(1,
> >> >>
> >"\377\253a)\307\10\230\6\360,,:\226Rq\204\343\2522&44\307\341\372\2
> >> >> 71\271/\224#?\346"..., 131072) = 131072
> >> >>
> >> >> And then used btrace to monitor the I/O requests sent to the device:
> >> >>
> >> >> 252,4    0      413     0.077274997 14645  Q   R 4408 + 8 [dd]
> >> >> 252,4    2       77     0.077355648  5529  C   R 4408 + 8 [0]
> >> >> 252,4    0      414     0.077393725 14645  Q   R 4416 + 8 [dd]
> >> >> 252,4    2       78     0.077630722  5529  C   R 4416 + 8 [0]
> >> >> 	...
> >> >>
> >> >> ... and indeed, the reads are being sent to the device in 4k chunks.
> >> >> That's indeed surprising.  I'd have to do some debugging with
> >> >> tracepoints to see what requests are being issued from the
> >> >> mm/filemap.c to the file system.
> >> >
> >> > And this is in fact expected. There are two basic ways how data can
> >> > appear in page cache: ->readpage and ->readpages filesystem
> >> > callbacks. The second one is what readahead (and only readahead)
> >> > uses, the first one is used as a fallback when readahead fails for
> >> > some reason. So if you disable readahead, you're left only with -
> >>readpage call which does only one-page (4k) reads.
> >>
> >> Even *with* readahead, why would we add the overhead of processing
> >> each page separately instead of handling all pages in a single batch via
> >readpages()?
> >
> >Hum, I don't understand. With readahead enabled, we should be submitting
> >larger batches of IO as generated by ->readpages call and ->readpage actually
> >never ends up issuing any IO (see how generic_file_buffered_read() calls
> >page_cache_sync_readahead() first which ends up locking pages and
> >submitting reads) and only then we go, search for the page again and lock it -
> >which effectively waits for the readahead to pull in the first page.
> >
> >								Honza
> >--
> >Jan Kara <jack@...e.com>
> >SUSE Labs, CR
> 
> 'read_ahead_kb' should be only used for the read ahead (second time read internal),
> should be used as a flag to change the first read request chunk size came from user space read.
> Even the 'read_ahead_kb' configured 0.

OK, so you made me look into details how the read request size gets
computed :).  The thing is: When read_ahead_kb is 0, we really do single
page reads as all the cleverness in trying to issue large read requests
gets disabled.  Once read_ahead_kb is >0 (you have to write there at least
PAGE_SIZE - i.e.  4 on x86_64), we will actually issue requests of size at
least requested in the syscall.

								Honza
-- 
Jan Kara <jack@...e.com>
SUSE Labs, CR