linux-kernel - Re: [PATCH v4 01/11] block: make generic_make

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <CAF1ivSa76LmWTNagR6waPVkcypZY=cd0x9woJCbBfecs40A1Pg@mail.gmail.com>
Date:	Tue, 26 May 2015 10:17:57 -0700
From:	Ming Lin <mlin@...nel.org>
To:	Mike Snitzer <snitzer@...hat.com>
Cc:	Ming Lin <mlin@...nel.org>, lkml <linux-kernel@...r.kernel.org>,
	Christoph Hellwig <hch@....de>,
	Kent Overstreet <kent.overstreet@...il.com>,
	Jens Axboe <axboe@...nel.dk>, Dongsu Park <dpark@...teo.net>,
	Christoph Hellwig <hch@...radead.org>,
	Al Viro <viro@...iv.linux.org.uk>,
	Ming Lei <ming.lei@...onical.com>, Neil Brown <neilb@...e.de>,
	Alasdair Kergon <agk@...hat.com>, dm-devel@...hat.com,
	Lars Ellenberg <drbd-dev@...ts.linbit.com>,
	drbd-user@...ts.linbit.com, Jiri Kosina <jkosina@...e.cz>,
	Geoff Levand <geoff@...radead.org>, Jim Paris <jim@...n.com>,
	Joshua Morris <josh.h.morris@...ibm.com>,
	Philip Kelleher <pjk1939@...ux.vnet.ibm.com>,
	Minchan Kim <minchan@...nel.org>,
	Nitin Gupta <ngupta@...are.org>,
	Oleg Drokin <oleg.drokin@...el.com>,
	Andreas Dilger <andreas.dilger@...el.com>
Subject: Re: [PATCH v4 01/11] block: make generic_make_request handle
 arbitrarily sized bios

On Tue, May 26, 2015 at 9:04 AM, Mike Snitzer <snitzer@...hat.com> wrote:
> On Tue, May 26 2015 at 11:02am -0400,
> Ming Lin <mlin@...nel.org> wrote:
>
>> On Tue, May 26, 2015 at 7:36 AM, Mike Snitzer <snitzer@...hat.com> wrote:
>> > On Fri, May 22 2015 at  2:18pm -0400,
>> > Ming Lin <mlin@...nel.org> wrote:
>> >
>> >> From: Kent Overstreet <kent.overstreet@...il.com>
>> >>
>> >> The way the block layer is currently written, it goes to great lengths
>> >> to avoid having to split bios; upper layer code (such as bio_add_page())
>> >> checks what the underlying device can handle and tries to always create
>> >> bios that don't need to be split.
>> >>
>> >> But this approach becomes unwieldy and eventually breaks down with
>> >> stacked devices and devices with dynamic limits, and it adds a lot of
>> >> complexity. If the block layer could split bios as needed, we could
>> >> eliminate a lot of complexity elsewhere - particularly in stacked
>> >> drivers. Code that creates bios can then create whatever size bios are
>> >> convenient, and more importantly stacked drivers don't have to deal with
>> >> both their own bio size limitations and the limitations of the
>> >> (potentially multiple) devices underneath them.  In the future this will
>> >> let us delete merge_bvec_fn and a bunch of other code.
>> >
>> > This series doesn't take any steps to train upper layers
>> > (e.g. filesystems) to size their bios larger (which is defined as
>> > "whatever size bios are convenient" above).
>> >
>> > bio_add_page(), and merge_bvec_fn, served as the means for upper layers
>> > (and direct IO) to build up optimally sized bios.  Without a replacement
>> > (that I can see anyway) how is this patchset making forward progress
>> > (getting Acks, etc)!?
>> >
>> > I like the idea of reduced complexity associated with these late bio
>> > splitting changes I'm just not seeing how this is ready given there are
>> > no upper layer changes that speak to building larger bios..
>> >
>> > What am I missing?
>>
>> See: [PATCH v4 02/11] block: simplify bio_add_page()
>> https://lkml.org/lkml/2015/5/22/754
>>
>> Now bio_add_page() can build lager bios.
>> And blk_queue_split() can split the bios in ->make_request() if needed.
>
> That'll result in quite large bios and always needing splitting.
>
> As Alasdair asked: please provide some performance data that justifies
> these changes.  E.g use a setup like: XFS on a DM striped target.  We
> can iterate on more complex setups once we have established some basic
> tests.

I'll test XFS on DM and also what Christoph suggested:
https://lkml.org/lkml/2015/5/25/226

>
> If you're just punting to reviewers to do the testing for you that isn't
> going to instill _any_ confidence in me for this patchset as a suitabe
> replacement relative to performance.

Kent's Direct IO rewrite patch depends on this series.
https://git.kernel.org/cgit/linux/kernel/git/mlin/linux.git/log/?h=block-dio-rewrite

I did test the dio patch on a 2 sockets(48 logical CPUs) server and
saw 40% improvement with 48 null_blks.
Here is the fio data of 4k read.

4.1-rc2
----------
Test 1: bw=50509MB/s, iops=12930K
Test 2: bw=49745MB/s, iops=12735K
Test 3: bw=50297MB/s, iops=12876K,
Average: bw=50183MB/s, iops=12847K

4.1-rc2-dio-rewrite
------------------------
Test 1: bw=70269MB/s, iops=17989K
Test 2: bw=70097MB/s, iops=17945K
Test 3: bw=70907MB/s, iops=18152K
Average: bw=70424MB/s, iops=18028K
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/