linux-kernel - Re: [PATCH REPOST blktests v2 7/9] nvme-rc: Calculate IO size for fio jobs

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <l5uvjenah7oso6batcdgvkrzpgw362c55hwxxvkscriy5xj225@o2qtmbvyg52s>
Date:   Fri, 28 Apr 2023 03:57:12 +0000
From:   Shinichiro Kawasaki <shinichiro.kawasaki@....com>
To:     Daniel Wagner <dwagner@...e.de>
CC:     Hannes Reinecke <hare@...e.de>,
        "linux-nvme@...ts.infradead.org" <linux-nvme@...ts.infradead.org>,
        "linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
        "linux-block@...r.kernel.org" <linux-block@...r.kernel.org>,
        Chaitanya Kulkarni <kch@...dia.com>,
        Shin'ichiro Kawasaki <shinichiro@...tmail.com>
Subject: Re: [PATCH REPOST blktests v2 7/9] nvme-rc: Calculate IO size for fio
 jobs

On Apr 21, 2023 / 09:03, Daniel Wagner wrote:
> On Fri, Apr 21, 2023 at 08:33:46AM +0200, Hannes Reinecke wrote:
>  > +_nvme_calc_rand_io_size() {
> > > +	local img_size_mb
> > > +	local io_size_mb
> > > +
> > > +	img_size_mb="$(convert_to_mb "$1")"
> > > +	io_size_mb="$(printf "%d" $((((img_size_mb * 1024 * 1024) / $(nproc) - 1) / 1024)))"
> > > +
> > 
> > ... ending with ridiculous small io sizes on machines with lots of CPUs.
> > Please cap nproc by something sane like 32.
> 
> Yeah, propably not really good long time strategy. I was wondering if we should
> make run_fio() variants smarter and do the size callculation there and not by
> the callee. If we do this, we could make the number of jobs dependend on CPUs
> and image size a bit nicer.

The usage of _run_fio_rand_io() look different for each test case. nvme/032
kills the fio process when it is no longer required. Then IO size reduction with
_nvme_calc_io_size() will not reduce runtime of nvme/032. I think nvme/040 has
same story, since _nvme_delete_ctrl will stop the fio process with I/O error.

On the other hand, nvme/045 and nvme/047 may have different usage. I'm not sure
if these test case needs I/O with all CPUs. It would be better to have other
run_fio() variant as Daniel mentioned, so that their runtime will not depend on
number of CPUs.