linux-kernel - Re: [PATCH] block: per-cpu counters for in-flight IO accounting

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <536D0537.7010905@kernel.dk>
Date:	Fri, 09 May 2014 10:41:27 -0600
From:	Jens Axboe <axboe@...nel.dk>
To:	Matias Bjørling <m@...rling.me>,
	sbradshaw@...ron.com
CC:	linux-kernel@...r.kernel.org
Subject: Re: [PATCH] block: per-cpu counters for in-flight IO accounting

On 05/09/2014 08:12 AM, Jens Axboe wrote:
> On 05/09/2014 03:17 AM, Matias Bjørling wrote:
>> With multi-million IOPS and multi-node workloads, the atomic_t in_flight
>> tracking becomes a bottleneck. Change the in-flight accounting to per-cpu
>> counters to elevate.
> 
> The part stats are a pain in the butt, I've tried to come up with a
> great fix for them too. But I don't think the percpu conversion is
> necessarily the right one. The summing is part of the hotpath, so percpu
> counters aren't necessarily the right way to go. I don't have a better
> answer right now, otherwise it would have been fixed :-)

Actual data point - this slows my test down ~14% compared to the stock
kernel. Also, if you experiment with this, you need to watch for the
out-of-core users of the part stats (like DM).

-- 
Jens Axboe

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/