linux-kernel - Re: [PATCH] vfs: Avoid IPI storm due to bh LRU invalidation

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <4F2FF70D.5020603@suse.de>
Date:	Mon, 06 Feb 2012 16:51:41 +0100
From:	Hannes Reinecke <hare@...e.de>
To:	"Srivatsa S. Bhat" <srivatsa.bhat@...ux.vnet.ibm.com>
Cc:	Jan Kara <jack@...e.cz>, linux-fsdevel@...r.kernel.org,
	LKML <linux-kernel@...r.kernel.org>,
	Andrew Morton <akpm@...ux-foundation.org>,
	Al Viro <viro@...IV.linux.org.uk>,
	Christoph Hellwig <hch@...radead.org>,
	Gilad Ben-Yossef <gilad@...yossef.com>
Subject: Re: [PATCH] vfs: Avoid IPI storm due to bh LRU invalidation

On 02/06/2012 04:42 PM, Srivatsa S. Bhat wrote:
> On 02/06/2012 07:25 PM, Jan Kara wrote:
> 
>> When discovery of lots of disks happen in parallel, we call
>> invalidate_bh_lrus() once for each disk from partitioning code resulting in a
>> storm of IPIs and causing a softlockup detection to fire (it takes several
>> *minutes* for a machine to execute all the invalidate_bh_lrus() calls).
>>
>> Fix the issue by allowing only single invalidation to run using a mutex and let
>> waiters for mutex figure out whether someone invalidated LRUs for them while
>> they were waiting.
>>
>> Signed-off-by: Jan Kara <jack@...e.cz>
>> ---
>>  fs/buffer.c |   23 ++++++++++++++++++++++-
>>  1 files changed, 22 insertions(+), 1 deletions(-)
>>
>>   I feel this is slightly hacky approach but it works. If someone has better
>> idea, please speak up.
>>
> 
> 
> Something related that you might be interested in:
> https://lkml.org/lkml/2012/2/5/109
> 
> (This is part of Gilad's patchset that tries to reduce cross-CPU IPI
> interference.)
> 
Yes, but this is only part of the equation.

When booting a machine with lots of disks chances are that each CPU
_will_ have LRU BHs attached to it (due to partitions table
reading). However, these LRU BHs have nothing to do with the device
in question. So we wouldn't even need to send IPIs here.
Sadly we seem to lack the facilities to figure that out
(I'm not an expert in that area to tell for sure :-).

So the best we can hope for is to serialise the IPIs to not overload
system with tons of IPIs.

Cheers,

Hannes
-- 
Dr. Hannes Reinecke		      zSeries & Storage
hare@...e.de			      +49 911 74053 688
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg
GF: J. Hawn, J. Guild, F. Imendörffer, HRB 16746 (AG Nürnberg)
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/