linux-kernel - Re: [RFC PATCH 2/3] softirq: implement interrupt flood detection

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20191231034806.GB20062@ming.t460p>
Date:   Tue, 31 Dec 2019 11:48:06 +0800
From:   Ming Lei <ming.lei@...hat.com>
To:     Daniel Wagner <dwagner@...e.de>
Cc:     Peter Zijlstra <peterz@...radead.org>,
        Thomas Gleixner <tglx@...utronix.de>,
        Jens Axboe <axboe@...nel.dk>, linux-kernel@...r.kernel.org,
        linux-block@...r.kernel.org, Long Li <longli@...rosoft.com>,
        Ingo Molnar <mingo@...hat.com>, Christoph Hellwig <hch@....de>,
        Keith Busch <keith.busch@...el.com>,
        Sagi Grimberg <sagi@...mberg.me>,
        John Garry <john.garry@...wei.com>,
        Hannes Reinecke <hare@...e.com>
Subject: Re: [RFC PATCH 2/3] softirq: implement interrupt flood detection

On Thu, Dec 19, 2019 at 11:43:47AM +0100, Daniel Wagner wrote:
> Hi,
> 
> On Thu, Dec 19, 2019 at 10:23:19AM +0100, Peter Zijlstra wrote:
> > On Thu, Dec 19, 2019 at 09:59:48AM +0800, Ming Lei wrote:
> > > > So pray tell, why did you not integrate this with IRQ_TIME_ACCOUNTING ?
> > > > That already takes a timestamp and does most of what you need.
> > > 
> > > Yeah, that was the 1st approach I thought of, but IRQ_TIME_ACCOUNTING
> > > may be disabled, and enabling it may cause observable effect on IO
> > > performance.
> > 
> > Is that an actual concern, are people disabling it?
> 
> In SLE and openSUSE kernels it is disabled for x86_64 at this
> point. And if I am not completely misstaken only x86_64 supports it at
> this point. I was looking at enable_sched_clock_irqtime() which is
> only called from x86_64.
> 
> Another thing I noticed get_util_irq() is defined in
> kernel/sched/sched.h. I don't think the block/blq-mq.c driver should
> include it direclty.

get_util_irq() only works in case of HAVE_SCHED_AVG_IRQ which depends
on IRQ_TIME_ACCOUNTING or PARAVIRT_TIME_ACCOUNTING.

Also rq->avg_irq.util_avg is only updated when there is scheduler
activities. However, when interrupt flood happens, scheduler can't
have chance to be called. Looks get_util_irq() can't be relied on
for this task.

> 
> Thanks,
> Daniel
> 
> ps: A customer observes the same problem as Ming is reporting.

Actually this issue should be more serious on ARM64 system, in which
there are more CPU cores, and each CPU core is often slower than
x86's, and each interrupt is only delivered to single CPU target.

Meantime the storage device performance is same for the two kinds of
systems.


Thanks,
Ming