linux-kernel - Re: [PATCH RFC] nohz,blk-mq: do not create blk-mq workqueues on nohz dedicated CPUs

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20150331231700.GB4161@amt.cnet>
Date:	Tue, 31 Mar 2015 20:17:01 -0300
From:	Marcelo Tosatti <mtosatti@...hat.com>
To:	Rik van Riel <riel@...hat.com>
Cc:	axboe@...nel.org, fweisbec@...hat.com, mingo@...nel.org,
	linux-kernel@...r.kernel.org, lcapitulino@...hat.com
Subject: Re: [PATCH RFC] nohz,blk-mq: do not create blk-mq workqueues on nohz
 dedicated CPUs

On Tue, Mar 31, 2015 at 10:27:26AM -0400, Rik van Riel wrote:
> CPUs with nohz_full do not want disruption from timer interrupts,
> or other random system things.  This includes block mq work.
> 
> There is another issue with block mq vs. realtime tasks that run
> 100% of the time, which is not uncommon on systems that have CPUs
> dedicated to real time use with isolcpus= and nohz_full=
> 
> Specifically, on systems like that, a block work item may never
> get to run, which could lead to filesystems getting stuck forever.
> 
> We can avoid both issues by not scheduling blk-mq workqueues on
> cpus in nohz_full mode.
> 
> Question for Jens: should we try to spread out the load for
> currently offline and nohz CPUs across the remaining CPUs in
> the system, to get the full benefit of blk-mq in these situations?
> 
> If so, do you have any preference on how I should implement that?
> 
> Cc: Frederic Weisbecker <fweisbec@...hat.com>
> Cc: Ingo Molnar <mingo@...nel.org>
> Cc: Jens Axboe <axboe@...nel.org>
> Signed-off-by: Rik van Riel <riel@...hat.com>
> ---
>  block/blk-mq.c | 5 +++++
>  1 file changed, 5 insertions(+)
> 
> diff --git a/block/blk-mq.c b/block/blk-mq.c
> index 4f4bea21052e..1004d6817fa4 100644
> --- a/block/blk-mq.c
> +++ b/block/blk-mq.c
> @@ -21,6 +21,7 @@
>  #include <linux/sched/sysctl.h>
>  #include <linux/delay.h>
>  #include <linux/crash_dump.h>
> +#include <linux/tick.h>
>  
>  #include <trace/events/block.h>
>  
> @@ -1760,6 +1761,10 @@ static void blk_mq_init_cpu_queues(struct request_queue *q,
>  		if (!cpu_online(i))
>  			continue;
>  
> +		/* Do not schedule work on nohz full dedicated CPUs. */
> +		if (tick_nohz_full_cpu(i))
> +			continue;
> +
>  		hctx = q->mq_ops->map_queue(q, i);
>  		cpumask_set_cpu(i, hctx->cpumask);
>  		hctx->nr_ctx++;

Rik,

I suppose any bound workqueue queued on isolated CPUs should be moved at 
queue time to other CPUs (sacrifficing performance).

So that by doing "queue_work" on an isolated CPU would move that 
work somewhere else.


--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/