linux-kernel - Re: [RFC PATCH 0/6] Convert all tasklets to workqueues

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20070622210013.GA1353@infradead.org>
Date:	Fri, 22 Jun 2007 22:00:13 +0100
From:	Christoph Hellwig <hch@...radead.org>
To:	Ingo Molnar <mingo@...e.hu>
Cc:	Linus Torvalds <torvalds@...ux-foundation.org>,
	Steven Rostedt <rostedt@...dmis.org>,
	LKML <linux-kernel@...r.kernel.org>,
	Andrew Morton <akpm@...ux-foundation.org>,
	Thomas Gleixner <tglx@...utronix.de>,
	Christoph Hellwig <hch@...radead.org>,
	john stultz <johnstul@...ibm.com>,
	Oleg Nesterov <oleg@...sign.ru>,
	"Paul E. McKenney" <paulmck@...ux.vnet.ibm.com>,
	Dipankar Sarma <dipankar@...ibm.com>,
	"David S. Miller" <davem@...emloft.net>, matthew.wilcox@...com,
	kuznet@....inr.ac.ru
Subject: Re: [RFC PATCH 0/6] Convert all tasklets to workqueues

On Fri, Jun 22, 2007 at 10:40:58PM +0200, Ingo Molnar wrote:
> when it comes to 'deferred processing', we've basically got two 'prime' 
> choices for deferred processing:
> 
>  - if it's high-performance then it goes into a softirq.
> 
>  - if performance is not important, or robustness and flexibility is 
>    more important than performance, then workqueues are used.
> 
> basically tasklets do _neither_ really well. They are too 'global' to 
> scale really well on SMP (even the RCU tasklet wasnt a real tasklet: it 
> was a _per CPU tasklet_, which almost by definition is equivalent to a 
> softirq, some some extra glue overhead ...), and tasklets are also too 
> much tied to softirqs to be used as a generic processing context.
> 
> that's why i'd like them to be gently but firmly phased out =B-)

Note that we also have a lot of inefficiency in the way we do deferred
processing.  Think of a setup where you run a XFS filesystem runs over
a megaraid adapter.

 (1) we get a real hardirq, which just clears the interrupt and then
     deferes to a tasklet
 (2) tasklet walks the producer / consumer queue and then calls scsi_done
     for each completeted scsi command which ends up doing
     raise_softirq_irqoff(BLOCK_SOFTIRQ);
 (3) block softirq does the heavy lifting for command completion and finally
     calls back into the bio's completion routine
 (4) xfs wants to avoid irq safe locking and thus deferes the command to a
     kthread

This is rather inefficient due to all the (semi-)context switches already
and not by far the worst setup given that a lot of dm modules can involve
another thread in the process.

Now if just plain convert tasklets to a thread based abstraction this
existing code becomes really dumb because we go from hardirq to process
context to go back to softirq context to go back to process context.

Ouch!

I think we need to put a little more though into how we can optimize our
irq path for the full stack.  Using irqthreads in an intelligent way might
be one option, but we'll need a lot of heavy benchmarking whatever way
we go.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/