linux-kernel - Re: [PATCH] libata: use single threaded work queue

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20090819120458.GZ12579@kernel.dk>
Date:	Wed, 19 Aug 2009 14:04:59 +0200
From:	Jens Axboe <jens.axboe@...cle.com>
To:	Jeff Garzik <jeff@...zik.org>
Cc:	linux-ide@...r.kernel.org, linux-kernel@...r.kernel.org,
	htejun@...il.com, Benjamin Herrenschmidt <benh@...nel.crashing.org>
Subject: Re: [PATCH] libata: use single threaded work queue

On Wed, Aug 19 2009, Jeff Garzik wrote:
> On 08/19/2009 07:25 AM, Jens Axboe wrote:
>> Hi,
>>
>> On boxes with lots of CPUs, we have so many kernel threads it's not
>> funny. The basic problem is that create_workqueue() creates a per-cpu
>> thread, where we could easily get by with a single thread for a lot of
>> cases.
>>
>> One such case appears to be ata_wq. You want at most one per pio drive,
>> not one per CPU. I'd suggest just dropping it to a single threaded wq.
>>
>> Signed-off-by: Jens Axboe<jens.axboe@...cle.com>
>>
>> diff --git a/drivers/ata/libata-core.c b/drivers/ata/libata-core.c
>> index 072ba5e..0d78628 100644
>> --- a/drivers/ata/libata-core.c
>> +++ b/drivers/ata/libata-core.c
>> @@ -6580,7 +6580,7 @@ static int __init ata_init(void)
>>   {
>>   	ata_parse_force_param();
>>
>> -	ata_wq = create_workqueue("ata");
>> +	ata_wq = create_singlethread_workqueue("ata");
>>   	if (!ata_wq)
>>   		goto free_force_tbl;
>
>
> I agree with one-thread-per-cpu is too much, in these modern multi-core  
> times, but one thread is too little.  You have essentially re-created  
> simplex DMA -- blocking and waiting such that one drive out of ~4 can be  
> used at any one time.
>
> ata_pio_task() is in a workqueue so that it can sleep and/or spend a  
> long time polling ATA registers.  That means an active task can  
> definitely starve all other tasks in the workqueue, if only one thread  
> is available.  If starvation occurs, it will potentially pause the  
> unrelated task for several seconds.
>
> The proposed patch actually expands an existing problem -- uniprocessor  
> case, where there is only one workqueue thread.  For the reasons  
> outlined above, we actually want multiple threads even in the UP case.  
> If you have more than one PIO device, latency is bloody awful, with  
> occasional multi-second "hiccups" as one PIO devices waits for another.  
> It's an ugly wart that users DO occasionally complain about; luckily  
> most users have at most one PIO polled device.
>
> It would be nice if we could replace this workqueue with a thread pool,  
> where thread count inside the pool ranges from zero to $N depending on  
> level of thread pool activity.  Our common case in libata would be  
> _zero_ threads, if so...

That would be ideal, N essentially be:

        N = min(nr_cpus, nr_drives_that_need_pio);

How can I easily test whether we will ever need a pio thread for a
drive in libata? For a simple patch, I would suggest simply creating a
single threaded workqueue per ap instead, if that ata_port would ever
want PIO.

-- 
Jens Axboe

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/