linux-kernel - Re: [PATCH 2/2] ufs: don't use the fair tag sharings

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <a26e28a6-91e0-e803-749e-2ce957711c64@huaweicloud.com>
Date:   Wed, 17 May 2023 15:49:15 +0800
From:   Yu Kuai <yukuai1@...weicloud.com>
To:     Bart Van Assche <bvanassche@....org>,
        Yu Kuai <yukuai1@...weicloud.com>,
        Christoph Hellwig <hch@...radead.org>
Cc:     Ed Tsai <ed.tsai@...iatek.com>, axboe@...nel.dk,
        linux-block@...r.kernel.org, linux-scsi@...r.kernel.org,
        linux-kernel@...r.kernel.org, martin.petersen@...cle.com,
        stanley.chu@...iatek.com, peter.wang@...iatek.com,
        chun-hung.wu@...iatek.com, alice.chao@...iatek.com,
        powen.kao@...iatek.com, naomi.chu@...iatek.com,
        wsd_upstream@...iatek.com, "yukuai (C)" <yukuai3@...wei.com>
Subject: Re: [PATCH 2/2] ufs: don't use the fair tag sharings

Hi,

在 2023/05/16 23:12, Bart Van Assche 写道:
> On 5/12/23 20:09, Yu Kuai wrote:
>> 在 2023/05/13 2:12, Bart Van Assche 写道:
>>> The fair tag sharing algorithm has a negative impact on all SCSI 
>>> devices with multiple logical units. This is because logical units 
>>> are considered active until (request timeout) seconds have elapsed 
>>> after the logical unit stopped being used (see also the 
>>> blk_mq_tag_idle() call in blk_mq_timeout_work()). UFS users are hit 
>>> by this because UFS 3.0 devices have a limited queue depth (32) and 
>>> because power management commands are submitted to a logical unit 
>>> (WLUN). Hence, it happens often that the block layer "active queue" 
>>> counter is equal to 2 while only one logical unit is being used 
>>> actively (a logical unit backed by NAND flash). The performance 
>>> difference between queue depths 16 and 32 for UFS devices is 
>>> significant.
>>
>> We meet similiar problem before, but I think remove tag fair sharing
>> might cause some problems, because get tag is not fair currently, for
>> example 2 devices share 32 tag, while device a issue large amount of
>> io concurrently, and device b only issue one io, in this case, if fair
>> tag sharing is removed, device b can get bad io latency.
>>
>> By the way, I tried to propose a way to workaround this by following:
>>
>> 1) disable fair tag sharing untill get tag found no tag is avaiable;
>> 2) enable fair tag sharing again if the disk donesn't faild to get tag
>> for a period of time;
>>
>> Can this approch be considered?
> 
> I'm afraid that this approach won't help for the UFS driver since it is 
> likely that all tags are in use by a single logical unit during an IOPS 
> test. Hence, fair sharing would be enabled even when we don't want it to 
> be enabled.

It's right my original method is not flexible.

> 
> I propose that we switch to one of these two approaches:

How about a smoothing method that the device with more io will share
more tag, and each device will get at least one tag?

Thanks,
Kuai

> * Either remove the fair tag sharing code entirely and rely on the 
> fairness mechanism provided by the sbitmap code. I'm referring to how 
> __sbitmap_queue_wake_up() uses the wake_index member variable.
> * Or make the behavior of the fairness algorithm configurable from user 
> space. One possible approach is to make the proportion of tags for a 
> logical unit / NVMe namespace configurable via sysfs. This will allow to 
> reduce the number of tags for the WLUN of UFS devices.
> 
> Thanks,
> 
> Bart.
> 
> 
> .
>