linux-kernel - Re: 6.13/regression/bisected

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-Id: <D8OFRDU1AVWC.1G6DIBZPLJ78@bsdbackstore.eu>
Date: Mon, 24 Mar 2025 11:48:39 +0100
From: "Maurizio Lombardi" <mlombard@...backstore.eu>
To: "Mikhail Gavrilov" <mikhail.v.gavrilov@...il.com>, "Keith Busch"
 <kbusch@...nel.org>
Cc: "Christoph Hellwig" <hch@....de>, <axboe@...nel.dk>,
 <linux-nvme@...ts.infradead.org>, "Linux List Kernel Mailing"
 <linux-kernel@...r.kernel.org>, "Linux regressions mailing list"
 <regressions@...ts.linux.dev>
Subject: Re: 6.13/regression/bisected - new nvme timeout errors

On Sun Mar 23, 2025 at 10:49 AM CET, Mikhail Gavrilov wrote:
> On Thu, Mar 6, 2025 at 8:19 PM Keith Busch <kbusch@...nel.org> wrote:
>>
>> >
>> >     nvme-pci: reverse request order in nvme_queue_rqs
>>
>> The patch here uses the order recieved to dispatch commands in
>> consequetive submission queue entries, which is supposed to be the
>> desired behavior for any device. I did some testing on mailine, and it
>> sure looks like the order the driver does this is optimal, so I'm not
>> sure what's going on with your observation.
>>
>> Do you have a scheduler enabled on your device?
>>
>> How are you generating IO? Is it a pattern I should be able to replicate
>> with 'fio'?
>
> Sorry for not answering for a long time, but all because I was testing
> the off-list patch that was sent to me by Maurizio Lombardi.
>
> diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
> index 950289405ef2..990d9ef1eef9 100644
> --- a/drivers/nvme/host/pci.c
> +++ b/drivers/nvme/host/pci.c
> @@ -989,6 +989,9 @@ static void nvme_submit_cmds(struct nvme_queue
> *nvmeq, struct rq_list *rqlist)
>  {
>         struct request *req;
>
> +       if (rq_list_empty(rqlist))
> +               return;
> +
>         spin_lock(&nvmeq->sq_lock);
>         while ((req = rq_list_pop(rqlist))) {
>                 struct nvme_iod *iod = blk_mq_rq_to_pdu(req);
>
> And I want to say right away that this patch solved my problem. I
> double checked, and both times the system worked for 4 days without
> errors:
>

Glad to hear it worked!

Keith, I think the problem here is simply that nvme_write_sq_db() ends up
being called even when the rqlist is empty and this couldn't happen before
the changes introduced by commit "nvme-pci: reverse request order in
nvme_queue_rqs".

If you say that this change makes sense I can submit a formal patch.

Maurizio