linux-kernel - Re: [PATCH] nvme: Check for readiness more quickly, to speed up boot time

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <BYAPR04MB5749363E3AC8C583F5CB076786E60@BYAPR04MB5749.namprd04.prod.outlook.com>
Date:   Sun, 1 Mar 2020 02:01:05 +0000
From:   Chaitanya Kulkarni <Chaitanya.Kulkarni@....com>
To:     Josh Triplett <josh@...htriplett.org>,
        Keith Busch <kbusch@...nel.org>, Jens Axboe <axboe@...com>,
        Christoph Hellwig <hch@....de>,
        Sagi Grimberg <sagi@...mberg.me>
CC:     "linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
        "linux-nvme@...ts.infradead.org" <linux-nvme@...ts.infradead.org>
Subject: Re: [PATCH] nvme: Check for readiness more quickly, to speed up boot
 time

Nit:- please have a look at the patch subject line and make
sure it is not exceeding the required length.

One question though, have you seen similar kind of performance 
improvements when system is booted ?

I took some numbers and couldn't see similar benefit. See [1] :-

Without :-

714.532560-714.456099 = .076461
721.189886-721.110845 = .079041
727.836938-727.765572 = .071366
734.589886-734.519779 = .070107
741.244296-741.173503 = .070793

With this patch :-

805.549656-805.461924 = .087732
812.199549-812.124364 = .075185
818.868111-818.793779 = .074332
825.636130-825.554311 = .081819
832.287043-832.205882 = .081161

Regards,
Chaitanya

[1] Detail log :-

Without this patch :-

[  714.456099] nvme_init 3133
[  714.458501] nvme nvme0: pci function 0000:61:00.0
[  714.532560] nvme nvme0: 32/0/0 default/read/poll queues
[  721.110845] nvme_init 3133
[  721.114112] nvme nvme0: pci function 0000:61:00.0
[  721.189886] nvme nvme0: 32/0/0 default/read/poll queues
[  727.765572] nvme_init 3133
[  727.767814] nvme nvme0: pci function 0000:61:00.0
[  727.836938] nvme nvme0: 32/0/0 default/read/poll queues
[  734.519779] nvme_init 3133
[  734.522099] nvme nvme0: pci function 0000:61:00.0
[  734.589886] nvme nvme0: 32/0/0 default/read/poll queues
[  741.173503] nvme_init 3133
[  741.176089] nvme nvme0: pci function 0000:61:00.0
[  741.244296] nvme nvme0: 32/0/0 default/read/poll queues

With this patch :-

[  805.461924] nvme_init 3133
[  805.464521] nvme nvme0: pci function 0000:61:00.0
[  805.549656] nvme nvme0: 32/0/0 default/read/poll queues
[  812.124364] nvme_init 3133
[  812.126975] nvme nvme0: pci function 0000:61:00.0
[  812.199549] nvme nvme0: 32/0/0 default/read/poll queues
[  818.793779] nvme_init 3133
[  818.796581] nvme nvme0: pci function 0000:61:00.0
[  818.868111] nvme nvme0: 32/0/0 default/read/poll queues
[  825.554311] nvme_init 3133
[  825.556551] nvme nvme0: pci function 0000:61:00.0
[  825.636130] nvme nvme0: 32/0/0 default/read/poll queues
[  832.205882] nvme_init 3133
[  832.208934] nvme nvme0: pci function 0000:61:00.0
[  832.287043] nvme nvme0: 32/0/0 default/read/poll queues


On 02/28/2020 06:52 PM, Josh Triplett wrote:
> After initialization, nvme_wait_ready checks for readiness every 100ms,
> even though the drive may be ready far sooner than that. This delays
> system boot by hundreds of milliseconds. Reduce the delay, checking for
> readiness every millisecond instead.
>
> Boot-time tests on an AWS c5.12xlarge:
>
> Before:
> [    0.546936] initcall nvme_init+0x0/0x5b returned 0 after 37 usecs
> ...
> [    0.764178] nvme nvme0: 2/0/0 default/read/poll queues
> [    0.768424]  nvme0n1: p1
> [    0.774132] EXT4-fs (nvme0n1p1): mounted filesystem with ordered data mode. Opts: (null)
> [    0.774146] VFS: Mounted root (ext4 filesystem) on device 259:1.
> ...
> [    0.788141] Run /sbin/init as init process
>
> After:
> [    0.537088] initcall nvme_init+0x0/0x5b returned 0 after 37 usecs
> ...
> [    0.543457] nvme nvme0: 2/0/0 default/read/poll queues
> [    0.548473]  nvme0n1: p1
> [    0.554339] EXT4-fs (nvme0n1p1): mounted filesystem with ordered data mode. Opts: (null)
> [    0.554344] VFS: Mounted root (ext4 filesystem) on device 259:1.
> ...
> [    0.567931] Run /sbin/init as init process
>
> Signed-off-by: Josh Triplett <josh@...htriplett.org>
> ---
>   drivers/nvme/host/core.c | 2 +-
>   1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
> index a4d8c90ee7cc..04174a40b9b0 100644
> --- a/drivers/nvme/host/core.c
> +++ b/drivers/nvme/host/core.c
> @@ -2074,7 +2074,7 @@ static int nvme_wait_ready(struct nvme_ctrl *ctrl, u64 cap, bool enabled)
>   		if ((csts & NVME_CSTS_RDY) == bit)
>   			break;
>
> -		msleep(100);
> +		usleep_range(1000, 2000);
>   		if (fatal_signal_pending(current))
>   			return -EINTR;
>   		if (time_after(jiffies, timeout)) {
>