linux-kernel - Re: [RFC RESEND 16/16] nvme-pci: use blk_rq_dma

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <06787e6a-4e78-4524-960d-ec24b9f38191@kernel.dk>
Date: Tue, 5 Mar 2024 09:08:43 -0700
From: Jens Axboe <axboe@...nel.dk>
To: Keith Busch <kbusch@...nel.org>, Leon Romanovsky <leon@...nel.org>
Cc: Christoph Hellwig <hch@....de>, Robin Murphy <robin.murphy@....com>,
 Marek Szyprowski <m.szyprowski@...sung.com>, Joerg Roedel <joro@...tes.org>,
 Will Deacon <will@...nel.org>, Jason Gunthorpe <jgg@...pe.ca>,
 Chaitanya Kulkarni <chaitanyak@...dia.com>,
 Chaitanya Kulkarni <kch@...dia.com>, Jonathan Corbet <corbet@....net>,
 Sagi Grimberg <sagi@...mberg.me>, Yishai Hadas <yishaih@...dia.com>,
 Shameer Kolothum <shameerali.kolothum.thodi@...wei.com>,
 Kevin Tian <kevin.tian@...el.com>,
 Alex Williamson <alex.williamson@...hat.com>,
 Jérôme Glisse <jglisse@...hat.com>,
 Andrew Morton <akpm@...ux-foundation.org>, linux-doc@...r.kernel.org,
 linux-kernel@...r.kernel.org, linux-block@...r.kernel.org,
 linux-rdma@...r.kernel.org, iommu@...ts.linux.dev,
 linux-nvme@...ts.infradead.org, kvm@...r.kernel.org, linux-mm@...ck.org,
 Bart Van Assche <bvanassche@....org>,
 Damien Le Moal <damien.lemoal@...nsource.wdc.com>,
 Amir Goldstein <amir73il@...il.com>,
 "josef@...icpanda.com" <josef@...icpanda.com>,
 "Martin K. Petersen" <martin.petersen@...cle.com>,
 "daniel@...earbox.net" <daniel@...earbox.net>,
 Dan Williams <dan.j.williams@...el.com>, "jack@...e.com" <jack@...e.com>,
 Leon Romanovsky <leonro@...dia.com>, Zhu Yanjun <zyjzyj2000@...il.com>
Subject: Re: [RFC RESEND 16/16] nvme-pci: use blk_rq_dma_map() for NVMe SGL

On 3/5/24 8:51 AM, Keith Busch wrote:
> On Tue, Mar 05, 2024 at 01:18:47PM +0200, Leon Romanovsky wrote:
>> @@ -236,7 +236,9 @@ struct nvme_iod {
>>  	unsigned int dma_len;	/* length of single DMA segment mapping */
>>  	dma_addr_t first_dma;
>>  	dma_addr_t meta_dma;
>> -	struct sg_table sgt;
>> +	struct dma_iova_attrs iova;
>> +	dma_addr_t dma_link_address[128];
>> +	u16 nr_dma_link_address;
>>  	union nvme_descriptor list[NVME_MAX_NR_ALLOCATIONS];
>>  };
> 
> That's quite a lot of space to add to the iod. We preallocate one for
> every request, and there could be millions of them. 

Yeah, that's just a complete non-starter. As far as I can tell, this
ends up adding 1052 bytes per request. Doing the quick math on my test
box (24 drives), that's just a smidge over 3GB of extra memory. That's
not going to work, not even close.

-- 
Jens Axboe