linux-kernel - RE: [PATCH 1/1] swiotlb: Track and report io_tlb

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <BYAPR21MB1688F17200D973E799E88C31D7869@BYAPR21MB1688.namprd21.prod.outlook.com>
Date:   Wed, 22 Mar 2023 14:52:42 +0000
From:   "Michael Kelley (LINUX)" <mikelley@...rosoft.com>
To:     Petr Tesarik <petrtesarik@...weicloud.com>,
        "hch@....de" <hch@....de>,
        "m.szyprowski@...sung.com" <m.szyprowski@...sung.com>,
        "robin.murphy@....com" <robin.murphy@....com>,
        Dexuan Cui <decui@...rosoft.com>,
        Tianyu Lan <Tianyu.Lan@...rosoft.com>,
        "iommu@...ts.linux.dev" <iommu@...ts.linux.dev>,
        "linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>
Subject: RE: [PATCH 1/1] swiotlb: Track and report io_tlb_used high water mark
 in debugfs

From: Petr Tesarik <petrtesarik@...weicloud.com> Sent: Wednesday, March 22, 2023 7:33 AM
> 
> On 2/27/2023 5:41 PM, Michael Kelley wrote:
> > swiotlb currently reports the total number of slabs and the instantaneous
> > in-use slabs in debugfs. But with increased usage of swiotlb for all I/O
> > in Confidential Computing (coco) VMs, it has become difficult to know
> > how much memory to allocate for swiotlb bounce buffers, either via the
> > automatic algorithm in the kernel or by specifying a value on the
> > kernel boot line. The current automatic algorithm generously allocates
> > swiotlb bounce buffer memory, and may be wasting significant memory in
> > many use cases.
> 
> I like the idea. Tracking and exposing the high watermark is definitely a step in the right
> direction. I would also appreciate an indicator of fragmentation, but that can be
> implemented later.
> 
> However, you apparently have a scenario where swiotlb is used intensely. Are you able
> to measure swiotlb performance? If yes, can you suggest a suitable method? I'm asking
> for my work on making swiotlb more flexible (able to grow and shrink).
> 

Yes, in the CoCo VM scenario, all DMA transfers must use bounce buffers
because the hardware doesn't support I/O devices doing DMA to encrypted
memory.  Our measurements have primarily been of disk I/O bandwidth
and latency using 'fio'.  We compare the measurements in a normal VM
vs. a CoCo VM, with the delta mostly being the overhead of doing the
bounce buffering.  The delta includes the cost of swiotlb allocate/free,
plus the data copying.  Admittedly, this is a more macro-level measurement,
but it is directly relevant because it's ultimately the impact that users of
CoCo VMs see.  We have not done micro-level measurements of just the
swiotlb allocate/free functions.

Making the swiotlb able to grow and shrink would likely be very desirable
for CoCo VMs.  I'll look forward to seeing how that comes out. :-)

Michael