Date: Mon, 6 May 2024 13:16:45 -0700
From: "T.J. Mercier" <tjmercier@...gle.com>
To: Michael Kelley <mhklinux@...look.com>
Cc: Petr Tesařík <petr@...arici.cz>, 
	Jonathan Corbet <corbet@....net>, Thomas Gleixner <tglx@...utronix.de>, Ingo Molnar <mingo@...hat.com>, 
	Borislav Petkov <bp@...en8.de>, Dave Hansen <dave.hansen@...ux.intel.com>, 
	"x86@...nel.org" <x86@...nel.org>, "H. Peter Anvin" <hpa@...or.com>, "hch@...radead.org" <hch@...radead.org>, 
	"robin.murphy@....com" <robin.murphy@....com>, "joro@...tes.org" <joro@...tes.org>, 
	"will@...nel.org" <will@...nel.org>, "akpm@...ux-foundation.org" <akpm@...ux-foundation.org>, 
	"isaacmanjarres@...gle.com" <isaacmanjarres@...gle.com>, "iommu@...ts.linux.dev" <iommu@...ts.linux.dev>, 
	"linux-doc@...r.kernel.org" <linux-doc@...r.kernel.org>, 
	"linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>
Subject: Re: [PATCH] swiotlb: iommu/dma: Clarify swiotlb options apply only to dma-direct

On Mon, May 6, 2024 at 12:58 PM Michael Kelley <mhklinux@...look.com> wrote:
>
> From: T.J. Mercier <tjmercier@...gle.com> Sent: Monday, May 6, 2024 10:23 AM
> >
> > On Sat, May 4, 2024 at 1:16 AM Petr Tesařík <petr@...arici.cz> wrote:
> > >
> > > On Fri,  3 May 2024 18:35:26 +0000
> > > "T.J. Mercier" <tjmercier@...gle.com> wrote:
> > >
> > > > IOMMU implementations now sometimes bounce memory through SWIOTLB to
> > > > achieve cacheline alignment [1], or prevent DMA attacks by untrusted
> > > > devices [2]. These uses of SWIOTLB differ conceptually from historical
> > > > use which was a solution to the problem of device addressing
> > > > limitations that prevent DMA to some portion of system memory
> > > > (typically beyond 4 GiB). IOMMUs also solve the problem of device
> > > > addressing limitations and therefore eliminate the need for SWIOTLB for
> > > > that purpose. However as mentioned, IOMMUs can use SWIOTLB for other
> > > > purposes.
> > > >
> > > > The swiotlb kernel command line parameter does not impact IOMMU related
> > > > use of SWIOTLB, and that is intentional. IOMMUs cannot be forced to use
> > > > SWIOTLB for all buffers. Update the documentation for the swiotlb
> > > > parameter to clarify that SWIOTLB use can only be forced in scenarios
> > > > where an IOMMU is not involved.
> > > >
> > > > [1] https://lore.kernel.org/all/20230612153201.554742-16-catalin.marinas@arm.com/
> > > > [2] https://lore.kernel.org/all/20190906061452.30791-1-baolu.lu@linux.intel.com/
> > > > Signed-off-by: T.J. Mercier <tjmercier@...gle.com>
> > > > ---
> > > >  Documentation/admin-guide/kernel-parameters.txt | 1 +
> > > >  Documentation/arch/x86/x86_64/boot-options.rst  | 2 +-
> > > >  2 files changed, 2 insertions(+), 1 deletion(-)
> > > >
> > > > diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
> > > > index 213d0719e2b7..84c582ac246c 100644
> > > > --- a/Documentation/admin-guide/kernel-parameters.txt
> > > > +++ b/Documentation/admin-guide/kernel-parameters.txt
> > > > @@ -6486,6 +6486,7 @@
> > > >                                to a power of 2.
> > > >                       force -- force using of bounce buffers even if they
> > > >                                wouldn't be automatically used by the kernel
> > > > +                              where a hardware IOMMU is not involved
> > > >                       noforce -- Never use bounce buffers (for debugging)
> > > >
> > > >       switches=       [HW,M68k,EARLY]
> > >
> > > Yes, this part is correct. SWIOTLB cannot be forced if there is an IOMMU.
> > >
> > > > diff --git a/Documentation/arch/x86/x86_64/boot-options.rst b/Documentation/arch/x86/x86_64/boot-options.rst
> > > > index 137432d34109..066b4bc81583 100644
> > > > --- a/Documentation/arch/x86/x86_64/boot-options.rst
> > > > +++ b/Documentation/arch/x86/x86_64/boot-options.rst
> > > > @@ -285,7 +285,7 @@ iommu options only relevant to the AMD GART hardware IOMMU:
> > > >        Always panic when IOMMU overflows.
> > > >
> > > >  iommu options only relevant to the software bounce buffering (SWIOTLB) IOMMU
> > > > -implementation:
> > > > +implementation where a hardware IOMMU is not involved:
> > > >
> > > >      swiotlb=<slots>[,force,noforce]
> > > >        <slots>
> > >
> > > But this part needs some improvement. The "swiotlb" option is not
> > > entirely ignored if there is a hardware IOMMU. For example, the size of
> > > the SWIOTLB can be adjusted using "swiotlb=<slots>", and since SWIOTLB
> > > can be used by IOMMUs for other purposes (as you correctly note in the
> > > commit message), this setting is relevant even where a hardware IOMMU
> > > is involved.
> > >
> > > Petr T
> >
> > Thanks. I think I should also update the commit message:
> > "The swiotlb=force kernel command line parameter does not impact IOMMU
> > related use of SWIOTLB"
> > and title:
> > "Clarify swiotlb=force option applies only to dma-direct"
> >
> > As for fixing boot-options.txt, I think it makes the most sense to
> > expand on just the force option rather than the section summary like
> > above:
> >        force
> >          Force all IO through the software TLB.
> > +        Hardware IOMMU implementations can use SWIOTLB bounce buffering in
> > +        some circumstances, but they cannot be forced to always use them, so
> > +        this option only has an effect when no hardware IOMMU is involved.
> >        noforce
>
> Note also that the documentation for swiotlb= in boot-options.rst is somewhat
> out-of-date.  It doesn't have the optional second integer parameter to specify
> the number of "areas" that have their own lock.  Perhaps that could be fixed
> at the same time?
>
> Michael

Thanks, I could add this as a second patch.

I also noticed that several of the iommu options (soft, allowed,
nofullflush, panic) are not listed in the set of possible options at
the top of the IOMMU section. Rebooting is the only other section that
does that, so I'm wondering if we're better off getting rid of it to
be like other sections.

Powered by blists - more mailing lists