Message-ID: <35f818e2cf3f15b7dc7028f543e0eea94c799efd.camel@infradead.org>
Date: Mon, 12 May 2025 19:02:20 -0700
From: David Woodhouse <dwmw2@...radead.org>
To: Andrew Morton <akpm@...ux-foundation.org>, Alexander Graf
 <graf@...zon.com>
Cc: kexec@...ts.infradead.org, linux-kernel@...r.kernel.org, Baoquan He
	 <bhe@...hat.com>, Pasha Tatashin <pasha.tatashin@...een.com>, 
	nh-open-source@...zon.com
Subject: Re: [PATCH] kexec: Enable CMA based contiguous allocation

On Mon, 2025-05-12 at 16:59 -0700, Andrew Morton wrote:
> On Mon, 12 May 2025 22:57:52 +0000 Alexander Graf <graf@...zon.com> wrote:
> 
> > When booting a new kernel with kexec_file, the kernel picks a target
> > location that the kernel should live at, then allocates random pages,
> > checks whether any of those patches magically happens to coincide with
> > a target address range and if so, uses them for that range.
> > 
> > For every page allocated this way, it then creates a page list that the
> > relocation code - code that executes while all CPUs are off and we are
> > just about to jump into the new kernel - copies to their final memory
> > location. We can not put them there before, because chances are pretty
> > good that at least some page in the target range is already in use by
> > the currently running Linux environment.
> > 
> > All of this is inefficient.
> > 
> > Since kexec got introduced, Linux has gained the CMA framework which
> > can perform physically contiguous memory mappings, while keeping that
> > memory available for movable memory when it is not needed for contiguous
> > allocations. The default CMA allocator is for DMA allocations.
> > 
> > This patch adds logic to the kexec file loader to attempt to place the
> > target payload at a location allocated from CMA. If successful, it uses
> > that memory range directly instead of creating copy instructions during
> > the hot phase. To ensure that there is a safety net in case anything goes
> > wrong with the CMA allocation, it also adds a flag for user space to force
> > disable CMA allocations.
> > 
> > Using CMA allocations has two advantages:
> > 
> >   1) Faster. There is no more need to copy in the hot phase.
> 
> How much faster?  Kinda matters as "fast" is the whole point of the patch!
> 
> >   2) More robust. Even if by accident some page is still in use for DMA,
> >      the new kernel image will be safe from that access because it resides
> >      in a memory region that is considered allocated in the old kernel and
> >      has a chance to reinitialize that component.
> 
> 
https://lore.kernel.org/all/20250512140909.3464-1-dssauerw@amazon.de/> Is this known to be a problem in current code?

Oh $DEITY yes. The Arm Generic Interrupt Controller is, to quote a dear
friend, "a cautionary tale of how not to approach a hardware design".

It does a whole bunch of arbitrary DMA all over the place, and doesn't
even live behind an IOMMU. And doesn't *stop* doing DMA unless you ask
it *really* nicely; merely shutting down the offending high-level
components isn't always enough, because they might still to write back
some caches.

Here's one of the latest examples (not actually the one which has been
breaking kexec for us, as far as we know, but an example of the genre):
https://lore.kernel.org/all/20250512140909.3464-1-dssauerw@amazon.de/

So putting the new kernel into a physical memory region which was
considered 'free' by the previous kernel, as Alex explains, is actually
a very good defence-in-depth mechanism to protect against such issues.


Download attachment "smime.p7s" of type "application/pkcs7-signature" (5069 bytes)

Powered by blists - more mailing lists