Message-ID: <07b21458-832f-4b15-9bc8-43f21f902e34@amazon.com>
Date: Tue, 22 Jul 2025 00:01:25 +0200
From: Alexander Graf <graf@...zon.com>
To: Pasha Tatashin <pasha.tatashin@...een.com>
CC: <kexec@...ts.infradead.org>, <linux-kernel@...r.kernel.org>,
	<nh-open-source@...zon.com>, Baoquan He <bhe@...hat.com>, Zhongkun He
	<hezhongkun.hzk@...edance.com>, Andrew Morton <akpm@...ux-foundation.org>
Subject: Re: [PATCH v5] kexec: Enable CMA based contiguous allocation


On 10.06.25 13:31, Pasha Tatashin wrote:
> On Tue, Jun 10, 2025 at 4:53 AM Alexander Graf <graf@...zon.com> wrote:
>> When booting a new kernel with kexec_file, the kernel picks a target
>> location that the kernel should live at, then allocates random pages,
>> checks whether any of those patches magically happens to coincide with
>> a target address range and if so, uses them for that range.
>>
>> For every page allocated this way, it then creates a page list that the
>> relocation code - code that executes while all CPUs are off and we are
>> just about to jump into the new kernel - copies to their final memory
>> location. We can not put them there before, because chances are pretty
>> good that at least some page in the target range is already in use by
>> the currently running Linux environment. Copying is happening from a
>> single CPU at RAM rate, which takes around 4-50 ms per 100 MiB.
>>
>> All of this is inefficient and error prone.
>>
>> To successfully kexec, we need to quiesce all devices of the outgoing
>> kernel so they don't scribble over the new kernel's memory. We have seen
>> cases where that does not happen properly (*cough* GIC *cough*) and hence
>> the new kernel was corrupted. This started a month long journey to root
>> cause failing kexecs to eventually see memory corruption, because the new
>> kernel was corrupted severely enough that it could not emit output to
>> tell us about the fact that it was corrupted. By allocating memory for the
>> next kernel from a memory range that is guaranteed scribbling free, we can
>> boot the next kernel up to a point where it is at least able to detect
>> corruption and maybe even stop it before it becomes severe. This increases
>> the chance for successful kexecs.
>>
>> Since kexec got introduced, Linux has gained the CMA framework which
>> can perform physically contiguous memory mappings, while keeping that
>> memory available for movable memory when it is not needed for contiguous
>> allocations. The default CMA allocator is for DMA allocations.
>>
>> This patch adds logic to the kexec file loader to attempt to place the
>> target payload at a location allocated from CMA. If successful, it uses
>> that memory range directly instead of creating copy instructions during
>> the hot phase. To ensure that there is a safety net in case anything goes
>> wrong with the CMA allocation, it also adds a flag for user space to force
>> disable CMA allocations.
>>
>> Using CMA allocations has two advantages:
>>
>>    1) Faster by 4-50 ms per 100 MiB. There is no more need to copy in the
>>       hot phase.
>>    2) More robust. Even if by accident some page is still in use for DMA,
>>       the new kernel image will be safe from that access because it resides
>>       in a memory region that is considered allocated in the old kernel and
>>       has a chance to reinitialize that component.
>>
>> Signed-off-by: Alexander Graf <graf@...zon.com>
>> Acked-by: Baoquan He <bhe@...hat.com>
> Reviewed-by: Pasha Tatashin <pasha.tatashin@...een.com>


Andrew, I don't see this patch in linus/master. Is it still in your 
queue? :)

Alex




Amazon Web Services Development Center Germany GmbH
Tamara-Danz-Str. 13
10243 Berlin
Geschaeftsfuehrung: Christian Schlaeger, Jonathan Weiss
Eingetragen am Amtsgericht Charlottenburg unter HRB 257764 B
Sitz: Berlin
Ust-ID: DE 365 538 597

Powered by blists - more mailing lists