[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <8734lgpuoi.fsf@yhuang6-desk2.ccr.corp.intel.com>
Date: Tue, 01 Oct 2024 07:51:09 +0800
From: "Huang, Ying" <ying.huang@...el.com>
To: David Hildenbrand <david@...hat.com>
Cc: Thomas Gleixner <tglx@...utronix.de>, Ingo Molnar <mingo@...hat.com>,
Borislav Petkov <bp@...en8.de>, Dave Hansen
<dave.hansen@...ux.intel.com>, "Kirill A . Shutemov"
<kirill.shutemov@...ux.intel.com>, x86@...nel.org, Andrew Morton
<akpm@...ux-foundation.org>, Oscar Salvador <osalvador@...e.de>,
linux-coco@...ts.linux.dev, linux-mm@...ck.org,
linux-kernel@...r.kernel.org, Dan Williams <dan.j.williams@...el.com>,
Kai Huang <kai.huang@...el.com>, "H. Peter Anvin" <hpa@...or.com>, Andy
Lutomirski <luto@...nel.org>
Subject: Re: [PATCH] tdx, memory hotplug: Check whole hot-adding memory
range for TDX
Hi, David,
Thanks a lot for comments!
David Hildenbrand <david@...hat.com> writes:
> On 30.09.24 07:51, Huang Ying wrote:
>> On systems with TDX (Trust Domain eXtensions) enabled, memory ranges
>> hot-added must be checked for compatibility by TDX. This is currently
>> implemented through memory hotplug notifiers for each memory_block.
>> If a memory range which isn't TDX compatible is hot-added, for
>> example, some CXL memory, the command line as follows,
>> $ echo 1 > /sys/devices/system/node/nodeX/memoryY/online
>> will report something like,
>> bash: echo: write error: Operation not permitted
>> If pr_debug() is enabled, the error message like below will be shown
>> in the kernel log,
>> online_pages [mem 0xXXXXXXXXXX-0xXXXXXXXXXX] failed
>> Both are too general to root cause the problem. This will confuse
>> users. One solution is to print some error messages in the TDX memory
>> hotplug notifier. However, memory hotplug notifiers are called for
>> each memory block, so this may lead to a large volume of messages in
>> the kernel log if a large number of memory blocks are onlined with a
>> script or automatically. For example, the typical size of memory
>> block is 128MB on x86_64, when online 64GB CXL memory, 512 messages
>> will be logged.
>
> ratelimiting would likely help here a lot, but I agree that it is
> suboptimal.
>
>> Therefore, in this patch, the whole hot-adding memory range is
>> checked
>> for TDX compatibility through a newly added architecture specific
>> function (arch_check_hotplug_memory_range()). If rejected, the memory
>> hot-adding will be aborted with a proper kernel log message. Which
>> looks like something as below,
>> virt/tdx: Reject hot-adding memory range: 0xXXXXXXXX-0xXXXXXXXX
>> for TDX compatibility.
>> > The target use case is to support CXL memory on TDX enabled systems.
>> If the CXL memory isn't compatible with TDX, the whole CXL memory
>> range hot-adding will be rejected. While the CXL memory can still be
>> used via devdax interface.
>
> I'm curious, why can that memory be used through devdax but not
> through the buddy? I'm probably missing something important :)
Because only TDX compatible memory can be used for TDX guest. The buddy
is used to allocate memory for TDX guest. While devdax will not be used
for that.
>> This also makes the original TDX memory hotplug notifier useless, so
>> delete it.
>
> The online-notifier would even be too late when used with the
> memmap-on-memory feature I assume, as we might be touching that memory
> even before being able to call memory online notifiers.
This should be OK. Because we will not use the memory for TDX guest in
this way.
> One way to handle that would be to switch to the MEM_PREPARE_ONLINE
> notifier, but it's still called per-memory block.
>
> Nothing jumped at me, so
>
> Acked-by: David Hildenbrand <david@...hat.com>
Thank you very much!
--
Best Regards,
Huang, Ying
Powered by blists - more mailing lists