[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <20241201223920.GA32487@ranerica-svr.sc.intel.com>
Date: Sun, 1 Dec 2024 14:39:21 -0800
From: Ricardo Neri <ricardo.neri-calderon@...ux.intel.com>
To: Radu Rendec <rrendec@...hat.com>
Cc: x86@...nel.org, Andreas Herrmann <aherrmann@...e.com>,
Catalin Marinas <catalin.marinas@....com>,
Chen Yu <yu.c.chen@...el.com>, Len Brown <len.brown@...el.com>,
Pierre Gondois <Pierre.Gondois@....com>, Pu Wen <puwen@...on.cn>,
"Rafael J. Wysocki" <rafael.j.wysocki@...el.com>,
Sudeep Holla <sudeep.holla@....com>,
Srinivas Pandruvada <srinivas.pandruvada@...ux.intel.com>,
Will Deacon <will@...nel.org>, Zhang Rui <rui.zhang@...el.com>,
Nikolay Borisov <nik.borisov@...e.com>,
Huang Ying <ying.huang@...el.com>,
Ricardo Neri <ricardo.neri@...el.com>, linux-kernel@...r.kernel.org
Subject: Re: [PATCH v8 1/2] cacheinfo: Allocate memory during CPU hotplug if
not done from the primary CPU
On Fri, Nov 29, 2024 at 11:12:52AM -0500, Radu Rendec wrote:
> On Wed, 2024-11-27 at 16:22 -0800, Ricardo Neri wrote:
> > Commit 5944ce092b97 ("arch_topology: Build cacheinfo from primary CPU")
> > adds functionality that architectures can use to optionally allocate and
> > build cacheinfo early during boot. Commit 6539cffa9495 ("cacheinfo: Add
> > arch specific early level initializer") lets secondary CPUs correct (and
> > reallocate memory) cacheinfo data if needed.
> >
> > If the early build functionality is not used and cacheinfo does not need
> > correction, memory for cacheinfo is never allocated. x86 does not use the
> > early build functionality. Consequently, during the cacheinfo CPU hotplug
> > callback, last_level_cache_is_valid() attempts to dereference a NULL
> > pointer:
> >
> > BUG: kernel NULL pointer dereference, address: 0000000000000100
> > #PF: supervisor read access in kernel mode
> > #PF: error_code(0x0000) - not present page
> > PGD 0 P4D 0
> > Oops: 0000 [#1] PREEPMT SMP NOPTI
> > CPU: 0 PID 19 Comm: cpuhp/0 Not tainted 6.4.0-rc2 #1
> > RIP: 0010: last_level_cache_is_valid+0x95/0xe0a
> >
> > Allocate memory for cacheinfo during the cacheinfo CPU hotplug callback if
> > not done earlier.
> >
> > Moreover, before determining the validity of the last-level cache info,
> > ensure that it has been allocated. Simply checking for non-zero
> > cache_leaves() is not sufficient, as some architectures (e.g., Intel
> > processors) have non-zero cache_leaves() before allocation.
> >
> > Dereferencing NULL cacheinfo can occur in update_per_cpu_data_slice_size().
> > This function iterates over all online CPUs. However, a CPU may have come
> > online recently, but its cacheinfo may not have been allocated yet.
> >
> > While here, remove an unnecessary indentation in allocate_cache_info().
> >
> > Reviewed-by: Andreas Herrmann <aherrmann@...e.de>
> > Reviewed-by: Nikolay Borisov <nik.borisov@...e.com>
> > Reviewed-by: Radu Rendec <rrendec@...hat.com>
> > Reviewed-by: Sudeep Holla <sudeep.holla@....com>
> > Tested-by: Andreas Herrmann <aherrmann@...e.de>
> > Fixes: 6539cffa9495 ("cacheinfo: Add arch specific early level initializer")
> > Signed-off-by: Ricardo Neri <ricardo.neri-calderon@...ux.intel.com>
> > ---
> > Cc: Andreas Herrmann <aherrmann@...e.com>
> > Cc: Catalin Marinas <catalin.marinas@....com>
> > Cc: Chen Yu <yu.c.chen@...el.com>
> > Cc: Huang Ying <ying.huang@...el.com>
> > Cc: Len Brown <len.brown@...el.com>
> > Cc: Nikolay Borisov <nik.borisov@...e.com>
> > Cc: Radu Rendec <rrendec@...hat.com>
> > Cc: Pierre Gondois <Pierre.Gondois@....com>
> > Cc: Pu Wen <puwen@...on.cn>
> > Cc: "Rafael J. Wysocki" <rafael.j.wysocki@...el.com>
> > Cc: Sudeep Holla <sudeep.holla@....com>
> > Cc: Srinivas Pandruvada <srinivas.pandruvada@...ux.intel.com>
> > Cc: Will Deacon <will@...nel.org>
> > Cc: Zhang Rui <rui.zhang@...el.com>
> > Cc: linux-arm-kernel@...ts.infradead.org
> > Cc: stable@...r.kernel.org # 6.3+
> > ---
> > Change since v7:
> > * None
> >
> > Changes since v6:
> > * Merged patches 1 and 2 of v6 into one. (Borislav)
> > * Merged the history of patches 1 and 2ino this patch.
> > * Kept the Reviewed-by and Tested-by tags from the two merged patches.
> > * Fixed a formatting issue in allocate_cache_info(). (Borislav)
> >
> > Changes since v5:
> > * Fixed nonsensical subject (Nikolay).
> > * Added Reviewed-by and Tested-by tags from Andreas. Thanks!
> > * Added Reviewed-by tag from Nikolay. Thanks!
> >
> > Changes since v4:
> > * Combined checks for per_cpu_cacheinfo() and cache_leaves() in a single
> > line. (Sudeep)
> > * Added Reviewed-by tag from Sudeep. Thanks!
> >
> > Changes since v3:
> > * Added Reviewed-by tag from Radu and Sudeep. Thanks!
> >
> > Changes since v2:
> > * Introduced this patch.
> >
> > Changes since v1:
> > * N/A
> >
> > ---
> > The motivation for commit 5944ce092b97 was to prevent a BUG splat in
> > PREEMPT_RT kernels during memory allocation. This splat is not observed on
> > x86 because the memory allocation for cacheinfo happens in
> > detect_cache_attributes() from the cacheinfo CPU hotplug callback.
> >
> > The dereference of a NULL cacheinfo is not observed today because
> > cache_leaves(cpu) is zero until after init_cache_level() is called
> > (during the CPU hotplug callback). A subsequent changeset will set
> > the number of cache leaves earlier and the NULL-pointer dereference
> > will be observed.
> > ---
> > drivers/base/cacheinfo.c | 11 +++++++----
> > 1 file changed, 7 insertions(+), 4 deletions(-)
> >
> > diff --git a/drivers/base/cacheinfo.c b/drivers/base/cacheinfo.c
> > index 7a7609298e18..a1afc478e0e8 100644
> > --- a/drivers/base/cacheinfo.c
> > +++ b/drivers/base/cacheinfo.c
> > @@ -58,7 +58,7 @@ bool last_level_cache_is_valid(unsigned int cpu)
> > {
> > struct cacheinfo *llc;
> >
> > - if (!cache_leaves(cpu))
> > + if (!cache_leaves(cpu) || !per_cpu_cacheinfo(cpu))
> > return false;
> >
> > llc = per_cpu_cacheinfo_idx(cpu, cache_leaves(cpu) - 1);
> > @@ -466,8 +466,7 @@ int __weak populate_cache_leaves(unsigned int cpu)
> > static inline
> > int allocate_cache_info(int cpu)
> > {
> > - per_cpu_cacheinfo(cpu) = kcalloc(cache_leaves(cpu),
> > - sizeof(struct cacheinfo), GFP_ATOMIC);
> > + per_cpu_cacheinfo(cpu) = kcalloc(cache_leaves(cpu), sizeof(struct cacheinfo), GFP_ATOMIC);
> > if (!per_cpu_cacheinfo(cpu)) {
> > cache_leaves(cpu) = 0;
> > return -ENOMEM;
> > @@ -539,7 +538,11 @@ static inline int init_level_allocate_ci(unsigned int cpu)
> > */
> > ci_cacheinfo(cpu)->early_ci_levels = false;
> >
> > - if (cache_leaves(cpu) <= early_leaves)
> > + /*
> > + * Some architectures (e.g., x86) do not use early initialization.
> > + * Allocate memory now in such case.
> > + */
> > + if (cache_leaves(cpu) <= early_leaves && per_cpu_cacheinfo(cpu))
> > return 0;
> >
> > kfree(per_cpu_cacheinfo(cpu));
>
> Since Borislav explicitly said you were not supposed to keep the tags,
> I reviewed it again, and I'm OK to keep mine. Thanks!
>
> Reviewed-by: Radu Rendec <rrendec@...hat.com>
Thank you!
Powered by blists - more mailing lists