linux-kernel - Re: [PATCH] ARM64: Dynamically allocate cpumasks and increase supported CPUs to 512

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date: Tue, 16 Jan 2024 13:06:33 -0800
From: Eric Mackay <eric.mackay@...cle.com>
To: mark.rutland@....com
Cc: Jonathan.Cameron@...wei.com, Matteo.Carlini@....com,
        Valentin.Schneider@....com, akpm@...ux-foundation.org,
        anshuman.khandual@....com, catalin.marinas@....com, cl@...ux.com,
        dave.kleikamp@...cle.com, linux-arm-kernel@...ts.infradead.org,
        linux-kernel@...r.kernel.org, linux-mm@...ck.org,
        linux@...linux.org.uk, robin.murphy@....com,
        vanshikonda@...amperecomputing.com, yang@...amperecomputing.com
Subject: Re: [PATCH] ARM64: Dynamically allocate cpumasks and increase supported CPUs to 512

On Tue, Jan 16, 2024 1:08:20PM +0000, Mark Rutland wrote:
> On Mon, Jan 15, 2024 at 03:39:00PM +0000, Russell King (Oracle) wrote:
> > On Thu, Dec 14, 2023 at 04:05:56PM -0800, Christoph Lameter (Ampere) wrote:
> > > +# Determines the placement of cpumasks.
> > > +#
> > > +# With CPUMASK_OFFSTACK the cpumasks are dynamically allocated.
> > > +# Useful for machines with lots of core because it avoids increasing
> > > +# the size of many of the data structures in the kernel.
> > > +#
> > > +# If this is off then the cpumasks have a static sizes and are
> > > +# embedded within data structures.
> > > +#
> > > +config CPUMASK_OFFSTACK
> > > +	def_bool y
> > > +	depends on NR_CPUS > 256
> > 
> > Should that be ">= 256" ?
> 
> I don't think that ">= 256" makes sense. Note that since the cpumasks are
> arrays of unsigned long, they're chunked into groups of 64 bits:
> 
>     2 to  64 cpus:  1 x unsigned long =>  8 bytes
>    65 to 128 cpus:  2 x unsigned long => 16 bytes
>   129 to 192 cpus:  3 x unsigned long => 24 bytes
>   193 to 256 cpus:  4 x unsigned long => 32 bytes
>   257 to 320 cpus:  5 x unsigned long => 40 bytes
> 
> ... and so if a mask for 256 CPUs is too big to go in the stack, so is any mask
> for 193+ CPUs, and so ">= 256" should be clamped down to ">= 193" or "> 192".
> The boundary should be just after a multiple of 64.
> 
> How did we choose 256 specifically? I note that x86-64 allows 512 CPUs before
> requiring CPUMASK_OFFSTACK, and I see that powerpc selects CPUMASK_OFFSTACK
> when NR_CPUS >= 8192.
> 
> Mark.

The suggestion for >= 256 may have been a zero-index/one-index mixup.

It seems > 256 was chosen as the cutoff simply because it preserves existing behavior.
The patch description seems to imply there was pushback from distro maintainers on just increasing
the default NR_CPUS.

The existing default value of 256 is probably already a strain for smaller ARM systems, to which
x86-64 isn't a reasonable comparison. I'm not sure what the reaction to increasing from 64 to 256
in 2019 was like, but picking a pivot point for CPUMASK_OFFSTACK beyond 256 may skew the balance
even less in favor of smaller ARM systems.