linux-kernel - RE: [regression] linux-6.6.y, minmax: virtual memory exhausted in i586 chroot during kernel compilation

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date: Tue, 13 Feb 2024 15:13:10 +0000
From: David Laight <David.Laight@...LAB.COM>
To: Linux regressions mailing list <regressions@...ts.linux.dev>, Greg KH
	<gregkh@...uxfoundation.org>
CC: Sasha Levin <sashal@...nel.org>, "stable@...r.kernel.org"
	<stable@...r.kernel.org>, LKML <linux-kernel@...r.kernel.org>
Subject: RE: [regression] linux-6.6.y, minmax: virtual memory exhausted in
 i586 chroot during kernel compilation

From: Linux regression tracking (Thorsten Leemhuis)
> Sent: 13 February 2024 15:01
> 
> On 13.02.24 15:50, Greg KH wrote:
> > On Mon, Feb 12, 2024 at 05:16:58PM +0100, Linux regression tracking (Thorsten Leemhuis) wrote:
> >>
> >> I noticed a regression report in bugzilla.kernel.org that seems to be
> >> specific to the linux-6.6.y series:
> >>
> >> Quoting from https://bugzilla.kernel.org/show_bug.cgi?id=218484 :
> >>
> >>> After upgrading to version 6.6.16, the kernel compilation on a i586
> >>> arch (on a 32bit chroot in a 64bit host) fails with a message:
> >>>
> >>> virtual memory exhausted: Cannot allocate memory
> >>>
> >>> this happens even lowering the number of parallel compilation
> >>> threads. On a x86_64 arch the same problem doesn't occur. It's not
> >>> clear whether some weird recursion is triggered that exhausts the
> >>> memory, but it seems that the problem is caused by the patchset
> >>> 'minmax' added to the 6.6.16 version, in particular it seems caused
> >>> by these patches:
> >>>
> >>> - minmax-allow-min-max-clamp-if-the-arguments-have-the-same-signedness.patch
> >>> - minmax-fix-indentation-of-__cmp_once-and-__clamp_once.patch
> >>> - minmax-allow-comparisons-of-int-against-unsigned-char-short.patch
> >>> - minmax-relax-check-to-allow-comparison-between-unsigned-arguments-and-signed-constants.patch
> >>>
> >>> Reverting those patches fixes the memory exhaustion problem during compilation.
> >>
> >> The reporter later added:
> >>
> >>> From a quick test the same problem doesn't occur in 6.8-rc4.
> >> See the ticket for more details.
> >
> > I think this was already fixed in 6.7 or Linus's tree, but I can't seem
> > to find the commit at the moment.
> 
> I thought so as well, but was in the same situation. But your comment
> made me look again and now I found it: that was 31e97d7c9ae3de ("media:
> solo6x10: replace max(a, min(b, c)) by clamp(b, a, c)"), which indeed is
> not yet in 6.6.y.

The code is actually (now) doing:
	clamp(b, clamp(c, a, d), d)
but previously was four nested min()/max().
Even the a/b/c/d aren't trivial.
It always was a pretty long line, but the longer expansions made it explode.

I was mildly surprised to see the minmax changes backported.
Not complaining though.

But 31e97d7c9ae3de needs backporting as well.

(Someone really ought to look at the solo6x10 driver.
It really ought to be caching some of those values.
I also suspect (not looked) that if the values do get clamped (above)
it just wont work at all.)

	David

-
Registered Address Lakeside, Bramley Road, Mount Farm, Milton Keynes, MK1 1PT, UK
Registration No: 1397386 (Wales)