linux-kernel - Re: [PATCH 6.10 000/809] 6.10.3-rc3 review

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <4f71784a-e896-4512-b1e5-600d0497037c@roeck-us.net>
Date: Tue, 6 Aug 2024 11:09:52 -0700
From: Guenter Roeck <linux@...ck-us.net>
To: Vlastimil Babka <vbabka@...e.cz>,
 Linus Torvalds <torvalds@...ux-foundation.org>
Cc: linux-kernel@...r.kernel.org, Linux-MM <linux-mm@...ck.org>,
 Thomas Gleixner <tglx@...utronix.de>
Subject: Re: [PATCH 6.10 000/809] 6.10.3-rc3 review

On 8/6/24 04:02, Vlastimil Babka wrote:
> On 8/6/24 04:40, Linus Torvalds wrote:
>> [ Let's drop random people and bring in Vlastimil ]
> 
> tglx was reproducing it so I add him back
> 
>> Vlastimil,
>>   it turns out that the "this patch" is entirely a red herring, and the
>> problem comes and goes randomly with just some code layout issues. See
>>
>>     http://server.roeck-us.net/qemu/parisc64-6.10.3/
>>
>> for more detail, particularly you'll see the "log.bad.gz" with the full log.
> 
> [    0.000000] BUG kmem_cache_node (Not tainted): objects 21 > max 16
> [    0.000000] Slab 0x0000000041ed0000 objects=21 used=5 fp=0x00000000434003d0 flags=0x200(workingset|section=0|zone=0)
> 
> flags tell us this came from the partial list (workingset), there's no head flag so order-0
> 
> since the error was detected it basically throws the slab page away and tries another one
> 
> [    0.000000] BUG kmem_cache (Tainted: G    B             ): objects 25 > max 16
> [    0.000000] Slab 0x0000000041ed0080 objects=25 used=6 fp=0x0000000043402790 flags=0x240(workingset|head|section=0|zone=0)
> 
> this was also from the partial list but head flag so at least order-1, two things are weird:
> - max=16 is same as above even though it should be at least double as
> slab page's order is larger
> - objects=25 also isn't at least twice than objects=21
> 
> All the following are:
> [    0.000000] BUG kmem_cache (Tainted: G    B             ): objects 25 > max 16
> [    0.000000] Slab 0x0000000041ed0300 objects=25 used=1 fp=0x000000004340c150 flags=0x40(head|section=0|zone=0)
> 
> we depleted the partial list so it's allocating new slab pages, that are
> also at least order-1
> 
> It looks like maxobj calculation is bogus, would be useful to see what values it
> calculates from. I'm attaching a diff, but maybe it will also hide the issue...
> 

If I am really careful I can catch the problem. Adding a parameter to slab_err()
makes it disappear. Calling slab_order() multiple times makes it disappear. But I can
assign the result of slab_order() to a variable and go from there, by changing the
parameters to slab_err() one at a time. Here is what I get by combining multiple
test runs.

objects 21 max 16 order 0 size 192
  with:
   Slab 0x0000000041ed0000 objects=21 used=5 fp=0x00000000434003d0 flags=0x200(workingset|section=0|zone=0)
objects 25 max 16 order 1 size 320 (many times)
  with:
   Slab 0x0000000041ed0080 objects=25 used=6 fp=0x0000000043402790 flags=0x240(workingset|head|section=0|zone=0)
   Slab 0x0000000041ed0300 objects=25 used=1 fp=0x000000004340c150 flags=0x40(head|section=0|zone=0)
   Slab 0x0000000041ed0380 objects=25 used=1 fp=0x000000004340e150 flags=0x40(head|section=0|zone=0)
   and so on

If I add a log message into check_slab() and display every check I get:

objects 21 max 21 (order 0 size 192)
objects 25 max 25 (order 1 size 320)
objects 25 max 25 (order 1 size 320)
objects 25 max 25 (order 1 size 320)
objects 21 max 21 (order 0 size 192)

and so on. I guess that confirms that the maxobj calculation is wrong in the failure case.
That seems weird, though, given that order_objects() should return a constant based on
order and size. PAGE_SIZE is 4096, meaning order_objects() should return 21 and 25, never
16. That does make me really wonder if there is an emulation problem, though I don't
really understand why that would not be persistent.

Any idea what else I could test ?

Thanks,
Guenter