| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Message-ID: <8705882e-83fb-efbf-b48d-2a5661732151@xs4all.nl>
Date: Sat, 18 Sep 2021 14:39:21 +0200
From: Udo van den Heuvel <udovdh@...all.nl>
To: "linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>
Subject: Re: 5.13.14 page allocation failure
People,
Why do these pagefaults start to occur when a machine goes from 16 to 32 GB?
Does the amount of CPU threads matter?
Why doesn't vm.min_free_kbytes really help yet?
Why doesn't the kernel simply use a bit of the unused memory?
Why isn't the mm infrastructure not so 'automagic' as it once appeared?
Please explain as it might help to fix the allocation issue.
Udo
PS:
# free
total used free shared buff/cache
available
Mem: 32325764 6612532 1365524 430068 24347708
24110776
Swap: 18874360 331776 18542584
On 10-09-2021 07:46, Udo van den Heuvel wrote:
> Again:
>
> Sep 9 15:06:03 surfplank2 rtkit-daemon[296496]: Successfully made
> thread 311799 of process 311302 (/usr/lib64/firefox/firefox) owned by
> '500' RT at priority 10.
> Sep 9 15:12:41 knalp3 kernel: [120876.114705] warn_alloc: 2 callbacks
> suppressed
> Sep 9 15:12:41 knalp3 kernel: [120876.114708] Web Content: page
> allocation failure: order:7, mode:0x40a20(GFP_ATOMIC|__GFP_COMP),
> nodemask=(null)
> Sep 9 15:12:41 knalp3 kernel: [120876.114715] CPU: 12 PID: 312097 Comm:
> Web Content Not tainted 5.13.14 #5
> Sep 9 15:12:41 knalp3 kernel: [120876.114717] Hardware name: Gigabyte
> Technology Co., Ltd. X570 AORUS PRO/X570 AORUS PRO, BIOS F34 07/08/2021
> Sep 9 15:12:41 knalp3 kernel: [120876.114718] Call Trace:
> Sep 9 15:12:41 knalp3 kernel: [120876.114721] dump_stack+0x64/0x7c
> Sep 9 15:12:41 knalp3 kernel: [120876.114725] warn_alloc.cold+0x6f/0xd3
> Sep 9 15:12:41 knalp3 kernel: [120876.114728]
> __alloc_pages_slowpath.constprop.0+0x6c3/0xa80
> Sep 9 15:12:41 knalp3 kernel: [120876.114731] ?
> get_page_from_freelist+0x223/0x9d0
> Sep 9 15:12:41 knalp3 kernel: [120876.114733] __alloc_pages+0x169/0x1d0
> Sep 9 15:12:41 knalp3 kernel: [120876.114735]
> cache_alloc_refill+0x58f/0x810
> Sep 9 15:12:41 knalp3 kernel: [120876.114737] ? _raw_spin_unlock+0xd/0x20
> Sep 9 15:12:41 knalp3 kernel: [120876.114740]
> kmem_cache_alloc+0x109/0x120
> Sep 9 15:12:41 knalp3 kernel: [120876.114742] p_dump_task_f+0x19/0xf0
> [p_lkrg]
> Sep 9 15:12:41 knalp3 kernel: [120876.114749]
> p_wake_up_new_task_entry+0x6b/0xe0 [p_lkrg]
> Sep 9 15:12:41 knalp3 kernel: [120876.114753]
> pre_handler_kretprobe+0x8a/0x160
> Sep 9 15:12:41 knalp3 kernel: [120876.114756] opt_pre_handler+0x4a/0x80
> Sep 9 15:12:41 knalp3 kernel: [120876.114758]
> optimized_callback+0xbe/0xf0
> Sep 9 15:12:41 knalp3 kernel: [120876.114761] 0xffffffffc0076294
> Sep 9 15:12:41 knalp3 kernel: [120876.114763] ?
> wake_up_new_task+0x1/0x280
> Sep 9 15:12:41 knalp3 kernel: [120876.114765] ? kernel_clone+0xd6/0x380
> Sep 9 15:12:41 knalp3 kernel: [120876.114768] ? __do_sys_clone+0x60/0x80
> Sep 9 15:12:41 knalp3 kernel: [120876.114769] ? do_syscall_64+0x61/0x80
> Sep 9 15:12:41 knalp3 kernel: [120876.114771] ?
> exit_to_user_mode_prepare+0x19/0xf0
> Sep 9 15:12:41 knalp3 kernel: [120876.114773] ?
> entry_SYSCALL_64_after_hwframe+0x44/0xae
> Sep 9 15:12:41 knalp3 kernel: [120876.114775] Mem-Info:
> Sep 9 15:12:41 knalp3 kernel: [120876.114776] active_anon:2937
> inactive_anon:1070473 isolated_anon:0
> Sep 9 15:12:41 knalp3 kernel: [120876.114776] active_file:2166685
> inactive_file:4129380 isolated_file:0
> Sep 9 15:12:41 knalp3 kernel: [120876.114776] unevictable:6793
> dirty:773 writeback:0
> Sep 9 15:12:41 knalp3 kernel: [120876.114776] slab_reclaimable:58285
> slab_unreclaimable:200826
> Sep 9 15:12:41 knalp3 kernel: [120876.114776] mapped:263480
> shmem:109170 pagetables:10726 bounce:0
> Sep 9 15:12:41 knalp3 kernel: [120876.114776] free:116342
> free_pcp:7303 free_cma:0
> Sep 9 15:12:41 knalp3 kernel: [120876.114779] Node 0
> active_anon:11748kB inactive_anon:4281892kB active_file:8666740kB
> inactive_file:16517520kB unevictable:27172kB isolated(anon):0kB
> isolated(file):0kB mapped:1053920kB dirty:3092kB writeback:0kB
> shmem:436680kB shmem_thp: 0kB shmem_pmdmapped: 0kB anon_thp: 13352
> 96kB writeback_tmp:0kB kernel_stack:26276kB pagetables:42904kB
> all_unreclaimable? no
> Sep 9 15:12:41 knalp3 kernel: [120876.114782] DMA free:11264kB min:32kB
> low:44kB high:56kB reserved_highatomic:0KB active_anon:0kB
> inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB
> writepending:0kB present:15984kB managed:15360kB mlocked:0kB bounce:0kB
> free_pcp:0kB local_pcp:0kB free_cma:0kB
> Sep 9 15:12:41 knalp3 kernel: [120876.114785] lowmem_reserve[]: 0 2357
> 31506 31506
> Sep 9 15:12:41 knalp3 kernel: [120876.114787] DMA32 free:118884kB
> min:5052kB low:7464kB high:9876kB reserved_highatomic:26624KB
> active_anon:4144kB inactive_anon:713188kB active_file:581608kB
> inactive_file:865364kB unevictable:0kB writepending:208kB
> present:2521312kB managed:2455776kB mlocked:0kB bounce:0kB free
> _pcp:15980kB local_pcp:1344kB free_cma:0kB
> Sep 9 15:12:41 knalp3 kernel: [120876.114790] lowmem_reserve[]: 0 0
> 29148 29148
> Sep 9 15:12:41 knalp3 kernel: [120876.114791] Normal free:335220kB
> min:64544kB low:94388kB high:124232kB reserved_highatomic:0KB
> active_anon:7604kB inactive_anon:3569388kB active_file:8085132kB
> inactive_file:15652668kB unevictable:27172kB writepending:2884kB
> present:30379008kB managed:29854648kB mlocked:24700kB
> bounce:0kB free_pcp:13080kB local_pcp:688kB free_cma:0kB
> Sep 9 15:12:41 knalp3 kernel: [120876.114794] lowmem_reserve[]: 0 0 0 0
> Sep 9 15:12:41 knalp3 kernel: [120876.114795] DMA: 0*4kB 0*8kB 0*16kB
> 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 1*1024kB (U) 1*2048kB (M) 2*4096kB
> (M) = 11264kB
> Sep 9 15:12:41 knalp3 kernel: [120876.114801] DMA32: 2599*4kB (UM)
> 4207*8kB (UM) 781*16kB (M) 144*32kB (M) 114*64kB (UM) 38*128kB (UM)
> 20*256kB (UM) 15*512kB (UM) 2*1024kB (M) 5*2048kB (UM) 5*4096kB (M) =
> 118884kB
> Sep 9 15:12:41 knalp3 kernel: [120876.114807] Normal: 2676*4kB (UME)
> 1619*8kB (UME) 5501*16kB (UME) 2818*32kB (UME) 867*64kB (UME) 455*128kB
> (UME) 78*256kB (UM) 0*512kB 0*1024kB 0*2048kB 0*4096kB = 335544kB
> Sep 9 15:12:41 knalp3 kernel: [120876.114813] Node 0 hugepages_total=0
> hugepages_free=0 hugepages_surp=0 hugepages_size=2048kB
> Sep 9 15:12:41 knalp3 kernel: [120876.114814] 6409485 total pagecache
> pages
> Sep 9 15:12:41 knalp3 kernel: [120876.114815] 2 pages in swap cache
> Sep 9 15:12:41 knalp3 kernel: [120876.114816] Swap cache stats: add
> 211, delete 209, find 61/84
> Sep 9 15:12:41 knalp3 kernel: [120876.114817] Free swap = 18873080kB
> Sep 9 15:12:41 knalp3 kernel: [120876.114818] Total swap = 18874360kB
> Sep 9 15:12:41 knalp3 kernel: [120876.114819] 8229076 pages RAM
> Sep 9 15:12:41 knalp3 kernel: [120876.114819] 0 pages HighMem/MovableOnly
> Sep 9 15:12:41 knalp3 kernel: [120876.114820] 147630 pages reserved
> Sep 9 15:12:41 knalp3 kernel: [120876.114820] [p_lkrg]
> p_alloc_ed_pids() returned NULL for pid 312164 :(
> Sep 9 15:12:41 knalp3 kernel: [120876.114822] [p_lkrg] <Exploit
> Detection> Error[-1] when trying to add process[312164 |Web Content] for
> tracking!
> Sep 9 15:12:41 knalp3 kernel: [120876.184547] [p_lkrg]
> p_alloc_ed_pids() returned NULL for pid 312165 :(
> Sep 9 15:12:41 knalp3 kernel: [120876.184551] [p_lkrg] <Exploit
> Detection> Error[-1] when trying to add process[312165 |Socket Thread]
> for tracking!
> Sep 9 15:12:41 knalp3 kernel: warn_alloc: 2 callbacks suppressed
>
>
>
> Udo
>
>
> On 08-09-2021 17:17, Udo van den Heuvel wrote:
>> Hello,
>>
>> Ever since I recently migrated from AMD Ryzen 3400g to 5700g and went
>> from 16GB to 32GB we get these:
>>
>> Sep 8 16:40:44 knalp3 kernel: chrome: page allocation failure:
>> order:7, mode:0x40a20(GFP_ATOMIC|__GFP_COMP), nodemask=(null)
>> Sep 8 16:40:44 knalp3 kernel: CPU: 0 PID: 156918 Comm: chrome Not
>> tainted 5.13.14 #5
>> Sep 8 16:40:44 knalp3 kernel: Hardware name: Gigabyte Technology Co.,
>> Ltd. X570 AORUS PRO/X570 AORUS PRO, BIOS F34 07/08/2021
>> Sep 8 16:40:44 knalp3 kernel: Call Trace:
>> Sep 8 16:40:44 knalp3 kernel: dump_stack+0x64/0x7c
>> Sep 8 16:40:44 knalp3 kernel: warn_alloc.cold+0x6f/0xd3
>> Sep 8 16:40:44 knalp3 kernel:
>> __alloc_pages_slowpath.constprop.0+0x6c3/0xa80
>> Sep 8 16:40:44 knalp3 kernel: ? get_page_from_freelist+0x223/0x9d0
>> Sep 8 16:40:44 knalp3 kernel: __alloc_pages+0x169/0x1d0
>> Sep 8 16:40:44 knalp3 kernel: cache_alloc_refill+0x58f/0x810
>> Sep 8 16:40:44 knalp3 kernel: ? _raw_spin_unlock+0xd/0x20
>> Sep 8 16:40:44 knalp3 kernel: kmem_cache_alloc+0x109/0x120
>> Sep 8 16:40:44 knalp3 kernel: p_dump_task_f+0x19/0xf0 [p_lkrg]
>> Sep 8 16:40:44 knalp3 kernel: p_wake_up_new_task_entry+0x6b/0xe0
>> [p_lkrg]
>> Sep 8 16:40:44 knalp3 kernel: pre_handler_kretprobe+0x8a/0x160
>> Sep 8 16:40:44 knalp3 kernel: opt_pre_handler+0x4a/0x80
>> Sep 8 16:40:44 knalp3 kernel: optimized_callback+0xbe/0xf0
>> Sep 8 16:40:44 knalp3 kernel: 0xffffffffc0076294
>> Sep 8 16:40:44 knalp3 kernel: ? wake_up_new_task+0x1/0x280
>> Sep 8 16:40:44 knalp3 kernel: ? kernel_clone+0xd6/0x380
>> Sep 8 16:40:44 knalp3 kernel: ? __do_sys_clone+0x60/0x80
>> Sep 8 16:40:44 knalp3 kernel: ? do_syscall_64+0x61/0x80
>> Sep 8 16:40:44 knalp3 kernel: ? entry_SYSCALL_64_after_hwframe+0x44/0xae
>> Sep 8 16:40:44 knalp3 kernel: Mem-Info:
>> Sep 8 16:40:44 knalp3 kernel: active_anon:5244 inactive_anon:1494070
>> isolated_anon:0#012 active_file:1098885 inactive_file:4703460
>> isolated_file:0#012 unevictable:639 dirty:948 writeback:0#012
>> slab_reclaimable:74606 slab_unreclaimable:245109#012 mapped:347626
>> shmem:160084 pagetables:16000 bounce:0#012 free:98085 free_pcp:4475
>> free_cma:0
>> Sep 8 16:40:44 knalp3 kernel: Node 0 active_anon:20976kB
>> inactive_anon:5976280kB active_file:4395540kB inactive_file:18813840kB
>> unevictable:2556kB isolated(anon):0kB isolated(file):0kB
>> mapped:1390504kB dirty:3792kB writeback:0kB shmem:640336kB shmem_thp:
>> 0kB shmem_pmdmapped: 0kB anon_thp: 1681408kB writeback_tmp:0kB
>> kernel_stack:31376kB pagetables:64000kB all_unreclaimable? no
>> Sep 8 16:40:44 knalp3 kernel: DMA free:11264kB min:32kB low:44kB
>> high:56kB reserved_highatomic:0KB active_anon:0kB inactive_anon:0kB
>> active_file:0kB inactive_file:0kB unevictable:0kB writepending:0kB
>> present:15984kB managed:15360kB mlocked:0kB bounce:0kB free_pcp:0kB
>> local_pcp:0kB free_cma:0kB
>> Sep 8 16:40:44 knalp3 kernel: lowmem_reserve[]: 0 2357 31506 31506
>> Sep 8 16:40:44 knalp3 kernel: DMA32 free:118968kB min:5052kB
>> low:7464kB high:9876kB reserved_highatomic:16384KB active_anon:124kB
>> inactive_anon:772432kB active_file:166476kB inactive_file:1282188kB
>> unevictable:16kB writepending:368kB present:2521312kB
>> managed:2455776kB mlocked:16kB bounce:0kB free_pcp:3528kB
>> local_pcp:0kB free_cma:0kB
>> Sep 8 16:40:44 knalp3 kernel: lowmem_reserve[]: 0 0 29148 29148
>> Sep 8 16:40:44 knalp3 kernel: Normal free:262612kB min:64544kB
>> low:94388kB high:124232kB reserved_highatomic:0KB active_anon:20488kB
>> inactive_anon:5202876kB active_file:4228640kB inactive_file:17532192kB
>> unevictable:2540kB writepending:3424kB present:30379008kB
>> managed:29854648kB mlocked:68kB bounce:0kB free_pcp:14248kB
>> local_pcp:64kB free_cma:0kB
>> Sep 8 16:40:44 knalp3 kernel: lowmem_reserve[]: 0 0 0 0
>> Sep 8 16:40:44 knalp3 kernel: DMA: 0*4kB 0*8kB 0*16kB 0*32kB 0*64kB
>> 0*128kB 0*256kB 0*512kB 1*1024kB (U) 1*2048kB (M) 2*4096kB (M) = 11264kB
>> Sep 8 16:40:44 knalp3 kernel: DMA32: 494*4kB (UM) 132*8kB (UM)
>> 194*16kB (M) 710*32kB (M) 224*64kB (UM) 24*128kB (UM) 18*256kB (UM)
>> 13*512kB (M) 6*1024kB (M) 9*2048kB (M) 9*4096kB (M) = 118968kB
>> Sep 8 16:40:44 knalp3 kernel: Normal: 570*4kB (UME) 2546*8kB (UME)
>> 2503*16kB (UME) 1391*32kB (UME) 848*64kB (UME) 549*128kB (UME)
>> 114*256kB (U) 1*512kB (M) 0*1024kB 0*2048kB 0*4096kB = 261448kB
>> Sep 8 16:40:44 knalp3 kernel: Node 0 hugepages_total=0
>> hugepages_free=0 hugepages_surp=0 hugepages_size=2048kB
>> Sep 8 16:40:44 knalp3 kernel: 5963328 total pagecache pages
>> Sep 8 16:40:44 knalp3 kernel: 0 pages in swap cache
>> Sep 8 16:40:44 knalp3 kernel: Swap cache stats: add 170, delete 169,
>> find 61/83
>> Sep 8 16:40:44 knalp3 kernel: Free swap = 18873848kB
>> Sep 8 16:40:44 knalp3 kernel: Total swap = 18874360kB
>> Sep 8 16:40:44 knalp3 kernel: 8229076 pages RAM
>> Sep 8 16:40:44 knalp3 kernel: 0 pages HighMem/MovableOnly
>> Sep 8 16:40:44 knalp3 kernel: 147630 pages reserved
>> Sep 8 16:40:44 knalp3 kernel: [p_lkrg] p_alloc_ed_pids() returned
>> NULL for pid 156929 :(
>>
>> This did not happen (unless there really was an OOM) when using 16GB
>> of RAM.
>>
>> The RAM is from AMD's HCL for AMD Ryzen 3000+. The RAM was tested with
>> memtest86 with no errors. (only a night, but still)
>>
>> How can I find out what is going on?
>> Did I misconfigure my kernel?
>> Do I make to configure something to use the 'immense' amount of 32GB
>> with no issues?
>>
>> Please let me know.
>>
>>
>> Kind regards,
>> Udo
>
Powered by blists - more mailing lists