linux-kernel - Re: [PATCH] x86-32: Allocate irq stacks seperate from percpu area

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <1288186405.2709.117.camel@edumazet-laptop>
Date:	Wed, 27 Oct 2010 15:33:25 +0200
From:	Eric Dumazet <eric.dumazet@...il.com>
To:	Peter Zijlstra <peterz@...radead.org>
Cc:	Brian Gerst <brgerst@...il.com>, tj@...nel.org, x86@...nel.org,
	linux-kernel@...r.kernel.org, torvalds@...ux-foundation.org,
	mingo@...e.hu
Subject: Re: [PATCH] x86-32: Allocate irq stacks seperate from percpu area

Le mercredi 27 octobre 2010 à 11:57 +0200, Peter Zijlstra a écrit :
> On Wed, 2010-10-27 at 08:07 +0200, Eric Dumazet wrote:
> > > -     irqctx = &per_cpu(hardirq_stack, cpu);
> > > +     irqctx = (union irq_ctx *)__get_free_pages(THREAD_FLAGS, THREAD_ORDER);
> > 
> > Hmm, then we lose NUMA affinity for stacks. 
> 
> I guess we could use:
> 
>   alloc_pages_node(cpu_to_node(cpu), THREAD_FLAGS, THREAD_ORDER);
> 
> 

Anyway, I just discovered per_cpu data on my machine (NUMA capable) all
sit on a single node, if 32bit kernel used.

# cat /proc/buddyinfo 
Node 0, zone      DMA      0      1      0      1      2      1      1      0      1      1      3 
Node 0, zone   Normal     94    251     81     16      3      2      1      2      1      2    187 
Node 0, zone  HighMem    113     88     47     36     18      5      4      3      2      0    268 
Node 1, zone  HighMem    154     97     43     16      9      4      3      2      3      2    482 

# dmesg | grep pcpu
[    0.000000] pcpu-alloc: s41920 r0 d23616 u65536 alloc=1*2097152
[    0.000000] pcpu-alloc: [0] 00 01 02 03 04 05 06 07 08 09 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 


dual socket machine (E5540  @ 2.53GHz), total of 8 cores, 16 threads.

# dmesg | grep irqstack
[    0.000000] CPU 0 irqstacks, hard=f4a00000 soft=f4a02000
[    0.173397] CPU 1 irqstacks, hard=f4a10000 soft=f4a12000
[    0.284939] CPU 2 irqstacks, hard=f4a20000 soft=f4a22000
[    0.392908] CPU 3 irqstacks, hard=f4a30000 soft=f4a32000
[    0.500757] CPU 4 irqstacks, hard=f4a40000 soft=f4a42000
[    0.608713] CPU 5 irqstacks, hard=f4a50000 soft=f4a52000
[    0.716665] CPU 6 irqstacks, hard=f4a60000 soft=f4a62000
[    0.828668] CPU 7 irqstacks, hard=f4a70000 soft=f4a72000
[    0.936555] CPU 8 irqstacks, hard=f4a80000 soft=f4a82000
[    1.044525] CPU 9 irqstacks, hard=f4a90000 soft=f4a92000
[    1.152470] CPU 10 irqstacks, hard=f4aa0000 soft=f4aa2000
[    1.260367] CPU 11 irqstacks, hard=f4ab0000 soft=f4ab2000
[    1.368313] CPU 12 irqstacks, hard=f4ac0000 soft=f4ac2000
[    1.476313] CPU 13 irqstacks, hard=f4ad0000 soft=f4ad2000
[    1.584167] CPU 14 irqstacks, hard=f4ae0000 soft=f4ae2000
[    1.692222] CPU 15 irqstacks, hard=f4af0000 soft=f4af2000


With a 64bit kernel its fine :

[    0.000000] pcpu-alloc: s76992 r8192 d21312 u131072 alloc=1*2097152
[    0.000000] pcpu-alloc: [0] 00 02 04 06 08 10 12 14 17 19 21 23 25 27 29 31 
[    0.000000] pcpu-alloc: [1] 01 03 05 07 09 11 13 15 16 18 20 22 24 26 28 30 

I presume node 1 having only HighMem could be the reason ?



--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/