netdev - Re: Fw: [PATCH] sunrpc: use better NUMA affinities

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Message-ID: <4E324DB4.7060600@fastmail.fm>
Date:	Fri, 29 Jul 2011 16:05:40 +1000
From:	Greg Banks <gnb@...tmail.fm>
To:	NeilBrown <neilb@...e.de>
CC:	linux-nfs@...r.kernel.org, David Miller <davem@...emloft.net>,
	linux-kernel <linux-kernel@...r.kernel.org>,
	netdev <netdev@...r.kernel.org>,
	Eric Dumazet <eric.dumazet@...il.com>
Subject: Re: Fw: [PATCH] sunrpc: use better NUMA affinities

On 29/07/11 15:32, NeilBrown wrote:
>
> Hi Greg,
>   I saw this patch float past and thought of you... You may not be interested
>   any more, and it may be a perfectly good patch that does not need any
>   comment, but I thought I would let you know anyway.

Thanks Neil.

I've trimmed the cc list to limit the number of copies Trond and Bruce get:)

> From: Eric Dumazet<eric.dumazet@...il.com>
> To: Trond Myklebust<Trond.Myklebust@...app.com>
> Cc: "J. Bruce Fields"<bfields@...ldses.org>, Neil Brown<neilb@...e.de>,
> David Miller<davem@...emloft.net>, linux-nfs@...r.kernel.org, netdev
> <netdev@...r.kernel.org>, linux-kernel<linux-kernel@...r.kernel.org>
> Subject: [PATCH] sunrpc: use better NUMA affinities
>
>
> Use NUMA aware allocations to reduce latencies and increase throughput.

Briefly looking at the patch, it doesn't seem wrong but I'm surprised 
it's (still) necessary.

Some years ago at SGI we encountered that same problem; we solved it by 
delaying all the allocation of data structures associated with a thread 
so that they were performed in the thread itself, after the thread had 
been limited to run on a certain set of CPUs.  Thus the thread's normal 
allocation behaviour resulted in all of it's allocations being from 
node-local pages.  It was a pretty ugly patch, but it worked and made a 
huge difference to NFS throughput on large NUMA boxes.

Later Jeff Layton converted the sunrpc svc startup code to use kthreads 
and at the time I read his patches, pointed out this problem, and posted 
my patch for comparison

http://linux-nfs.org/pipermail/nfsv4/2008-May/008760.html

I seem to remember coming to the conclusion that Jeff eventually 
addressed this problem...am I misremembering or did something regress?

-- 
Greg.

--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html