linux-kernel - Re: [PATCH v3 0/3] epoll: introduce round robin wakeup mode

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <54FAF076.3090308@akamai.com>
Date:	Sat, 07 Mar 2015 07:35:02 -0500
From:	Jason Baron <jbaron@...mai.com>
To:	Ingo Molnar <mingo@...nel.org>
CC:	Andrew Morton <akpm@...ux-foundation.org>, peterz@...radead.org,
	mingo@...hat.com, viro@...iv.linux.org.uk, normalperson@...t.net,
	davidel@...ilserver.org, mtk.manpages@...il.com,
	luto@...capital.net, linux-kernel@...r.kernel.org,
	linux-fsdevel@...r.kernel.org, linux-api@...r.kernel.org,
	Linus Torvalds <torvalds@...ux-foundation.org>,
	Alexander Viro <viro@....linux.org.uk>
Subject: Re: [PATCH v3 0/3] epoll: introduce round robin wakeup mode

On 03/05/2015 04:15 AM, Ingo Molnar wrote:
> * Jason Baron <jbaron@...mai.com> wrote:
>
>> 2) We are using the wakeup in this case to 'assign' work more 
>> permanently to the thread. That is, in the case of a listen socket 
>> we then add the connected socket to the woken up threads local set 
>> of epoll events. So the load persists past the wake up. And in this 
>> case, doing the round robin wakeups, simply allows us to access more 
>> cpu bandwidth. (I'm also looking into potentially using cpu affinity 
>> to do the wakeups as well as you suggested.)
> So this is the part that I still don't understand.

Here's maybe another way to frame this. Epoll sets add
a waiter on the wait queue in a fixed order when epoll sets
are added (via EPOLL_CTL_ADD). This order does not change
modulo adds/dels which are usually not common. So if
we don't want to wake all threads, when say an interrupt
occurs at some random point, we can either:

1) Walk the list, wake up the first epoll set that has idle
threads (queued via epoll_wait()) and return.

or:

2) Walk the list and wake up the first epoll set that has idle
threads, but then 'rotate' or move this epoll set to the tail
of the queue before returning.

So because the epoll sets are in a fixed order there is
an extreme bias to pick the same epoll sets over and over
regardless of the order in which threads return to wait
via (epoll_wait()). So I think the rotate makes sense for
the case where I am trying to assign work to threads that
may persist past the wake up point, and for cases where
the threads can finish all their work before returning
back to epoll_wait().

Thanks,

-Jason

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/