linux-kernel - The thundering herd like problem when multi epolls on one fd

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Message-ID: <4F116350.6090704@gmail.com>
Date:	Sat, 14 Jan 2012 19:13:20 +0800
From:	Li Yu <raise.sail@...il.com>
To:	linux-kernel@...r.kernel.org
CC:	davidel@...ilserver.org
Subject: The thundering herd like problem when multi epolls on one fd

Hi, 

	My buddy reported a thundering herd problem about using epoll 
on TCP listen sockets. He said their usage like below:

	1. sk = new tcp_listen_socket();
	2. create many child processes or threads.
	3. in new created processes (threads), use epoll API on listen
sk to provide HTTP service.

	Such using pattern means we have multi wait queues when 
accepting one socket, and it is not exclusive waking up, so we get a 
thundering herd like problem. And, so I heard many popular applications
can use such pattern, which includes nginx, lighttpd, haproxy at least.
So should we change this waking up behavior to exclusive too ? 

	Below is a simple patch (tested and works) for epoll() to do it,
of course, we also should fix select() and poll() syscalls if it is right.

	Thanks.

Yu	

diff --git a/fs/eventpoll.c b/fs/eventpoll.c
index 828e750..a3d6ab4 100644
--- a/fs/eventpoll.c
+++ b/fs/eventpoll.c
@@ -898,7 +899,7 @@ static void ep_ptable_queue_proc(struct file *file, wait_queue_head_t *whead,
                init_waitqueue_func_entry(&pwq->wait, ep_poll_callback);
                pwq->whead = whead;
                pwq->base = epi;
-               add_wait_queue(whead, &pwq->wait);
+               add_wait_queue_exclusive(whead, &pwq->wait);
                list_add_tail(&pwq->llink, &epi->pwqlist);
                epi->nwait++;
        } else {
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/