>From c060a5db92274402a0178d7c777a1e37c15eadb9 Mon Sep 17 00:00:00 2001 From: Willy Tarreau Date: Tue, 15 Dec 2015 16:40:00 +0100 Subject: net: make lingering sockets score less in compute_score() When multiple processes use SO_REUSEPORT for a seamless restart operation, there's a tiny window during which both the old and the new process are bound to the same port, and there's no way for the old process to gracefully stop receiving connections without dropping the few that are left in the queue between the last poll() and the shutdown() or close() operation. Incoming connections are distributed between multiple listening sockets in inet_lookup_listener() according to multiple criteria. The first criterion is a score based on a number of attributes for each socket, then a hash computation ensures that the connections are evenly distributed between sockets of equal weight. This patch provides a simple approach by which the old process can simply decrease its score by disabling SO_REUSEPORT on its listening sockets. Thus, the sockets from the new process always score higher and are always preferred. The old process can then safely drain incoming connections and stop after meeting the -1 EAGAIN condition, as shown in the example below : process A (old one) | process B (new one) | setsockopt(SO_REUSEPORT, 1) | listen() >= 0 | ... | accept() | ... | setsockopt(SO_REUSEPORT, 1) ... | listen() From now on, both processes receive incoming connections ... | kill(process A, go_away) setsockopt(SO_REUSEPORT, 0) | accept() >= 0 Here process A stops receiving new connections accept() >= 0 | accept() >= 0 ... | accept() = -1 EAGAIN | accept() >= 0 close() | exit() | Signed-off-by: Willy Tarreau --- net/ipv4/inet_hashtables.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/net/ipv4/inet_hashtables.c b/net/ipv4/inet_hashtables.c index ccc5980..1c950ba 100644 --- a/net/ipv4/inet_hashtables.c +++ b/net/ipv4/inet_hashtables.c @@ -189,6 +189,8 @@ static inline int compute_score(struct sock *sk, struct net *net, return -1; score += 4; } + if (sk->sk_reuseport) + score++; if (sk->sk_incoming_cpu == raw_smp_processor_id()) score++; } -- 1.7.12.1