netdev - Re: [patch] socket.7: Document SO_INCOMING

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <326b99c3-cc86-3abd-1069-2b3a52d9ba47@gmail.com>
Date:   Wed, 19 Apr 2017 20:48:12 +0200
From:   "Michael Kerrisk (man-pages)" <mtk.manpages@...il.com>
To:     Eric Dumazet <eric.dumazet@...il.com>
Cc:     mtk.manpages@...il.com,
        Francois Saint-Jacques <fsaintjacques@...il.com>,
        linux-man@...r.kernel.org, netdev@...r.kernel.org,
        Eric Dumazet <edumazet@...gle.com>
Subject: Re: [patch] socket.7: Document SO_INCOMING_CPU

Hi Eric,

[reodering for clarity]

>> On 02/19/2017 09:55 PM, Michael Kerrisk (man-pages) wrote:
>>> [CC += Eric, so that he might review]
>>>
>>> Hello Francois,
>>>
>>> On 02/18/2017 05:06 AM, Francois Saint-Jacques wrote:
>>>> This socket option is undocumented. Applies on the latest version
>>>> (man-pages-4.09-511).
>>>>
>>>> diff --git a/man7/socket.7 b/man7/socket.7
>>>> index 3efd7a5d8..1a3ffa253 100644
>>>> --- a/man7/socket.7
>>>> +++ b/man7/socket.7
>>>> @@ -490,6 +490,26 @@ flag on a socket
>>>>  operation.
>>>>  Expects an integer boolean flag.
>>>>  .TP
>>>> +.BR SO_INCOMING_CPU " (getsockopt since Linux 3.19, setsockopt since
>>>> Linux 4.4)"
>>>> +.\" getsocktop 2c8c56e15df3d4c2af3d656e44feb18789f75837
>>>> +.\" setsocktop 70da268b569d32a9fddeea85dc18043de9d89f89
>>>> +Sets or gets the cpu affinity of a socket. Expects an integer flag.
>>>> +.sp
>>>> +.in +4n
>>>> +.nf
>>>> +int cpu = 1;
>>>> +socklen_t len = sizeof(cpu);
>>>> +setsockopt(fd, SOL_SOCKET, SO_INCOMING_CPU, &cpu, &len);
>>>> +.fi
>>>> +.in
>>>> +.sp
>>>> +The typical use case is one listener per RX queue, as the associated listener
>>>> +should only accept flows handled in softirq by the same cpu.  This provides
>>>> +optimal NUMA behavior and keep cpu caches hot.
>>>> +.TP
>>>>  .B SO_KEEPALIVE
>>>>  Enable sending of keep-alive messages on connection-oriented sockets.
>>>>  Expects an integer boolean flag.
>>>
>>> Thank you! Patch applied.
>>>
>>> I have tried to enhance the description somewhat. I'm not sure whether
>>> what I've written is quite correct (or whether it should be further
>>> extended). Eric, could you please take a look at the following, and let 
>>> me know if anything needs fixing:
>>>
>>>        SO_INCOMING_CPU  (gettable  since Linux 3.19, settable since Linux
>>>        4.4)
>>>               Sets or gets the CPU affinity  of  a  socket.   Expects  an
>>>               integer flag.
>>>
>>>                   int cpu = 1;
>>>                   socklen_t len = sizeof(cpu);
>>>                   setsockopt(fd, SOL_SOCKET, SO_INCOMING_CPU, &cpu, &len);
>>>
>>>               Because  all  of the packets for a single stream (i.e., all
>>>               packets for the same 4-tuple) arrive on the single RX queue
>>>               that  is  associated with a particular CPU, the typical use
>>>               case is to employ one listening process per RX queue,  with
>>>               the  incoming  flow being handled by a listener on the same
>>>               CPU that is handling the RX queue.  This  provides  optimal
>>>               NUMA behavior and keeps CPU caches hot.

> Hi Michael
> 
> Sorry for the delay.

Thanks for the reply, but I think you are assuming I know more than 
I do. I'd like you to elaborate a little please. See below.

> Note that setting the option is not supported if SO_REUSEPORT is used.

Please define "not supported". Does this yield an API diagnostic?
If so, what is it?

> Socket will be selected from an array, either by a hash or BPF program
> that has no access to this information.

Sorry -- I'm lost here. How does this comment relate to the proposed
man page text above?

Thanks,

Michael


-- 
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
Linux/UNIX System Programming Training: http://man7.org/training/