netdev - Re: [PATCH iproute2] ss: Render buffer to output every time a number of chunks are allocated

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:   Thu, 14 Feb 2019 08:55:03 -0800
From:   Eric Dumazet <eric.dumazet@...il.com>
To:     Stefano Brivio <sbrivio@...hat.com>,
        Stephen Hemminger <stephen@...workplumber.org>
Cc:     Phil Sutter <phil@....cc>, David Ahern <dsahern@...il.com>,
        Sabrina Dubroca <sd@...asysnail.net>, netdev@...r.kernel.org
Subject: Re: [PATCH iproute2] ss: Render buffer to output every time a number
 of chunks are allocated



On 02/13/2019 04:58 PM, Stefano Brivio wrote:
> Eric reported that, with 10 million sockets, ss -emoi (about 1000 bytes
> output per socket) can easily lead to OOM (buffer would grow to 10GB of
> memory).
> 
> Limit the maximum size of the buffer to five chunks, 1M each. Render and
> flush buffers whenever we reach that.
> 
> This might make the resulting blocks slightly unaligned between them, with
> occasional loss of readability on lines occurring every 5k to 50k sockets
> approximately. Something like (from ss -tu):
> 
> [...]
> CLOSE-WAIT   32       0           192.168.1.50:35232           10.0.0.1:https
> ESTAB        0        0           192.168.1.50:53820           10.0.0.1:https
> ESTAB       0        0           192.168.1.50:46924            10.0.0.1:https
> CLOSE-WAIT  32       0           192.168.1.50:35228            10.0.0.1:https
> [...]
> 
> However, I don't actually expect any human user to scroll through that
> amount of sockets, so readability should be preserved when it matters.
> 
> The bulk of the diffstat comes from moving field_next() around, as we now
> call render() from it. Functionally, this is implemented by six lines of
> code, most of them in field_next().
> 
> Reported-by: Eric Dumazet <eric.dumazet@...il.com>
> Fixes: 691bd854bf4a ("ss: Buffer raw fields first, then render them as a table")
> Signed-off-by: Stefano Brivio <sbrivio@...hat.com>
> ---
> Eric, it would be nice if you could test this with your bazillion sockets,
> I checked this with -emoi and "only" 500,000 sockets.

Thanks, this seems reasonable enough to me.

# /usr/bin/time misc/ss -t |head -1
State       Recv-Q   Send-Q       Local Address:Port        Peer Address:Port   
Command terminated by signal 13
0.05user 0.00system 0:00.05elapsed 100%CPU (0avgtext+0avgdata 5836maxresident)k
0inputs+0outputs (0major+1121minor)pagefaults 0swaps