netdev - Re: [PATCH iproute2 net-next v2 3/4] ss: Buffer raw fields first, then render them as a table

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:   Wed, 13 Feb 2019 09:37:11 +0100
From:   Stefano Brivio <sbrivio@...hat.com>
To:     Eric Dumazet <eric.dumazet@...il.com>
Cc:     Stephen Hemminger <stephen@...workplumber.org>,
        netdev@...r.kernel.org, Sabrina Dubroca <sd@...asysnail.net>
Subject: Re: [PATCH iproute2 net-next v2 3/4] ss: Buffer raw fields first,
 then render them as a table

On Tue, 12 Feb 2019 16:42:04 -0800
Eric Dumazet <eric.dumazet@...il.com> wrote:

> I do not get it.
> 
> "ss -emoi " uses almost 1KB per socket.
> 
> 10,000,000 sockets -> we need about 10GB of memory  ???
> 
> This is a serious regression.

I guess this is rather subjective: the worst case I considered back then
was the output of 'ss -tei0' (less than 500 bytes) for one million
sockets, which gives 500M of memory, which should in turn be fine on a
machine handling one million sockets.

Now, if 'ss -emoi' on 10 million sockets is an actual use case (out of
curiosity: how are you going to process that output? Would JSON help?),
I see two easy options to solve this:

1. flush the output every time we reach a given buffer size (1M
   perhaps). This might make the resulting blocks slightly unaligned,
   with occasional loss of readability on lines occurring every 1k to
   10k sockets approximately, even though after 1k sockets column sizes
   won't change much (it looks anyway better than the original), and I
   don't expect anybody to actually scroll that output

2. add a switch for unbuffered output, but then you need to remember to
   pass it manually, and the whole output would be as bad as the
   original in case you need the switch.

I'd rather go with 1., it's easy to implement (we already have partial
flushing with '--events') and it looks like a good compromise on
usability. Thoughts?

-- 
Stefano