netdev - Re: [PATCH 2/2] net/9p: increase default msize to 128k

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Message-ID: <YTVb3K37JxUWUdXN@codewreck.org>
Date:   Mon, 6 Sep 2021 09:07:56 +0900
From:   Dominique Martinet <asmadeus@...ewreck.org>
To:     Eric Van Hensbergen <ericvh@...il.com>
Cc:     Christian Schoenebeck <linux_oss@...debyte.com>,
        Greg Kurz <groug@...d.org>,
        Latchesar Ionkov <lucho@...kov.net>, netdev@...r.kernel.org,
        v9fs-developer@...ts.sourceforge.net
Subject: Re: [PATCH 2/2] net/9p: increase default msize to 128k

Eric Van Hensbergen wrote on Sun, Sep 05, 2021 at 06:44:13PM -0500:
> there will likely be a tradeoff with tcp in terms of latency to first
> message so while
> absolute bw may be higher processing time may suffer.  8k was default msize
> to more closely match it to jumbo frames on an ethernet.  of course all
> that intuition is close to 30 years out of date….

It's not because the max size is 128k (or 1MB) that this much is sent
over the wire everytime -- if a message used to fit in 8KB, then its
on-the-wire size won't change and speed/latency won't be affected for
these.

For messages that do require more than 8KB (read/write/readdir) then you
can fit more data per message, so for a given userspace request (feed me
xyz amount of data) you'll have less client-server round-trips, and the
final user-reflected latency will be better as well -- that's why
e.g. NFS has been setting a max size of 1MB by default for a while now,
and they allow even more (32MB iirc? not sure)

I've only had done these tests years ago and no longer have access to
the note that was written back then, but TCP also definitely benefits
from > 64k msize as long as there's enough memory available.

The downside (because it's not free) is there though, you need more
memory for 9p with big buffers even if we didn't need so much in the
first place.
The code using a slab now means that the memory is not locked per mount
as it used to, but that also means allocations can fail if there is a
big pressure after not having been released. OTOH as long as it's
consistently used the buffers will be recycled so it's not necessarily
too bad performance-wise in hot periods.

Ideally we'd need to rework transports to allow scatter-gather (iovec or
similar API), and work with smaller allocations batched together on
send, but I don't have time for something like that... If we do that we
can probably get the best of both worlds -- and could consider >1MB, but
that's unrealistic as of now, regardless of the transport.

-- 
Dominique