netdev - Re: [RFC PATCH net-next 0/3] sctp: add GSO support

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <20160128205551.GC6602@mrl.redhat.com>
Date:	Thu, 28 Jan 2016 18:55:52 -0200
From:	"'Marcelo Ricardo Leitner'" <marcelo.leitner@...il.com>
To:	David Laight <David.Laight@...LAB.COM>
Cc:	"netdev@...r.kernel.org" <netdev@...r.kernel.org>,
	Neil Horman <nhorman@...driver.com>,
	Vlad Yasevich <vyasevich@...il.com>,
	David Miller <davem@...emloft.net>,
	"brouer@...hat.com" <brouer@...hat.com>,
	"alexander.duyck@...il.com" <alexander.duyck@...il.com>,
	"alexei.starovoitov@...il.com" <alexei.starovoitov@...il.com>,
	"borkmann@...earbox.net" <borkmann@...earbox.net>,
	"marek@...udflare.com" <marek@...udflare.com>,
	"hannes@...essinduktion.org" <hannes@...essinduktion.org>,
	"fw@...len.de" <fw@...len.de>,
	"pabeni@...hat.com" <pabeni@...hat.com>,
	"john.r.fastabend@...el.com" <john.r.fastabend@...el.com>,
	"linux-sctp@...r.kernel.org" <linux-sctp@...r.kernel.org>
Subject: Re: [RFC PATCH net-next 0/3] sctp: add GSO support

On Thu, Jan 28, 2016 at 05:30:24PM +0000, David Laight wrote:
> From: 'Marcelo Ricardo Leitner'
> > Sent: 28 January 2016 15:53
> > On Thu, Jan 28, 2016 at 01:51:02PM +0000, David Laight wrote:
> > > From: Marcelo Ricardo Leitner
> > > > Sent: 27 January 2016 17:07
> > > > This patchset is merely a RFC for the moment. There are some
> > > > controversial points that I'd like to discuss before actually proposing
> > > > the patches.
> > >
> > > You also need to look at how a 'user' can actually get SCTP to
> > > merge data chunks in the first place.
> > >
> > > With Nagle disabled (and it probably has to be since the data flow
> > > is unlikely to be 'command-response' or 'unidirectional bulk')
> > > it is currently almost impossible to get more than one chunk
> > > into an ethernet frame.
> > >
> > > Support for MSG_MORE would help.
> > >
> > > Given the current implementation you can get almost the required
> > > behaviour by turning nagle off and on repeatedly.
> > 
> > That's pretty much expected, I think. Without Nagle, if bandwidth and
> > cwnd allow, segment will be sent. GSO by itself shouldn't cause a
> > buffering to protect from that.
> > 
> > If something causes a bottleneck, tx may get queue up.  Like if I do a
> > stress test in my system, generally receiver side is slower than sender,
> > so I end up having tx buffers pretty easily. It mimics bandwidth
> > restrictions.
> 
> Imagine a using M2UA to connect local machines (one running mtp3, the other mtp2).
> Configure two linksets of 16 signalling links and perform a double-reflect
> loopback test.
> The SCTP connection won't ever saturate, so every msu ends up in its own
> ethernet packet.
> It is easy to generate 1000's of ethernet frames/sec on a single connection.
> 
> (We do this with something not entirely quite like M2UA over TCP,
> even then it is very hard to get multiple message into a single
> ethernet frame.)
> 

Agreed, GSO won't help much in there without a corking feature like
MSG_MORE or that on/off switch on Nagle you mentioned.

Thing about this (and also GRO) is on identifying how much can be spent
on waiting for the next chunk/packet without causing issues to the
application. Nagle is there and helps quite a lot, but timing-sensitive
applications will turn it off.

GSO will then rely on a bottleneck for causing tx to get buffered (hmm
there goes the timing sensitivity), which then can be GSOed without
problems, or it will require that application to provide some tips, like
MSG_MORE. But well, GSO is mainly meant for bulk stuff.

> > There is also the case of sending large data chunks, where
> > sctp_sendmsg() will segment it into smaller chunks already.
> 
> I presume they are merged before being passed to the receiving socket?

Yes. SCTP will reassemble the chunk and only deliver it to receiving
application when all pieces needed are there.

> > But yes, agreed, MSG_MORE is at least a welcomed compliment here,
> > specially for applications generating a train of chunks. Will put that in
> > my ToDo here, thanks.
> 
> I've posted a patch in the past for MSG_MORE, didn't quite work.

Ahh cool. Can you share the archive link please? Maybe I can take it
from there then.

> > > I did wonder whether the queued data could actually be picked up
> > > be a Heartbeat chunk that is probing a different remote address
> > > (which would be bad news).
> > 
> > I don't follow. You mean if a heartbeat may get stuck in queue or if
> > sending of a heartbeat can end up carrying additional data by accident?
> 
> My suspicion was that the heartbeat would carry the queued data.

I'm afraid I'm  still not following, sorry. You mean that this GSO patch
would cause the heartbeat to carry queued data? If yes, no, because for
SCTP side of it it mangles the packet size and make it look bigger
instead of handling multiple packets. It will then break this large
sctp_packet into several sk_buff and glue them together as if they were
GROed, allowing skb_segment to just split them back. The reason the
sctp_packet is generated, being it due to user data or control chunks
like heartbeats, is not modified.

  Marcelo