Message-ID: <20150302180659.GC7971@gospo.home.greyhouse.net>
Date:	Mon, 2 Mar 2015 13:06:59 -0500
From:	Andy Gospodarek <gospo@...ulusnetworks.com>
To:	Scott Feldman <sfeldma@...il.com>
Cc:	Tom Herbert <therbert@...gle.com>,
	Neil Horman <nhorman@...driver.com>,
	Simon Horman <simon.horman@...ronome.com>,
	John Fastabend <john.r.fastabend@...el.com>,
	Thomas Graf <tgraf@...g.ch>, Jiri Pirko <jiri@...nulli.us>,
	Linux Netdev List <netdev@...r.kernel.org>,
	David Miller <davem@...emloft.net>,
	Andy Gospodarek <andy@...yhouse.net>,
	Daniel Borkmann <dborkman@...hat.com>,
	Or Gerlitz <ogerlitz@...lanox.com>,
	Jesse Gross <jesse@...ira.com>, jpettit@...ira.com,
	Joe Stringer <joestringer@...ira.com>,
	Jamal Hadi Salim <jhs@...atatu.com>,
	Florian Fainelli <f.fainelli@...il.com>,
	Roopa Prabhu <roopa@...ulusnetworks.com>,
	John Linville <linville@...driver.com>,
	Shrijeet Mukherjee <shrijeet@...il.com>,
	Benjamin LaHaise <bcrl@...ck.org>
Subject: Re: Flows! Offload them.

On Mon, Mar 02, 2015 at 08:54:06AM -0800, Scott Feldman wrote:
> On Mon, Mar 2, 2015 at 5:49 AM, Andy Gospodarek
> <gospo@...ulusnetworks.com> wrote:
> > On Thu, Feb 26, 2015 at 05:52:16PM -0800, Tom Herbert wrote:
> >> On Thu, Feb 26, 2015 at 5:22 PM, Neil Horman <nhorman@...driver.com> wrote:
> > [...]
> >> > Yes, exactly that, for the general traditional networking use case, that is
> >> > exactly what we want, to opportunistically move traffic faster with less load on
> >> > the cpu.  We don't nominally care what traffic is offloaded, as long as the
> >> > hardware does a better job than just software alone.  If we get an occasional
> >> > miss and have to do stuff in software, so be it.
> >> >
> >> +1 on an in kernel "Network Resource Manager". This also came up in
> >> Sunil's plan to configure RPS affinities from a driver so I'm taking
> >> liberty by generalizing the concept :-).
> >
> > I agree completely that there is a need for what you both describe.  Not
> > only to handle what items to offload for users looking for that level
> > of granularity, but also to allow driver implementers to decide if their
> > hardware/driver implementation may allow for async write of data to
> > hardware tables, and any other implementation specific detail they may
> > want to provide.
> 
> Can you elaborate on "allow for async write of data to hardware
> tables"?  Is this the trampoline model where user's request goes to
> the kernel, and then back to user-space, and finally to the hardware
> via an user-space SDK?I think we should exclude that model from
> discussions about resource management.  With the recent L2/L3 offload
> work, I'm advocating a synchronous call path from user to kernel to
> hardware so we can return a actionable result code, and put the burden
> of resource management in user-space, not in the kernel.

No, this would not be to enable a trampoline driver.  While that is
interesting to some, developing explicit infra upstream to enable such a
thing is not interesting to me.

This comes on the heels of recent discussions I've had with more than
one vendor who have discussed the algorithmic complexity that often
exists in their driver and the concern about whether or not they could
allow some calls down to hardware to be async when it is known that
there is no real pressure on the available space in any given table.

When a driver would alert that limited space would be available, I would
see these calls going synchronous to be positive that they would be
allowed to complete and to allow a more standard error path.  Since
behavior might change slightly depending on available resources, it
seemed like a good topic to discuss when discussing a resource manager.  

This may be an optimization that is not needed in the base version of a
resource manager and if that is the case, that is fine with me. :)

--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Powered by blists - more mailing lists