netdev - Re: Question regarding {G,S}CHANNELS API

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20160108114139.74a35521@jkicinski-Precision-T1700>
Date:	Fri, 8 Jan 2016 11:41:39 +0000
From:	Jakub Kicinski <jakub.kicinski@...ronome.com>
To:	Prashant Sreedharan <prashant@...adcom.com>
Cc:	"netdev@...r.kernel.org" <netdev@...r.kernel.org>,
	David Miller <davem@...emloft.net>,
	Michael Chan <mchan@...adcom.com>,
	Ben Hutchings <ben@...adent.org.uk>, siva.kallam@...adcom.com
Subject: Re: Question regarding {G,S}CHANNELS API

On Thu, 7 Jan 2016 13:24:24 -0800, Prashant Sreedharan wrote:
> On Thu, 2016-01-07 at 15:43 +0000, Jakub Kicinski wrote:
> > Hi!
> >
> > I'm trying to understand how number of "separate" rx/tx vs combined
> > channels should be configured.  I'd like to express asymmetric but
> > mostly combined queue configuration (i.e. min(rx, tx) is combined the
> > rest is separate).  Since default number of RX queues is just 8 it is
> > tempting to allocate 8 RX queues but num_online_cpus() TX queues.  Does
> > this configuration make sense?  What should ethtool -l report?
> >
> > I had a look through the drivers and it seems that most of them fall
> > nicely into the all combined and all separate categories. Two
> > exceptions I found are bnxt_en (recent Michael's changes) and tg3.
> > bnxt_en can switch between combined and separate mode.  tg3 is more
> > interesting, it uses separate rx/tx parameters but combines first
> > min(rx, tx) of the queues as I would like to.
>
> It does not combine it takes the max(rx, tx)
> u32 irq_cnt = max(tp->rxq_cnt, tp->txq_cnt);
>
> >
> > The problem is if I hijack the "separate" rx/tx queues parameters like
> > tg3 does there is no way for the user to express that (s)he truly wants
> > them separate.  Also it seems that tg3 goes against the ethtool manual
> > which states:
> > >  rx N   Changes the number of channels with only receive queues.
> > >  tx N   Changes the number of channels with only transmit queues.
> > Which indicates that if user wants 8 rx, 12 tx and combining (s)he
> > should do:
> >   ethtool -L ethX combined 8 tx 4
>
> there is no combined option only separate tx/rx option is supported by
> tg3 driver. And at max only 4 RX, 4 TX rings are supported by the tg3
> family of chips in non-iov mode.
>
> Each status block (which is similar to completion ring) supports 1 RX or
> 1 TX or both. So at max there are 4 status blocks plus 1 additional
> status block for async events. And each status block is assigned a irq.

Thanks for response Prashant.

I think what you describe is exactly what is meant by combined rx/tx
rings.  Single status block used for both RX and TX. tg3 can allocate
up to max_rx RX queues and up to max_tx TX queues, the numbers are
independent but then first min(rx, tx) will get combined and use the
same block and IRQ, right?

The point of my email was that it's a bit cumbersome to express such
configuration with the current ethtool API.  Therefore I'm asking if
it's preferred to use:
  ethtool ethX -L rx N tx M
in this case, or:
  ethtool ethX -L combined N tx/rx abs(M-N)
If the second then I think I'll post a patch to extend ethtool's help
since it may be non-obvious to users.