Message-ID: <20090521072050.GA2892@ami.dom.local>
Date:	Thu, 21 May 2009 09:20:50 +0200
From:	Jarek Poplawski <jarkao2@...il.com>
To:	Eric Dumazet <dada1@...mosbay.com>
Cc:	Vladimir Ivashchenko <hazard@...ncoudi.com>,
	netdev@...r.kernel.org, kaber@...sh.net, davem@...emloft.net,
	devik@....cz, Antonio Almeida <vexwek@...il.com>,
	Corey Hickey <bugfood-ml@...ooh.org>
Subject: Re: HTB accuracy for high speed

On Thu, May 21, 2009 at 12:46:16AM +0200, Eric Dumazet wrote:
> Vladimir Ivashchenko a écrit :
> >>>> I guess you should send some logs. Your previous report seem to show
> >>> Can you give some hints on which logs you would like to see?
> >> Similarly to Antonio's: ifconfigs and tc -s for qdiscs and classes at
> >> the beginning and at the end of testing.
> > 
> > Ok, it seems that I finally found what is causing my HTB on 2.6.29 not
> > to reach full throughput: dst hashing on sfq with high divisor value.
> > 
> > 2.6.21 esfq divisor 13 depth 4096 hash dst - 680 mbps
> > 2.6.29 sfq WITHOUT "flow hash keys dst ... " (default sfq) - 680 mbps
> > 2.6.29 sfq + "flow hash keys dst divisor 64" filter - 680 mbps
> > 2.6.29 sfq + "flow hash keys dst divisor 256" filter - 660 mbps
> > 2.6.29 sfq + "flow hash keys dst divisor 2048" filters - 460 mbps
> > 
> > I'm using high sfq hash divisor in order to decrease the number of
> > collisions, there are several thousands of hosts behind each of the
> > classes. 
> > 
> > Any ideas why increasing the sfq divisor size results in drop of
> > throughput ?
> > 
> > Attached are diagnostics gathered in case of divisor 2048.
> > 
> 
> 
> But... it appears sfq currently supports a fixed divisor of 1024
> 
> net/sched/sch_sfq.c
> 
>  IMPLEMENTATION:
>  This implementation limits maximal queue length to 128;
>  maximal mtu to 2^15-1; number of hash buckets to 1024.
>  The only goal of this restrictions was that all data
>  fit into one 4K page :-). Struct sfq_sched_data is
>  organized in anti-cache manner: all the data for a bucket
>  are scattered over different locations. This is not good,
>  but it allowed me to put it into 4K.
> 
>  It is easy to increase these values, but not in flight.  */
> 
> #define SFQ_DEPTH   128
> #define SFQ_HASH_DIVISOR    1024
> 
> 
> Apparently Corey Hickey 2007 work on SFQ was not merged.
> 
> http://kerneltrap.org/mailarchive/linux-netdev/2007/9/28/325048

Yes, sfq has its design limits, and as a matter of fact, because of
max length (127) it should be treated as a toy or "personal" qdisc.

I don't know why more of esfq wasn't merged, anyway similar
functionality could be achieved in current kernels with sch_drr +
cls_flow, alas not enough documented. Here is some hint:
http://markmail.org/message/h24627xkrxyqxn4k

Jarek P.

PS: I guess, you wasn't very consistent if your main problem was
exceeding or not reaching htb rate, and there is quite a difference.

Vladimir Ivashchenko wrote, On 05/08/2009 10:46 PM:

> Exporting HZ=1000 doesn't help. However, even if I recompile the kernel
> to 1000 Hz and the burst is calculated correctly, for some reason HTB on
> 2.6.29 is still worse at rate control than 2.6.21.
> 
> With 2.6.21, ceil of 775 mbits, burst 99425b -> actual rate 825 mbits.
> With 2.6.29, same ceil/burst -> actual rate 890 mbits.
...

Vladimir Ivashchenko wrote, On 05/17/2009 10:29 PM:

> Hi Antonio,
> 
> FYI, these are exactly the same problems I get in real life.
> Check the later posts in "bond + tc regression" thread.
...
--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Powered by blists - more mailing lists