[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-Id: <20080328073854.M73368@visp.net.lb>
Date: Fri, 28 Mar 2008 09:38:54 +0200
From: "Denys Fedoryshchenko" <denys@...p.net.lb>
To: netdev@...r.kernel.org, netfilter-devel@...r.kernel.org
Subject: Re: kernel 2.6.25-rc7 highly unstable on high load
Already patched and tested, it doesn't change anything.
On Fri, 28 Mar 2008 06:49:53 +0100, Eric Dumazet wrote
> Denys Fedoryshchenko a :
> > Just to make sure 2.6.24.3 is stable and it is regression i am supplying
> > output from it.
> > Do you want me to submit summary to bugzilla and regression list as well?
> >
> > And in short, IMHO 2.6.25 have major issues on routing that have to be
fixed
> > before release. TRIE is crashing, and even with HASH there is leak. I am
> > trying my best to bisect it, but it is major router and i cannot take
much
> > risk on it, so i wish i can simulate in my home mini-lab. Still i am not
able
> > to get even proper switch (Lebanon difficult country for IT).
> >
> > Kup ~ # uname -a
> > Linux Kup 2.6.24.3-build-0023 #3 SMP Sat Mar 8 13:01:35 EET 2008 i686
unknown
> >
> > up ~ # rtstat -i60 -c6000
> > rt_cache|rt_cache|rt_cache|rt_cache|rt_cache|rt_cache|rt_cache|rt_cache|
> > rt_cache|rt_cache|rt_cache|rt_cache|rt_cache|rt_cache|rt_cache|rt_cache|
> > rt_cache|
> > entries| in_hit|in_slow_|in_slow_|in_no_ro| in_brd|in_marti|in_marti|
> > out_hit|out_slow|out_slow|gc_total|gc_ignor|gc_goal_|gc_dst_o|in_hlist|
> > out_hlis|
> > | | tot| mc| ute| | an_dst|
> > an_src| | _tot| _mc| | ed| miss| verflow|
> > _search|t_search|
> > 54750| 4430| 1128| 0| 12| 0| 0|
0|
> > 263| 190| 0| 709| 708| 0| 0| 3545|
313|
> > 92913| 8829| 1211| 0| 1| 0| 0|
0|
> > 343| 163| 0| 1375| 1373| 0| 0| 12545|
724|
> > 115323| 8232| 906| 0| 0| 0| 0|
0|
> > 299| 128| 0| 1035| 1033| 0| 0| 18069|
813|
> > 128985| 8650| 839| 0| 0| 0| 0|
0|
> > 289| 115| 0| 954| 952| 0| 0| 22515|
845|
> > 116682| 8911| 861| 0| 0| 0| 0|
0|
> > 288| 117| 0| 978| 976| 0| 0| 23433|
775|
> > 99969| 9164| 889| 0| 0| 0| 0|
0|
> > 280| 113| 0| 1002| 1000| 0| 0| 26741|
839|
> > 124602| 9395| 1012| 0| 0| 0| 0|
0|
> > 271| 122| 0| 1134| 1132| 0| 0| 27381|
787|
> > 110051| 10036| 824| 0| 0| 0| 0|
0|
> > 279| 120| 0| 944| 942| 0| 0| 28558|
783|
> > 126835| 10631| 772| 0| 0| 0| 0|
0|
> > 274| 117| 0| 888| 886| 0| 0| 29451|
780|
> > 111881| 10357| 762| 0| 0| 0| 0|
0|
> > 275| 117| 0| 879| 877| 0| 0| 28235|
751|
> > 127018| 10178| 796| 0| 0| 0| 0|
0|
> > 283| 117| 0| 913| 911| 0| 0| 29480|
807|
> > 112242| 9839| 814| 0| 0| 0| 0|
0|
> > 293| 115| 0| 929| 927| 0| 0| 28095|
796|
> > 41267| 9493| 1217| 0| 1| 0| 0|
0|
> > 269| 138| 0| 811| 810| 0| 0| 18545|
548|
> > 76380| 9722| 1060| 0| 1| 0| 0|
0|
> > 250| 135| 0| 1195| 1193| 0| 0| 14786|
414|
> > 99922| 9811| 779| 0| 0| 0| 0|
0|
> > 281| 124| 0| 902| 900| 0| 0| 21853|
589|
> >
> > Kup ~ # rtstat -i60 -c6000
> > rt_cache|rt_cache|rt_cache|rt_cache|rt_cache|rt_cache|rt_cache|rt_cache|
> > rt_cache|rt_cache|rt_cache|rt_cache|rt_cache|rt_cache|rt_cache|rt_cache|
> > rt_cache|
> > entries| in_hit|in_slow_|in_slow_|in_no_ro| in_brd|in_marti|in_marti|
> > out_hit|out_slow|out_slow|gc_total|gc_ignor|gc_goal_|gc_dst_o|in_hlist|
> > out_hlis|
> > | | tot| mc| ute| | an_dst|
> > an_src| | _tot| _mc| | ed| miss| verflow|
> > _search|t_search|
> >
> > 122053| 150955| 14888| 0| 25| 1| 0|
0|
> > 4611| 2090| 0| 15820| 15789| 0| 0| 369513|
11562|
> > 105226| 10215| 872| 0| 0| 0| 0|
0|
> > 279| 116| 0| 988| 986| 0| 0| 30343|
799|
> > 126236| 10462| 924| 0| 0| 0| 0|
0|
> > 260| 120| 0| 1044| 1042| 0| 0| 31699|
782|
> > 114492| 9782| 884| 0| 0| 0| 0|
0|
> > 253| 120| 0| 1005| 1003| 0| 0| 29695|
722|
> >
> > After ip route flush cache
> > Kup ~ # rtstat -i60 -c6000
> > rt_cache|rt_cache|rt_cache|rt_cache|rt_cache|rt_cache|rt_cache|rt_cache|
> > rt_cache|rt_cache|rt_cache|rt_cache|rt_cache|rt_cache|rt_cache|rt_cache|
> > rt_cache|
> > entries| in_hit|in_slow_|in_slow_|in_no_ro| in_brd|in_marti|in_marti|
> > out_hit|out_slow|out_slow|gc_total|gc_ignor|gc_goal_|gc_dst_o|in_hlist|
> > out_hlis|
> > | | tot| mc| ute| | an_dst|
> > an_src| | _tot| _mc| | ed| miss| verflow|
> > _search|t_search|
> > 9088| 202136| 19262| 0| 29| 1| 0|
0|
> > 5976| 2696| 0| 20647| 20606| 0| 0| 521714|
15415|
> >
> >
> > !!!!!
> > I am not wrong, ip route flush cache doesn't work at 2.6.25-rc7. I will
make
> > sure about that now.
> >
> Maybe you are a litle bit too fast for "ip route flush cache" :)
>
> It used to work like that : schedule a timer to start a flush in
> about 2 seconds. A flush meaning : scan the whole table and delete
> all entries.
>
> On machines with 4 millions dst entries, this was using too much
> time and eventually crashing.
>
> On recent kernels, each rtable entry has a special field named
> rt_genid, so that "ip route flush cache" doesnt have to scan the
> whole table, but only change the global genid. rtables entries will
> be deleted later, when their rt_genid is found to be different than
> the global genid.
>
> Please try the patch that was suggested yesterday, as it is probably
> the cure your router needs.
>
> http://git2.kernel.org/?p=linux/kernel/git/davem/net-
2.6.git;a=commitdiff;h=7c0ecc4c4f8fd90988aab8a95297b9c0038b6160
>
> Thank you
>
> --
> To unsubscribe from this list: send the line "unsubscribe netdev" in
> the body of a message to majordomo@...r.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
--
Denys Fedoryshchenko
Technical Manager
Virtual ISP S.A.L.
--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Powered by blists - more mailing lists