netdev - Strange CPU load when flushing route cache (kernel 2.6.31.6)

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Message-ID: <1258970332.29747.262.camel@jdb-workstation>
Date:	Mon, 23 Nov 2009 10:58:52 +0100
From:	Jesper Dangaard Brouer <jdb@...x.dk>
To:	Eric Dumazet <eric.dumazet@...il.com>
Cc:	Linux Kernel Network Hackers <netdev@...r.kernel.org>,
	Robert Olsson <robert@...julf.net>
Subject: Strange CPU load when flushing route cache (kernel 2.6.31.6)

Hi Eric and netdev,

I have observed a strange route cache behaviour when I upgraded some
of my production Linux routers (1Gbit/s tg3) to kernel 2.6.31.6 (from
kernel 2.6.25.7).

Every time the route cache is flushed I get a CPU spike (in softirq)
with a tail.  I have attached some graphs that illustrate the issue
(hope vger.kernel.org will allow these attachments...)

I have done some tuning of the route cache:

 # From /etc/sysctl.conf
 #
 # Adjusting the route cache flush interval
 net/ipv4/route/secret_interval = 1200

 # Limiting the route cache size
 # ip_dst_cache slab objects is 256 bytes.
 # 2000000 * 256 bytes = 512 MB
 net/ipv4/route/max_size = 2000000

Boot parameters: "rhash_entries=262143 vmalloc=256M"

The rhash_entries is for the route cache hash size.  The vmalloc is
needed because I have _very_ large iptables rulesets (and is running
on a 32-bit kernel, due to old hardware).

Any thoughs on how to avoid these CPU spikes?
Or where the issue occurs in the code?

-- 
Med venlig hilsen / Best regards
  Jesper Brouer
  ComX Networks A/S
  Linux Network Kernel Developer
  Cand. Scient Datalog / MSc.CS
  Author of http://adsl-optimizer.dk
  LinkedIn: http://www.linkedin.com/in/brouer

Download attachment "CPU_usage.png" of type "image/png" (16710 bytes)

Download attachment "CPU_usage_softirq.png" of type "image/png" (15344 bytes)

Download attachment "PPS_eth1-rx.png" of type "image/png" (10930 bytes)

Download attachment "route_cache.png" of type "image/png" (17499 bytes)

Download attachment "softnet_time_squeeze.png" of type "image/png" (15145 bytes)