netdev - Re: kernel mode pppoe ppp if + ifb + mirred redirect, ethernet packets in ifb?!

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-Id: <200910131545.36946.denys@visp.net.lb>
Date:	Tue, 13 Oct 2009 15:45:36 +0300
From:	Denys Fedoryschenko <denys@...p.net.lb>
To:	hadi@...erus.ca
Cc:	netdev@...r.kernel.org
Subject: Re: kernel mode pppoe ppp if + ifb + mirred redirect, ethernet packets in ifb?!


> I bet pppd in user space is probably your biggest problem in terms of
> performance.
No. :-)
First problem, on restart and massive users login i am getting some weird 
locking problem with modprobe,probably related to busybox/mdev.
When interface come up, maybe it is problem of mdev, it is trying to do 
modprobe with ip address as attribute. Probably they must not modprobe on 
virtual interface. It is not easy to track who is reason of such call.

As result:
[  174.564503] request_module: runaway loop modprobe 172.16.3.1
[  174.564702] request_module: runaway loop modprobe 172.16.3.1
[  174.801355] request_module: runaway loop modprobe 172.16.106.1
[  174.801487] request_module: runaway loop modprobe 172.16.106.1
[  175.011415] request_module: runaway loop modprobe 172.16.106.1
load average (even it doesn't mean anything in terms of CPU load) jumps to 
72-80.

Another bottleneck is u32 (i can optimize but) and some strange locks 
appearing at top of perf, maybe same as logon case.
And yes, pppd also appearing, but seems just registering new sysctls (for new 
interface?) takes a lot of resources.

I can post perf -a -f g if you are interested for "logging in" case and 
regular operation.

>
> > After switching to skbedit things improve a lot (before 1k users was near
> > max)
>
> Not using netfilter will improve your numbers. So can skbedit do fwmark
> as well?
I dont need it, i am using skb->priority as a key for flow classifier.
It looks weird, a lot of obsolete code there, but it is very nice, i dont need 
to touch almost anything on ifb, and i can predefine even on startup required 
amount of classes and just change rate when required.

Here is key part of shaper code:

$TC filter add dev ifb0 protocol ip pref 32 parent 1: handle 1 flow map key 
priority baseclass 1:64

Then i create classes for id's i need (at my case id related to ppp interface 
number).

and when ppp interface come up also:
(lowid is related to ppp interface or user id)

filter add dev $2 parent ffff: protocol ip prio 10 u32 \
match u32 0 0 flowid 1:1 \
action skbedit priority 0x${lowid} pipe \
action mirred egress redirect dev ifb0

>
> yes, something like that.
> It may be easier to tcpdump -x on both pppoe and ifb and see how the
> packets look like at what offset. If that doesnt work well, I will work
> on a patch...
Well another way is just to use  as you suggest - egress on output 
interface(s).
>
> cheers,
> jamal


--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html