| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Date: Fri, 17 May 2024 18:15:53 +0200 From: Sebastian Andrzej Siewior <bigeasy@...utronix.de> To: Jesper Dangaard Brouer <hawk@...nel.org> Cc: Alexei Starovoitov <alexei.starovoitov@...il.com>, Toke Høiland-Jørgensen <toke@...hat.com>, LKML <linux-kernel@...r.kernel.org>, Network Development <netdev@...r.kernel.org>, "David S. Miller" <davem@...emloft.net>, Boqun Feng <boqun.feng@...il.com>, Daniel Borkmann <daniel@...earbox.net>, Eric Dumazet <edumazet@...gle.com>, Frederic Weisbecker <frederic@...nel.org>, Ingo Molnar <mingo@...hat.com>, Jakub Kicinski <kuba@...nel.org>, Paolo Abeni <pabeni@...hat.com>, Peter Zijlstra <peterz@...radead.org>, Thomas Gleixner <tglx@...utronix.de>, Waiman Long <longman@...hat.com>, Will Deacon <will@...nel.org>, Alexei Starovoitov <ast@...nel.org>, Andrii Nakryiko <andrii@...nel.org>, Eduard Zingerman <eddyz87@...il.com>, Hao Luo <haoluo@...gle.com>, Jiri Olsa <jolsa@...nel.org>, John Fastabend <john.fastabend@...il.com>, KP Singh <kpsingh@...nel.org>, Martin KaFai Lau <martin.lau@...ux.dev>, Song Liu <song@...nel.org>, Stanislav Fomichev <sdf@...gle.com>, Yonghong Song <yonghong.song@...ux.dev>, bpf <bpf@...r.kernel.org> Subject: Re: [PATCH net-next 14/15 v2] net: Reference bpf_redirect_info via task_struct on PREEMPT_RT. On 2024-05-14 14:20:03 [+0200], Jesper Dangaard Brouer wrote: > Trick for CPU-map to do early drop on remote CPU: > > # ./xdp-bench redirect-cpu --cpu 3 --remote-action drop ixgbe1 > > I recommend using Ctrl+\ while running to show more info like CPUs being > used and what kthread consumes. To catch issues e.g. if you are CPU > redirecting to same CPU as RX happen to run on. Okay. So I reworked the last two patches make the struct part of task_struct and then did as you suggested: Unpatched: |Sending: |Show adapter(s) (eno2np1) statistics (ONLY that changed!) |Ethtool(eno2np1 ) stat: 952102520 ( 952,102,520) <= port.tx_bytes /sec |Ethtool(eno2np1 ) stat: 14876602 ( 14,876,602) <= port.tx_size_64 /sec |Ethtool(eno2np1 ) stat: 14876602 ( 14,876,602) <= port.tx_unicast /sec |Ethtool(eno2np1 ) stat: 446045897 ( 446,045,897) <= tx-0.bytes /sec |Ethtool(eno2np1 ) stat: 7434098 ( 7,434,098) <= tx-0.packets /sec |Ethtool(eno2np1 ) stat: 446556042 ( 446,556,042) <= tx-1.bytes /sec |Ethtool(eno2np1 ) stat: 7442601 ( 7,442,601) <= tx-1.packets /sec |Ethtool(eno2np1 ) stat: 892592523 ( 892,592,523) <= tx_bytes /sec |Ethtool(eno2np1 ) stat: 14876542 ( 14,876,542) <= tx_packets /sec |Ethtool(eno2np1 ) stat: 2 ( 2) <= tx_restart /sec |Ethtool(eno2np1 ) stat: 2 ( 2) <= tx_stopped /sec |Ethtool(eno2np1 ) stat: 14876622 ( 14,876,622) <= tx_unicast /sec | |Receive: |eth1->? 8,732,508 rx/s 0 err,drop/s | receive total 8,732,508 pkt/s 0 drop/s 0 error/s | cpu:10 8,732,508 pkt/s 0 drop/s 0 error/s | enqueue to cpu 3 8,732,510 pkt/s 0 drop/s 7.00 bulk-avg | cpu:10->3 8,732,510 pkt/s 0 drop/s 7.00 bulk-avg | kthread total 8,732,506 pkt/s 0 drop/s 205,650 sched | cpu:3 8,732,506 pkt/s 0 drop/s 205,650 sched | xdp_stats 0 pass/s 8,732,506 drop/s 0 redir/s | cpu:3 0 pass/s 8,732,506 drop/s 0 redir/s | redirect_err 0 error/s | xdp_exception 0 hit/s I verified that the "drop only" case hits 14M packets/s while this redirect part reports 8M packets/s. Patched: |Sending: |Show adapter(s) (eno2np1) statistics (ONLY that changed!) |Ethtool(eno2np1 ) stat: 952635404 ( 952,635,404) <= port.tx_bytes /sec |Ethtool(eno2np1 ) stat: 14884934 ( 14,884,934) <= port.tx_size_64 /sec |Ethtool(eno2np1 ) stat: 14884928 ( 14,884,928) <= port.tx_unicast /sec |Ethtool(eno2np1 ) stat: 446496117 ( 446,496,117) <= tx-0.bytes /sec |Ethtool(eno2np1 ) stat: 7441602 ( 7,441,602) <= tx-0.packets /sec |Ethtool(eno2np1 ) stat: 446603461 ( 446,603,461) <= tx-1.bytes /sec |Ethtool(eno2np1 ) stat: 7443391 ( 7,443,391) <= tx-1.packets /sec |Ethtool(eno2np1 ) stat: 893086506 ( 893,086,506) <= tx_bytes /sec |Ethtool(eno2np1 ) stat: 14884775 ( 14,884,775) <= tx_packets /sec |Ethtool(eno2np1 ) stat: 14 ( 14) <= tx_restart /sec |Ethtool(eno2np1 ) stat: 14 ( 14) <= tx_stopped /sec |Ethtool(eno2np1 ) stat: 14884937 ( 14,884,937) <= tx_unicast /sec | |Receive: |eth1->? 8,735,198 rx/s 0 err,drop/s | receive total 8,735,198 pkt/s 0 drop/s 0 error/s | cpu:6 8,735,198 pkt/s 0 drop/s 0 error/s | enqueue to cpu 3 8,735,193 pkt/s 0 drop/s 7.00 bulk-avg | cpu:6->3 8,735,193 pkt/s 0 drop/s 7.00 bulk-avg | kthread total 8,735,191 pkt/s 0 drop/s 208,054 sched | cpu:3 8,735,191 pkt/s 0 drop/s 208,054 sched | xdp_stats 0 pass/s 8,735,191 drop/s 0 redir/s | cpu:3 0 pass/s 8,735,191 drop/s 0 redir/s | redirect_err 0 error/s | xdp_exception 0 hit/s This looks to be in the same range/ noise level. top wise I have ksoftirqd at 100% and cpumap/./map at ~60% so I hit CPU speed limit on a 10G link. perf top shows | 18.37% bpf_prog_4f0ffbb35139c187_cpumap_l4_hash [k] bpf_prog_4f0ffbb35139c187_cpumap_l4_hash | 13.15% [kernel] [k] cpu_map_kthread_run | 12.96% [kernel] [k] ixgbe_poll | 6.78% [kernel] [k] page_frag_free | 5.62% [kernel] [k] xdp_do_redirect for the top 5. Is this something that looks reasonable? Sebastian
Powered by blists - more mailing lists