netdev - RE: [PATCH net-next 1/2] net: Keep sk->sk_forward

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <CH3PR11MB7345DBA6F79282169AAFE9E0FC759@CH3PR11MB7345.namprd11.prod.outlook.com>
Date: Fri, 12 May 2023 05:51:40 +0000
From: "Zhang, Cathy" <cathy.zhang@...el.com>
To: Shakeel Butt <shakeelb@...gle.com>
CC: Eric Dumazet <edumazet@...gle.com>, Linux MM <linux-mm@...ck.org>, Cgroups
	<cgroups@...r.kernel.org>, Paolo Abeni <pabeni@...hat.com>,
	"davem@...emloft.net" <davem@...emloft.net>, "kuba@...nel.org"
	<kuba@...nel.org>, "Brandeburg@...gle.com" <Brandeburg@...gle.com>,
	"Brandeburg, Jesse" <jesse.brandeburg@...el.com>, "Srinivas, Suresh"
	<suresh.srinivas@...el.com>, "Chen, Tim C" <tim.c.chen@...el.com>, "You,
 Lizhen" <lizhen.you@...el.com>, "eric.dumazet@...il.com"
	<eric.dumazet@...il.com>, "netdev@...r.kernel.org" <netdev@...r.kernel.org>
Subject: RE: [PATCH net-next 1/2] net: Keep sk->sk_forward_alloc as a proper
 size



> -----Original Message-----
> From: Shakeel Butt <shakeelb@...gle.com>
> Sent: Friday, May 12, 2023 1:07 PM
> To: Zhang, Cathy <cathy.zhang@...el.com>
> Cc: Eric Dumazet <edumazet@...gle.com>; Linux MM <linux-
> mm@...ck.org>; Cgroups <cgroups@...r.kernel.org>; Paolo Abeni
> <pabeni@...hat.com>; davem@...emloft.net; kuba@...nel.org;
> Brandeburg@...gle.com; Brandeburg, Jesse <jesse.brandeburg@...el.com>;
> Srinivas, Suresh <suresh.srinivas@...el.com>; Chen, Tim C
> <tim.c.chen@...el.com>; You, Lizhen <lizhen.you@...el.com>;
> eric.dumazet@...il.com; netdev@...r.kernel.org
> Subject: Re: [PATCH net-next 1/2] net: Keep sk->sk_forward_alloc as a proper
> size
> 
> On Fri, May 12, 2023 at 03:23:45AM +0000, Zhang, Cathy wrote:
> > Remove the invalid mail addr added unintentionally.
> >
> 
> Sorry that was my buggy script.
> 
> [...]
> > >
> > > Hi Shakeel,
> > >
> > > Run with the temp change you provided,  the output shows it comes to
> > > drain_stock_1(), Here is the call trace:
> > >
> > >      8.96%  mc-worker        [kernel.vmlinux]            [k] page_counter_cancel
> > >             |
> > >              --8.95%--page_counter_cancel
> > >                        |
> > >                         --8.95%--page_counter_uncharge
> > >                                   drain_stock_1
> > >                                   __refill_stock
> > >                                   refill_stock
> > >                                   |
> > >                                    --8.88%--try_charge_memcg
> > >                                              mem_cgroup_charge_skmem
> > >                                              |
> > >                                               --8.87%--__sk_mem_raise_allocated
> > >                                                         __sk_mem_schedule
> > >                                                         |
> > >                                                         |--5.37%--tcp_try_rmem_schedule
> > >                                                         |          tcp_data_queue
> > >                                                         |          tcp_rcv_established
> > >                                                         |          tcp_v4_do_rcv
> >
> 
> Thanks a lot. This tells us that one or both of following scenarios are
> happening:
> 
> 1. In the softirq recv path, the kernel is processing packets from multiple
> memcgs.
> 
> 2. The process running on the CPU belongs to memcg which is different from
> the memcgs whose packets are being received on that CPU.

Thanks for sharing the points, Shakeel! Is there any trace records you want to
collect?

> 
> BTW have you seen this performance issue when you run the client and
> server on different machines? I am wondering if RFS would be good enough
> for such scenario and we only need to worry about the same machine case.

The same test scenario is tried between two machines which are on the same
network but with network topology, both of the two machines are with 1Gbit NIC.
There is no hot paths observed, but it might because that the systems are
not so busy, for memtier shows the average transfer speed is down to a few
tenth of original on the same machine.