netdev - Re: [PATCH RFC v1 net-next 0/4] net: Qdisc backpressure infrastructure

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20220510230347.GA11152@bytedance>
Date:   Tue, 10 May 2022 16:03:47 -0700
From:   Peilin Ye <yepeilin.cs@...il.com>
To:     Eric Dumazet <eric.dumazet@...il.com>
Cc:     "David S. Miller" <davem@...emloft.net>,
        Jakub Kicinski <kuba@...nel.org>,
        Paolo Abeni <pabeni@...hat.com>,
        Hideaki YOSHIFUJI <yoshfuji@...ux-ipv6.org>,
        David Ahern <dsahern@...nel.org>,
        Jamal Hadi Salim <jhs@...atatu.com>,
        Cong Wang <xiyou.wangcong@...il.com>,
        Jiri Pirko <jiri@...nulli.us>,
        Peilin Ye <peilin.ye@...edance.com>, netdev@...r.kernel.org,
        linux-kernel@...r.kernel.org, Cong Wang <cong.wang@...edance.com>
Subject: Re: [PATCH RFC v1 net-next 0/4] net: Qdisc backpressure
 infrastructure

Hi Eric,

On Mon, May 09, 2022 at 08:26:27PM -0700, Eric Dumazet wrote:
> On 5/6/22 12:43, Peilin Ye wrote:
> > From: Peilin Ye <peilin.ye@...edance.com>
> > 
> > Hi all,
> > 
> > Currently sockets (especially UDP ones) can drop a lot of skbs at TC
> > egress when rate limited by shaper Qdiscs like HTB.  This experimental
> > patchset tries to improve this by introducing a backpressure mechanism, so
> > that sockets are temporarily throttled when they "send too much".
> > 
> > For now it takes care of TBF, HTB and CBQ, for UDP and TCP sockets.  Any
> > comments, suggestions would be much appreciated.  Thanks!
> 
> This very much looks like trying to solve an old problem to me.
> 
> If you were using EDT model, a simple eBPF program could get rid of the
> HTB/TBF qdisc
> 
> and you could use MQ+FQ as the packet schedulers, with the true multiqueue
> sharding.
> 
> FQ provides fairness, so a flow can not anymore consume all the qdisc limit.

This RFC tries to solve the "when UDP starts to drop (whether because of
per-flow or per-Qdisc limit), it drops a lot" issue described in [I] of
the cover letter; its main goal is not to improve fairness.

> (If your UDP sockets send packets all over the place (not connected
> sockets),
> 
> then the eBPF can also be used to rate limit them)

I was able to reproduce the same issue using EDT: default sch_fq
flow_limit (100 packets), with a 1 Gbit/sec rate limit.  Now if I run
this:

  $ iperf -u -c 1.2.3.4 -p 5678 -l 3K -i 0.5 -t 30 -b 3g

  [ ID] Interval       Transfer     Bandwidth
  [  3]  0.0- 0.5 sec   137 MBytes  2.29 Gbits/sec
  [  3]  0.5- 1.0 sec   142 MBytes  2.38 Gbits/sec
  [  3]  1.0- 1.5 sec   117 MBytes  1.96 Gbits/sec
  [  3]  1.5- 2.0 sec   105 MBytes  1.77 Gbits/sec
  [  3]  2.0- 2.5 sec   132 MBytes  2.22 Gbits/sec
  <...>                             ^^^^^^^^^^^^^^

On average it tries to send 2.31 Gbits per second, dropping 56.71% of
the traffic:

  $ tc -s qdisc show dev eth0
  <...>
  qdisc fq 5: parent 1:4 limit 10000p flow_limit 100p buckets 1024 orphan_mask 1023 quantum 18030 initial_quantum 90150 low_rate_threshold 550Kbit refill_delay 40.0ms
   Sent 16356556 bytes 14159 pkt (dropped 2814461, overlimits 0 requeues 0)
                                  ^^^^^^^^^^^^^^^

This RFC does not cover EDT though, since it does not use Qdisc watchdog
or friends.

Thanks,
Peilin Ye