netdev - Re: [RFC v3] net: sched: implement TCQ_F_CAN

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <20210323113900.g3c2mg5z4elamyz3@skbuf>
Date:   Tue, 23 Mar 2021 13:39:00 +0200
From:   Vladimir Oltean <olteanv@...il.com>
To:     Yunsheng Lin <linyunsheng@...wei.com>
Cc:     davem@...emloft.net, kuba@...nel.org, ast@...nel.org,
        daniel@...earbox.net, andriin@...com, edumazet@...gle.com,
        weiwan@...gle.com, cong.wang@...edance.com, ap420073@...il.com,
        netdev@...r.kernel.org, linux-kernel@...r.kernel.org,
        linuxarm@...neuler.org, mkl@...gutronix.de,
        linux-can@...r.kernel.org, jhs@...atatu.com,
        xiyou.wangcong@...il.com, jiri@...nulli.us, andrii@...nel.org,
        kafai@...com, songliubraving@...com, yhs@...com,
        john.fastabend@...il.com, kpsingh@...nel.org, bpf@...r.kernel.org,
        jonas.bonn@...rounds.com, pabeni@...hat.com, mzhivich@...mai.com,
        johunt@...mai.com, albcamus@...il.com, kehuan.feng@...il.com,
        a.fatoum@...gutronix.de
Subject: Re: [RFC v3] net: sched: implement TCQ_F_CAN_BYPASS for lockless
 qdisc

On Mon, Mar 22, 2021 at 10:00:33PM +0200, Vladimir Oltean wrote:
> Hi Yunsheng,
> 
> On Mon, Mar 22, 2021 at 05:09:16PM +0800, Yunsheng Lin wrote:
> > Currently pfifo_fast has both TCQ_F_CAN_BYPASS and TCQ_F_NOLOCK
> > flag set, but queue discipline by-pass does not work for lockless
> > qdisc because skb is always enqueued to qdisc even when the qdisc
> > is empty, see __dev_xmit_skb().
> > 
> > This patch calls sch_direct_xmit() to transmit the skb directly
> > to the driver for empty lockless qdisc too, which aviod enqueuing
> > and dequeuing operation. qdisc->empty is set to false whenever a
> > skb is enqueued, see pfifo_fast_enqueue(), and is set to true when
> > skb dequeuing return NULL, see pfifo_fast_dequeue().
> > 
> > There is a data race between enqueue/dequeue and qdisc->empty
> > setting, qdisc->empty is only used as a hint, so we need to call
> > sch_may_need_requeuing() to see if the queue is really empty and if
> > there is requeued skb, which has higher priority than the current
> > skb.
> > 
> > The performance for ip_forward test increases about 10% with this
> > patch.
> > 
> > Signed-off-by: Yunsheng Lin <linyunsheng@...wei.com>
> > ---
> > Hi, Vladimir and Ahmad
> > 	Please give it a test to see if there is any out of order
> > packet for this patch, which has removed the priv->lock added in
> > RFC v2.
> > 
> > There is a data race as below:
> > 
> >       CPU1                                   CPU2
> > qdisc_run_begin(q)                            .
> >         .                                q->enqueue()
> > sch_may_need_requeuing()                      .
> >     return true                               .
> >         .                                     .
> >         .                                     .
> >     q->enqueue()                              .
> > 
> > When above happen, the skb enqueued by CPU1 is dequeued after the
> > skb enqueued by CPU2 because sch_may_need_requeuing() return true.
> > If there is not qdisc bypass, the CPU1 has better chance to queue
> > the skb quicker than CPU2.
> > 
> > This patch does not take care of the above data race, because I
> > view this as similar as below:
> > 
> > Even at the same time CPU1 and CPU2 write the skb to two socket
> > which both heading to the same qdisc, there is no guarantee that
> > which skb will hit the qdisc first, becuase there is a lot of
> > factor like interrupt/softirq/cache miss/scheduling afffecting
> > that.
> > 
> > So I hope the above data race will not cause problem for Vladimir
> > and Ahmad.
> > ---
> 
> Preliminary results on my test setup look fine, but please allow me to
> run the canfdtest overnight, since as you say, races are still
> theoretically possible.

I haven't found any issues during the overnight test and until now.

Tested-by: Vladimir Oltean <vladimir.oltean@....com> # flexcan