netdev - RE: [RFC -next 2/2] net: sched: red: don't reset the backlog on every stat dump

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <DB5PR05MB18954F403296865ABE464362AC130@DB5PR05MB1895.eurprd05.prod.outlook.com>
Date:   Mon, 8 Jan 2018 09:18:08 +0000
From:   Nogah Frankel <nogahf@...lanox.com>
To:     Jakub Kicinski <jakub.kicinski@...ronome.com>,
        "john.fastabend@...il.com" <john.fastabend@...il.com>,
        "jiri@...nulli.us" <jiri@...nulli.us>,
        "xiyou.wangcong@...il.com" <xiyou.wangcong@...il.com>,
        Yuval Mintz <yuvalm@...lanox.com>
CC:     "netdev@...r.kernel.org" <netdev@...r.kernel.org>,
        "oss-drivers@...ronome.com" <oss-drivers@...ronome.com>,
        "edumazet@...gle.com" <edumazet@...gle.com>
Subject: RE: [RFC -next 2/2] net: sched: red: don't reset the backlog on every
 stat dump

> -----Original Message-----
> From: Jakub Kicinski [mailto:jakub.kicinski@...ronome.com]
> Sent: Thursday, January 04, 2018 10:19 PM
> To: john.fastabend@...il.com; jiri@...nulli.us; xiyou.wangcong@...il.com;
> Nogah Frankel <nogahf@...lanox.com>; Yuval Mintz
> <yuvalm@...lanox.com>
> Cc: netdev@...r.kernel.org; oss-drivers@...ronome.com;
> edumazet@...gle.com; Jakub Kicinski <jakub.kicinski@...ronome.com>
> Subject: [RFC -next 2/2] net: sched: red: don't reset the backlog on every stat
> dump
> 
> Commit 0dfb33a0d7e2 ("sch_red: report backlog information") copied
> child's backlog into RED's backlog.  Back then RED did not maintain
> its own backlog counts.  This has changed after commit 2ccccf5fb43f
> ("net_sched: update hierarchical backlog too") and commit d7f4f332f082
> ("sch_red: update backlog as well").  Copying is no longer necessary.
> 
> Tested:
> 
> $ tc -s qdisc show dev veth0
> qdisc red 1: root refcnt 2 limit 400000b min 30000b max 30000b ecn
>  Sent 20942 bytes 221 pkt (dropped 0, overlimits 0 requeues 0)
>  backlog 1260b 14p requeues 14
>   marked 0 early 0 pdrop 0 other 0
> qdisc tbf 2: parent 1: rate 1Kbit burst 15000b lat 3585.0s
>  Sent 20942 bytes 221 pkt (dropped 0, overlimits 138 requeues 0)
>  backlog 1260b 14p requeues 14
> 
> Recently RED offload was added.  We need to make sure drivers don't
> depend on resetting the stats.  This means backlog should be treated
> like any other statistic:
> 
>   total_stat = new_hw_stat - prev_hw_stat;
> 
> Unlike for other statistics new_hw_stat < prev_hw_stat can be true.
> Adjust mlxsw.

There is one problem with this patch, and that is that we can fail in 
changing RED that is offloaded. In this case, we delete RED from the driver
but the backlog will still include the hardware backlog.
The solution is to send in the offload-replace command a pointer to the
backlog, so failure in updating the hardware can be follow by backlog update,
if needed.

Thanks

Nogah

> 
> Signed-off-by: Jakub Kicinski <jakub.kicinski@...ronome.com>
> ---
>  drivers/net/ethernet/mellanox/mlxsw/spectrum.h       | 1 +
>  drivers/net/ethernet/mellanox/mlxsw/spectrum_qdisc.c | 9 +++++++--
>  net/sched/sch_red.c                                  | 1 -
>  3 files changed, 8 insertions(+), 3 deletions(-)
> 
> diff --git a/drivers/net/ethernet/mellanox/mlxsw/spectrum.h
> b/drivers/net/ethernet/mellanox/mlxsw/spectrum.h
> index ff8d32bc852c..6755050e4ee0 100644
> --- a/drivers/net/ethernet/mellanox/mlxsw/spectrum.h
> +++ b/drivers/net/ethernet/mellanox/mlxsw/spectrum.h
> @@ -237,6 +237,7 @@ struct mlxsw_sp_qdisc {
>  	u64 tx_packets;
>  	u64 drops;
>  	u64 overlimits;
> +	u64 backlog;
>  };
> 
>  /* No need an internal lock; At worse - miss a single periodic iteration */
> diff --git a/drivers/net/ethernet/mellanox/mlxsw/spectrum_qdisc.c
> b/drivers/net/ethernet/mellanox/mlxsw/spectrum_qdisc.c
> index c33beac5def0..d5091740bd40 100644
> --- a/drivers/net/ethernet/mellanox/mlxsw/spectrum_qdisc.c
> +++ b/drivers/net/ethernet/mellanox/mlxsw/spectrum_qdisc.c
> @@ -212,6 +212,7 @@ mlxsw_sp_qdisc_get_red_stats(struct mlxsw_sp_port
> *mlxsw_sp_port, u32 handle,
>  	u64 tx_bytes, tx_packets, overlimits, drops;
>  	struct mlxsw_sp_port_xstats *xstats;
>  	struct rtnl_link_stats64 *stats;
> +	s64 backlog;
> 
>  	if (mlxsw_sp_qdisc->handle != handle ||
>  	    mlxsw_sp_qdisc->type != MLXSW_SP_QDISC_RED)
> @@ -226,17 +227,21 @@ mlxsw_sp_qdisc_get_red_stats(struct
> mlxsw_sp_port *mlxsw_sp_port, u32 handle,
>  		     mlxsw_sp_qdisc->overlimits;
>  	drops = xstats->wred_drop[tclass_num] + xstats-
> >tail_drop[tclass_num] -
>  		mlxsw_sp_qdisc->drops;
> +	backlog = mlxsw_sp_cells_bytes(mlxsw_sp_port->mlxsw_sp,
> +				       xstats->backlog[tclass_num]) -
> +		mlxsw_sp_qdisc->backlog;
> 
>  	_bstats_update(res->bstats, tx_bytes, tx_packets);
>  	res->qstats->overlimits += overlimits;
>  	res->qstats->drops += drops;
> -	res->qstats->backlog += mlxsw_sp_cells_bytes(mlxsw_sp_port-
> >mlxsw_sp,
> -						xstats->backlog[tclass_num]);
> +	res->qstats->backlog +=	backlog;
> 
>  	mlxsw_sp_qdisc->drops +=  drops;
>  	mlxsw_sp_qdisc->overlimits += overlimits;
>  	mlxsw_sp_qdisc->tx_bytes += tx_bytes;
>  	mlxsw_sp_qdisc->tx_packets += tx_packets;
> +	mlxsw_sp_qdisc->backlog += backlog;
> +
>  	return 0;
>  }
> 
> diff --git a/net/sched/sch_red.c b/net/sched/sch_red.c
> index a392eaa4a0b4..caebb7e37551 100644
> --- a/net/sched/sch_red.c
> +++ b/net/sched/sch_red.c
> @@ -322,7 +322,6 @@ static int red_dump(struct Qdisc *sch, struct sk_buff
> *skb)
>  	};
>  	int err;
> 
> -	sch->qstats.backlog = q->qdisc->qstats.backlog;
>  	err = red_dump_offload_stats(sch, &opt);
>  	if (err)
>  		goto nla_put_failure;
> --
> 2.15.1