| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Message-ID: <a5be329e-39e3-fdfc-500d-383953546d40@mellanox.com>
Date: Thu, 30 Apr 2020 20:07:43 +0300
From: Tariq Toukan <tariqt@...lanox.com>
To: Jesper Dangaard Brouer <brouer@...hat.com>, sameehj@...zon.com
Cc: Tariq Toukan <tariqt@...lanox.com>,
Saeed Mahameed <saeedm@...lanox.com>, netdev@...r.kernel.org,
bpf@...r.kernel.org, zorik@...zon.com, akiyano@...zon.com,
gtzalik@...zon.com,
Toke Høiland-Jørgensen <toke@...hat.com>,
Daniel Borkmann <borkmann@...earbox.net>,
Alexei Starovoitov <alexei.starovoitov@...il.com>,
John Fastabend <john.fastabend@...il.com>,
Alexander Duyck <alexander.duyck@...il.com>,
Jeff Kirsher <jeffrey.t.kirsher@...el.com>,
David Ahern <dsahern@...il.com>,
Willem de Bruijn <willemdebruijn.kernel@...il.com>,
Ilias Apalodimas <ilias.apalodimas@...aro.org>,
Lorenzo Bianconi <lorenzo@...nel.org>,
steffen.klassert@...unet.com
Subject: Re: [PATCH net-next v2 28/33] mlx5: rx queue setup time determine
frame_sz for XDP
On 4/30/2020 2:22 PM, Jesper Dangaard Brouer wrote:
> The mlx5 driver have multiple memory models, which are also changed
> according to whether a XDP bpf_prog is attached.
>
> The 'rx_striding_rq' setting is adjusted via ethtool priv-flags e.g.:
> # ethtool --set-priv-flags mlx5p2 rx_striding_rq off
>
> On the general case with 4K page_size and regular MTU packet, then
> the frame_sz is 2048 and 4096 when XDP is enabled, in both modes.
>
> The info on the given frame size is stored differently depending on the
> RQ-mode and encoded in a union in struct mlx5e_rq union wqe/mpwqe.
> In rx striding mode rq->mpwqe.log_stride_sz is either 11 or 12, which
> corresponds to 2048 or 4096 (MLX5_WQ_TYPE_LINKED_LIST_STRIDING_RQ).
> In non-striding mode (MLX5_WQ_TYPE_CYCLIC) the frag_stride is stored
> in rq->wqe.info.arr[0].frag_stride, for the first fragment, which is
> what the XDP case cares about.
>
> To reduce effect on fast-path, this patch determine the frame_sz at
> setup time, to avoid determining the memory model runtime. Variable
> is named first_frame_sz to make it clear that this is only the frame
> size of the first fragment.
>
> This mlx5 driver does a DMA-sync on XDP_TX action, but grow is safe
> as it have done a DMA-map on the entire PAGE_SIZE. The driver also
> already does a XDP length check against sq->hw_mtu on the possible
> XDP xmit paths mlx5e_xmit_xdp_frame() + mlx5e_xmit_xdp_frame_mpwqe().
>
> V2: Fix that frag_size need to be recalc before creating SKB.
>
> Cc: Tariq Toukan <tariqt@...lanox.com>
> Cc: Saeed Mahameed <saeedm@...lanox.com>
> Signed-off-by: Jesper Dangaard Brouer <brouer@...hat.com>
> ---
> drivers/net/ethernet/mellanox/mlx5/core/en.h | 1 +
> drivers/net/ethernet/mellanox/mlx5/core/en/xdp.c | 1 +
> drivers/net/ethernet/mellanox/mlx5/core/en_main.c | 6 ++++++
> drivers/net/ethernet/mellanox/mlx5/core/en_rx.c | 2 ++
> 4 files changed, 10 insertions(+)
>
> diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en.h b/drivers/net/ethernet/mellanox/mlx5/core/en.h
> index 23701c0e36ec..ba6a0ee297c6 100644
> --- a/drivers/net/ethernet/mellanox/mlx5/core/en.h
> +++ b/drivers/net/ethernet/mellanox/mlx5/core/en.h
> @@ -652,6 +652,7 @@ struct mlx5e_rq {
> struct {
> u16 umem_headroom;
> u16 headroom;
> + u32 first_frame_sz;
> u8 map_dir; /* dma map direction */
> } buff;
>
> diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/xdp.c b/drivers/net/ethernet/mellanox/mlx5/core/en/xdp.c
> index f049e0ac308a..b63abaf51253 100644
> --- a/drivers/net/ethernet/mellanox/mlx5/core/en/xdp.c
> +++ b/drivers/net/ethernet/mellanox/mlx5/core/en/xdp.c
> @@ -137,6 +137,7 @@ bool mlx5e_xdp_handle(struct mlx5e_rq *rq, struct mlx5e_dma_info *di,
> if (xsk)
> xdp.handle = di->xsk.handle;
> xdp.rxq = &rq->xdp_rxq;
> + xdp.frame_sz = rq->buff.first_frame_sz;
>
> act = bpf_prog_run_xdp(prog, &xdp);
> if (xsk) {
> diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
> index 47396f1b02f4..1d04ed3feead 100644
> --- a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
> +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
> @@ -462,6 +462,8 @@ static int mlx5e_alloc_rq(struct mlx5e_channel *c,
> rq->mpwqe.num_strides =
> BIT(mlx5e_mpwqe_get_log_num_strides(mdev, params, xsk));
>
> + rq->buff.first_frame_sz = (1 << rq->mpwqe.log_stride_sz);
> +
> err = mlx5e_create_rq_umr_mkey(mdev, rq);
> if (err)
> goto err_rq_wq_destroy;
> @@ -485,6 +487,8 @@ static int mlx5e_alloc_rq(struct mlx5e_channel *c,
> num_xsk_frames = wq_sz << rq->wqe.info.log_num_frags;
>
> rq->wqe.info = rqp->frags_info;
> + rq->buff.first_frame_sz = rq->wqe.info.arr[0].frag_stride;
> +
> rq->wqe.frags =
> kvzalloc_node(array_size(sizeof(*rq->wqe.frags),
> (wq_sz << rq->wqe.info.log_num_frags)),
> @@ -522,6 +526,8 @@ static int mlx5e_alloc_rq(struct mlx5e_channel *c,
> }
>
> if (xsk) {
> + rq->buff.first_frame_sz = xsk_umem_xdp_frame_sz(umem);
> +
> err = mlx5e_xsk_resize_reuseq(umem, num_xsk_frames);
> if (unlikely(err)) {
> mlx5_core_err(mdev, "Unable to allocate the Reuse Ring for %u frames\n",
> diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c b/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c
> index e2beb89c1832..04671ed977a5 100644
> --- a/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c
> +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c
> @@ -1084,6 +1084,7 @@ mlx5e_skb_from_cqe_linear(struct mlx5e_rq *rq, struct mlx5_cqe64 *cqe,
> if (consumed)
> return NULL; /* page/packet was consumed by XDP */
>
> + frag_size = MLX5_SKB_FRAG_SZ(rx_headroom + cqe_bcnt);
This is a re-calculation of frag_size, using the exact same command used
earlier in this function, but with a newer value of rx_headroom.
This wasn't part of the previous patchset. I understand the need.
However, this code repetition looks weird and non-optimal to me. I think
we can come up with something better.
> skb = mlx5e_build_linear_skb(rq, va, frag_size, rx_headroom, cqe_bcnt);
> if (unlikely(!skb))
> return NULL;
> @@ -1385,6 +1386,7 @@ mlx5e_skb_from_cqe_mpwrq_linear(struct mlx5e_rq *rq, struct mlx5e_mpw_info *wi,
> return NULL; /* page/packet was consumed by XDP */
> }
>
> + frag_size = MLX5_SKB_FRAG_SZ(rx_headroom + cqe_bcnt32);
Same here.
> skb = mlx5e_build_linear_skb(rq, va, frag_size, rx_headroom, cqe_bcnt32);
> if (unlikely(!skb))
> return NULL;
>
>
My suggetion is:
Pass &frag_size to mlx5e_xdp_handle(), and update it within it, just
next to the update of rx_headroom.
All the needed information is there: the new rx_headroom, and cqe_bcnt.
Thanks,
Tariq
Powered by blists - more mailing lists