netdev - Re: [PATCH net-next] mlx4: optimize xmit path

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Date:	Sun, 28 Sep 2014 11:07:16 -0700
From:	Alexei Starovoitov <ast@...mgrid.com>
To:	Eric Dumazet <eric.dumazet@...il.com>
Cc:	Or Gerlitz <gerlitz.or@...il.com>,
	"David S. Miller" <davem@...emloft.net>,
	Jesper Dangaard Brouer <brouer@...hat.com>,
	Eric Dumazet <edumazet@...gle.com>,
	John Fastabend <john.r.fastabend@...el.com>,
	Linux Netdev List <netdev@...r.kernel.org>,
	Amir Vadai <amirv@...lanox.com>,
	Or Gerlitz <or.gerlitz@...il.com>
Subject: Re: [PATCH net-next] mlx4: optimize xmit path

On Sat, Sep 27, 2014 at 3:56 PM, Eric Dumazet <eric.dumazet@...il.com> wrote:
> From: Eric Dumazet <edumazet@...gle.com>
>
> First I implemented skb->xmit_more support, and pktgen throughput
> went from ~5Mpps to ~10Mpps.
>
> Then, looking closely at this driver I found false sharing problems that
> should be addressed by this patch, as my pktgen now reaches 14.7 Mpps
> on a single TX queue, with a burst factor of 8.
>
> So this patch in a whole permits to improve raw performance on a single
> TX queue from about 5 Mpps to 14.7 Mpps.

this is great improvement!
Thank you for leading this effort.
10G line rate is definitely nice :)
Hopefully Or can demo similar numbers with 40G nic as well :)

> +       if (ring->bf_enabled && desc_size <= MAX_BF && !bounce &&
> +           !vlan_tx_tag_present(skb) && send_doorbell) {

feels something wrong here, since it checks for
send_doorbell, but iowrite() happens in the 'else' part
of this branch with another 'if (send_doorbell)'

The previous code seems equally confusing to me.

> +               tx_desc->ctrl.bf_qpn = ring->doorbell_qpn |
> +                                      cpu_to_be32(real_size);
>
>                 op_own |= htonl((bf_index & 0xffff) << 8);
> -               /* Ensure new descirptor hits memory
> -               * before setting ownership of this descriptor to HW */
> +               /* Ensure new descriptor hits memory
> +                * before setting ownership of this descriptor to HW
> +                */
>                 wmb();
>                 tx_desc->ctrl.owner_opcode = op_own;
>
>                 wmb();
>
> -               mlx4_bf_copy(ring->bf.reg + ring->bf.offset, (unsigned long *) &tx_desc->ctrl,
> -                    desc_size);
> +               mlx4_bf_copy(ring->bf.reg + ring->bf.offset,
> +                            &tx_desc->ctrl,
> +                            desc_size);
>
>                 wmb();
>
>                 ring->bf.offset ^= ring->bf.buf_size;
>         } else {
> -               /* Ensure new descirptor hits memory
> -               * before setting ownership of this descriptor to HW */
> +               tx_desc->ctrl.vlan_tag = cpu_to_be16(vlan_tag);
> +               tx_desc->ctrl.ins_vlan = MLX4_WQE_CTRL_INS_VLAN *
> +                       !!vlan_tx_tag_present(skb);
> +               tx_desc->ctrl.fence_size = real_size;
> +
> +               /* Ensure new descriptor hits memory
> +                * before setting ownership of this descriptor to HW
> +                */
>                 wmb();
>                 tx_desc->ctrl.owner_opcode = op_own;
> -               wmb();
> -               iowrite32be(ring->doorbell_qpn, ring->bf.uar->map + MLX4_SEND_DOORBELL);
> +
> +               if (send_doorbell) {
> +                       wmb(); /* ensure owner_opcode is written */
> +                       iowrite32(ring->doorbell_qpn,
> +                                 ring->bf.uar->map + MLX4_SEND_DOORBELL);
> +               }

shinfo, prefetch and access_once additions all look useful to me.

Thanks!
--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html