| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Message-ID: <ZLJ109eXtIge6eJ9@corigine.com>
Date: Sat, 15 Jul 2023 11:32:51 +0100
From: Simon Horman <simon.horman@...igine.com>
To: Aurelien Aptel <aaptel@...dia.com>
Cc: linux-nvme@...ts.infradead.org, netdev@...r.kernel.org,
sagi@...mberg.me, hch@....de, kbusch@...nel.org, axboe@...com,
chaitanyak@...dia.com, davem@...emloft.net, kuba@...nel.org,
Yoray Zack <yorayz@...dia.com>, aurelien.aptel@...il.com,
smalin@...dia.com, malin1024@...il.com, ogerlitz@...dia.com,
borisp@...dia.com, galshalom@...dia.com, mgurtovoy@...dia.com
Subject: Re: [PATCH v12 13/26] Documentation: add ULP DDP offload
documentation
On Wed, Jul 12, 2023 at 04:15:00PM +0000, Aurelien Aptel wrote:
> From: Yoray Zack <yorayz@...dia.com>
>
> Document the new ULP DDP API and add it under "networking".
> Use NVMe-TCP implementation as an example.
...
> diff --git a/Documentation/networking/ulp-ddp-offload.rst b/Documentation/networking/ulp-ddp-offload.rst
> new file mode 100644
> index 000000000000..b8d7c9c4d6b0
> --- /dev/null
> +++ b/Documentation/networking/ulp-ddp-offload.rst
...
> +Device configuration
> +====================
> +
> +During driver initialization the driver sets the following
> +:c:type:`struct net_device <net_device>` properties:
> +
> +* The ULP DDP capabilities it supports
> + in :c:type:`struct ulp_ddp_netdev_caps <ulp_ddp_caps>`
> +* The ULP DDP operations pointer in :c:type:`struct ulp_ddp_dev_ops`.
'make htmldocs' seems a little unhappy about this for some reason:
.../ulp-ddp-offload.rst:74: WARNING: Unparseable C cross-reference: 'struct ulp_ddp_dev_ops'
Invalid C declaration: Expected identifier in nested name, got keyword: struct [error at 6]
struct ulp_ddp_dev_ops
------^
> +
> +The current list of capabilities is represented as a bitset:
> +
> +.. code-block:: c
> +
> + enum {
> + ULP_DDP_C_NVME_TCP_BIT,
> + ULP_DDP_C_NVME_TCP_DDGST_RX_BIT,
> + /* add capabilities above */
> + ULP_DDP_C_COUNT,
> + };
> +
> +The enablement of capabilities can be controlled from userspace via
> +netlink. See Documentation/networking/ethtool-netlink.rst for more
> +details.
> +
> +Later, after the L5P completes its handshake, the L5P queries the
> +driver for its runtime limitations via the :c:member:`limits` operation:
> +
> +.. code-block:: c
> +
> + int (*limits)(struct net_device *netdev,
> + struct ulp_ddp_limits *lim);
> +
> +
> +All L5P share a common set of limits and parameters (:c:type:`struct ulp_ddp_limits`):
Likewise, here:
.../ulp-ddp-offload.rst:100: WARNING: Unparseable C cross-reference: 'struct ulp_ddp_limits'
Invalid C declaration: Expected identifier in nested name, got keyword: struct [error at 6]
struct ulp_ddp_limits
------^
> +
> +.. code-block:: c
> +
> + /**
> + * struct ulp_ddp_limits - Generic ulp ddp limits: tcp ddp
> + * protocol limits.
> + * Add new instances of ulp_ddp_limits in the union below (nvme-tcp, etc.).
> + *
> + * @type: type of this limits struct
> + * @max_ddp_sgl_len: maximum sgl size supported (zero means no limit)
> + * @io_threshold: minimum payload size required to offload
> + * @tls: support for ULP over TLS
> + * @nvmeotcp: NVMe-TCP specific limits
> + */
> + struct ulp_ddp_limits {
> + enum ulp_ddp_type type;
> + int max_ddp_sgl_len;
> + int io_threshold;
> + bool tls:1;
> + union {
> + /* ... protocol-specific limits ... */
> + struct nvme_tcp_ddp_limits nvmeotcp;
> + };
> + };
...
> +Asynchronous teardown
> +---------------------
> +
> +To teardown the association between tags and buffers and allow tag reuse NIC HW
> +is called by the NIC driver during `teardown`. This operation may be
> +performed either synchronously or asynchronously. In asynchronous teardown,
> +`teardown` returns immediately without unmapping NIC HW buffers. Later,
> +when the unmapping completes by NIC HW, the NIC driver will call up to L5P
> +using :c:member:`ddp_teardown_done` of :c:type:`struct ulp_ddp_ulp_ops`:
And here:
.../ulp-ddp-offload.rst:283: WARNING: Unparseable C cross-reference: 'struct ulp_ddp_ulp_ops'
Invalid C declaration: Expected identifier in nested name, got keyword: struct [error at 6]
struct ulp_ddp_ulp_ops
------^
> +
> +.. code-block:: c
> +
> + void (*ddp_teardown_done)(void *ddp_ctx);
> +
> +The `ddp_ctx` parameter passed in `ddp_teardown_done` is the same on provided
> +in `teardown` and it is used to carry some context about the buffers
> +and tags that are released.
> +
> +Resync handling
> +===============
> +
> +RX
> +--
> +In presence of packet drops or network packet reordering, the device may lose
> +synchronization between the TCP stream and the L5P framing, and require a
> +resync with the kernel's TCP stack. When the device is out of sync, no offload
> +takes place, and packets are passed as-is to software. Resync is very similar
> +to TLS offload (see documentation at Documentation/networking/tls-offload.rst)
> +
> +If only packets with L5P data are lost or reordered, then resynchronization may
> +be avoided by NIC HW that keeps tracking PDU headers. If, however, PDU headers
> +are reordered, then resynchronization is necessary.
> +
> +To resynchronize hardware during traffic, we use a handshake between hardware
> +and software. The NIC HW searches for a sequence of bytes that identifies L5P
> +headers (i.e., magic pattern). For example, in NVMe-TCP, the PDU operation
> +type can be used for this purpose. Using the PDU header length field, the NIC
> +HW will continue to find and match magic patterns in subsequent PDU headers. If
> +the pattern is missing in an expected position, then searching for the pattern
> +starts anew.
> +
> +The NIC will not resume offload when the magic pattern is first identified.
> +Instead, it will request L5P software to confirm that indeed this is a PDU
> +header. To request confirmation the NIC driver calls up to L5P using
> +:c:member:`*resync_request` of :c:type:`struct ulp_ddp_ulp_ops`:
And here:
.../ulp-ddp-offload.rst:321: WARNING: Unparseable C cross-reference: '*resync_request'
Invalid C declaration: Expected identifier in nested name. [error at 0]
*resync_request
^
.../ulp-ddp-offload.rst:321: WARNING: Unparseable C cross-reference: 'struct ulp_ddp_ulp_ops'
Invalid C declaration: Expected identifier in nested name, got keyword: struct [error at 6]
struct ulp_ddp_ulp_ops
------^
> +
> +.. code-block:: c
> +
> + bool (*resync_request)(struct sock *sk, u32 seq, u32 flags);
> +
> +The `seq` parameter contains the TCP sequence of the last byte in the PDU header.
> +The `flags` parameter contains a flag (`ULP_DDP_RESYNC_PENDING`) indicating whether
> +a request is pending or not.
> +L5P software will respond to this request after observing the packet containing
> +TCP sequence `seq` in-order. If the PDU header is indeed there, then L5P
> +software calls the NIC driver using the :c:member:`resync` function of
> +the :c:type:`struct ulp_ddp_dev_ops <ulp_ddp_ops>` inside the :c:type:`struct
> +net_device <net_device>` while passing the same `seq` to confirm it is a PDU
> +header.
> +
> +.. code-block:: c
> +
> + void (*resync)(struct net_device *netdev,
> + struct sock *sk, u32 seq);
...
Powered by blists - more mailing lists