Message-ID: <20220620152647.2498927-1-dchumak@nvidia.com>
Date:   Mon, 20 Jun 2022 18:26:42 +0300
From:   Dima Chumak <dchumak@...dia.com>
To:     Jakub Kicinski <kuba@...nel.org>
CC:     Jiri Pirko <jiri@...dia.com>,
        "David S. Miller" <davem@...emloft.net>,
        Eric Dumazet <edumazet@...gle.com>,
        Paolo Abeni <pabeni@...hat.com>, <netdev@...r.kernel.org>,
        Dima Chumak <dchumak@...dia.com>
Subject: [PATCH net-next 0/5] devlink rate police limiter

Currently, kernel provides a way to limit tx rate of a VF via devlink
rate function of a port. The underlying mechanism is a shaper applied to
all traffic passing through the target VF or a group of VFs. By its
essence, a shaper naturally works with outbound traffic, and in
practice, it's rarely seen to be implemented for inbound traffic.
Nevertheless, there is a user request to have a mechanism for limiting
inbound traffic as well. It is usually done by using some form of
traffic policing, dropping excess packets over the configured limit that
set by a user. Thus, introducing another limiting mechanism to the port
function can help close this gap.

This series introduces devlink attrs, along with their ops, to manage
rate policing of a single port as well as a port group. It is based on
the existing notion of leaf and node rate objects, and extends their
attributes to support both RX and TX limiting, for a number of packets
per second and/or a number of bytes per second. Additionally, there is a
second set of parameters for specifying the size of buffering performed,
called "burst", that controls the allowed level of spikes in traffic
before it starts getting dropped.

A new sub-type of a devlink_rate object is introduced, called
"limit_type". It can be either "shaping", the default, or "police".
A single leaf or a node object can be switched from one limit type to
another, but it cannot do both types of rate limiting simultaneously.
A node and a leaf object that have parent-child relationship must have
the same limit type. In other words, it's only possible to group rate
objects of the same limit type as their group's limit_type.

devlink_ops extended with following callbacks:
- rate_{leaf|node}_tx_{burst|pkts|pkts_burst}_set
- rate_{leaf|node}_rx_{max|burst|pkts|pkts_burst}_set

UAPI provides:
- setting tx_{burst|pkts|pkts_burst} and rx_{max|burst|pkts|pkts_burst}
  of a rate object

Added devlink_rate police attrs support for netdevsim driver.

Issues/open questions:
- Current implementation requires a user to set both "rate" and "burst"
  parameters explicitly, in order to activate police rate limiting. For
  example, "rx_max 200Mbit rx_burst 16mb". Is it necessary to
  automagically deduce "burst" value when it's omitted by the user?
  For example when user only sets "rx_max 200Mbit".
- If answer is positive to the first question, at which level it's
  better to be done, at user-space iproute2, at kernel devlink core or
  at vendor driver that implements devlink_ops for police attrs?

CLI examples:

  $ devlink port function rate show
  netdevsim/netdevsim10/128: type leaf limit_type unset
  netdevsim/netdevsim10/129: type leaf limit_type unset
  netdevsim/netdevsim10/130: type leaf limit_type unset

  # Set police rate limiting of inbound traffic
  $ devlink port function rate set netdevsim/netdevsim10/128 \
            limit_type police rx_max 100mbit rx_burst 10mbit
  $ devlink port function rate show
  netdevsim/netdevsim10/128: type leaf limit_type police rx_max 100Mbit rx_burst 10485Kbit

  # Set shaping rate limiting of outbound traffic (default limit_type)
  $ devlink port function rate set netdevsim/netdevsim10/129 tx_max 200mbit
  $ devlink port function rate show
  netdevsim/netdevsim10/129: type leaf limit_type shaping tx_max 200Mbit

  # Attempt to set police attr with the default shaping limit_type
  $ devlink port function rate set netdevsim/netdevsim10/129 rx_max 400mbit
  Unsupported option "rx_max" for limit_type "shaping"

  # Set police rate attr for a port that already has active shaping
  $ devlink port function rate set netdevsim/netdevsim10/129 limit_type police rx_max 400mbit
  Error: devlink: Cannot change limit_type of the rate leaf object, reset current rate attributes first.
  kernel answers: Device or resource busy

  # Create a rate group
  $ devlink port function rate add netdevsim/netdevsim10/g1 \
            limit_type police rx_max 1Gbit
  $ devlink port function rate show
  netdevsim/netdevsim10/g1: type node limit_type police rx_max 1Gbit

  # Add port to the group
  $ devlink port function rate set netdevsim/netdevsim10/128 parent g1
  $ devlink port function rate show
  netdevsim/netdevsim10/g1: type node limit_type police rx_max 1Gbit
  netdevsim/netdevsim10/128: type leaf limit_type police rx_max 100Mbit rx_burst 10485Kbit parent g1
  netdevsim/netdevsim10/129: type leaf limit_type shaping tx_max 200Mbit
  netdevsim/netdevsim10/130: type leaf limit_type unset

  # Try to add a port with a non-matching limit_type to the group
  $ devlink port function rate set netdevsim/netdevsim10/129 parent g1
  Error: devlink: Parent and object should be of the same limit_type.
  kernel answers: Invalid argument

  # Adding a port with "unset" limit_type to a group inherits the
  # group's limit_type
  $ devlink port function rate set netdevsim/netdevsim10/130 parent g1
  $ devlink port function rate show
  netdevsim/netdevsim10/130: type leaf limit_type police parent g1

  # Set all police parameters
  $ devlink port func rate set netdevsim/netdevsim10/130 \
            limit_type police tx_max 10GBps tx_burst 1gb \
                              rx_max 25GBps rx_burst 2gb \
                              tx_pkts 10000 tx_pkts_burst 1gb \
                              rx_pkts 20000 rx_pkts_burst 2gb

Dima Chumak (5):
  devlink: Introduce limit_type attr for rate objects
  devlink: Introduce police rate limit type
  netdevsim: Support devlink rate limit_type police
  selftest: netdevsim: Add devlink rate police sub-test
  Documentation: devlink rate objects limit_type

 .../networking/devlink/devlink-port.rst       |  44 ++-
 .../networking/devlink/netdevsim.rst          |   3 +-
 .../net/ethernet/mellanox/mlx5/core/esw/qos.c |  28 +-
 drivers/net/netdevsim/dev.c                   | 211 ++++++++++-
 drivers/net/netdevsim/netdevsim.h             |  11 +-
 include/net/devlink.h                         |  52 ++-
 include/uapi/linux/devlink.h                  |  15 +
 net/core/devlink.c                            | 336 ++++++++++++++++--
 .../drivers/net/netdevsim/devlink.sh          | 215 ++++++++++-
 9 files changed, 853 insertions(+), 62 deletions(-)

-- 
2.36.1

Powered by blists - more mailing lists