Message-ID: <12318cdb-afa9-4fdb-b6d7-6568b31c1817@intel.com>
Date: Thu, 29 May 2025 10:23:23 +0300
From: Mor Bar-Gabay <morx.bar.gabay@...el.com>
To: "Abdul Rahim, Faizal" <faizal.abdul.rahim@...el.com>, Tony Nguyen
	<anthony.l.nguyen@...el.com>, Przemek Kitszel <przemyslaw.kitszel@...el.com>,
	Andrew Lunn <andrew+netdev@...n.ch>, "David S . Miller"
	<davem@...emloft.net>, Eric Dumazet <edumazet@...gle.com>, Jakub Kicinski
	<kuba@...nel.org>, Paolo Abeni <pabeni@...hat.com>, Vladimir Oltean
	<vladimir.oltean@....com>
CC: <intel-wired-lan@...ts.osuosl.org>, <netdev@...r.kernel.org>,
	<linux-kernel@...r.kernel.org>, Simon Horman <horms@...nel.org>, Faizal Rahim
	<faizal.abdul.rahim@...ux.intel.com>, Aleksandr Loktionov
	<aleksandr.loktionov@...el.com>, Chwee-Lin Choong
	<chwee.lin.choong@...el.com>
Subject: Re: [Intel-wired-lan] [PATCH iwl-next v3 5/7] igc: add private flag
 to reverse TX queue priority in TSN mode

On 19/05/2025 10:19, Abdul Rahim, Faizal wrote:
> From: Faizal Rahim <faizal.abdul.rahim@...ux.intel.com>
> 
> By default, igc assigns TX hw queue 0 the highest priority and queue 3
> the lowest. This is opposite of most NICs, where TX hw queue 3 has the
> highest priority and queue 0 the lowest.
> 
> mqprio in igc already uses TX arbitration unconditionally to reverse TX
> queue priority when mqprio is enabled. The TX arbitration logic does not
> require a private flag, because mqprio was added recently and no known
> users depend on the default queue ordering, which differs from the typical
> convention.
> 
> taprio does not use TX arbitration, so it inherits the default igc TX
> queue priority order. This causes tc command inconsistencies when
> configuring frame preemption with taprio compared to mqprio in igc.
> Other tc command inconsistencies and configuration issues already exist
> when using taprio on igc compared to other network controllers. These
> issues are described in a later section.
> 
> To harmonize TX queue priority behavior between taprio and mqprio, and
> to fix these issues without breaking long-standing taprio use cases,
> this patch adds a new private flag, called reverse-tsn-txq-prio, to
> reverse the TX queue priority. It makes queue 3 the highest and queue 0
> the lowest, reusing the TX arbitration logic already used by mqprio.
> 
> Users must set the private flag when enabling frame preemption with
> taprio to follow the standard convention. Doing so promotes adoption of
> the correct priority model for new features while preserving
> compatibility with legacy configurations.
> 
> This new private flag addresses:
> 
> 1.  Non-standard socket → tc → TX hw queue mapping for taprio in igc
> 
> Without the private flag:
> - taprio maps (socket → tc → TX hardware queue) differently on igc
>    compared to other network controllers
> - On igc, mqprio maps tc differently from taprio, since mqprio already
>    uses TX arbitration
> 
> The following examples compare taprio configuration on igc and other
> network controllers:
> a)  On other NICs (TX hw queue 3 is highest priority):
>      taprio num_tc 4 map 0 1 2 3 .... \
>      queues 1@0 1@1 1@2 1@3
> 
>      Mapping translates to:
>      socket 0 → tc 0 → queue 0
>      socket 3 → tc 3 → queue 3
> 
>      This is the normal mapping that respects the standard convention:
>      higher socket number → higher tc -> higher priority TX hw queue
> 
> b)  On igc (TX hw queue 0 is highest priority by default):
>      taprio num_tc 4 map 3 2 1 0 .... \
>      queues 1@0 1@1 1@2 1@3
> 
>      Mapping translates to:
>      socket 0 → tc 3 → queue 3
>      socket 3 → tc 0 → queue 0
> 
>      This igc tc mapping example is based on Intel's TSN validation test
>      case, where a higher socket priority maps to a higher priority queue.
>      It respects the mapping:
>        higher socket number -> higher priority TX hw queue
>      but breaks the expected ordering:
>        higher tc -> higher priority TX hw queue
>      as defined in [Ref1]. This custom mapping complicates common taprio
>      setup across NICs.
> 
> 2.  Non-standard frame preemption mapping for taprio in igc
> 
> Without the private flag:
> - Compared to other network controllers, taprio on igc must flip the
>    expected fp sequence, since express traffic is expected to map to the
>    highest priority queue and preemptible traffic to lower ones
> - On igc, frame preemption configuration for mqprio differs from taprio,
>    since mqprio already uses TX arbitration
> 
> The following examples compare taprio frame preemption configuration on
> igc and other network controllers:
> a)  On other NICs (TX hw queue 3 is highest priority):
>      taprio num_tc 4 map ..... \
>      queues 1@0 1@1 1@2 1@3 \
>      fp P P P E
> 
>      Mapping translates to:
>      tc0, tc1, tc2 → preemptible → queue 0, 1, 2
>      tc3           → express     → queue 3
> 
>      This is the normal mapping that respects the standard convention:
>      higher tc -> express traffic -> higher priority TX hw queue
>      lower tc  -> preemptible traffic -> lower priority TX hw queue
> 
> b)  On igc (TX hw queue 0 is highest priority by default):
>      taprio num_tc 4 map ...... \
>      queues 1@0 1@1 1@2 1@3 \
>      fp E P P P
> 
>      Mapping translates to:
>      tc0           → express     → queue 0
>      tc1, tc2, tc3 → preemptible → queue 1, 2, 3
> 
>      This inversion respects the mapping of:
>        express traffic -> higher priority TX hw queue
>      but breaks the expected ordering:
>        higher tc -> express traffic
>      as defined in [Ref1] where higher tc indicates higher priority. In
>      this case, the lower tc0 is assigned to express traffic. This custom
>      mapping further complicates common preemption setup across NICs.
> 
> Tests were performed on taprio with the following combinations, where
> two apps send traffic simultaneously on different queues:
> 
>    Private Flag   Traffic Sent By           Traffic Sent By
>    ----------------------------------------------------------------
>    enabled        iperf3 (queue 3)          iperf3 (queue 0)
>    disabled       iperf3 (queue 0)          iperf3 (queue 3)
>    enabled        iperf3 (queue 3)          real-time app (queue 0)
>    disabled       iperf3 (queue 0)          real-time app (queue 3)
>    enabled        real-time app (queue 3)   iperf3 (queue 0)
>    disabled       real-time app (queue 0)   iperf3 (queue 3)
>    enabled        real-time app (queue 3)   real-time app (queue 0)
>    disabled       real-time app (queue 0)   real-time app (queue 3)
> 
> Private flag is controlled with:
>   ethtool --set-priv-flags enp1s0 reverse-tsn-txq-prio <on|off>
> 
> [Ref1]
> IEEE 802.1Q clause 8.6.8 Transmission selection:
> "For a given Port and traffic class, frames are selected from the
> corresponding queue for transmission if and only if:
> ...
> b) For each queue corresponding to a numerically higher value of traffic
> class supported by the Port, the operation of the transmission selection
> algorithm supported by that queue determines that there is no frame
> available for transmission."
> 
> Reviewed-by: Simon Horman <horms@...nel.org>
> Signed-off-by: Faizal Rahim <faizal.abdul.rahim@...ux.intel.com>
> ---
>   drivers/net/ethernet/intel/igc/igc.h         |  1 +
>   drivers/net/ethernet/intel/igc/igc_ethtool.c | 12 ++++++++++--
>   drivers/net/ethernet/intel/igc/igc_main.c    |  3 ++-
>   drivers/net/ethernet/intel/igc/igc_tsn.c     |  3 ++-
>   4 files changed, 15 insertions(+), 4 deletions(-)
> 
Tested-by: Mor Bar-Gabay <morx.bar.gabay@...el.com>

Powered by blists - more mailing lists