Message-ID: <20181022122743.20384-1-Ariel.Elior@cavium.com>
Date:   Mon, 22 Oct 2018 15:27:37 +0300
From:   Ariel Elior <Ariel.Elior@...ium.com>
To:     <davem@...emloft.net>
CC:     <netdev@...r.kernel.org>, Ariel Elior <Ariel.Elior@...ium.com>,
        "Michal Kalderon" <Michal.Kalderon@...ium.com>,
        Tomer Tayar <Tomer.Tayar@...ium.com>
Subject: [PATCH net-next 0/6] qed*: Doorbell overflow recovery

Doorbell Overflow
If sufficient CPU cores will send doorbells at a sufficiently high rate, they
can cause an overflow in the doorbell queue block message fifo. When fill level
reaches maximum, the device stops accepting all doorbells from that PF until a
recovery procedure has taken place.

Doorbell Overflow Recovery
The recovery procedure basically means resending the last doorbell for every
doorbelling entity. A doorbelling entity is anything which may send doorbells:
L2 tx ring, rdma sq/rq/cq, light l2, vf l2 tx ring, spq, etc. This relies on
the design assumption that all doorbells are aggregative, so last doorbell
carries the information of all previous doorbells.

APIs
All doorbelling entities need to register with the mechanism before sending
doorbells. The registration entails providing the doorbell address the entity
would be using, and a virtual address where last doorbell data can be found.
Typically fastpath structures already have this construct.

Executing the recovery procedure
Handling the attentions, iterating over all the registered entities and
resending their doorbells, is all handled within qed core module.

Relevance
All doorbelling entities in all protocols need to register with the mechanism,
via the new APIs. Technically this is quite simple (just call the API). Some
protocol fastpath implementation may not have the doorbell data stored anywhere
(compute it from scratch every time) and will have to add such a place.
This is rare and is also better practice (save some cycles on the fastpath).

Performance Penalty
No performance penalty should incur as a result of this feature. If anything
performance can improve by avoiding recalcualtion of doorbell data everytime
doorbell is sent (in some flows).

Add the database used to register doorbelling entities, and APIs for adding
and deleting entries, and logic for traversing the database and doorbelling
once on behalf of all entities.

Please consider applying to net-next.
Thanks,
Ariel

Signed-off-by: Ariel Elior <Ariel.Elior@...ium.com>
Signed-off-by: Michal Kalderon <Michal.Kalderon@...ium.com>
Signed-off-by: Tomer Tayar <Tomer.Tayar@...ium.com>

Ariel Elior (6):
  qed: Add doorbell overflow recovery mechanism
  qed: Use the doorbell overflow recovery mechanism in case of doorbell
    overflow
  qed: Register slowpath queue doorbell with doorbell overflow recovery
    mechanism
  qed: Register light L2 queues with doorbell overflow recovery
    mechanism
  qed: Expose the doorbell overflow recovery mechanism to the protocol
    drivers
  qede: Register l2 queues with doorbell overflow recovery mechanism

 drivers/net/ethernet/qlogic/qed/qed.h          |  31 ++-
 drivers/net/ethernet/qlogic/qed/qed_dev.c      | 334 ++++++++++++++++++++++++-
 drivers/net/ethernet/qlogic/qed/qed_dev_api.h  |  28 +++
 drivers/net/ethernet/qlogic/qed/qed_int.c      | 152 +++++++++--
 drivers/net/ethernet/qlogic/qed/qed_int.h      |  10 +
 drivers/net/ethernet/qlogic/qed/qed_ll2.c      |  30 ++-
 drivers/net/ethernet/qlogic/qed/qed_ll2.h      |   1 +
 drivers/net/ethernet/qlogic/qed/qed_main.c     |  66 ++++-
 drivers/net/ethernet/qlogic/qed/qed_reg_addr.h |  50 ++++
 drivers/net/ethernet/qlogic/qed/qed_sp.h       |   4 +-
 drivers/net/ethernet/qlogic/qed/qed_spq.c      |  22 ++
 drivers/net/ethernet/qlogic/qede/qede_main.c   |   9 +
 include/linux/qed/qed_if.h                     |  42 ++++
 13 files changed, 744 insertions(+), 35 deletions(-)

-- 
1.8.3.1

Powered by blists - more mailing lists