netdev - [PATCH net-next 00/14] ionic: putting ionic on a diet

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Message-ID: <20240306232959.17316-1-shannon.nelson@amd.com>
Date: Wed, 6 Mar 2024 15:29:45 -0800
From: Shannon Nelson <shannon.nelson@....com>
To: <netdev@...r.kernel.org>, <davem@...emloft.net>, <kuba@...nel.org>,
	<edumazet@...gle.com>, <pabeni@...hat.com>
CC: <brett.creeley@....com>, <drivers@...sando.io>, Shannon Nelson
	<shannon.nelson@....com>
Subject: [PATCH net-next 00/14] ionic: putting ionic on a diet

Building on the performance work done in the previous patchset
    [Link] https://lore.kernel.org/netdev/20240229193935.14197-1-shannon.nelson@amd.com/
this patchset puts the ionic driver on a diet, decreasing the memory
requirements per queue, and simplifies a few more bits of logic.

We trimmed the queue management structs and gained some ground, but
the most savings came from trimming the individual buffer descriptors.
The original design used a single generic buffer descriptor for Tx, Rx and
Adminq needs, but the Rx and Adminq descriptors really don't need all the
info that the Tx descriptors track.  By splitting up the descriptor types
we can significantly reduce the descriptor sizes for Rx and Adminq use.

There is a small reduction in the queue management structs, saving about
3 cachelines per queuepair:

    ionic_qcq:
	Before:	/* size: 2176, cachelines: 34, members: 23 */
	After:	/* size: 2048, cachelines: 32, members: 23 */

We also remove an array of completion descriptor pointers, or about
8 Kbytes per queue.

But the biggest savings came from splitting the desc_info struct into
queue specific structs and trimming out what was unnecessary.

    Before:
	ionic_desc_info:
		/* size: 496, cachelines: 8, members: 10 */
    After:
	ionic_tx_desc_info:
		/* size: 496, cachelines: 8, members: 6 */
	ionic_rx_desc_info:
		/* size: 224, cachelines: 4, members: 2 */
	ionic_admin_desc_info:
		/* size: 8, cachelines: 1, members: 1 */

In a 64 core host the ionic driver will default to 64 queuepairs of
1024 descriptors for Rx, 1024 for Tx, and 80 for Adminq and Notifyq. 

The total memory usage for 64 queues:
    Before:
	  65 * sizeof(ionic_qcq)			   141,440
	+ 64 * 1024 * sizeof(ionic_desc_info)		32,505,856
	+ 64 * 1024 * sizeof(ionic_desc_info)		32,505,856
	+ 64 * 1024 * 2 * sizeof(ionic_qc_info)		    16,384
	+  1 *   80 * sizeof(ionic_desc_info)		    39,690
							----------
							65,201,038

    After:
	  65 * sizeof(ionic_qcq)			   133,120
	+ 64 * 1024 * sizeof(ionic_tx_desc_info)	32,505,856
	+ 64 * 1024 * sizeof(ionic_rx_desc_info)	14,680,064
	+                           (removed)		         0
	+  1 *   80 * sizeof(ionic_admin desc_info)	       640
							----------
							47,319,680

This saves us approximately 18 Mbytes per port in a 64 core machine,
a 28% savings in our memory needs.

In addition, this improves our simple single thread / single queue
iperf case on a 9100 MTU connection from 86.7 to 95 Gbits/sec.


Shannon Nelson (14):
  ionic: remove desc, sg_desc and cmb_desc from desc_info
  ionic: drop q mapping
  ionic: move adminq-notifyq handling to main file
  ionic: remove callback pointer from desc_info
  ionic: remove the cq_info to save more memory
  ionic: use specialized desc info structs
  ionic: fold adminq clean into service routine
  ionic: refactor skb building
  ionic: carry idev in ionic_cq struct
  ionic: rearrange ionic_qcq
  ionic: rearrange ionic_queue for better layout
  ionic: remove unnecessary NULL test
  ionic: better dma-map error handling
  ionic: keep stats struct local to error handling

 drivers/net/ethernet/pensando/ionic/ionic.h   |   2 +
 .../net/ethernet/pensando/ionic/ionic_dev.c   | 105 +----
 .../net/ethernet/pensando/ionic/ionic_dev.h   |  79 ++--
 .../net/ethernet/pensando/ionic/ionic_lif.c   | 165 +++-----
 .../net/ethernet/pensando/ionic/ionic_lif.h   |   8 +-
 .../net/ethernet/pensando/ionic/ionic_main.c  | 115 ++++-
 .../net/ethernet/pensando/ionic/ionic_txrx.c  | 399 ++++++++----------
 .../net/ethernet/pensando/ionic/ionic_txrx.h  |   2 +-
 8 files changed, 371 insertions(+), 504 deletions(-)

-- 
2.17.1