lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
Open Source and information security mailing list archives
| ||
|
Message-ID: <dde54385fcb38c1a66b7d09d6975d449dc5bdfde.camel@microchip.com> Date: Thu, 5 Oct 2023 05:17:29 +0000 From: <VishvambarPanth.S@...rochip.com> To: <jacob.e.keller@...el.com>, <netdev@...r.kernel.org>, <linux-kernel@...r.kernel.org> CC: <andrew@...n.ch>, <pabeni@...hat.com>, <edumazet@...gle.com>, <davem@...emloft.net>, <UNGLinuxDriver@...rochip.com>, <Bryan.Whitehead@...rochip.com>, <kuba@...nel.org> Subject: Re: [PATCH net] net: microchip: lan743x : bidirectional throughuput improvement On Fri, 2023-09-29 at 10:35 -0700, Jacob Keller wrote: > EXTERNAL EMAIL: Do not click links or open attachments unless you > know the content is safe > > On 9/27/2023 4:16 AM, Vishvambar Panth S wrote: > > The LAN743x/PCI11xxx DMA descriptors are always 4 dwords long, but > > the > > device supports placing the descriptors in memory back to back or > > reserving space in between them using its DMA_DESCRIPTOR_SPACE > > (DSPACE) > > configurable hardware setting. Currently DSPACE is unnecessarily > > set to > > match the host's L1 cache line size, resulting in space reserved in > > between descriptors in most platforms and causing a suboptimal > > behavior > > (single PCIe Mem transaction per descriptor). By changing the > > setting > > to DSPACE=16 many descriptors can be packed in a single PCIe Mem > > transaction resulting in a massive performance improvement in > > bidirectional tests without any negative effects. > > Tested and verified improvements on x64 PC and several ARM > > platforms > > (typical data below) > > > > I assume here the choice of 16 is to get the 16 bytes from 4 dwords? Thanks for the comments. Yes, it is set to 16 bytes to match the size of the descriptors which are 4 dwords long. > > > Test setup 1: x64 PC with LAN7430 ---> x64 PC > > > > iperf3 UDP bidirectional with DSPACE set to L1 CACHE Size: > > - - - - - - - - - - - - - - - - - - - - - - - - - > > [ ID][Role] Interval Transfer Bitrate > > [ 5][TX-C] 0.00-10.00 sec 170 MBytes 143 Mbits/sec sender > > [ 5][TX-C] 0.00-10.04 sec 169 MBytes 141 Mbits/sec > > receiver > > [ 7][RX-C] 0.00-10.00 sec 1.02 GBytes 876 Mbits/sec sender > > [ 7][RX-C] 0.00-10.04 sec 1.02 GBytes 870 Mbits/sec > > receiver > > > > iperf3 UDP bidirectional with DSPACE set to 16 Bytes > > - - - - - - - - - - - - - - - - - - - - - - - - - > > [ ID][Role] Interval Transfer Bitrate > > [ 5][TX-C] 0.00-10.00 sec 1.11 GBytes 956 Mbits/sec sender > > [ 5][TX-C] 0.00-10.04 sec 1.11 GBytes 951 Mbits/sec > > receiver > > [ 7][RX-C] 0.00-10.00 sec 1.10 GBytes 948 Mbits/sec sender > > [ 7][RX-C] 0.00-10.04 sec 1.10 GBytes 942 Mbits/sec > > receiver > > > > Going from barely transmitting to hitting the line rate *and* > improving > both Tx and Rx slightly is fantastic. Huge win just avoiding all that > unnecessary wasted PCIe bandwidth. > > > Test setup 2 : RK3399 with LAN7430 ---> x64 PC > > > > RK3399 Spec: > > The SOM-RK3399 is ARM module designed and developed by > > FriendlyElec. > > Cores: 64-bit Dual Core Cortex-A72 + Quad Core Cortex-A53 > > Frequency: Cortex-A72(up to 2.0GHz), Cortex-A53(up to 1.5GHz) > > PCIe: PCIe x4, compatible with PCIe 2.1, Dual operation mode > > > > iperf3 UDP bidirectional with DSPACE set to L1 CACHE Size: > > - - - - - - - - - - - - - - - - - - - - - - - - - > > [ ID][Role] Interval Transfer Bitrate > > [ 5][TX-C] 0.00-10.00 sec 534 MBytes 448 Mbits/sec sender > > [ 5][TX-C] 0.00-10.05 sec 534 MBytes 446 Mbits/sec > > receiver > > [ 7][RX-C] 0.00-10.00 sec 1.12 GBytes 961 Mbits/sec sender > > [ 7][RX-C] 0.00-10.05 sec 1.11 GBytes 946 Mbits/sec > > receiver > > > > iperf3 UDP bidirectional with DSPACE set to 16 Bytes > > - - - - - - - - - - - - - - - - - - - - - - - - - > > [ ID][Role] Interval Transfer Bitrate > > [ 5][TX-C] 0.00-10.00 sec 966 MBytes 810 Mbits/sec sender > > [ 5][TX-C] 0.00-10.04 sec 965 MBytes 806 Mbits/sec > > receiver > > [ 7][RX-C] 0.00-10.00 sec 1.11 GBytes 956 Mbits/sec sender > > [ 7][RX-C] 0.00-10.04 sec 1.07 GBytes 919 Mbits/sec > > receiver > > > > Fixes: 23f0703c125b ("lan743x: Add main source files for new > > lan743x driver") > > Signed-off-by: Vishvambar Panth S <vishvambarpanth.s@...rochip.com> > > --- > > Always fun to see a massive win like this! Even just the 2x > improvement > on the RK3399 is huge, let alone 6x improvement on the x86 platform. > > Reviewed-by: Jacob Keller <jacob.e.keller@...el.com> > > > drivers/net/ethernet/microchip/lan743x_main.h | 2 +- > > 1 file changed, 1 insertion(+), 1 deletion(-) > > > > diff --git a/drivers/net/ethernet/microchip/lan743x_main.h > > b/drivers/net/ethernet/microchip/lan743x_main.h > > index 52609fc13ad9..6dac6fef7d24 100644 > > --- a/drivers/net/ethernet/microchip/lan743x_main.h > > +++ b/drivers/net/ethernet/microchip/lan743x_main.h > > @@ -1067,7 +1067,7 @@ struct lan743x_adapter { > > #define DMA_DESCRIPTOR_SPACING_32 (32) > > #define DMA_DESCRIPTOR_SPACING_64 (64) > > #define DMA_DESCRIPTOR_SPACING_128 (128) > > -#define DEFAULT_DMA_DESCRIPTOR_SPACING (L1_CACHE_BYTES) > > +#define DEFAULT_DMA_DESCRIPTOR_SPACING > > (DMA_DESCRIPTOR_SPACING_16) > > > > #define DMAC_CHANNEL_STATE_SET(start_bit, stop_bit) \ > > (((start_bit) ? 2 : 0) | ((stop_bit) ? 1 : 0))
Powered by blists - more mailing lists