| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Message-ID: <CAL8zT=g67JLZZjLF81+wxe9pmuPZ4EdpY1fHJMQNgcy9R6oBzg@mail.gmail.com> Date: Thu, 7 Jun 2012 14:27:42 +0200 From: Jean-Michel Hautbois <jhautbois@...il.com> To: Eric Dumazet <eric.dumazet@...il.com> Cc: Sathya.Perla@...lex.com, netdev@...r.kernel.org Subject: Re: Difficulties to get 1Gbps on be2net ethernet card 2012/6/6 Jean-Michel Hautbois <jhautbois@...il.com>: > 2012/6/6 Jean-Michel Hautbois <jhautbois@...il.com>: >> 2012/6/6 Jean-Michel Hautbois <jhautbois@...il.com>: >>> 2012/6/6 Eric Dumazet <eric.dumazet@...il.com>: >>>> On Wed, 2012-06-06 at 12:04 +0200, Jean-Michel Hautbois wrote: >>>> >>>>> Well, well, well, after having tested several configurations, several >>>>> drivers, I have a big difference between an old 2.6.26 kernel and a >>>>> newer one (I tried 3.2 and 3.4). >>>>> >>>>> Here is my stream : UDP packets (multicast), 4000 bytes length, MTU >>>>> set to 4096. I am sending packets only, nothing on RX. >>>>> I send from 1Gbps upto 2.4Gbps and I see no drops in tc with 2.6.26 >>>>> kernel, but a lot of drops with a newer kernel. >>>>> So, I don't know if I missed something in my kernel configuration, but >>>>> I have used the 2.6.26 one as a reference, in order to set the same >>>>> options (DMA related, etc). >>>>> >>>>> I easily reproduce this problem and setting a bigger txqueuelen solves >>>>> it partially. >>>>> 1Gbps requires a txqueulen of 9000, 2.4Gbps requires more than 20000 ! >>>>> >>>>> If you have any idea, I am interested, as this is a big issue for my use case. >>>>> >>>> >>>> Yep. >>>> >>>> This driver wants to limit number of tx completions, thats just wrong. >>>> >>>> Fix and dirty patch: >>>> >>>> >>>> diff --git a/drivers/net/ethernet/emulex/benet/be.h b/drivers/net/ethernet/emulex/benet/be.h >>>> index c5c4c0e..1e8f8a6 100644 >>>> --- a/drivers/net/ethernet/emulex/benet/be.h >>>> +++ b/drivers/net/ethernet/emulex/benet/be.h >>>> @@ -105,7 +105,7 @@ static inline char *nic_name(struct pci_dev *pdev) >>>> #define MAX_TX_QS 8 >>>> #define MAX_ROCE_EQS 5 >>>> #define MAX_MSIX_VECTORS (MAX_RSS_QS + MAX_ROCE_EQS) /* RSS qs + RoCE */ >>>> -#define BE_TX_BUDGET 256 >>>> +#define BE_TX_BUDGET 65535 >>>> #define BE_NAPI_WEIGHT 64 >>>> #define MAX_RX_POST BE_NAPI_WEIGHT /* Frags posted at a time */ >>>> #define RX_FRAGS_REFILL_WM (RX_Q_LEN - MAX_RX_POST) >>>> >>> >>> I will try that in a few minutes. >>> I also have a mlx4 driver (mlx4_en) which has a similar behaviour, and >>> a broadcom (bnx2x). >>> >> >> And it is not really better, still need about 18000 at 2.4Gbps in >> order to avoid drops... >> I really think there is something in the networking stack or in my >> configuration (DMA ? Something else ?)... >> As it doesn't seem to be driver related as I said... >> > > If it can help, on a 3.0 kernel a txqueuelen of 9000 is sufficient in > order to get this bandwith on TX. > > JM All, I made some tests, and I didn't mention it : I am using the bonding driver over my ethernet drivers (be2net/mlx4 etc.). When I am using bonding, I need a big txqeuelen in order to send 2.4Gbps. When I disable bonding, and use directly the NIC then I don't see any drops in qdisc and it works well. So, I think there is something between 2.6.26 and 3.0 in the bonding driver which causes this issue. Any help would be appreciated :). Regards, JM -- To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to majordomo@...r.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Powered by blists - more mailing lists