Message-ID: <CACna6rzMMUA_MQemYyP93+TjP4aocL1WaYQUJFwQm3+DLNAefg@mail.gmail.com>
Date:	Thu, 23 Apr 2015 20:18:01 +0200
From:	Rafał Miłecki <zajec5@...il.com>
To:	Network Development <netdev@...r.kernel.org>
Cc:	Felix Fietkau <nbd@...nwrt.org>
Subject: NAPI: Polling function requesting extra calls - difference between
 3.18.11 and 4.0.0

Hi,

Recently Felix improved bgmac driver to be smarter in situation where
new packets were Tx/Rx-ed during bgmac_poll execution. It was handled
in:
eb64e29 bgmac: leave interrupts disabled as long as there is work to do
http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=eb64e2923a886441c7b322f138b36029f3fa6a36
With above change, after handling all skb-s in bgmac_poll, bgmac
checks if there are new sbk-s to be handled. If so, it doesn't call
napi_complete & it doesn't enable interrupts.

Above commit was merged for 4.1 kernel, but I work with OpenWrt based
on older releases, so I have all bgmac changes backported to 3.8.11
and 4.0.


With 3.8.11 Felix's change seems to be working fine, I did some debugging:

[   44.970000] [bgmac_interrupt] int_status & int_mask = 0x01000000

[   44.980000] [bgmac_poll] START weight:64
[   44.990000] [bgmac_poll] Read handled:0 skb-s
[   44.990000] [bgmac_poll] Read IRQ status again BGMAC_IS_TX0:0 BGMAC_IS_RX:0
[   45.000000] [bgmac_poll] Read skb-s < weight, calling napi_complete, en IRQs
[   45.010000] [bgmac_poll] END returning handled:0

[   48.040000] [bgmac_interrupt] int_status & int_mask = 0x01000000

[   48.050000] [bgmac_poll] START weight:64
[   48.060000] [bgmac_poll] Read handled:0 skb-s
[   48.060000] [bgmac_poll] Read IRQ status again BGMAC_IS_TX0:0 BGMAC_IS_RX:0
[   48.070000] [bgmac_poll] Read skb-s < weight, calling napi_complete, en IRQs
[   48.080000] [bgmac_poll] END returning handled:0

[   48.100000] [bgmac_poll] START weight:64
[   48.110000] [bgmac_poll] Read handled:1 skb-s
[   48.110000] [bgmac_poll] Read IRQ status again BGMAC_IS_TX0:1 BGMAC_IS_RX:1
[   48.120000] [bgmac_poll] END more skb-s to handle, returning handled:1

[   48.130000] [bgmac_poll] START weight:64
[   48.130000] [bgmac_poll] Read handled:1 skb-s
[   48.140000] [bgmac_poll] Read IRQ status again BGMAC_IS_TX0:0 BGMAC_IS_RX:0
[   48.150000] [bgmac_poll] Read skb-s < weight, calling napi_complete, en IRQs
[   48.160000] [bgmac_poll] END returning handled:1

Please note that at 48.120000 bgmac decided bgmac_poll should be
called again and it didn't call napi_complete & didn't enable IRQs. It
simply returned 1.
NAPI behaved just like we expected, bgmac_poll was called again a
moment later. It handled skb-s that appeared meanwhile and ended
calling napi_complete and returning 1.


Now, it fails badly for me at all when using kernel 4.0.0:

[   52.800000] [bgmac_interrupt] int_status & int_mask = 0x01000000

[   52.800000] [bgmac_poll] START weight:64
[   52.810000] [bgmac_poll] Read handled:0 skb-s
[   52.810000] [bgmac_poll] Read IRQ status again BGMAC_IS_TX0:0 BGMAC_IS_RX:0
[   52.820000] [bgmac_poll] Read skb-s < weight, calling napi_complete, en IRQs
[   52.830000] [bgmac_poll] END returning handled:0

[   54.610000] [bgmac_interrupt] int_status & int_mask = 0x01000000

[   54.620000] [bgmac_poll] START weight:64
[   54.630000] [bgmac_poll] Read handled:0 skb-s
[   54.630000] [bgmac_poll] Read IRQ status again BGMAC_IS_TX0:0 BGMAC_IS_RX:0
[   54.640000] [bgmac_poll] Read skb-s < weight, calling napi_complete, en IRQs
[   54.650000] [bgmac_poll] END returning handled:0

[   55.900000] [bgmac_interrupt] int_status & int_mask = 0x01000000

[   55.910000] [bgmac_poll] START weight:64
[   55.920000] [bgmac_poll] Read handled:1 skb-s
[   55.920000] [bgmac_poll] Read IRQ status again BGMAC_IS_TX0:1 BGMAC_IS_RX:1
[   55.930000] [bgmac_poll] END more skb-s to handle, returning handled:1

As you can see, at 55.930000 bgmac also decided bgmac_poll should be
called again. It didn't call napi_complete, didn't enable IRQs. It
simply returned 1. Just like with kernel 3.8.11.
It this case however, NAPI never called bgmac_poll again. It resulted
in bgmac hanging (IRQs are off, bgmac expects bgmac_poll to be
called).


Can you help us with this, please? Does bgmac use NAPI incorrectly?
Were there some important changes in 3.19 or 4.0?

-- 
Rafał

View attachment "tmp.diff" of type "text/plain" (1843 bytes)

Powered by blists - more mailing lists