netdev - Kernel 5.4 regression - memory leak in network layer

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Date:   Tue, 19 Nov 2019 11:09:10 -0500 (EST)
From:   Byron Stanoszek <gandalf@...ds.org>
To:     "David S. Miller" <davem@...emloft.net>
cc:     linux-kernel@...r.kernel.org, netdev@...r.kernel.org
Subject: Kernel 5.4 regression - memory leak in network layer

Hi Dave,

I'm experiencing a large memory leak in kernel 5.4-rc* when receiving network
packets using (at least) the intel igb driver. The leak is not present in
kernel 5.3. I've bisected this down to a set of ~900 commits, but I'm having
trouble bisecting any further due to several of the commits causing a kernel
panic on my system.

Git bisect shows the problem started somewhere in this range of commits:

a18670f4617d41829ce88ab3e47bbc406e4dc5e8 028db79cf46ae860fbdc5aa4f431b6969ea72c7b

I've traced the unreferenced objects using kmemleak. They all look like the
following:

unreferenced object 0xffff88821a48a180 (size 64):
   comm "softirq", pid 0, jiffies 4294709480 (age 192.558s)
   hex dump (first 32 bytes):
     01 00 00 00 01 06 ff ff 00 00 00 00 00 00 00 00  ................
     00 20 72 3d 82 88 ff ff 00 00 00 00 00 00 00 00  . r=............
   backtrace:
     [<00000000edf73c5e>] skb_ext_add+0xc0/0xf0
     [<00000000ca960770>] br_nf_pre_routing+0x171/0x489
     [<0000000063a55d83>] br_handle_frame+0x171/0x300
     [<0000000023337088>] __netif_receive_skb_core+0x12a/0x840
     [<0000000072de0b8c>] __netif_receive_skb_one_core+0x21/0x50
     [<0000000006be8eb7>] netif_receive_skb_internal+0x55/0xa0
     [<000000006ba3ef5b>] napi_gro_receive+0x45/0x80
     [<00000000f960dfcf>] igb_poll+0x434/0xeb0
     [<0000000002710716>] net_rx_action+0xcb/0x240
     [<00000000657ca2bd>] __do_softirq+0xc8/0x209
     [<000000004860e2aa>] irq_exit+0x9a/0xa0
     [<0000000024397c0c>] do_IRQ+0x4a/0xc0
     [<000000008ecdeb2a>] ret_from_intr+0x0/0x14

I am currently using this patch to igb_main.c to disable MSI-X (just in case it
matters):

--- linux/drivers/net/ethernet/intel/igb/igb_main.c.bak 2019-06-19 16:39:12.000000000 -0400
+++ linux/drivers/net/ethernet/intel/igb/igb_main.c     2019-06-19 16:39:21.000000000 -0400
@@ -1097,7 +1097,7 @@ static void igb_set_interrupt_capability
 	int err;
 	int numvecs, i;

-	if (!msix)
+//	if (!msix)
 		goto msi_only;
 	adapter->flags |= IGB_FLAG_HAS_MSIX;

Please let me know if I can assist further to help you track this leak down.

I've attached a copy of my kernel config.

Thanks,
  -Byron

View attachment "config-5.4" of type "text/plain" (100211 bytes)