| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Message-ID: <771471B8871B5044A6CA7CCD9C26EEE10117E265CE84@xmail2.se.axis.com>
Date: Fri, 7 Mar 2014 16:45:17 +0100
From: Lars Persson <lars.persson@...s.com>
To: Eric Dumazet <eric.dumazet@...il.com>
CC: "netdev@...r.kernel.org" <netdev@...r.kernel.org>
Subject: RE: [PATCH] tcp: tcp_release_cb() should release socket ownership
So far no deadlock on the socket lock is seen with this patch. I have seen two out-of-memory crashes which leaves the system deadlocked trying to allocate memory.
Stack traces and kernel log from one of the oom crashes is attached. I was thinking this may be caused by having the dirty_ratio set too high given our product's heavy use of memory. Currently re-testing with dirty_ratio reduced to 5%.
[11515.530000] Recording maint invoked oom-killer: gfp_mask=0x201da, order=0, oom_score_adj=0
[11515.540000] CPU: 0 PID: 1304 Comm: Recording maint Tainted: G O 3.10.32 #10
[11515.550000] Stack : 80666e52 00000049 80660000 00000000 805563dc 8b1876f8 8054af80 805d2227
00000518 00000000 80656d40 8b1876f8 000007ce 00000002 00000000 8047d858
00000000 8003bb64 00000006 00000000 8054d024
[11515.580000] 8b189a84 8b189a84 8054af80
00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
00000000 00000000 00000000 00000000 00000000 00000000 00000000 8b189a18
...
[11515.600000] Call Trace:
[11515.600000] [<8001d574>] show_stack+0x64/0x7c
[11515.610000] [<8047f438>] dump_header.isra.15+0x80/0x234
[11515.610000] [<800dad58>] oom_kill_process+0x2b8/0x464
[11515.620000] [<800db480>] out_of_memory+0x330/0x35c
[11515.620000] [<800defb8>] __alloc_pages_nodemask+0x748/0x75c
[11515.630000] [<800d9ab0>] filemap_fault+0x1b8/0x4b0
[11515.640000] [<800f8da8>] __do_fault+0x88/0x638
[11515.650000] [<800fc360>] handle_pte_fault+0xc4/0x1328
[11515.650000] [<800fd6ec>] handle_mm_fault+0x128/0x188
[11515.660000] [<800253ac>] do_page_fault+0x10c/0x520
[11515.660000] [<80017ee0>] ret_from_exception+0x0/0x10
[11515.670000]
[11515.670000] Mem-Info:
[11515.670000] Normal per-cpu:
[11515.680000] CPU 0: hi: 90, btch: 15 usd: 12
[11515.680000] CPU 1: hi: 90, btch: 15 usd: 88
[11515.690000] active_anon:8293 inactive_anon:12641 isolated_anon:0
[11515.690000] active_file:9 inactive_file:247 isolated_file:0
[11515.690000] unevictable:0 dirty:0 writeback:0 unstable:0
[11515.690000] free:2034 slab_reclaimable:1371 slab_unreclaimable:20541
[11515.690000] mapped:729 shmem:14591 pagetables:330 bounce:0
[11515.690000] free_cma:0
[11515.730000] Normal free:8136kB min:8192kB low:10240kB high:12288kB active_anon:33172kB inactive_anon:50564kB active_file:36kB inactive_file:1024kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:204800kB managed:190524kB mlocked:0kB dirty:0kB writeback:0kB mapped:2916kB shmem:58364kB slab_reclaimable:5484kB slab_unreclaimable:82164kB kernel_stack:1256kB pagetables:1320kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:340 all_unreclaimable? yes
[11515.780000] lowmem_reserve[]: 0 0
[11515.780000] Normal: 32*4kB (UM) 0*8kB 0*16kB 0*32kB 1*64kB (R) 1*128kB (R) 1*256kB (R) 1*512kB (R) 1*1024kB (R) 1*2048kB (R) 1*4096kB (R) = 8256kB
[11515.800000] 14940 total pagecache pages
[11515.820000] 51200 pages RAM
[11515.820000] 3486 pages reserved
[11515.830000] 263787 pages shared
[11515.830000] 44113 pages non-shared
Kernel thread cifsd:
-000 |__schedule()
-001 |__lock_sock(sk = 0x8BFD7920)
-002 |lock_sock_nested(sk = 0x8BFD7920, subclass = 0)
-003 |sk_wait_data(sk = 0x8BFD7920, timeo = 0x8AFD3C58)
-004 |tcp_recvmsg(?, sk = 0x8BFD7920, msg = 0x8AFD3DE0, len = 4, nonblock = 0, f
-005 |inet_recvmsg(iocb = 0x8AFD3CF8, ?, msg = 0x8AFD3DE0, size = 4, flags = 0)
-006 |sock_recvmsg(sock = 0x87C0A8E0, msg = 0x8AFD3DE0, size = 4, flags = 0)
-007 |kernel_recvmsg(?, ?, ?, ?, size = 4, flags = 0)
-008 |cifs_readv_from_socket(server = 0x8B072800, iov_orig = 0x8AFD3E40, nr_segs
-009 |cifs_read_from_socket(?, ?, ?)
-010 |cifs_demultiplex_thread(p = 0x8B072800)
-011 |kthread(_create = 0x8C7DFD40)
-012 |ret_from_kernel_thread(asm)
--> |exception
--- |end of frame
Kernel thread kworker/1:1
-000 |__schedule()
-001 |__cond_resched()
-002 |cond_resched()
-003 |shrink_slab(shrink = 0x8C6C799C, nr_pages_scanned = 32, lru_pages = 5)
-004 |try_to_free_pages(?, ?, ?, ?)
-005 |__alloc_pages_nodemask(?, order = 0, ?, nodemask = 0x0)
-006 |cache_alloc_refill(cachep = 0x8C414780, flags = 80, ?)
-007 |kmem_cache_alloc(cachep = 0x8C414780, flags = 80)
-008 |__alloc_skb(size = 1856, gfp_mask = 80, ?, ?)
-009 |sk_stream_alloc_skb(sk = 0x8BFD7920, size = 1648, ?)
-010 |tcp_sendmsg(?, sk = 0x8BFD7920, ?, ?)
-011 |sock_sendmsg(sock = 0x87C0A8E0, msg = 0x8C6C7CB8, size = 42)
-012 |kernel_sendmsg(?, ?, ?, ?, size = 42)
-013 |smb_send_kvec(server = 0x8B072800, iov = 0x8C6C7DC8, n_vec = 1, sent = 0x8C6C7D2C)
-014 |smb_send_rqst(server = 0x8B072800, rqst = 0x8C6C7DB0)
-015 |cifs_call_async(server = 0x8B072800, rqst = 0x8C6C7DB0, receive = 0x0, callback = 0x802194DC, cb
-016 |CIFSSMBEcho(server = 0x8B072800)
-017 |cifs_echo_request(work = 0x8B072A90)
-018 |process_one_work(worker = 0x8C6F5B80, work = 0x8B072A90)
-019 |worker_thread(__worker = 0x8C6F5B80)
-020 |kthread(_create = 0x8C47DD90)
-021 |ret_from_kernel_thread(asm)
--> |exception
--- |end of frame
sk = 0x8BFD7920 -> (
__sk_common = (skc_addrpair = 8575039233117432000, skc_daddr = 16820416, skc_rcv_saddr = 1996531904, skc_hash = 670815510, skc_u16hashes = (54550, 10235), skc_portpair = 3106389249, skc_dport = 48385, skc_num = 47399, skc_f
sk_lock = (
slock = (
rlock = (
raw_lock = (
lock = 0x00DC00DC,
h = (serving_now = 220, ticket = 220)),
magic = 0xDEAD4EAD,
owner_cpu = 0xFFFFFFFF,
owner = 0xFFFFFFFF,
dep_map = (key = 0x80B87FC0, class_cache = (0x0, 0x0), name = 0x80568D0C)),
__padding = (220, 0, 220, 0, 173, 78, 173, 222, 255, 255, 255, 255, 255, 255, 255, 255),
dep_map = (key = 0x80B87FC0, class_cache = (0x0, 0x0), name = 0x80568D0C)),
owned = 1,
wq = (lock = (rlock = (raw_lock = (lock = 196611, h = (serving_now = 3, ticket = 3)), magic = 3735899821, owner_cpu = 4294967295, owner = 0xFFFFFFFF, dep_map = (key = 0x80B87FE0, class_cache = (0x0, 0x0), name = 0x80568CB
dep_map = (key = 0x80B87FD0, class_cache = (0x0, 0x0), name = 0x80568D20)),
> -----Original Message-----
> From: Eric Dumazet [mailto:eric.dumazet@...il.com]
> Sent: den 6 mars 2014 03:21
> To: David Miller
> Cc: Lars Persson; netdev@...r.kernel.org
> Subject: Re: [PATCH] tcp: tcp_release_cb() should release socket
> ownership
>
> On Wed, 2014-03-05 at 21:15 -0500, David Miller wrote:
>
> > Targetting net-next for the assertion is fine.
> >
> > Did you get test results back yet?
>
> Lars gave the results this morning for the first patch.
>
> He said he was now testing the official patch.
>
> Lets wait a bit more ;)
>
>
Powered by blists - more mailing lists