| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Message-ID: <b95f4a06d7d9b13849ad277128690436@eikelenboom.it>
Date: Thu, 12 Nov 2015 16:16:45 +0100
From: Sander Eikelenboom <linux@...elenboom.it>
To: Eric Dumazet <eric.dumazet@...il.com>
Cc: netdev@...r.kernel.org, netfilter-devel@...r.kernel.org
Subject: Re: [linux-4.4-mw] BUG: unable to handle kernel paging request
ip_vs_out.constprop
On 2015-11-12 15:09, Eric Dumazet wrote:
> On Thu, 2015-11-12 at 11:08 +0100, Sander Eikelenboom wrote:
>> Hi All,
>>
>> Just got a crash with a linux-4.4-mw kernel.
>> I'm using a routed bridge and apart from the splat below i have got
>> some
>> interesting other messages that aren't there in 4.3 (and perhaps are
>> of
>> interest for the crash as well):
>> [ 207.033768] vif vif-1-0 vif1.0: set_features() failed (-1); wanted
>> 0x0000000400004803, left 0x0000000400114813
>> [ 207.033780] vif vif-1-0 vif1.0: set_features() failed (-1); wanted
>> 0x0000000400004803, left 0x0000000400114813
>> [ 207.245435] xen_bridge: error setting offload STP state on port
>> 1(vif1.0)
>> [ 207.245442] vif vif-1-0 vif1.0: failed to set HW ageing time
>> [ 207.245443] xen_bridge: error setting offload STP state on port
>> 1(vif1.0)
>> [ 207.245491] vif vif-1-0 vif1.0: set_features() failed (-1); wanted
>> 0x0000000400004803, left 0x0000000400114813
>>
>> The commit message for the commit that introduced the "set HW ageing
>> time" error message, doesn't seem to tell
>> me much about it's purpose. If it's not related i can reported as a
>> seperate issue.
>>
>> --
>> Sander
>>
>> The crash:
>> [ 354.328687] BUG: unable to handle kernel paging request at
>> ffff880049aa8000
>> [ 354.350206] IP: [<ffffffff81a074a7>]
>> ip_vs_out.constprop.25+0x47/0x60
>> [ 354.360882] PGD 2212067 PUD 25b4067 PMD 5ffb6067 PTE 0
>> [ 354.371587] Oops: 0000 [#1] SMP
>> [ 354.382143] Modules linked in:
>> [ 354.392537] CPU: 0 PID: 0 Comm: swapper/0 Not tainted
>> 4.3.0-mw-20151111-linus-doflr+ #1
>> [ 354.403105] Hardware name: MSI MS-7640/890FXA-GD70 (MS-7640) ,
>> BIOS
>> V1.8B1 09/13/2010
>> [ 354.413666] task: ffffffff82218580 ti: ffffffff82200000 task.ti:
>> ffffffff82200000
>> [ 354.424255] RIP: e030:[<ffffffff81a074a7>] [<ffffffff81a074a7>]
>> ip_vs_out.constprop.25+0x47/0x60
>> [ 354.434742] RSP: e02b:ffff88005f6034b0 EFLAGS: 00010246
>> [ 354.445006] RAX: 0000000000000001 RBX: ffff88005f6034f8 RCX:
>> ffff880049aa7ce0
>> [ 354.455262] RDX: ffff88003c0e5500 RSI: 0000000000000003 RDI:
>> ffff880004e0e800
>> [ 354.465422] RBP: ffff88005f6034b8 R08: 0000000000000014 R09:
>> 0000000000000003
>> [ 354.475508] R10: 0000000000000001 R11: ffff880040f394cc R12:
>> ffff88005f603528
>> [ 354.485567] R13: ffff88003c0e5500 R14: ffffffff822da2e8 R15:
>> ffff88003c0e5500
>> [ 354.495595] FS: 00007f0243c2b700(0000) GS:ffff88005f600000(0000)
>> knlGS:0000000000000000
>> [ 354.505474] CS: e033 DS: 0000 ES: 0000 CR0: 000000008005003b
>> [ 354.515135] CR2: ffff880049aa8000 CR3: 0000000059271000 CR4:
>> 0000000000000660
>> [ 354.524794] Stack:
>> [ 354.534319] ffffffff81a074fc ffff88005f6034e8 ffffffff8199e138
>> ffff88003c0e5500
>> [ 354.543981] ffff88005f603528 ffff88003c0e5500 0000000000000000
>> ffff88005f603518
>> [ 354.553577] ffffffff8199e1af ffff880005300048 ffff88003c0e5500
>> ffffffff822da2e8
>> [ 354.563160] Call Trace:
>> [ 354.572418] <IRQ>
>> [ 354.572480] [<ffffffff81a074fc>] ? ip_vs_local_reply4+0x1c/0x20
>> [ 354.590458] [<ffffffff8199e138>] nf_iterate+0x58/0x70
>> [ 354.599372] [<ffffffff8199e1af>] nf_hook_slow+0x5f/0xb0
>> [ 354.608245] [<ffffffff81a1c73e>] __ip_local_out+0x9e/0xb0
>> [ 354.617036] [<ffffffff81a1a940>] ? ip_forward_options+0x1a0/0x1a0
>> [ 354.625874] [<ffffffff81a1c767>] ip_local_out+0x17/0x40
>> [ 354.634383] [<ffffffff81a1c8d8>] ip_build_and_send_pkt+0x148/0x1c0
>> [ 354.642715] [<ffffffff81a39796>] tcp_v4_send_synack+0x56/0xa0
>> [ 354.650893] [<ffffffff81a22b88>] ?
>> inet_csk_reqsk_queue_hash_add+0x68/0x90
>> [ 354.659083] [<ffffffff81a2b98d>] tcp_conn_request+0x95d/0x970
>> [ 354.667196] [<ffffffff810ccfa6>] ? __local_bh_enable_ip+0x26/0x90
>> [ 354.675246] [<ffffffff81a38bc7>] tcp_v4_conn_request+0x47/0x50
>> [ 354.683254] [<ffffffff81a30663>] tcp_rcv_state_process+0x183/0xca0
>> [ 354.691004] [<ffffffff81a37a7c>] tcp_v4_do_rcv+0x5c/0x1f0
>> [ 354.698533] [<ffffffff81a3a2b7>] tcp_v4_rcv+0x987/0x9a0
>> [ 354.705968] [<ffffffff81a5deb8>] ? ipv4_confirm+0x78/0xf0
>> [ 354.713370] [<ffffffff81a172f4>]
>> ip_local_deliver_finish+0x84/0x120
>> [ 354.720739] [<ffffffff81a17842>] ip_local_deliver+0x42/0xd0
>> [ 354.728029] [<ffffffff81a17270>] ? inet_del_offload+0x40/0x40
>> [ 354.735270] [<ffffffff81a17496>] ip_rcv_finish+0x106/0x320
>> [ 354.742413] [<ffffffff81a17ae1>] ip_rcv+0x211/0x370
>> [ 354.749268] [<ffffffff81a17390>] ?
>> ip_local_deliver_finish+0x120/0x120
>> [ 354.755929] [<ffffffff8196cd9b>]
>> __netif_receive_skb_core+0x2cb/0x970
>> [ 354.762535] [<ffffffff819bb75a>] ? nf_nat_setup_info+0x7a/0x2f0
>> [ 354.769131] [<ffffffff8196f381>] __netif_receive_skb+0x11/0x70
>> [ 354.775481] [<ffffffff8196f3fe>]
>> netif_receive_skb_internal+0x1e/0x80
>> [ 354.781638] [<ffffffff8199e1af>] ? nf_hook_slow+0x5f/0xb0
>> [ 354.787771] [<ffffffff8196f469>] netif_receive_skb+0x9/0x10
>> [ 354.793916] [<ffffffff81a7a1a8>]
>> br_handle_frame_finish+0x178/0x4b0
>> [ 354.800077] [<ffffffff81a5ec07>] ? nf_nat_ipv4_fn+0x167/0x1e0
>> [ 354.806260] [<ffffffff81a7a020>] ?
>> br_handle_local_finish+0x50/0x50
>> [ 354.812405] [<ffffffff81a85193>]
>> br_nf_pre_routing_finish+0x183/0x360
>> [ 354.818574] [<ffffffff81a7a030>] ? br_netif_receive_skb+0x10/0x10
>> [ 354.824775] [<ffffffff81a85707>] br_nf_pre_routing+0x2a7/0x380
>> [ 354.830780] [<ffffffff81a85010>] ? br_nf_forward_ip+0x3f0/0x3f0
>> [ 354.836567] [<ffffffff8199e138>] nf_iterate+0x58/0x70
>> [ 354.842281] [<ffffffff8199e1af>] nf_hook_slow+0x5f/0xb0
>> [ 354.847886] [<ffffffff81a7a682>] br_handle_frame+0x1a2/0x290
>> [ 354.853520] [<ffffffff81a7a030>] ? br_netif_receive_skb+0x10/0x10
>> [ 354.859206] [<ffffffff81a7a4e0>] ?
>> br_handle_frame_finish+0x4b0/0x4b0
>> [ 354.864824] [<ffffffff8196cbfb>]
>> __netif_receive_skb_core+0x12b/0x970
>> [ 354.870350] [<ffffffff810fe841>] ?
>> __raw_callee_save___pv_queued_spin_unlock+0x11/0x20
>> [ 354.875880] [<ffffffff8196f381>] __netif_receive_skb+0x11/0x70
>> [ 354.881293] [<ffffffff8196f3fe>]
>> netif_receive_skb_internal+0x1e/0x80
>> [ 354.886653] [<ffffffff8196f469>] netif_receive_skb+0x9/0x10
>> [ 354.891918] [<ffffffff8173c693>] xenvif_tx_action+0x693/0x820
>> [ 354.897170] [<ffffffff8173ebf9>] xenvif_poll+0x29/0x70
>> [ 354.902426] [<ffffffff819706e7>] net_rx_action+0x1f7/0x300
>> [ 354.907636] [<ffffffff810ccda3>] __do_softirq+0x103/0x210
>> [ 354.912837] [<ffffffff810cd0ab>] irq_exit+0x4b/0xa0
>> [ 354.917940] [<ffffffff814de7d0>] xen_evtchn_do_upcall+0x30/0x40
>> [ 354.923051] [<ffffffff81af173e>]
>> xen_do_hypervisor_callback+0x1e/0x40
>> [ 354.928089] <EOI>
>> [ 354.928175] [<ffffffff810013aa>] ? xen_hypercall_sched_op+0xa/0x20
>> [ 354.938047] [<ffffffff810013aa>] ? xen_hypercall_sched_op+0xa/0x20
>> [ 354.942985] [<ffffffff81009420>] ? xen_safe_halt+0x10/0x20
>> [ 354.947859] [<ffffffff810193c3>] ? default_idle+0x13/0x20
>> [ 354.952664] [<ffffffff810198fa>] ? arch_cpu_idle+0xa/0x10
>> [ 354.957470] [<ffffffff810fc25e>] ? default_idle_call+0x2e/0x50
>> [ 354.962291] [<ffffffff810fc4f2>] ? cpu_startup_entry+0x272/0x2e0
>> [ 354.967063] [<ffffffff81ae89c7>] ? rest_init+0x77/0x80
>> [ 354.971854] [<ffffffff82316f43>] ? start_kernel+0x438/0x445
>> [ 354.976640] [<ffffffff823164ef>] ?
>> x86_64_start_reservations+0x2a/0x2c
>> [ 354.981457] [<ffffffff82319fad>] ? xen_start_kernel+0x555/0x561
>> [ 354.986277] Code: 48 f7 42 58 fe ff ff ff b8 01 00 00 00 74 13 8b
>> 4f
>> 04 85 c9 74 0a 55 48 89 e5 e8 05 fa ff ff 5d f3 c3 f3 c3 66 83 79 10
>> 02
>> 75 d5 <80> b9 20 03 00 00 00 79 cc c3 66 66 66 66 66 66 2e 0f 1f 84 00
>> [ 354.996803] RIP [<ffffffff81a074a7>]
>> ip_vs_out.constprop.25+0x47/0x60
>> [ 355.002021] RSP <ffff88005f6034b0>
>> [ 355.007159] CR2: ffff880049aa8000
>> [ 355.012294] ---[ end trace 5b3b3b699aee4fc6 ]---
>> [ 355.017424] Kernel panic - not syncing: Fatal exception in
>> interrupt
>> [ 355.022732] Kernel Offset: disabled
>> (XEN) [2015-11-11 15:45:14.718] Hardware Dom0 crashed: rebooting
>> machine
>> in 5 seconds.
>>
>> (gdb) list *0xffffffff81a074a7
>> 0xffffffff81a074a7 is in ip_vs_out
>> (net/netfilter/ipvs/ip_vs_core.c:1192).
>> 1187 if (unlikely(skb->sk != NULL && hooknum == NF_INET_LOCAL_OUT &&
>> 1188 af == AF_INET)) {
>> 1189 struct sock *sk = skb->sk;
>> 1190 struct inet_sock *inet = inet_sk(skb->sk);
>> 1191
>> 1192 if (inet && sk->sk_family == PF_INET && inet->nodefrag)
>> 1193 return NF_ACCEPT;
>> 1194 }
>> 1195
>> 1196 if (unlikely(!skb_dst(skb)))
>>
>
> Thanks for the report, please try following patch :
Hi Eric,
Thanks for the patch!
Got it up and running at the moment, but since i don't have a clear
trigger it
will take 1 or 2 days before i can report something back.
--
Sander
> diff --git a/net/netfilter/ipvs/ip_vs_core.c
> b/net/netfilter/ipvs/ip_vs_core.c
> index 1e24fff53e4b..f57b4dcdb233 100644
> --- a/net/netfilter/ipvs/ip_vs_core.c
> +++ b/net/netfilter/ipvs/ip_vs_core.c
> @@ -1176,6 +1176,7 @@ ip_vs_out(struct netns_ipvs *ipvs, unsigned int
> hooknum, struct sk_buff *skb, in
> struct ip_vs_protocol *pp;
> struct ip_vs_proto_data *pd;
> struct ip_vs_conn *cp;
> + struct sock *sk;
>
> EnterFunction(11);
>
> @@ -1183,13 +1184,12 @@ ip_vs_out(struct netns_ipvs *ipvs, unsigned
> int hooknum, struct sk_buff *skb, in
> if (skb->ipvs_property)
> return NF_ACCEPT;
>
> + sk = skb_to_full_sk(skb);
> /* Bad... Do not break raw sockets */
> - if (unlikely(skb->sk != NULL && hooknum == NF_INET_LOCAL_OUT &&
> + if (unlikely(sk && hooknum == NF_INET_LOCAL_OUT &&
> af == AF_INET)) {
> - struct sock *sk = skb->sk;
> - struct inet_sock *inet = inet_sk(skb->sk);
>
> - if (inet && sk->sk_family == PF_INET && inet->nodefrag)
> + if (sk->sk_family == PF_INET && inet_sk(sk)->nodefrag)
> return NF_ACCEPT;
> }
>
> @@ -1681,6 +1681,7 @@ ip_vs_in(struct netns_ipvs *ipvs, unsigned int
> hooknum, struct sk_buff *skb, int
> struct ip_vs_conn *cp;
> int ret, pkts;
> int conn_reuse_mode;
> + struct sock *sk;
>
> /* Already marked as IPVS request or reply? */
> if (skb->ipvs_property)
> @@ -1708,12 +1709,11 @@ ip_vs_in(struct netns_ipvs *ipvs, unsigned int
> hooknum, struct sk_buff *skb, int
> ip_vs_fill_iph_skb(af, skb, false, &iph);
>
> /* Bad... Do not break raw sockets */
> - if (unlikely(skb->sk != NULL && hooknum == NF_INET_LOCAL_OUT &&
> + sk = skb_to_full_sk(skb);
> + if (unlikely(sk && hooknum == NF_INET_LOCAL_OUT &&
> af == AF_INET)) {
> - struct sock *sk = skb->sk;
> - struct inet_sock *inet = inet_sk(skb->sk);
>
> - if (inet && sk->sk_family == PF_INET && inet->nodefrag)
> + if (sk->sk_family == PF_INET && inet_sk(sk)->nodefrag)
> return NF_ACCEPT;
> }
--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Powered by blists - more mailing lists