| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Message-ID: <20181116184800.GY16768@wantstofly.org>
Date: Fri, 16 Nov 2018 20:48:00 +0200
From: Lennert Buytenhek <buytenh@...tstofly.org>
To: steffen.klassert@...unet.com, herbert@...dor.apana.org.au
Cc: Jean-Philippe Menil <jpmenil@...il.com>, davem@...emloft.net,
netdev@...r.kernel.org, kuznet@....inr.ac.ru,
yoshfuji@...ux-ipv6.org
Subject: Re: [BUG] xfrm: unable to handle kernel NULL pointer dereference
On Sat, Nov 10, 2018 at 08:34:34PM +0100, Jean-Philippe Menil wrote:
> we're seeing unexpected crashes from kernel 4.15 to 4.18.17, using
> IPsec VTI interfaces, on several vpn hosts, since upgrade from 4.4.
I looked into this with Jean-Philippe, and it appears to be crashing
on a NULL pointer dereference in the inlined xfrm_policy_check() call
in vti_rcv_cb(), and specifically on the skb_dst(skb) dereference in
__xfrm_policy_check2():
return (!net->xfrm.policy_count[dir] && !skb->sp) ||
(skb_dst(skb)->flags & DST_NOPOLICY) || <=====
__xfrm_policy_check(sk, ndir, skb, family);
Commit 9e1437937807 ("xfrm: Fix NULL pointer dereference when
skb_dst_force clears the dst_entry.") fixes a very similar problem on
the output and forward paths, but our issue seems to be triggering on
the input path.
This hack patch seems to make the crashes go away, and the printk added
triggers with approximately the same regularity as the crashes used
to occur, so the fix from 9e1437937807 probably needs to be extended
to the input path somewhat like this.
Thanks!
diff --git a/net/xfrm/xfrm_input.c b/net/xfrm/xfrm_input.c
index 352abca2605f..c666e29441b4 100644
--- a/net/xfrm/xfrm_input.c
+++ b/net/xfrm/xfrm_input.c
@@ -381,6 +381,12 @@ int xfrm_input(struct sk_buff *skb, int nexthdr, __be32 spi, int encap_type)
XFRM_SKB_CB(skb)->seq.input.hi = seq_hi;
skb_dst_force(skb);
+ if (!skb_dst(skb)) {
+ if (net_ratelimit())
+ printk(KERN_CRIT "OH CRAP\n");
+ goto drop;
+ }
+
dev_hold(skb->dev);
if (crypto_done)
> Attached, the offended oops against 4.18.
>
> Output of decodedecode:
>
> [ 37.134864] Code: 8b 44 24 70 0f c8 89 87 b4 00 00 00 48 8b 86 20 05 00 00
> 8b 80 f8 14 00 00 85 c0 75 05 48 85 d2 74 0e 48 8b 43 58 48 83 e0 fe <f6> 40
> 38 04 74 7d 44 89 b3 b4 00 00 00 49 8b 44 24 20 48 39 86 20
> All code
> ========
> 0: 8b 44 24 70 mov 0x70(%rsp),%eax
> 4: 0f c8 bswap %eax
> 6: 89 87 b4 00 00 00 mov %eax,0xb4(%rdi)
> c: 48 8b 86 20 05 00 00 mov 0x520(%rsi),%rax
> 13: 8b 80 f8 14 00 00 mov 0x14f8(%rax),%eax
> 19: 85 c0 test %eax,%eax
> 1b: 75 05 jne 0x22
> 1d: 48 85 d2 test %rdx,%rdx
> 20: 74 0e je 0x30
> 22: 48 8b 43 58 mov 0x58(%rbx),%rax
> 26: 48 83 e0 fe and $0xfffffffffffffffe,%rax
> 2a:* f6 40 38 04 testb $0x4,0x38(%rax) <-- trapping
> instruction
> 2e: 74 7d je 0xad
> 30: 44 89 b3 b4 00 00 00 mov %r14d,0xb4(%rbx)
> 37: 49 8b 44 24 20 mov 0x20(%r12),%rax
> 3c: 48 rex.W
> 3d: 39 .byte 0x39
> 3e: 86 20 xchg %ah,(%rax)
>
> Code starting with the faulting instruction
> ===========================================
> 0: f6 40 38 04 testb $0x4,0x38(%rax)
> 4: 74 7d je 0x83
> 6: 44 89 b3 b4 00 00 00 mov %r14d,0xb4(%rbx)
> d: 49 8b 44 24 20 mov 0x20(%r12),%rax
> 12: 48 rex.W
> 13: 39 .byte 0x39
> 14: 86 20 xchg %ah,(%rax)
>
>
> if my understanding is correct, we fail here:
>
> /build/linux-hwe-edge-yHKLQJ/linux-hwe-edge-4.18.0/include/net/xfrm.h:
> 1169 return (!net->xfrm.policy_count[dir] && !skb->sp) ||
> 0x0000000000000b19 <+185>: testb $0x4,0x38(%rax)
> 0x0000000000000b1d <+189>: je 0xb9c <vti_rcv_cb+316>
>
> (gdb) list *0x0000000000000b19
> 0xb19 is in vti_rcv_cb
> (/build/linux-hwe-edge-yHKLQJ/linux-hwe-edge-4.18.0/include/net/xfrm.h:1169).
> 1164 int ndir = dir | (reverse ? XFRM_POLICY_MASK + 1 : 0);
> 1165
> 1166 if (sk && sk->sk_policy[XFRM_POLICY_IN])
> 1167 return __xfrm_policy_check(sk, ndir, skb, family);
> 1168
> 1169 return (!net->xfrm.policy_count[dir] && !skb->sp) ||
> 1170 (skb_dst(skb)->flags & DST_NOPOLICY) ||
> 1171 __xfrm_policy_check(sk, ndir, skb, family);
> 1172 }
> 1173
>
> I really have hard time to understand why skb seem to be freed twice.
>
> I'm not able to repeat the bug in lab, but it happened regulary in prod,
> seem to depend of the workload.
>
> Any help will be appreciated.
>
> Let me know if you need further informations.
>
> Regards,
>
> Jean-Philippe
> [ 31.154360] BUG: unable to handle kernel NULL pointer dereference at 0000000000000038
> [ 31.162233] PGD 0 P4D 0
> [ 31.164786] Oops: 0000 [#1] SMP PTI
> [ 31.168291] CPU: 5 PID: 42 Comm: ksoftirqd/5 Not tainted 4.18.0-11-generic #12~18.04.1-Ubuntu
> [ 31.176854] Hardware name: Supermicro Super Server/X10SDV-4C-7TP4F, BIOS 1.0b 11/21/2016
> [ 31.184980] RIP: 0010:vti_rcv_cb+0xb9/0x1a0 [ip_vti]
> [ 31.189962] Code: 8b 44 24 70 0f c8 89 87 b4 00 00 00 48 8b 86 20 05 00 00 8b 80 f8 14 00 00 85 c0 75 05 48 85 d2 74 0e 48 8b 43 58 48 83 e0 fe <f6> 40 38 04 74 7d 44 89 b3 b4 00 00 00 49 8b 44 24 20 48 39 86 20
> [ 31.208916] RSP: 0018:ffffbc61832e3920 EFLAGS: 00010246
> [ 31.214160] RAX: 0000000000000000 RBX: ffff9a3504964a00 RCX: 0000000000000002
> [ 31.221328] RDX: ffff9a351add4080 RSI: ffff9a351aa08000 RDI: ffff9a3504964a00
> [ 31.228485] RBP: ffffbc61832e3940 R08: 0000000000000004 R09: ffffffffc0aa612b
> [ 31.235643] R10: 0008f09b99881884 R11: 1884bd4e2d6b1fac R12: ffff9a3507b31900
> [ 31.242803] R13: ffff9a3507b31000 R14: 0000000000000000 R15: ffff9a3504964a00
> [ 31.249964] FS: 0000000000000000(0000) GS:ffff9a35bfd40000(0000) knlGS:0000000000000000
> [ 31.258077] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [ 31.263848] CR2: 0000000000000038 CR3: 000000041a40a003 CR4: 00000000003606e0
> [ 31.271004] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 31.278163] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
> [ 31.285320] Call Trace:
> [ 31.287789] xfrm4_rcv_cb+0x4a/0x70
> [ 31.291297] xfrm_input+0x58f/0x8f0
> [ 31.294807] vti_input+0xaa/0x110 [ip_vti]
> [ 31.298926] vti_rcv+0x33/0x3c [ip_vti]
> [ 31.302783] xfrm4_esp_rcv+0x39/0x50
> [ 31.306375] ip_local_deliver_finish+0x62/0x200
> [ 31.310923] ip_local_deliver+0xdf/0xf0
> [ 31.314775] ? ip_rcv_finish+0x420/0x420
> [ 31.318718] ip_rcv_finish+0x126/0x420
> [ 31.322486] ip_rcv+0x28f/0x360
> [ 31.325655] ? inet_del_offload+0x40/0x40
> [ 31.329686] __netif_receive_skb_core+0x48c/0xb70
> [ 31.334413] ? kmem_cache_alloc+0xb4/0x1d0
> [ 31.338532] ? __build_skb+0x2b/0xf0
> [ 31.342128] __netif_receive_skb+0x18/0x60
> [ 31.346244] ? __netif_receive_skb+0x18/0x60
> [ 31.350536] netif_receive_skb_internal+0x45/0xe0
> [ 31.355263] napi_gro_receive+0xc5/0xf0
> [ 31.359141] mlx5e_handle_rx_cqe+0x1b2/0x5d0 [mlx5_core]
> [ 31.364476] ? skb_release_all+0x24/0x30
> [ 31.368430] mlx5e_poll_rx_cq+0xd3/0x990 [mlx5_core]
> [ 31.373432] mlx5e_napi_poll+0x9b/0xc60 [mlx5_core]
> [ 31.378333] ? __switch_to_asm+0x34/0x70
> [ 31.382270] ? __switch_to_asm+0x40/0x70
> [ 31.386214] ? __switch_to_asm+0x34/0x70
> [ 31.391056] ? __switch_to_asm+0x40/0x70
> [ 31.395905] ? __switch_to_asm+0x34/0x70
> [ 31.400743] net_rx_action+0x140/0x3a0
> [ 31.405379] ? __switch_to+0xad/0x500
> [ 31.409887] __do_softirq+0xe4/0x2bb
> [ 31.414448] run_ksoftirqd+0x2b/0x40
> [ 31.418862] smpboot_thread_fn+0xfc/0x170
> [ 31.423700] kthread+0x121/0x140
> [ 31.427701] ? sort_range+0x30/0x30
> [ 31.432040] ? kthread_create_worker_on_cpu+0x70/0x70
> [ 31.437816] ret_from_fork+0x35/0x40
> [ 31.442219] Modules linked in: esp6 authenc echainiv xfrm6_mode_tunnel xfrm4_mode_tunnel xfrm_user xfrm4_tunnel tunnel4 ipcomp xfrm_ipcomp esp4 ah4 af_key xfrm_algo ip_vti ip_tunnel ip6_vti ip6_tunnel tunnel6 8021q garp mrp stp llc bonding ipt_REJECT nf_reject_ipv4 nfnetlink_log n
> fnetlink xt_NFLOG xt_hl xt_limit xt_nat xt_TCPMSS xt_HL xt_comment xt_tcpudp xt_multiport xt_conntrack iptable_filter iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat xt_connmark xt_mark iptable_mangle xt_CT nf_conntrack xt_addrtype iptable_raw bpfilter ipmi_ssif gpio_
> ich intel_rapl sb_edac x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel kvm irqbypass intel_cstate intel_rapl_perf input_leds joydev mei_me intel_pch_thermal ioatdma mei lpc_ich ipmi_si ipmi_devintf ipmi_msghandler acpi_pad mac_hid sch_fq_codel
> [ 31.519488] ib_iser rdma_cm iw_cm ib_cm iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi ip_tables x_tables autofs4 btrfs zstd_compress raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c raid0 multipath linear mlx5_ib ib_uverbs ib
> _core raid1 hid_generic usbhid hid crct10dif_pclmul crc32_pclmul ghash_clmulni_intel ast pcbc ttm drm_kms_helper aesni_intel syscopyarea aes_x86_64 sysfillrect mxm_wmi crypto_simd sysimgblt cryptd glue_helper fb_sys_fops mlx5_core ixgbe igb mpt3sas drm ahci tls libahci i2c_algo_bit m
> lxfw raid_class dca devlink mdio scsi_transport_sas wmi
> [ 31.578877] CR2: 0000000000000038
> [ 31.583249] ---[ end trace c4bada38847a0075 ]---
> [ 31.737166] RIP: 0010:vti_rcv_cb+0xb9/0x1a0 [ip_vti]
> [ 31.737167] Code: 8b 44 24 70 0f c8 89 87 b4 00 00 00 48 8b 86 20 05 00 00 8b 80 f8 14 00 00 85 c0 75 05 48 85 d2 74 0e 48 8b 43 58 48 83 e0 fe <f6> 40 38 04 74 7d 44 89 b3 b4 00 00 00 49 8b 44 24 20 48 39 86 20
> [ 31.737209] RSP: 0018:ffffbc61832e3920 EFLAGS: 00010246
> [ 31.737212] RAX: 0000000000000000 RBX: ffff9a3504964a00 RCX: 0000000000000002
> [ 31.737213] RDX: ffff9a351add4080 RSI: ffff9a351aa08000 RDI: ffff9a3504964a00
> [ 31.737216] RBP: ffffbc61832e3940 R08: 0000000000000004 R09: ffffffffc0aa612b
> [ 31.737219] R10: 0008f09b99881884 R11: 1884bd4e2d6b1fac R12: ffff9a3507b31900
> [ 31.737220] R13: ffff9a3507b31000 R14: 0000000000000000 R15: ffff9a3504964a00
> [ 31.737222] FS: 0000000000000000(0000) GS:ffff9a35bfd40000(0000) knlGS:0000000000000000
> [ 31.737224] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [ 31.737225] CR2: 0000000000000038 CR3: 000000041a40a003 CR4: 00000000003606e0
> [ 31.737227] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 31.737228] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
> [ 31.737230] Kernel panic - not syncing: Fatal exception in interrupt
> [ 31.737264] Kernel Offset: 0x3c00000 from 0xffffffff81000000 (relocation range: 0xffffffff80000000-0xffffffffbfffffff)
> [ 36.558279] ---[ end Kernel panic - not syncing: Fatal exception in interrupt ]---
> [ 36.566769] ------------[ cut here ]------------
> [ 36.572306] sched: Unexpected reschedule of offline CPU#3!
> [ 36.578714] WARNING: CPU: 5 PID: 42 at /build/linux-hwe-edge-yHKLQJ/linux-hwe-edge-4.18.0/arch/x86/kernel/smp.c:128 native_smp_send_resched
> ule+0x3a/0x40
> [ 36.594262] Modules linked in: esp6 authenc echainiv xfrm6_mode_tunnel xfrm4_mode_tunnel xfrm_user xfrm4_tunnel tunnel4 ipcomp xfrm_ipcomp
> esp4 ah4 af_key xfrm_algo ip_vti ip_tunnel ip6_vti ip6_tunnel tunnel6 8021q garp mrp stp llc bonding ipt_REJECT nf_reject_ipv4 nfnetlink_log n
> fnetlink xt_NFLOG xt_hl xt_limit xt_nat xt_TCPMSS xt_HL xt_comment xt_tcpudp xt_multiport xt_conntrack iptable_filter iptable_nat nf_conntrack
> _ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat xt_connmark xt_mark iptable_mangle xt_CT nf_conntrack xt_addrtype iptable_raw bpfilter ipmi_ssif gpio_
> ich intel_rapl sb_edac x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel kvm irqbypass intel_cstate intel_rapl_perf input_leds joydev m
> ei_me intel_pch_thermal ioatdma mei lpc_ich ipmi_si ipmi_devintf ipmi_msghandler acpi_pad mac_hid sch_fq_codel
> [ 36.673307] ib_iser rdma_cm iw_cm ib_cm iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi ip_tables x_tables autofs4 btrfs zstd_compres
> s raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c raid0 multipath linear mlx5_ib ib_uverbs ib
> _core raid1 hid_generic usbhid hid crct10dif_pclmul crc32_pclmul ghash_clmulni_intel ast pcbc ttm drm_kms_helper aesni_intel syscopyarea aes_x
> 86_64 sysfillrect mxm_wmi crypto_simd sysimgblt cryptd glue_helper fb_sys_fops mlx5_core ixgbe igb mpt3sas drm ahci tls libahci i2c_algo_bit m
> lxfw raid_class dca devlink mdio scsi_transport_sas wmi
> [ 36.733827] CPU: 5 PID: 42 Comm: ksoftirqd/5 Tainted: G D 4.18.0-11-generic #12~18.04.1-Ubuntu
> [ 36.745027] Hardware name: Supermicro Super Server/X10SDV-4C-7TP4F, BIOS 1.0b 11/21/2016
> [ 36.754392] RIP: 0010:native_smp_send_reschedule+0x3a/0x40
> [ 36.761143] Code: c6 62 01 73 17 48 8b 05 24 d2 17 01 be fd 00 00 00 48 8b 40 30 e8 96 96 ba 00 5d c3 89 fe 48 c7 c7 90 2a cd 85 e8 26 4a 0
> 3 00 <0f> 0b 5d c3 66 90 0f 1f 44 00 00 55 48 89 e5 53 48 83 ec 20 65 48
> [ 36.782546] RSP: 0018:ffff9a35bfd43b68 EFLAGS: 00010082
> [ 36.789019] RAX: 0000000000000000 RBX: 0000000000000003 RCX: 0000000000000006
> [ 36.797379] RDX: 0000000000000007 RSI: 0000000000000092 RDI: ffff9a35bfd564b0
> [ 36.805721] RBP: ffff9a35bfd43b68 R08: 00000000000004b7 R09: 0000000000cdcdcd
> [ 36.814046] R10: 0000000000000324 R11: 00000000ffffffff R12: ffff9a35bfce2c40
> [ 36.822350] R13: ffff9a3598e35c00 R14: 0000000000000008 R15: 0000000000000003
> [ 36.830637] FS: 0000000000000000(0000) GS:ffff9a35bfd40000(0000) knlGS:0000000000000000
> [ 36.839869] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [ 36.846742] CR2: 0000000000000038 CR3: 000000041a40a003 CR4: 00000000003606e0
> [ 36.855009] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 36.863275] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
> [ 36.871510] Call Trace:
> [ 36.875048] <IRQ>
> [ 36.878136] resched_curr+0x5d/0xc0
> [ 36.882689] check_preempt_wakeup+0x130/0x240
> [ 36.888116] check_preempt_curr+0x2d/0x90
> [ 36.893184] ttwu_do_wakeup+0x1e/0x140
> [ 36.897984] ttwu_do_activate+0x77/0x80
> [ 36.902857] try_to_wake_up+0x1d6/0x450
> [ 36.907726] ? __netif_receive_skb_core+0x48c/0xb70
> [ 36.913625] default_wake_function+0x12/0x20
> [ 36.918914] __wake_up_common+0x73/0x130
> [ 36.923851] __wake_up_locked+0x16/0x20
> [ 36.928706] ep_poll_callback+0xcb/0x2b0
> [ 36.933634] __wake_up_common+0x73/0x130
> [ 36.938549] __wake_up_common_lock+0x80/0xc0
> [ 36.943787] ? tick_sched_do_timer+0x60/0x60
> [ 36.949011] __wake_up+0x13/0x20
> [ 36.953190] wake_up_klogd_work_func+0x40/0x60
> [ 36.958560] irq_work_run_list+0x52/0x80
> [ 36.963388] irq_work_tick+0x3b/0x50
> [ 36.967841] update_process_times+0x42/0x60
> [ 36.972883] tick_sched_handle+0x25/0x70
> [ 36.977644] tick_sched_timer+0x3c/0x80
> [ 36.982285] __hrtimer_run_queues+0x10f/0x280
> [ 36.987429] hrtimer_interrupt+0xe7/0x240
> [ 36.992202] ? rcu_irq_exit+0x1d/0x20
> [ 36.996602] smp_apic_timer_interrupt+0x6f/0x130
> [ 37.001929] apic_timer_interrupt+0xf/0x20
> [ 37.006712] </IRQ>
> [ 37.009476] RIP: 0010:panic+0x1fe/0x244
> [ 37.013951] Code: eb a6 83 3d 37 05 8f 01 00 74 05 e8 d0 73 02 00 48 c7 c6 20 f1 57 86 48 c7 c7 c8 cd cd 85 e8 f0 78 06 00 fb 66 0f 1f 44 0
> 0 00 <31> db e8 2f c6 0d 00 4c 39 eb 7c 1d 41 83 f4 01 48 8b 05 df 04 8f
> [ 37.034119] RSP: 0018:ffffbc61832e3668 EFLAGS: 00000286 ORIG_RAX: ffffffffffffff13
> [ 37.042324] RAX: 0000000000000046 RBX: 0000000000000000 RCX: 0000000000000006
> [ 37.050086] RDX: 0000000000000000 RSI: 0000000000000096 RDI: ffff9a35bfd564b0
> [ 37.057835] RBP: ffffbc61832e36e0 R08: 00000000000004b5 R09: 0000000000cdcdcd
> [ 37.065570] R10: 0000000000000000 R11: 00000000ffffffff R12: 0000000000000000
> [ 37.006712] </IRQ>
> [ 37.009476] RIP: 0010:panic+0x1fe/0x244
> [ 37.013951] Code: eb a6 83 3d 37 05 8f 01 00 74 05 e8 d0 73 02 00 48 c7 c6 20 f1 57 86 48 c7 c7 c8 cd cd 85 e8 f0 78 06 00 fb 66 0f 1f 44 00 00 <31> db e8 2f c6 0d 00 4c 39 eb 7c 1d 41 83 f4 01 48 8b 05 df 04 8f
> [ 37.034119] RSP: 0018:ffffbc61832e3668 EFLAGS: 00000286 ORIG_RAX: ffffffffffffff13
> [ 37.042324] RAX: 0000000000000046 RBX: 0000000000000000 RCX: 0000000000000006
> [ 37.050086] RDX: 0000000000000000 RSI: 0000000000000096 RDI: ffff9a35bfd564b0
> [ 37.057835] RBP: ffffbc61832e36e0 R08: 00000000000004b5 R09: 0000000000cdcdcd
> [ 37.065570] R10: 0000000000000000 R11: 00000000ffffffff R12: 0000000000000000
> [ 37.073287] R13: 0000000000000000 R14: 0000000000000009 R15: 0000000000000000
> [ 37.080999] ? panic+0x1f7/0x244
> [ 37.084812] oops_end+0xce/0xe0
> [ 37.088531] no_context+0x17c/0x400
> [ 37.092585] ? skcipher_walk_first+0x4c/0x110
> [ 37.097500] __bad_area_nosemaphore+0x115/0x1d0
> [ 37.102591] ? cbc_decrypt+0xb1/0xe0 [aesni_intel]
> [ 37.107926] bad_area_nosemaphore+0x14/0x20
> [ 37.112654] __do_page_fault+0xd4/0x4d0
> [ 37.117030] do_page_fault+0x2d/0xf0
> [ 37.121143] ? skb_copy_bits+0x61/0x260
> [ 37.125511] page_fault+0x1e/0x30
> [ 37.129362] RIP: 0010:vti_rcv_cb+0xb9/0x1a0 [ip_vti]
> [ 37.134864] Code: 8b 44 24 70 0f c8 89 87 b4 00 00 00 48 8b 86 20 05 00 00 8b 80 f8 14 00 00 85 c0 75 05 48 85 d2 74 0e 48 8b 43 58 48 83 e0 fe <f6> 40 38 04 74 7d 44 89 b3 b4 00 00 00 49 8b 44 24 20 48 39 86 20
> [ 37.154892] RSP: 0018:ffffbc61832e3920 EFLAGS: 00010246
> [ 37.160711] RAX: 0000000000000000 RBX: ffff9a3504964a00 RCX: 0000000000000002
> [ 37.168451] RDX: ffff9a351add4080 RSI: ffff9a351aa08000 RDI: ffff9a3504964a00
> [ 37.176196] RBP: ffffbc61832e3940 R08: 0000000000000004 R09: ffffffffc0aa612b
> [ 37.183950] R10: 0008f09b99881884 R11: 1884bd4e2d6b1fac R12: ffff9a3507b31900
> [ 37.191716] R13: ffff9a3507b31000 R14: 0000000000000000 R15: ffff9a3504964a00
> [ 37.199481] ? esp_input_done2+0x6b/0x320 [esp4]
> [ 37.204740] xfrm4_rcv_cb+0x4a/0x70
> [ 37.208873] xfrm_input+0x58f/0x8f0
> [ 37.212996] vti_input+0xaa/0x110 [ip_vti]
> [ 37.217735] vti_rcv+0x33/0x3c [ip_vti]
> [ 37.222209] xfrm4_esp_rcv+0x39/0x50
> [ 37.226424] ip_local_deliver_finish+0x62/0x200
> [ 37.231599] ip_local_deliver+0xdf/0xf0
> [ 37.236072] ? ip_rcv_finish+0x420/0x420
> [ 37.240635] ip_rcv_finish+0x126/0x420
> [ 37.245025] ip_rcv+0x28f/0x360
> [ 37.248801] ? inet_del_offload+0x40/0x40
> [ 37.253457] __netif_receive_skb_core+0x48c/0xb70
> [ 37.258807] ? kmem_cache_alloc+0xb4/0x1d0
> [ 37.263541] ? __build_skb+0x2b/0xf0
> [ 37.267749] __netif_receive_skb+0x18/0x60
> [ 37.272483] ? __netif_receive_skb+0x18/0x60
> [ 37.277391] netif_receive_skb_internal+0x45/0xe0
> [ 37.282732] napi_gro_receive+0xc5/0xf0
> [ 37.287232] mlx5e_handle_rx_cqe+0x1b2/0x5d0 [mlx5_core]
> [ 37.293170] ? skb_release_all+0x24/0x30
> [ 37.297708] mlx5e_poll_rx_cq+0xd3/0x990 [mlx5_core]
> [ 37.303281] mlx5e_napi_poll+0x9b/0xc60 [mlx5_core]
> [ 37.308752] ? __switch_to_asm+0x34/0x70
> [ 37.313266] ? __switch_to_asm+0x40/0x70
> [ 37.317768] ? __switch_to_asm+0x34/0x70
> [ 37.322259] ? __switch_to_asm+0x40/0x70
> [ 37.326742] ? __switch_to_asm+0x34/0x70
> [ 37.331214] net_rx_action+0x140/0x3a0
> [ 37.335519] ? __switch_to+0xad/0x500
> [ 37.339728] __do_softirq+0xe4/0x2bb
> [ 37.343852] run_ksoftirqd+0x2b/0x40
> [ 37.347971] smpboot_thread_fn+0xfc/0x170
> [ 37.352522] kthread+0x121/0x140
> [ 37.356282] ? sort_range+0x30/0x30
> [ 37.360303] ? kthread_create_worker_on_cpu+0x70/0x70
> [ 37.365895] ret_from_fork+0x35/0x40
> [ 37.370010] ---[ end trace c4bada38847a0076 ]---
Powered by blists - more mailing lists