Message-ID: <HE1P193MB01233D583E9A7B1418A77713A8859@HE1P193MB0123.EURP193.PROD.OUTLOOK.COM>
Date:   Sat, 9 Jul 2022 06:14:14 +0000
From:   Michelle Bies <mimbies@...look.com>
To:     "netdev@...r.kernel.org" <netdev@...r.kernel.org>,
        Eric Dumazet <edumazet@...gle.com>
Subject: Re: TPROXY + Attempt to release TCP socket in state 1

Hi Eric
unfortunately, nobody response to my problem :(
Did I report my problem to the right mailing list? 


On 05 July 2022 09:52, Michelle Bies wrote:
> Hi
>
> I'm trying to run squid with TPROXY enabled but the system reboots after logging these messages:
> 
>   IPv4: Attempt to release TCP socket in state 1 00000000dfe7f997
> 
> and after some seconds:
> 
>   rcu: INFO: rcu_sched detected stalls on CPUs/tasks:
>   rcu:5-....: (6 GPs behind) idle=f9e/0/0x1 softirq=184030/184030 fqs=1901
>   (detected by 1, t=6302 jiffies, g=894549, q=17504)
>   Sending NMI from CPU 1 to CPUs 5:
>   NMI backtrace for cpu 5
>   CPU: 5 PID: 0 Comm: swapper/5 Tainted: GO 5.4.181+ #9
>   Hardware name: Dell Inc. PowerEdge R630/02C2CP, BIOS 2.12.1 12/04/2020
>   RIP: 0010:__inet_lookup_established+0x4c/0xf7
>   Code: 48 89 f5 53 48 c1 e5 20 44 89 cb 48 09 c5 44 09 c3 e8 7b fe ff ff 41 89 c6 41 89 c5 49 8b 07 45 23 77 10 4e 8d 3c f0 49 8b 17 <f6> c2 01 0f 85 89 00 00 00 44 39 6a a0 75 7b 39 5a a4 75 76 48 39
>   RSP: 0018:ffffc90000208c48 EFLAGS: 00000a12
>   RAX: ffff88844ad00000 RBX: 0000000001bb18e1 RCX: 00000000baf65c60
>   RDX: 000000000002b145 RSI: 00000000e5cc7389 RDI: 000000009da543b1
>   RBP: 490e1a683c63a8c0 R08: 00000000000018e1 R09: 0000000001bb0000
>   R10: ffff8883c96da400 R11: 00000000000018e1 R12: ffffffff822b3740
>   R13: 00000000593d39c6 R14: 00000000000139c6 R15: ffff88844ad9ce30
>   FS:0000000000000000(0000) GS:ffff88844d940000(0000) knlGS:0000000000000000
>   CS:0010 DS: 0000 ES: 0000 CR0: 0000000080050033
>   CR2: 00007fd0297e8eb0 CR3: 00000003d37d4005 CR4: 00000000003606e0
>   DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
>   DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
>   Call Trace:
>  <IRQ>
>  nf_sk_lookup_slow_v4+0x224/0xfbe [nf_socket_ipv4]
>  ? do_raw_spin_lock+0x2b/0x52
>  socket_match.isra.0+0x2f/0xf9 [xt_socket]
>  ipt_do_table+0x26f/0x5c1 [ip_tables]
>  ? nf_ct_key_equal+0x38/0x5d [nf_conntrack]
>  ? nf_conntrack_in+0x2bd/0x46b [nf_conntrack]
>  nf_hook_slow+0x3c/0xb4
>  nf_hook.constprop.0+0xa5/0xc8
>  ? l3mdev_l3_rcv.constprop.0+0x50/0x50
>  ip_rcv+0x41/0x61
>  __netif_receive_skb_one_core+0x74/0x95
>  process_backlog+0x97/0x122
>  net_rx_action+0xf5/0x2a3
>  __do_softirq+0xc2/0x1c6
>  irq_exit+0x41/0x80
>  call_function_single_interrupt+0xf/0x20
>  </IRQ>
>   RIP: 0010:mwait_idle+0x5f/0x75
>   Code: f0 31 d2 48 89 d1 65 48 8b 04 25 40 ac 01 00 0f 01 c8 48 8b 08 48 c1 e9 03 83 e1 01 75 0e e8 2b c3 6c ff 48 89 c8 fb 0f 01 c9 <eb> 01 fb bf 15 00 00 00     65 48 8b 34 25 40 ac 01 00 e9 26 c3 6c ff
>   RSP: 0018:ffffc900000f3ee0 EFLAGS: 00000246 ORIG_RAX: ffffffffffffff04
>   RAX: 0000000000000000 RBX: ffff88844beee3c0 RCX: 0000000000000000
>   RDX: 0000000000000000 RSI: ffff88844beee3c0 RDI: 0000000000000015
>   RBP: 0000000000000000 R08: 0000000000840188 R09: 0000000000000000
>   R10: ffff88844d964b80 R11: ffff88844d964bc0 R12: 0000000000000000
>   R13: 0000000000000005 R14: 0000000000000000 R15: 0000000000000000
>  do_idle+0xcf/0x1da
>  ? do_idle+0x2/0x1da
>  cpu_startup_entry+0x1a/0x1c
>  start_secondary+0x14b/0x169
>  secondary_startup_64+0xa4/0xb0
>   ------------[ cut here ]------------
>   NETDEV WATCHDOG: eth2 (igb): transmit queue 4 timed out
>   WARNING: CPU: 1 PID: 0 at net/sched/sch_generic.c:480 dev_watchdog+0xcf/0x128
>   Modules linked in: xt_CLASSIFY xt_nfacct sch_sfq xt_IMQ xt_NFLOG xt_limit xt_pkttype xt_nat xt_MASQUERADE xt_REDIRECT xt_connlimit nf_conncount xt_time xt_geoip(O) xt_iprange xt_NFQUEUE xt_TPROXY nf_tproxy_ipv6 nf_tproxy_ipv4 xt_mac xt_mark 8021q garp mrp xt_multiport xt_socket nf_socket_ipv4 nf_socket_ipv6 ebtable_filter ebtable_nat ebtables xt_state xt_conntrack iptable_filter iptable_nat xt_set xt_connlabel xt_connmark iptable_mangle xt_recent iptable_raw sch_htb ip_set_hash_ipportip ip_set_hash_ip nfnetlink_acct nf_nat_pptp nf_conntrack_pptp nf_nat_irc nf_conntrack_irc nf_nat_tftp nf_conntrack_tftp nf_nat_ftp nf_conntrack_ftp nf_nat_h323 nf_conntrack_h323 nf_nat_sip nf_conntrack_sip nf_nat nfnetlink_log ip_set nfnetlink_queue nf_conntrack_netlink tun nfnetlink intel_lpss_pci intel_lpss imq igb sch_fq_codel nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 bonding llc e1000e e1000 ip_tables
>   CPU: 1 PID: 0 Comm: swapper/1 Tainted: G O5.4.181+ #9
>   Hardware name: Dell Inc. PowerEdge R630/02C2CP, BIOS 2.12.1 12/04/2020
>   RIP: 0010:dev_watchdog+0xcf/0x128
>   Code: 4b a1 00 00 75 38 48 89 ef c6 05 39 4b a1 00 01 e8 2e 7b fd ff 44 89 e1 48 89 ee 48 c7 c7 fb e6 0d 82 48 89 c2 e8 a0 98 0d 00 <0f> 0b eb 10 41 ff c4 48 05 40 01 00 00 41 39 f4 75 9d eb 13 48 8b
>   RSP: 0018:ffffc90000158ec0 EFLAGS: 00010282
>   RAX: 0000000000000000 RBX: ffff88844b234440 RCX: 0000000000000007
>   RDX: 00000000000003f2 RSI: ffffc90000158db4 RDI: ffff88844d85b5b0
>   RBP: ffff88844b234000 R08: 0000000000000001 R09: 0000000000014600
>   R10: 0000000000000000 R11: 000000000000005c R12: 0000000000000004
>   R13: ffffc90000158ef8 R14: ffffffff822050c0 R15: 0000000000000002
>   FS:0000000000000000(0000) GS:ffff88844d840000(0000) knlGS:0000000000000000
>   CS:0010 DS: 0000 ES: 0000 CR0: 0000000080050033
>   CR2: 00007faad50f2000 CR3: 0000000437d0e005 CR4: 00000000003606e0
>   DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
>   DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
>   Call Trace:
>  <IRQ>
>  call_timer_fn.isra.0+0x18/0x6f
>  ? netif_tx_lock+0x7a/0x7a
>  __run_timers.part.0+0x12d/0x163
>  ? hrtimer_forward+0x73/0x7b
>  ? tick_sched_timer+0x57/0x62
>  ? timerqueue_add+0x62/0x68
>  run_timer_softirq+0x21/0x43
>  __do_softirq+0xc2/0x1c6
>  irq_exit+0x41/0x80
>  smp_apic_timer_interrupt+0x6f/0x7a
>  apic_timer_interrupt+0xf/0x20
>  </IRQ>
>   RIP: 0010:mwait_idle+0x5f/0x75
>   Code: f0 31 d2 48 89 d1 65 48 8b 04 25 40 ac 01 00 0f 01 c8 48 8b 08 48 c1 e9 03 83 e1 01 75 0e e8 2b c3 6c ff 48 89 c8 fb 0f 01 c9 <eb> 01 fb bf 15 00 00 00 65 48 8b 34 25 40 ac 01 00 e9 26 c3 6c ff
>   RSP: 0018:ffffc900000d3ee0 EFLAGS: 00000246 ORIG_RAX: ffffffffffffff13
>   RAX: 0000000000000000 RBX: ffff88844beeaac0 RCX: 0000000000000000
>   RDX: 0000000000000000 RSI: ffff88844beeaac0 RDI: 0000000000000015
>   RBP: 0000000000000000 R08: 0000000000000001 R09: 0000000000000000
>   R10: ffff88844d864b80 R11: ffff88844d864bc0 R12: 0000000000000000
>   R13: 0000000000000001 R14: 0000000000000000 R15: 0000000000000000
>  do_idle+0xcf/0x1da
>  cpu_startup_entry+0x1a/0x1c
>  start_secondary+0x14b/0x169
>  secondary_startup_64+0xa4/0xb0
>   ---[ end trace 9e50b2e05e0ee06d ]---
> 
> My current kernel is 5.4 and these are my iptables config:
> 
>  iptables -t mangle -A PREROUTING -p tcp -m multiport --sport 80 -m socket -m conntrack --ctdir REPLY -j DIVERT
>  iptables -t mangle -A DIVERT -j MARK --set-mark 1
>  iptables -t mangle -A DIVERT -j ACCEPT
>  iptables -t mangle -A PREROUTING -p tcp -m multiport --dports 80 -j TPROXY --tproxy-mark 1 --on-port 3129
> 

Powered by blists - more mailing lists