linux-kernel - Spinlock debugging in the kernel

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [day] [month] [year] [list]

Message-ID: <CAF7cDAc8DTG1drUGYMPPVkM5CjjFqZGFodzqVOk4hmM1Bc9nnA@mail.gmail.com>
Date:   Mon, 30 Nov 2020 16:13:41 -0800
From:   Krishna M <krishnamurthi.rusty@...il.com>
To:     linux-kernel@...r.kernel.org
Subject: Spinlock debugging in the kernel

Please excuse me if this is a wrong group.
Please find the log of a system that I am trying to debug.

It is a "soft lockup" issue and the root cause seems to be coming from
the mm unit of the kernel.
I am relatively new to kernel and arm64 platforms.

How do you typically decipher a locking issue like this? Any ideas/
pointers that I can run with will greatly help.

[  512.522056] Rehashing will be triggered for ipset fqdn:pol_5_to of
elements 362 ahash_max 4 nsize 4
[  907.867755] BUG: soft lockup - CPU#0 stuck for 22s! [kswapd0:395]
[  919.861747] BUG: soft lockup - CPU#0 stuck for 33s! [kswapd0:395]
[  919.861752] Modules linked in: nfsv3 nfs lockd grace sunrpc
ath9k(O) ath9k_common(O) ath9k_hw(O) ath(O) mac80211(O) cfg80211(O)
compat(O) gproxy(O) bwdriver(O)            rndis_host cdc_ether usbnet
mii GobiSerial(O) cdc_acm option usb_wwan clstrio(O) clb(PO) kxp(PO)
ip6t_rpfilter ip6t_REJECT        nf_log_ipv6 nf_reject_ipv6 ip6t_frag
ip6table_mangle ip6table_filter ip6table_raw ip6t_ipv6header
nf_conntrack_ipv6 nf_defrag_ipv6 ifb act_mirred em_meta cls_basic
cls_flow cls_u32 sch_htb sch_prio sch_sfq arpt_ARPPROXY arpt_REPLY
arpt_mangle arptable_filter arp_tables xt_REDIRECT xt_nat ipt_REJECT
xt_recent xt_NETMAP ipt_MASQUERADE            nf_log_ipv4
nf_log_common xt_LOG xt_iprange ipt_ah xt_addrtype xt_TRACE xt_TCPMSS
xt_tcpmss xt_state xt_rateest xt_RATEEST xt_pkttype xt_physdev
[  919.862293]  xt_multiport xt_mark xt_mac xt_limit(O) xt_length
xt_ipv4options(O) xt_helper xt_hashlimit xt_esp xt_DSCP xt_dscp xt_CT
xt_conntrack xt_connmark             xt_connbytes xt_comment
xt_CLASSIFY nf_conntrack_netlink iptable_raw nf_nat_snmp_basic
nf_conntrack_snmp nf_conntrack_broadcast nf_nat_tftp nf_conntrack_tftp
nf_nat_pptp    nf_nat_proto_gre nf_conntrack_pptp
nf_conntrack_proto_gre nf_nat_irc nf_conntrack_irc nf_nat_ftp
nf_conntrack_ftp iptable_filter iptable_nat iptable_mangle
nf_reject_ipv4   nf_nat_redirect nf_nat_masquerade_ipv4 nf_nat_ipv4
nf_nat ip_tables nf_conntrack_ipv4 nf_defrag_ipv4 nf_conntrack
compat_xtables(O) ip_set_hash_netportnet(O)
ip_set_hash_netnet(O) ip_set_hash_netiface(O) ip_set_hash_mac(O)
ip_set_hash_ipmark(O) ip_set_list_set(O) ip_set_hash_netport(O)
ip_set_hash_net(O) ip_set_hash_ipportip(O)
[  919.862508]  ip_set_hash_ipportnet(O) ip_set_hash_ipport(O)
ip_set_hash_ip(O) ip_set_bitmap_port(O) ip_set_bitmap_ipmac(O)
ip_set_bitmap_ip(O) xt_set(O) ip_set(O)        nfnetlink
sha512_generic dummy bonding ip6_vti ip6_tunnel tunnel6 ebt_ip6
ebt_arpreply 8021q garp ip_vti ip_gre ip_tunnel gre ebt_mark_m
ebt_mark ebt_redirect ebt_dnat      ebt_ip ebt_arp ebt_snat ebt_vlan
ebt_log ebt_fpath(O) ebtable_broute ebtable_filter ebtable_nat
ebtables br_netfilter bridge stp llc alarm_panic(PO) alarm(PO)
l2tp_ppp l2tp_netlink l2tp_core ip6_udp_tunnel udp_tunnel tun pppoe
pppox ppp_deflate ppp_mppe ppp_async ppp_generic crc_ccitt slhc
usb_storage caam_pkc caamalg caamhash caamalg_desc caamhash_desc
caamrng caam_jr error(P) caam [last unloaded: nf_synproxy_core]
[  919.862794] CPU: 0 PID: 395 Comm: kswapd0 Tainted: P           O
4.14.83 #1
[  919.862802] task: 5a1b0000 task.stack: 5a0ba000
[  919.862808] NIP:  406f7f04 LR: 40163650 CTR: 40163d8c
[  919.862815] REGS: 5a0bbad0 TRAP: 0901   Tainted: P           O     (4.14.83)
[  919.862818] MSR:  00029000 <CE,EE,ME>  CR: 44002444  XER: 00000000
[  919.862843]  GPR00: 40163e08 5a0bbb80 5a1b0000 5aa16698 55f80540
1000f000 00000000 00000000  GPR08: 00000001 00000001 00000001 00000409
24004442
[  919.862915] NIP [406f7f04] _raw_spin_lock+0x30/0x5c
[  919.862934] LR [40163650] page_vma_mapped_walk+0x144/0x1e8

Thanks,
Krishna M