lists.openwall.net   lists  /  announce  owl-users  owl-dev  john-users  john-dev  passwdqc-users  yescrypt  popa3d-users  /  oss-security  kernel-hardening  musl  sabotage  tlsify  passwords  /  crypt-dev  xvendor  /  Bugtraq  Full-Disclosure  linux-kernel  linux-netdev  linux-ext4  linux-hardening  linux-cve-announce  PHC 
Open Source and information security mailing list archives
 
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [day] [month] [year] [list]
Message-ID: <20220209014709.GA26885@xsang-OptiPlex-9020>
Date:   Wed, 9 Feb 2022 09:48:06 +0800
From:   kernel test robot <oliver.sang@...el.com>
To:     Sebastian Andrzej Siewior <bigeasy@...utronix.de>
Cc:     0day robot <lkp@...el.com>, LKML <linux-kernel@...r.kernel.org>,
        lkp@...ts.01.org, cgroups@...r.kernel.org, linux-mm@...ck.org,
        Andrew Morton <akpm@...ux-foundation.org>,
        Johannes Weiner <hannes@...xchg.org>,
        Michal Hocko <mhocko@...nel.org>,
        Michal Koutný <mkoutny@...e.com>,
        Peter Zijlstra <peterz@...radead.org>,
        Thomas Gleixner <tglx@...utronix.de>,
        Vladimir Davydov <vdavydov.dev@...il.com>,
        Waiman Long <longman@...hat.com>,
        Sebastian Andrzej Siewior <bigeasy@...utronix.de>
Subject: [mm/memcg]  86895e1e85:
 WARNING:possible_circular_locking_dependency_detected



Greeting,

FYI, we noticed the following commit (built with gcc-9):

commit: 86895e1e85b3a8220936304841b20d01a6239976 ("[PATCH 3/4] mm/memcg: Add a local_lock_t for IRQ and TASK object.")
url: https://github.com/0day-ci/linux/commits/Sebastian-Andrzej-Siewior/mm-memcg-Address-PREEMPT_RT-problems-instead-of-disabling-it/20220126-004438
patch link: https://lore.kernel.org/linux-mm/20220125164337.2071854-4-bigeasy@linutronix.de

in testcase: kernel-selftests
version: kernel-selftests-x86_64-f050cde9-1_20220127
with following parameters:

	group: cgroup
	ucode: 0xe2

test-description: The kernel contains a set of "self tests" under the tools/testing/selftests/ directory. These are intended to be small unit tests to exercise individual code paths in the kernel.
test-url: https://www.kernel.org/doc/Documentation/kselftest.txt


on test machine: 8 threads Intel(R) Core(TM) i7-6700 CPU @ 3.40GHz with 16G memory

caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace):



If you fix the issue, kindly add following tag
Reported-by: kernel test robot <oliver.sang@...el.com>


[  698.190282][ T6895] WARNING: possible circular locking dependency detected
[  698.197143][ T6895] 5.17.0-rc1-00003-g86895e1e85b3 #1 Not tainted
[  698.203222][ T6895] ------------------------------------------------------
[  698.210077][ T6895] with_stress.sh/6895 is trying to acquire lock:
[ 698.216247][ T6895] ffff888100b42428 (&mm->mmap_lock#2){++++}-{3:3}, at: mpol_rebind_mm (arch/x86/include/asm/jump_label.h:27 include/linux/jump_label.h:212 include/linux/mmap_lock.h:35 include/linux/mmap_lock.h:72 mm/mempolicy.c:384) 
[  698.225194][ T6895]
[  698.225194][ T6895] but task is already holding lock:
[ 698.232394][ T6895] ffffffff8569b1f0 (&cpuset_rwsem){++++}-{0:0}, at: cpuset_attach (kernel/cgroup/cpuset.c:2257) 
[  698.240991][ T6895]
[  698.240991][ T6895] which lock already depends on the new lock.
[  698.240991][ T6895]
[  698.251226][ T6895]
[  698.251226][ T6895] the existing dependency chain (in reverse order) is:
[  698.260076][ T6895]
[  698.260076][ T6895] -> #2 (&cpuset_rwsem){++++}-{0:0}:
[ 698.267369][ T6895] lock_acquire (kernel/locking/lockdep.c:438 kernel/locking/lockdep.c:5641 kernel/locking/lockdep.c:5604) 
[ 698.272241][ T6895] percpu_down_write (kernel/locking/percpu-rwsem.c:220) 
[ 698.277455][ T6895] cpuset_css_online (include/linux/instrumented.h:86 include/asm-generic/bitops/instrumented-atomic.h:28 kernel/cgroup/cpuset.c:2820) 
[ 698.282667][ T6895] online_css (kernel/cgroup/cgroup.c:5250) 
[ 698.287277][ T6895] cgroup_apply_control_enable (kernel/cgroup/cgroup.c:5321 kernel/cgroup/cgroup.c:3124) 
[ 698.293448][ T6895] cgroup_apply_control (kernel/cgroup/cgroup.c:3207) 
[ 698.298830][ T6895] cgroup_subtree_control_write (kernel/cgroup/cgroup.c:3232 kernel/cgroup/cgroup.c:3365) 
[ 698.305080][ T6895] cgroup_file_write (kernel/cgroup/cgroup.c:3855) 
[ 698.310378][ T6895] kernfs_fop_write_iter (fs/kernfs/file.c:300) 
[ 698.316020][ T6895] new_sync_write (fs/read_write.c:504 (discriminator 1)) 
[ 698.321059][ T6895] vfs_write (fs/read_write.c:590) 
[ 698.325667][ T6895] ksys_write (fs/read_write.c:643) 
[ 698.330276][ T6895] do_syscall_64 (arch/x86/entry/common.c:50 arch/x86/entry/common.c:80) 
[ 698.335065][ T6895] entry_SYSCALL_64_after_hwframe (arch/x86/entry/entry_64.S:113) 
[  698.341314][ T6895]
[  698.341314][ T6895] -> #1 (cpu_hotplug_lock){++++}-{0:0}:
[ 698.348869][ T6895] lock_acquire (kernel/locking/lockdep.c:438 kernel/locking/lockdep.c:5641 kernel/locking/lockdep.c:5604) 
[ 698.353733][ T6895] cpus_read_lock (arch/x86/include/asm/preempt.h:80 include/linux/percpu-rwsem.h:53 kernel/cpu.c:309) 
[ 698.358600][ T6895] drain_all_stock (mm/memcontrol.c:2207 mm/memcontrol.c:2193) 
[ 698.363641][ T6895] try_charge_memcg (mm/memcontrol.c:2563) 
[ 698.368941][ T6895] charge_memcg (mm/memcontrol.c:6766) 
[ 698.373724][ T6895] __mem_cgroup_charge (include/linux/cgroup.h:403 mm/memcontrol.c:6787) 
[ 698.379116][ T6895] do_anonymous_page (mm/memory.c:3783) 
[ 698.384503][ T6895] __handle_mm_fault (mm/memory.c:4568 mm/memory.c:4705) 
[ 698.389976][ T6895] handle_mm_fault (mm/memory.c:4803) 
[ 698.395102][ T6895] do_user_addr_fault (arch/x86/mm/fault.c:1397) 
[ 698.400486][ T6895] exc_page_fault (arch/x86/include/asm/irqflags.h:29 arch/x86/include/asm/irqflags.h:70 arch/x86/include/asm/irqflags.h:130 arch/x86/mm/fault.c:1492 arch/x86/mm/fault.c:1540) 
[ 698.405440][ T6895] asm_exc_page_fault (arch/x86/include/asm/idtentry.h:568) 
[  698.410651][ T6895]
[  698.410651][ T6895] -> #0 (&mm->mmap_lock#2){++++}-{3:3}:
[ 698.418210][ T6895] check_prev_add (kernel/locking/lockdep.c:3064) 
[ 698.423341][ T6895] __lock_acquire (kernel/locking/lockdep.c:3187 kernel/locking/lockdep.c:3801 kernel/locking/lockdep.c:5027) 
[ 698.428552][ T6895] lock_acquire (kernel/locking/lockdep.c:438 kernel/locking/lockdep.c:5641 kernel/locking/lockdep.c:5604) 
[ 698.433419][ T6895] down_write (include/linux/instrumented.h:101 include/linux/atomic/atomic-instrumented.h:1779 kernel/locking/rwsem.c:254 kernel/locking/rwsem.c:1258 kernel/locking/rwsem.c:1268 kernel/locking/rwsem.c:1515) 
[ 698.438033][ T6895] mpol_rebind_mm (arch/x86/include/asm/jump_label.h:27 include/linux/jump_label.h:212 include/linux/mmap_lock.h:35 include/linux/mmap_lock.h:72 mm/mempolicy.c:384) 
[ 698.442995][ T6895] cpuset_attach (include/linux/instrumented.h:71 include/asm-generic/bitops/instrumented-non-atomic.h:134 kernel/cgroup/cpuset.c:258 kernel/cgroup/cpuset.c:2293) 
[ 698.447947][ T6895] cgroup_migrate_execute (kernel/cgroup/cgroup.c:2513) 
[ 698.453764][ T6895] cgroup_update_dfl_csses (kernel/cgroup/cgroup.c:2975) 
[ 698.459586][ T6895] cgroup_subtree_control_write (kernel/cgroup/cgroup.c:3232 kernel/cgroup/cgroup.c:3365) 
[ 698.465835][ T6895] cgroup_file_write (kernel/cgroup/cgroup.c:3855) 
[ 698.471130][ T6895] kernfs_fop_write_iter (fs/kernfs/file.c:300) 
[ 698.476775][ T6895] new_sync_write (fs/read_write.c:504 (discriminator 1)) 
[ 698.481811][ T6895] vfs_write (fs/read_write.c:590) 
[ 698.486417][ T6895] ksys_write (fs/read_write.c:643) 
[ 698.491022][ T6895] do_syscall_64 (arch/x86/entry/common.c:50 arch/x86/entry/common.c:80) 
[ 698.495802][ T6895] entry_SYSCALL_64_after_hwframe (arch/x86/entry/entry_64.S:113) 
[  698.502050][ T6895]
[  698.502050][ T6895] other info that might help us debug this:
[  698.502050][ T6895]
[  698.512106][ T6895] Chain exists of:
[  698.512106][ T6895]   &mm->mmap_lock#2 --> cpu_hotplug_lock --> &cpuset_rwsem
[  698.512106][ T6895]
[  698.524934][ T6895]  Possible unsafe locking scenario:
[  698.524934][ T6895]
[  698.532226][ T6895]        CPU0                    CPU1
[  698.537436][ T6895]        ----                    ----
[  698.542648][ T6895]   lock(&cpuset_rwsem);
[  698.546735][ T6895]                                lock(cpu_hotplug_lock);
[  698.553593][ T6895]                                lock(&cpuset_rwsem);
[  698.560197][ T6895]   lock(&mm->mmap_lock#2);
[  698.564550][ T6895]
[  698.564550][ T6895]  *** DEADLOCK ***
[  698.564550][ T6895]
[  698.572532][ T6895] 5 locks held by with_stress.sh/6895:
[ 698.577839][ T6895] #0: ffff8881013f2448 (sb_writers#7){.+.+}-{0:0}, at: ksys_write (fs/read_write.c:643) 
[ 698.586519][ T6895] #1: ffff88817c726088 (&of->mutex){+.+.}-{3:3}, at: kernfs_fop_write_iter (fs/kernfs/file.c:288) 
[ 698.596064][ T6895] #2: ffffffff8568d788 (cgroup_mutex){+.+.}-{3:3}, at: cgroup_lock_and_drain_offline (kernel/cgroup/cgroup.c:3001) 
[ 698.606385][ T6895] #3: ffffffff8568d550 (cgroup_threadgroup_rwsem){++++}-{0:0}, at: cgroup_update_dfl_csses (include/linux/spinlock.h:374 (discriminator 9) kernel/cgroup/cgroup.c:2951 (discriminator 9)) 
[ 698.617312][ T6895] #4: ffffffff8569b1f0 (&cpuset_rwsem){++++}-{0:0}, at: cpuset_attach (kernel/cgroup/cpuset.c:2257) 
[  698.626335][ T6895]
[  698.626335][ T6895] stack backtrace:
[  698.632074][ T6895] CPU: 1 PID: 6895 Comm: with_stress.sh Not tainted 5.17.0-rc1-00003-g86895e1e85b3 #1
[  698.641447][ T6895] Hardware name: HP HP Z240 SFF Workstation/802E, BIOS N51 Ver. 01.63 10/05/2017
[  698.650384][ T6895] Call Trace:
[  698.653526][ T6895]  <TASK>
[ 698.656319][ T6895] dump_stack_lvl (lib/dump_stack.c:107) 
[ 698.660669][ T6895] check_noncircular (kernel/locking/lockdep.c:2143) 
[ 698.665449][ T6895] ? print_circular_bug+0x480/0x480 
[ 698.671103][ T6895] ? mark_lock+0xca/0x13c0 
[ 698.675976][ T6895] ? mark_lock_irq (kernel/locking/lockdep.c:4564) 
[ 698.680765][ T6895] check_prev_add (kernel/locking/lockdep.c:3064) 
[ 698.685379][ T6895] ? __lock_acquire (arch/x86/include/asm/bitops.h:214 include/asm-generic/bitops/instrumented-non-atomic.h:135 kernel/locking/lockdep.c:199 kernel/locking/lockdep.c:3713 kernel/locking/lockdep.c:3769 kernel/locking/lockdep.c:5027) 
[ 698.690247][ T6895] __lock_acquire (kernel/locking/lockdep.c:3187 kernel/locking/lockdep.c:3801 kernel/locking/lockdep.c:5027) 
[ 698.694949][ T6895] ? lockdep_hardirqs_on_prepare (kernel/locking/lockdep.c:4885) 
[ 698.700772][ T6895] ? lock_is_held_type (kernel/locking/lockdep.c:5380 kernel/locking/lockdep.c:5682) 
[ 698.705645][ T6895] ? rcu_read_lock_sched_held (include/linux/lockdep.h:283 kernel/rcu/update.c:125) 
[ 698.711128][ T6895] lock_acquire (kernel/locking/lockdep.c:438 kernel/locking/lockdep.c:5641 kernel/locking/lockdep.c:5604) 
[ 698.715481][ T6895] ? mpol_rebind_mm (arch/x86/include/asm/jump_label.h:27 include/linux/jump_label.h:212 include/linux/mmap_lock.h:35 include/linux/mmap_lock.h:72 mm/mempolicy.c:384) 
[ 698.720096][ T6895] ? rcu_read_unlock (include/linux/rcupdate.h:723 (discriminator 5)) 
[ 698.724709][ T6895] ? get_task_mm (kernel/fork.c:1311) 
[ 698.728972][ T6895] ? rcu_read_lock_sched_held (include/linux/lockdep.h:283 kernel/rcu/update.c:125) 
[ 698.734445][ T6895] ? rcu_read_lock_bh_held (kernel/rcu/update.c:120) 
[ 698.739578][ T6895] ? lock_acquire (kernel/locking/lockdep.c:438 kernel/locking/lockdep.c:5641 kernel/locking/lockdep.c:5604) 
[ 698.744098][ T6895] ? lock_is_held_type (kernel/locking/lockdep.c:5380 kernel/locking/lockdep.c:5682) 
[ 698.748968][ T6895] down_write (include/linux/instrumented.h:101 include/linux/atomic/atomic-instrumented.h:1779 kernel/locking/rwsem.c:254 kernel/locking/rwsem.c:1258 kernel/locking/rwsem.c:1268 kernel/locking/rwsem.c:1515) 
[ 698.753063][ T6895] ? mpol_rebind_mm (arch/x86/include/asm/jump_label.h:27 include/linux/jump_label.h:212 include/linux/mmap_lock.h:35 include/linux/mmap_lock.h:72 mm/mempolicy.c:384) 
[ 698.757668][ T6895] ? down_write_killable (kernel/locking/rwsem.c:1512) 
[ 698.762797][ T6895] ? rwlock_bug+0xc0/0xc0 
[ 698.767581][ T6895] mpol_rebind_mm (arch/x86/include/asm/jump_label.h:27 include/linux/jump_label.h:212 include/linux/mmap_lock.h:35 include/linux/mmap_lock.h:72 mm/mempolicy.c:384) 
[ 698.772023][ T6895] cpuset_attach (include/linux/instrumented.h:71 include/asm-generic/bitops/instrumented-non-atomic.h:134 kernel/cgroup/cpuset.c:258 kernel/cgroup/cpuset.c:2293) 
[ 698.776463][ T6895] ? guarantee_online_cpus (kernel/cgroup/cpuset.c:2243) 
[ 698.781763][ T6895] ? lockdep_hardirqs_on_prepare (kernel/locking/lockdep.c:4232 kernel/locking/lockdep.c:4292 kernel/locking/lockdep.c:4244) 
[ 698.787587][ T6895] cgroup_migrate_execute (kernel/cgroup/cgroup.c:2513) 
[ 698.792895][ T6895] ? _raw_spin_unlock_irq (arch/x86/include/asm/irqflags.h:45 arch/x86/include/asm/irqflags.h:80 include/linux/spinlock_api_smp.h:159 kernel/locking/spinlock.c:202) 
[ 698.797942][ T6895] cgroup_update_dfl_csses (kernel/cgroup/cgroup.c:2975) 
[ 698.803249][ T6895] ? cgroup_apply_control_enable (kernel/cgroup/cgroup.c:2939) 
[ 698.809075][ T6895] ? css_next_descendant_pre (kernel/cgroup/cgroup.c:4394 kernel/cgroup/cgroup.c:4375) 
[ 698.814468][ T6895] cgroup_subtree_control_write (kernel/cgroup/cgroup.c:3232 kernel/cgroup/cgroup.c:3365) 
[ 698.820203][ T6895] ? cgroup_max_descendants_write (kernel/cgroup/cgroup.c:3283) 
[ 698.826116][ T6895] cgroup_file_write (kernel/cgroup/cgroup.c:3855) 
[ 698.830903][ T6895] ? css_release_work_fn (kernel/cgroup/cgroup.c:3855) 
[ 698.836028][ T6895] ? __kasan_slab_alloc (mm/kasan/common.c:431 mm/kasan/common.c:469) 
[ 698.840896][ T6895] ? lock_is_held_type (kernel/locking/lockdep.c:5380 kernel/locking/lockdep.c:5682) 
[ 698.845762][ T6895] ? css_release_work_fn (kernel/cgroup/cgroup.c:3855) 
[ 698.850886][ T6895] kernfs_fop_write_iter (fs/kernfs/file.c:300) 
[ 698.856015][ T6895] new_sync_write (fs/read_write.c:504 (discriminator 1)) 
[ 698.860544][ T6895] ? new_sync_read (fs/read_write.c:493) 
[ 698.865151][ T6895] ? ksys_write (fs/read_write.c:643) 
[ 698.869413][ T6895] ? rcu_read_unlock (include/linux/rcupdate.h:723 (discriminator 5)) 
[ 698.874027][ T6895] ? lock_is_held_type (kernel/locking/lockdep.c:5380 kernel/locking/lockdep.c:5682) 
[ 698.878895][ T6895] ? rcu_read_lock_held (kernel/rcu/update.c:341) 
[ 698.883772][ T6895] vfs_write (fs/read_write.c:590) 
[ 698.887867][ T6895] ksys_write (fs/read_write.c:643) 
[ 698.891963][ T6895] ? __ia32_sys_read (fs/read_write.c:633) 
[ 698.896570][ T6895] ? lockdep_hardirqs_on_prepare (kernel/locking/lockdep.c:4232 kernel/locking/lockdep.c:4292 kernel/locking/lockdep.c:4244) 
[ 698.902386][ T6895] ? syscall_enter_from_user_mode (arch/x86/include/asm/irqflags.h:45 arch/x86/include/asm/irqflags.h:80 kernel/entry/common.c:107) 
[ 698.908117][ T6895] do_syscall_64 (arch/x86/entry/common.c:50 arch/x86/entry/common.c:80) 
[ 698.912384][ T6895] ? lockdep_hardirqs_on_prepare (kernel/locking/lockdep.c:4232 kernel/locking/lockdep.c:4292 kernel/locking/lockdep.c:4244) 
[ 698.918203][ T6895] entry_SYSCALL_64_after_hwframe (arch/x86/entry/entry_64.S:113) 
[  698.923935][ T6895] RIP: 0033:0x7fe2d6264504
[ 698.928205][ T6895] Code: 00 f7 d8 64 89 02 48 c7 c0 ff ff ff ff eb b3 0f 1f 80 00 00 00 00 48 8d 05 f9 61 0d 00 8b 00 85 c0 75 13 b8 01 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 54 c3 0f 1f 00 41 54 49 89 d4 55 48 89 f5 53
All code
========
   0:	00 f7                	add    %dh,%bh
   2:	d8 64 89 02          	fsubs  0x2(%rcx,%rcx,4)
   6:	48 c7 c0 ff ff ff ff 	mov    $0xffffffffffffffff,%rax


To reproduce:

        git clone https://github.com/intel/lkp-tests.git
        cd lkp-tests
        sudo bin/lkp install job.yaml           # job file is attached in this email
        bin/lkp split-job --compatible job.yaml # generate the yaml file for lkp run
        sudo bin/lkp run generated-yaml-file

        # if come across any failure that blocks the test,
        # please remove ~/.lkp and /lkp dir to run from a clean state.



---
0DAY/LKP+ Test Infrastructure                   Open Source Technology Center
https://lists.01.org/hyperkitty/list/lkp@lists.01.org       Intel Corporation

Thanks,
Oliver Sang


View attachment "config-5.17.0-rc1-00003-g86895e1e85b3" of type "text/plain" (178547 bytes)

View attachment "job-script" of type "text/plain" (6389 bytes)

Download attachment "dmesg.xz" of type "application/x-xz" (30604 bytes)

View attachment "kernel-selftests" of type "text/plain" (39054 bytes)

View attachment "job.yaml" of type "text/plain" (5452 bytes)

View attachment "reproduce" of type "text/plain" (152 bytes)

Powered by blists - more mailing lists

Powered by Openwall GNU/*/Linux Powered by OpenVZ