lists.openwall.net   lists  /  announce  owl-users  owl-dev  john-users  john-dev  passwdqc-users  yescrypt  popa3d-users  /  oss-security  kernel-hardening  musl  sabotage  tlsify  passwords  /  crypt-dev  xvendor  /  Bugtraq  Full-Disclosure  linux-kernel  linux-netdev  linux-ext4  linux-hardening  linux-cve-announce  PHC 
Open Source and information security mailing list archives
 
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Date:	Wed, 30 Mar 2016 11:20:11 +0200
From:	Sedat Dilek <sedat.dilek@...il.com>
To:	Ingo Molnar <mingo@...nel.org>
Cc:	Peter Zijlstra <peterz@...radead.org>,
	Alfredo Alvarez Fernandez <alfredoalvarezfernandez@...il.com>,
	Linus Torvalds <torvalds@...ux-foundation.org>,
	"Theodore Ts'o" <tytso@....edu>,
	linux-fsdevel <linux-fsdevel@...r.kernel.org>,
	LKML <linux-kernel@...r.kernel.org>,
	Sasha Levin <sasha.levin@...cle.com>
Subject: Re: [Linux-v4.6-rc1] ext4: WARNING: CPU: 2 PID: 2692 at
 kernel/locking/lockdep.c:2017 __lock_acquire+0x180e/0x2260

On Tue, Mar 29, 2016 at 10:47 AM, Ingo Molnar <mingo@...nel.org> wrote:
>
> * Peter Zijlstra <peterz@...radead.org> wrote:
>
>> On Sun, Mar 27, 2016 at 05:03:44AM -0700, Linus Torvalds wrote:
>> > Anyway, I don't think that DEBUG_LOCKS_WARN_ON() in
>> >
>> >   kernel/locking/lockdep.c:2017 __lock_acquire
>> >
>> > would be an ext4 issue, it looks more like an internal lockdep issue.
>> >
>> > Adding in the lockdep people, who will set me right.
>>
>> You are right; this is lockdep running into a hash collision; which is a new
>> DEBUG_LOCKDEP test. See 9e4e7554e755 ("locking/lockdep: Detect chain_key
>> collisions").
>
> I've Cc:-ed Alfredo Alvarez Fernandez who added that test.
>
> Thanks,
>

[ CC Sasha (liblockdep maintainer) ]

I have added CONFIG_DEBUG_LOCKING_API_SELFTESTS=y to my 6th kernel-run.

$ egrep -i 'lockdep|locking' /boot/config-4.6.0-rc1-6-iniza-small
CONFIG_LOCKDEP_SUPPORT=y
CONFIG_FILE_LOCKING=y
CONFIG_MANDATORY_FILE_LOCKING=y
CONFIG_PROVE_LOCKING=y
CONFIG_LOCKDEP=y
CONFIG_DEBUG_LOCKDEP=y
CONFIG_DEBUG_LOCKING_API_SELFTESTS=y

[    0.000000] | Locking API testsuite:
[    0.000000] ----------------------------------------------------------------------------
[    0.000000]                                  | spin |wlock |rlock
|mutex | wsem | rsem |
[    0.000000]
--------------------------------------------------------------------------
[    0.000000]                      A-A deadlock:  ok  |  ok  |  ok  |
 ok  |  ok  |  ok  |
[    0.000000]                  A-B-B-A deadlock:  ok  |  ok  |  ok  |
 ok  |  ok  |  ok  |
[    0.000000]              A-B-B-C-C-A deadlock:  ok  |  ok  |  ok  |
 ok  |  ok  |  ok  |
[    0.000000]              A-B-C-A-B-C deadlock:  ok  |  ok  |  ok  |
 ok  |  ok  |  ok  |
[    0.000000]          A-B-B-C-C-D-D-A deadlock:  ok  |  ok  |  ok  |
 ok  |  ok  |  ok  |
[    0.000000]          A-B-C-D-B-D-D-A deadlock:  ok  |  ok  |  ok  |
 ok  |  ok  |  ok  |
[    0.000000]          A-B-C-D-B-C-D-A deadlock:  ok  |  ok  |  ok  |
 ok  |  ok  |  ok  |
[    0.000000]                     double unlock:  ok  |  ok  |  ok  |
 ok  |  ok  |  ok  |
[    0.000000]                   initialize held:  ok  |  ok  |  ok  |
 ok  |  ok  |  ok  |
[    0.000000]                  bad unlock order:  ok  |  ok  |  ok  |
 ok  |  ok  |  ok  |
[    0.000000]
--------------------------------------------------------------------------
[    0.000000]               recursive read-lock:             |  ok  |
            |  ok  |
[    0.000000]            recursive read-lock #2:             |  ok  |
            |  ok  |
[    0.000000]             mixed read-write-lock:             |  ok  |
            |  ok  |
[    0.000000]             mixed write-read-lock:             |  ok  |
            |  ok  |
[    0.000000]
--------------------------------------------------------------------------
[    0.000000]      hard-irqs-on + irq-safe-A/12:  ok  |  ok  |  ok  |
[    0.000000]      soft-irqs-on + irq-safe-A/12:  ok  |  ok  |  ok  |
[    0.000000]      hard-irqs-on + irq-safe-A/21:  ok  |  ok  |  ok  |
[    0.000000]      soft-irqs-on + irq-safe-A/21:  ok  |  ok  |  ok  |
[    0.000000]        sirq-safe-A => hirqs-on/12:  ok  |  ok  |  ok  |
[    0.000000]        sirq-safe-A => hirqs-on/21:  ok  |  ok  |  ok  |
[    0.000000]          hard-safe-A + irqs-on/12:  ok  |  ok  |  ok  |
[    0.000000]          soft-safe-A + irqs-on/12:  ok  |  ok  |  ok  |
[    0.000000]          hard-safe-A + irqs-on/21:  ok  |  ok  |  ok  |
[    0.000000]          soft-safe-A + irqs-on/21:  ok  |  ok  |  ok  |
[    0.000000]     hard-safe-A + unsafe-B #1/123:  ok  |  ok  |  ok  |
[    0.000000]     soft-safe-A + unsafe-B #1/123:  ok  |  ok  |  ok  |
[    0.000000]     hard-safe-A + unsafe-B #1/132:  ok  |  ok  |  ok  |
[    0.000000]     soft-safe-A + unsafe-B #1/132:  ok  |  ok  |  ok  |
[    0.000000]     hard-safe-A + unsafe-B #1/213:  ok  |  ok  |  ok  |
[    0.000000]     soft-safe-A + unsafe-B #1/213:  ok  |  ok  |  ok  |
[    0.000000]     hard-safe-A + unsafe-B #1/231:  ok  |  ok  |  ok  |
[    0.000000]     soft-safe-A + unsafe-B #1/231:  ok  |  ok  |  ok  |
[    0.000000]     hard-safe-A + unsafe-B #1/312:  ok  |  ok  |  ok  |
[    0.000000]     soft-safe-A + unsafe-B #1/312:  ok  |  ok  |  ok  |
[    0.000000]     hard-safe-A + unsafe-B #1/321:  ok  |  ok  |  ok  |
[    0.000000]     soft-safe-A + unsafe-B #1/321:  ok  |  ok  |  ok  |
[    0.000000]     hard-safe-A + unsafe-B #2/123:  ok  |  ok  |  ok  |
[    0.000000]     soft-safe-A + unsafe-B #2/123:  ok  |  ok  |  ok  |
[    0.000000]     hard-safe-A + unsafe-B #2/132:  ok  |  ok  |  ok  |
[    0.000000]     soft-safe-A + unsafe-B #2/132:  ok  |  ok  |  ok  |
[    0.000000]     hard-safe-A + unsafe-B #2/213:  ok  |  ok  |  ok  |
[    0.000000]     soft-safe-A + unsafe-B #2/213:  ok  |  ok  |  ok  |
[    0.000000]     hard-safe-A + unsafe-B #2/231:  ok  |  ok  |  ok  |
[    0.000000]     soft-safe-A + unsafe-B #2/231:  ok  |  ok  |  ok  |
[    0.000000]     hard-safe-A + unsafe-B #2/312:  ok  |  ok  |  ok  |
[    0.000000]     soft-safe-A + unsafe-B #2/312:  ok  |  ok  |  ok  |
[    0.000000]     hard-safe-A + unsafe-B #2/321:  ok  |  ok  |  ok  |
[    0.000000]     soft-safe-A + unsafe-B #2/321:  ok  |  ok  |  ok  |
[    0.000000]       hard-irq lock-inversion/123:  ok  |  ok  |  ok  |
[    0.000000]       soft-irq lock-inversion/123:  ok  |  ok  |  ok  |
[    0.000000]       hard-irq lock-inversion/132:  ok  |  ok  |  ok  |
[    0.000000]       soft-irq lock-inversion/132:  ok  |  ok  |  ok  |
[    0.000000]       hard-irq lock-inversion/213:  ok  |  ok  |  ok  |
[    0.000000]       soft-irq lock-inversion/213:  ok  |  ok  |  ok  |
[    0.000000]       hard-irq lock-inversion/231:  ok  |  ok  |  ok  |
[    0.000000]       soft-irq lock-inversion/231:  ok  |  ok  |  ok  |
[    0.000000]       hard-irq lock-inversion/312:  ok  |  ok  |  ok  |
[    0.000000]       soft-irq lock-inversion/312:  ok  |  ok  |  ok  |
[    0.000000]       hard-irq lock-inversion/321:  ok  |  ok  |  ok  |
[    0.000000]       soft-irq lock-inversion/321:  ok  |  ok  |  ok  |
[    0.000000]       hard-irq read-recursion/123:  ok  |
[    0.000000]       soft-irq read-recursion/123:  ok  |
[    0.000000]       hard-irq read-recursion/132:  ok  |
[    0.000000]       soft-irq read-recursion/132:  ok  |
[    0.000000]       hard-irq read-recursion/213:  ok  |
[    0.000000]       soft-irq read-recursion/213:  ok  |
[    0.000000]       hard-irq read-recursion/231:  ok  |
[    0.000000]       soft-irq read-recursion/231:  ok  |
[    0.000000]       hard-irq read-recursion/312:  ok  |
[    0.000000]       soft-irq read-recursion/312:  ok  |
[    0.000000]       hard-irq read-recursion/321:  ok  |
[    0.000000]       soft-irq read-recursion/321:  ok  |
[    0.000000]
--------------------------------------------------------------------------
[    0.000000]   | Wound/wait tests |
[    0.000000]   ---------------------
[    0.000000]                   ww api failures:  ok  |  ok  |  ok  |
[    0.000000]                ww contexts mixing:  ok  |  ok  |
[    0.000000]              finishing ww context:  ok  |  ok  |  ok  |  ok  |
[    0.000000]                locking mismatches:  ok  |  ok  |  ok  |
[    0.000000]                  EDEADLK handling:  ok  |  ok  |  ok  |
 ok  |  ok  |  ok  |  ok  |  ok  |  ok  |  ok  |
[    0.000000]            spinlock nest unlocked:  ok  |
[    0.000000]   -----------------------------------------------------
[    0.000000]                                  |block | try  |context|
[    0.000000]   -----------------------------------------------------
[    0.000000]                           context:  ok  |  ok  |  ok  |
[    0.000000]                               try:  ok  |  ok  |  ok  |
[    0.000000]                             block:  ok  |  ok  |  ok  |
[    0.000000]                          spinlock:  ok  |  ok  |  ok  |
[    0.000000] -------------------------------------------------------
[    0.000000] Good, all 253 testcases passed! |
[    0.000000] ---------------------------------

And it's all good?
( On shutdown I saw an(other) issue - will investigate, might not be related. )

With my patch "

liblockdep: Fix unsupported 'basename -s' in run_tests.sh" (see
<https://patchwork.kernel.org/patch/8693641/>),
tools/lib/lockdep/run_tests.sh runs fine, now.

$ cd linux/

$ LC_ALL=C make -C tools/ liblockdep
make: Entering directory `/home/wearefam/src/linux-kernel/linux/tools'
  DESCEND  lib/lockdep
make[1]: Entering directory
`/home/wearefam/src/linux-kernel/linux/tools/lib/lockdep'
  CC       fixdep.o
  LD       fixdep-in.o
  LINK     fixdep
  CC       common.o
  CC       lockdep.o
  CC       preload.o
  CC       rbtree.o
  LD       liblockdep-in.o
  LD       liblockdep.a
  LD       liblockdep.so.4.6.0-rc1
make[1]: Leaving directory
`/home/wearefam/src/linux-kernel/linux/tools/lib/lockdep'
make: Leaving directory `/home/wearefam/src/linux-kernel/linux/tools'

$ cd tools/lib/lockdep/

$ LC_ALL=C ./run_tests.sh
AA... PASSED!
ABA... PASSED!
ABBA... PASSED!
ABBA_2threads... PASSED!
ABBCCA... PASSED!
ABBCCDDA... PASSED!
ABCABC... PASSED!
ABCDBCDA... PASSED!
ABCDBDDA... PASSED!
WW... PASSED!
unlock_balance... PASSED!
(PRELOAD) AA... PASSED!
(PRELOAD) ABA... PASSED!
(PRELOAD) ABBA... PASSED!
(PRELOAD) ABBA_2threads... PASSED!
(PRELOAD) ABBCCA... PASSED!
(PRELOAD) ABBCCDDA... PASSED!
(PRELOAD) ABCABC... PASSED!
(PRELOAD) ABCDBCDA... PASSED!
(PRELOAD) ABCDBDDA... PASSED!
(PRELOAD) WW... PASSED!
(PRELOAD) unlock_balance... PASSED!

Thanks for all involved people (another interesting issue was caught
while dealing with this issue).

Linux-config and dmesg-log attached.

- Sedat -

View attachment "dmesg_4.6.0-rc1-6-iniza-small.txt" of type "text/plain" (63534 bytes)

Download attachment "config-4.6.0-rc1-6-iniza-small" of type "application/octet-stream" (134966 bytes)

Powered by blists - more mailing lists

Powered by Openwall GNU/*/Linux Powered by OpenVZ