linux-kernel - Re: [BUG][BISECT] NFSv4 root failures after "fs/locks: allow a lock request to block other requests."

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <87o9e2anwg.fsf@notabene.neil.brown.name>
Date:   Thu, 16 Aug 2018 15:19:59 +1000
From:   NeilBrown <neilb@...e.com>
To:     Jeff Layton <jlayton@...nel.org>,
        Krzysztof Kozlowski <krzk@...nel.org>
Cc:     Alexander Viro <viro@...iv.linux.org.uk>,
        "J. Bruce Fields" <bfields@...ldses.org>,
        linux-fsdevel@...r.kernel.org, linux-kernel@...r.kernel.org,
        "linux-samsung-soc\@vger.kernel.org" 
        <linux-samsung-soc@...r.kernel.org>
Subject: Re: [BUG][BISECT] NFSv4 root failures after "fs/locks: allow a lock request to block other requests."

On Wed, Aug 15 2018, Jeff Layton wrote:

> On Wed, 2018-08-15 at 14:28 +0200, Krzysztof Kozlowski wrote:
>> Hi,
>> 
>> Bisect pointed commit ce3147990450a68b3f549088b30f087742a08b5d
>> ("fs/locks: allow a lock request to block other requests.") to failure
>> boot of NFSv4 with root on several boards.
>> 
>> Log is here:
>> https://krzk.eu/#/builders/21/builds/836/steps/12/logs/serial0
>> 
>> With several errors:
>> kernel BUG at ../fs/locks.c:336!
>> Unable to handle kernel NULL pointer dereference at virtual address 00000004
>> 
>> Configuration:
>> 1. exynos_defconfig
>> 2. Arch ARM Linux
>> 3. Boards:
>> a. Odroid family (ARMv7, octa-core (Cortex-A7+A15), Exynos5422 SoC)
>> b. Toradex Colibri VF50 (ARMv7, UP, Cortex-A5)
>> 4. Systemd: v236, 238
>> 5. All boards boot from TFTP with NFS root (NFSv4)
>> 
>> On Colibri VF50 I got slightly different errors:
>> [   11.663204] Internal error: Oops - undefined instruction: 0 [#1] ARM
>> [   12.455273] Unable to handle kernel NULL pointer dereference at
>> virtual address 00000004
>> and only with some specific GCC (v6.3) or with other conditions which
>> I did not bisect yet. Maybe Colibri's failure is unrelated to that
>> commit.
>> 
>> Best regards,
>> Krzysztof

Thanks a lot for the report Krzysztof!!

>
> The BUG is due to a lock being freed when the fl_blocked list wasn't
> empty (implying that there were still blocked locks waiting on it).
>
> There are a number of calls to locks_delete_lock_ctx in posix_lock_inode
> and I don't think the fl_blocked list is being handled properly with all
> of them. It only transplants the blocked locks to a new lock when there
> are surviving locks on the list, and that may not be the case when the
> whole file is being unlocked.

locks_delete_lock_ctx() calls locks_unlink_lock_ctx() which calls
locks_wake_up_block() which doesn't only wake_up the blocks, but also
detached them. When that function completes, ->fl_blocked must be empty.

The trace shows the locks_free_lock() call at the end of fcntl_setlk64()
as the problematic call.
This suggests that do_lock_file_wait() exited with ->fl_blocked
non-empty, which it shouldn't.

I think we need to insert a call to locks_wake_up_block() in
do_lock_file_wait() before it returns.
I cannot find a sequence that would make this necessary, but
it isn't surprising that there might be one.

I'll dig through the code a bit more later and make sure I understand
what is happening.

Thanks,
NeilBrown

Download attachment "signature.asc" of type "application/pgp-signature" (833 bytes)