linux-kernel - Re: dm-crypt with no_read_workqueue and no_write

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <f3b3c90e-90e6-9228-f2e5-172997eebf85@maciej.szmigiero.name>
Date:   Thu, 24 Dec 2020 19:52:58 +0100
From:   "Maciej S. Szmigiero" <mail@...iej.szmigiero.name>
To:     Ignat Korchagin <ignat@...udflare.com>
Cc:     Alasdair G Kergon <agk@...hat.com>,
        Mike Snitzer <snitzer@...hat.com>,
        device-mapper development <dm-devel@...hat.com>,
        dm-crypt@...ut.de, linux-kernel <linux-kernel@...r.kernel.org>,
        Eric Biggers <ebiggers@...nel.org>,
        Damien Le Moal <Damien.LeMoal@....com>,
        Mikulas Patocka <mpatocka@...hat.com>,
        kernel-team <kernel-team@...udflare.com>,
        Nobuto Murata <nobuto.murata@...onical.com>,
        Chris Mason <clm@...com>, Josef Bacik <josef@...icpanda.com>,
        David Sterba <dsterba@...e.com>, linux-btrfs@...r.kernel.org,
        linux-crypto <linux-crypto@...r.kernel.org>,
        Herbert Xu <herbert@...dor.apana.org.au>
Subject: Re: dm-crypt with no_read_workqueue and no_write_workqueue + btrfs
 scrub = BUG()

On 24.12.2020 19:46, Ignat Korchagin wrote:
> On Wed, Dec 23, 2020 at 8:57 PM Herbert Xu <herbert@...dor.apana.org.au> wrote:
>>
>> On Wed, Dec 23, 2020 at 04:37:34PM +0100, Maciej S. Szmigiero wrote:
>>>
>>> It looks like to me that the skcipher API might not be safe to
>>> call from a softirq context, after all.
>>
>> skcipher is safe to use in a softirq.  The problem is only in
>> dm-crypt where it tries to allocate memory with GFP_NOIO.
> 
> Hm.. After eliminating the GFP_NOIO (as well as some other sleeping
> paths) from dm-crypt softirq code I still hit an occasional crash in
> my extreme setup (QEMU with 1 CPU and cryptd_max_cpu_qlen set to 1)
> (decoded with stacktrace_decode.sh):
(..)
> This happens when running dm-crypt with no_read_workqueues on top of
> an emulated NVME in QEMU (NVME driver "completes" IO in IRQ context).
> Somehow sending decryption requests to cryptd in some fashion in
> softirq context corrupts the crypto queue it seems.

You can try compiling your test kernel with KASAN, as it often
immediately points out where the memory starts to get corrupted
(if that's the bug).

Enabling other "checking" kernel debug options might help debugging
the root case of this, too.

> Regards,
> Ignat

Thanks,
Maciej