linux-kernel - Re: [PATCH v2] workqueue: Don't record workqueue stack holding raw_spin

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <YTW/emWbsNX7OkOh@elver.google.com>
Date:   Mon, 6 Sep 2021 09:12:58 +0200
From:   Marco Elver <elver@...gle.com>
To:     Shuah Khan <skhan@...uxfoundation.org>
Cc:     tj@...nel.org, jiangshanlai@...il.com, akpm@...ux-foundation.org,
        andreyknvl@...il.com, dvyukov@...gle.com,
        walter-zh.wu@...iatek.com, linux-kernel@...r.kernel.org,
        Sebastian Andrzej Siewior <bigeasy@...utronix.de>
Subject: Re: [PATCH v2] workqueue: Don't record workqueue stack holding
 raw_spin_lock

On Thu, Sep 02, 2021 at 05:46PM -0600, Shuah Khan wrote:
[...]
> > 3. Try to not allocate memory in stackdepot. Not sure this is feasible
> > without telling stackdepot to preallocate the max slabs on boot if RT.
> > 
> 
> We could. I have to ask though how much of the real world cases do we
> need to impact for the debug code to work?
> 
> > Anything else? Because I don't think any of the options are satisfying.
> 
> One option to consider is checking dry-run invalid nesting check and
> bail out if it is true in kasan_record_aux_stack()

Sadly, if lockdep is off, this won't work. And we need a way to
generically fix this, as otherwise we still have a bug (which may also
cause issues on RT kernels).

I propose we properly fix this and prevent stackdepot from replenishing
its "stack slab" pool if memory allocations cannot be done in the
current context. Specifically, I noticed technically it's a bug to use
either GFP_ATOMIC nor GFP_NOWAIT in certain non-preemptive contexts,
including raw_spin_locks (see gfp.h and ab00db216c9c7).

This is what kasan_record_aux_stack() via stackdepot does, and it's a
bug here regardless if lockdep is on or off.

I've prepared a series (see attached draft patches) that allows telling
stackdepot to not replenish its pool if alloc_pages() cannot be called
at all (where GFP_ATOMIC/NOWAIT doesn't even work).

The only downside is that saving a stack trace may fail if: stackdepot
runs out of space AND the same stack trace has not been recorded before.
I expect this to be unlikely, and a simple experiment (boot the kernel)
didn't result in any failure to record stack trace from insert_work().

I think this is a reasonable trade-off. And considering that we're
unsure if queuing work can or cannot be done from within an outer
raw_spin_lock'ed critical section, I don't see a better way.

If you agree, I'll send this series out for further review.

Thanks,
-- Marco

View attachment "0001-lib-stackdepot-introduce-__stack_depot_save.patch" of type "text/x-diff" (3982 bytes)

View attachment "0002-kasan-common-provide-can_alloc-in-kasan_save_stack.patch" of type "text/x-diff" (2537 bytes)

View attachment "0003-kasan-generic-introduce-kasan_record_aux_stack_noall.patch" of type "text/x-diff" (2359 bytes)

View attachment "0004-workqueue-kasan-avoid-alloc_pages-when-recording-sta.patch" of type "text/x-diff" (2941 bytes)