linux-kernel - Re: [PATCH 4/4] lockdep: Fix workqueue crossrelease annotation

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20170905070825.tovfkqvxpwosh5oa@hirez.programming.kicks-ass.net>
Date:   Tue, 5 Sep 2017 09:08:25 +0200
From:   Peter Zijlstra <peterz@...radead.org>
To:     Byungchul Park <byungchul.park@....com>
Cc:     Byungchul Park <max.byungchul.park@...il.com>,
        Ingo Molnar <mingo@...nel.org>, Tejun Heo <tj@...nel.org>,
        Boqun Feng <boqun.feng@...il.com>, david@...morbit.com,
        Johannes Berg <johannes@...solutions.net>, oleg@...hat.com,
        "linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
        kernel-team@....com
Subject: Re: [PATCH 4/4] lockdep: Fix workqueue crossrelease annotation

On Tue, Sep 05, 2017 at 09:38:45AM +0900, Byungchul Park wrote:
> On Mon, Sep 04, 2017 at 01:42:48PM +0200, Peter Zijlstra wrote:
> > On Mon, Sep 04, 2017 at 10:30:32AM +0900, Byungchul Park wrote:
> > > On Fri, Sep 01, 2017 at 06:38:52PM +0200, Peter Zijlstra wrote:
> > > > And get tangled up with the workqueue annotation again, no thanks.
> > > > Having the first few works see the thread setup isn't worth it.
> > > > 
> > > > And your work_id annotation had the same problem.
> > > 
> > > I keep asking you for an example because I really understand you.
> > > 
> > >    Fix my problematic example with your patches,
> > > 
> > >    or,
> > > 
> > >    Show me a problematic scenario with my original code, you expect.
> > > 
> > > Whatever, it would be helpful to understand you.
> > 
> > I _really_ don't understand what you're worried about. Is it the kthread
> > create and workqueue init or the pool->lock that is released/acquired in
> > process_one_work()?
> 
> s/in process_one_work()/in all worker code including setup code/
> 
> Original code was already designed to handle real dependencies well. But
> you invalidated it _w/o_ any reason, that's why I don't agree with your
> patches.

The reasons:

 - it avoids the interaction with the workqueue annotation
 - it makes each work consistent
 - its not different from what you did with work_id:

    https://lkml.kernel.org/r/1489479542-27030-6-git-send-email-byungchul.park@lge.com

    crossrelease_work_start() vs same_context_xhlock() { if
    (xhlock->work_id == curr->workid) ... }

> Your patches only do avoiding the wq issue now we focus on.
> 
> Look at:
> 
>  worker thread                         another context
>  -------------                         ---------------
>                                        wait_for_completion()
>        |
>        |      (1)
>        v
>   +---------+
>   | Work  A | (2)
>   +---------+
>        |
>        |      (3)
>        v
>   +---------+
>   | Work  B | (4)
>   +---------+
>        |
>        |      (5)
>        v
>   +---------+
>   | Work  C | (6)
>   +---------+
>        |
>        v
> 
> We have to consider whole context of the worker to build dependencies
> with a crosslock e.g. wait_for_commplete().
> 
> Only thing we have to care here is to make all works e.g. (2), (4) and
> (6) independent, because workqueue does _concurrency control_. As I said
> last year at the very beginning, for works not applied the control e.g.
> max_active == 1, we don't need that isolation. I said, it's a future work.
> 
> It would have been much easier to communicate with each other if you
> *tried* to understand my examples like now or you *tried* to give me one
> example at least. You didn't even *try*. Only thing I want to ask you
> for is to *try* to understand my opinions on conflicts.
> 
> Now, understand what I intended? Still unsufficient?

So you worry about max_active==1 ? Or you worry about pool->lock or
about the thread setup? I'm still not sure.