linux-kernel - Re: [PATCH 4/4] lockdep: Fix workqueue crossrelease annotation

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20170905003844.GO3240@X58A-UD3R>
Date:   Tue, 5 Sep 2017 09:38:45 +0900
From:   Byungchul Park <byungchul.park@....com>
To:     Peter Zijlstra <peterz@...radead.org>
Cc:     Byungchul Park <max.byungchul.park@...il.com>,
        Ingo Molnar <mingo@...nel.org>, Tejun Heo <tj@...nel.org>,
        Boqun Feng <boqun.feng@...il.com>, david@...morbit.com,
        Johannes Berg <johannes@...solutions.net>, oleg@...hat.com,
        "linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
        kernel-team@....com
Subject: Re: [PATCH 4/4] lockdep: Fix workqueue crossrelease annotation

On Mon, Sep 04, 2017 at 01:42:48PM +0200, Peter Zijlstra wrote:
> On Mon, Sep 04, 2017 at 10:30:32AM +0900, Byungchul Park wrote:
> > On Fri, Sep 01, 2017 at 06:38:52PM +0200, Peter Zijlstra wrote:
> > > And get tangled up with the workqueue annotation again, no thanks.
> > > Having the first few works see the thread setup isn't worth it.
> > > 
> > > And your work_id annotation had the same problem.
> > 
> > I keep asking you for an example because I really understand you.
> > 
> >    Fix my problematic example with your patches,
> > 
> >    or,
> > 
> >    Show me a problematic scenario with my original code, you expect.
> > 
> > Whatever, it would be helpful to understand you.
> 
> I _really_ don't understand what you're worried about. Is it the kthread
> create and workqueue init or the pool->lock that is released/acquired in
> process_one_work()?

s/in process_one_work()/in all worker code including setup code/

Original code was already designed to handle real dependencies well. But
you invalidated it _w/o_ any reason, that's why I don't agree with your
patches. Your patches only do avoiding the wq issue now we focus on.

Look at:

 worker thread                         another context
 -------------                         ---------------
                                       wait_for_completion()
       |
       |      (1)
       v
  +---------+
  | Work  A | (2)
  +---------+
       |
       |      (3)
       v
  +---------+
  | Work  B | (4)
  +---------+
       |
       |      (5)
       v
  +---------+
  | Work  C | (6)
  +---------+
       |
       v

We have to consider whole context of the worker to build dependencies
with a crosslock e.g. wait_for_commplete().

Only thing we have to care here is to make all works e.g. (2), (4) and
(6) independent, because workqueue does _concurrency control_. As I said
last year at the very beginning, for works not applied the control e.g.
max_active == 1, we don't need that isolation. I said, it's a future work.

It would have been much easier to communicate with each other if you
*tried* to understand my examples like now or you *tried* to give me one
example at least. You didn't even *try*. Only thing I want to ask you
for is to *try* to understand my opinions on conflicts.

Now, understand what I intended? Still unsufficient?