linux-kernel - Re: [RFC PATCH v2] sched_pair_cpu: Introduce scheduler task pairing system call

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <1747476740.14310.1593184577574.JavaMail.zimbra@efficios.com>
Date:   Fri, 26 Jun 2020 11:16:17 -0400 (EDT)
From:   Mathieu Desnoyers <mathieu.desnoyers@...icios.com>
To:     Peter Zijlstra <peterz@...radead.org>
Cc:     linux-kernel <linux-kernel@...r.kernel.org>,
        Thomas Gleixner <tglx@...utronix.de>,
        Joel Fernandes <joelaf@...gle.com>,
        "H. Peter Anvin" <hpa@...or.com>, Paul Turner <pjt@...gle.com>,
        Boqun Feng <boqun.feng@...il.com>,
        rostedt <rostedt@...dmis.org>,
        Andy Lutomirski <luto@...capital.net>,
        paulmck <paulmck@...nel.org>, Julien Desfossez <ju@...pix.org>
Subject: Re: [RFC PATCH v2] sched_pair_cpu: Introduce scheduler task pairing
 system call

----- On Jun 25, 2020, at 12:34 PM, Mathieu Desnoyers mathieu.desnoyers@...icios.com wrote:

> ----- On Jun 25, 2020, at 10:56 AM, Mathieu Desnoyers
> mathieu.desnoyers@...icios.com wrote:
> 
>> ----- On Jun 24, 2020, at 3:50 PM, Peter Zijlstra peterz@...radead.org wrote:
>> 
>>> On Wed, Jun 24, 2020 at 02:31:33PM -0400, Mathieu Desnoyers wrote:
>>> 
>> [...]
>>> The other alternative is using a preempt_notifier for the worker I
>>> suppose.
>> 
> [...]
>>> 
>>> preempt_notifier could work here too I suppose, install it on yourself
>>> when you do the pear syscall and take it away again when you're finished
>>> with it.
> 
> The issue I currently have with preempt notifiers is that I need to
> send an IPI from a sched_out notifier, which has interrupts off and
> hold the rq lock. smp_call_function_single() warns due to irq off, and
> indeed it triggers deadlocks.
> 
> Before using preempt notifiers, I was touching the "prev" task after
> irqs were reenabled and rq lock was released, which allowed me to
> send an IPI from that context.
> 
> Any thoughts on how to best solve this ?

I think I may have found a way out of this: I may not need to use
smp_call_function_single() at all.

When preempting a paired task, I think we can rely on memory barrier at the
beginning of scheduling of the paired task to match the memory barrier at
the end of scheduling of the kworker thread to provide memory ordering. Therefore,
the IPI is not needed at all in this case.

When preempting the kworker thread, things are a bit trickier. AFAIU I can simply
queue task work on the paired task directly without an IPI, and then use
kick_process() on the paired task.

The remaining concern is whether kick_process() (and thus smp_send_reschedule())
is sufficient to guarantee a memory barrier before smp_send_reschedule returns ?
I suspect not, because it only raises the IPI, and does not appear to wait for
its handler to complete. In that case I need a release on the paired task and
an acquire in sched_out of the kworker. The memory barrier at the end of schedule
fulfills the acquire, but I don't see how the acquire is done on the paired task,
because execution of its scheduler does not necessarily happen immediately when
the IPI is raised.

Thanks,

Mathieu

-- 
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com