linux-kernel - Re: [PATCH 1/2] IB/hfi1: Use preempt_{dis,en}able

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20171005163933.GJ647@jcartwri.amer.corp.natinst.com>
Date:   Thu, 5 Oct 2017 11:39:33 -0500
From:   Julia Cartwright <julia@...com>
To:     Thomas Gleixner <tglx@...utronix.de>
CC:     Steven Rostedt <rostedt@...dmis.org>,
        Arnaldo Carvalho de Melo <acme@...nel.org>,
        <bigeasy@...utronix.de>, <linux-rt-users@...r.kernel.org>,
        <linux-kernel@...r.kernel.org>,
        Arnaldo Carvalho de Melo <acme@...hat.com>,
        Clark Williams <williams@...hat.com>,
        Dean Luick <dean.luick@...el.com>,
        Dennis Dalessandro <dennis.dalessandro@...el.com>,
        Doug Ledford <dledford@...hat.com>,
        Kaike Wan <kaike.wan@...el.com>,
        Leon Romanovsky <leonro@...lanox.com>,
        <linux-rdma@...r.kernel.org>,
        Peter Zijlstra <peterz@...radead.org>,
        Sebastian Andrzej Siewior <sebastian.siewior@...utronix.de>,
        Sebastian Sanchez <sebastian.sanchez@...el.com>
Subject: Re: [PATCH 1/2] IB/hfi1: Use preempt_{dis,en}able_nort()

On Thu, Oct 05, 2017 at 06:16:23PM +0200, Thomas Gleixner wrote:
> On Thu, 5 Oct 2017, Julia Cartwright wrote:
> 
> > On Thu, Oct 05, 2017 at 11:55:39AM -0400, Steven Rostedt wrote:
> > > On Thu, 5 Oct 2017 10:37:59 -0500
> > > Julia Cartwright <julia@...com> wrote:
> > > 
> > > > On Thu, Oct 05, 2017 at 05:27:30PM +0200, Thomas Gleixner wrote:
> > > > > On Thu, 5 Oct 2017, Julia Cartwright wrote:  
> > > > > > On Tue, Oct 03, 2017 at 12:49:19PM -0300, Arnaldo Carvalho de Melo wrote:  
> > > > > > > -	preempt_disable();
> > > > > > > +	preempt_disable_nort();
> > > > > > >  	this_cpu_inc(*sc->buffers_allocated);  
> > > > > > 
> > > > > > Have you tried this on RT w/ CONFIG_DEBUG_PREEMPT?  I believe that the
> > > > > > this_cpu_* operations perform a preemption check, which we'd trip.  
> > > > > 
> > > > > Good point. Changing this to migrate_disable() would do the trick.  
> > > > 
> > > > Wouldn't we still trip the preempt check even with migration disabled?
> > > > In another thread I asked the same question: should the preemption
> > > > checks here be converted to migration-checks in RT?
> > > 
> > > Is it a "preemption check"?
> > 
> > Sorry if I was unclear, more precisely: the this_cpu_* family of
> > accessors, w/ CONFIG_DEBUG_PREEMPT currently spits out a warning when
> > the caller is invoked in a context where preemption is enabled.
> > 
> > The check is shared w/ the smp_processor_id() check, as implemented in
> > lib/smp_processor_id.c.  It effectively boils down to a check of
> > preempt_count() and irqs_disabled().
> 
> Except that on RT that check cares about the migrate disable state. You can
> invoke this_cpu_* and smp_processor_id() in preemptible/interruptible
> context because of:
> 
> 	if (cpumask_equal(current->cpus_ptr, cpumask_of(this_cpu)))
> 		goto out;
> 
> That's true even on mainline.
> 
> But you are right that this check needs some update because
> migrate_disable() does not immediately update the allowed cpumask IIRC.

Actually, I think it does:

   migrate_disable() ->
      p = current;
      ...
      migrate_disable_update_cpus_allowed(p) ->
          p->cpus_ptr = cpumask_of(smp_processor_id())
      ...

Perhaps it's worth a simple comment update, below.

   Julia

-- 8< --
Subject: [PATCH] kernel: sched: document smp_processor_id/this_cpu safety in
 migration-disabled context

On RT, users of smp_processor_id() and this_cpu_*() per-cpu accessors
are considered safe if the caller executes with migration disabled.  On
!RT, preempt_disable() is sufficient to make this guarantee, however on
RT, the lesser migrate_disable() is sufficient.

It is not entirely obvious which check in check_preemption_disabled()
makes this work, so document it.

Signed-off-by: Julia Cartwright <julia@...com>
---
 lib/smp_processor_id.c | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/lib/smp_processor_id.c b/lib/smp_processor_id.c
index de3b2d925473..c8091d9eb1b4 100644
--- a/lib/smp_processor_id.c
+++ b/lib/smp_processor_id.c
@@ -20,7 +20,11 @@ notrace static unsigned int check_preemption_disabled(const char *what1,
 
 	/*
 	 * Kernel threads bound to a single CPU can safely use
-	 * smp_processor_id():
+	 * smp_processor_id().
+	 *
+	 * In addition, threads which are currently executing within
+	 * a migration disabled region can safely use smp_processor_id() and
+	 * this_cpu accessors.
 	 */
 	if (cpumask_equal(current->cpus_ptr, cpumask_of(this_cpu)))
 		goto out;
-- 
2.14.1