linux-kernel - Re: tip: demise of tsk_cpus_allowed() and tsk_nr_cpus

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20170209065727.GA6902@gmail.com>
Date:   Thu, 9 Feb 2017 07:57:27 +0100
From:   Ingo Molnar <mingo@...nel.org>
To:     Peter Zijlstra <peterz@...radead.org>
Cc:     Thomas Gleixner <tglx@...utronix.de>,
        Mike Galbraith <efault@....de>, Ingo Molnar <mingo@...e.hu>,
        Sebastian Andrzej Siewior <bigeasy@...utronix.de>,
        LKML <linux-kernel@...r.kernel.org>
Subject: Re: tip: demise of tsk_cpus_allowed() and tsk_nr_cpus_allowed()


* Ingo Molnar <mingo@...nel.org> wrote:

> 
> * Peter Zijlstra <peterz@...radead.org> wrote:
> 
> > On Wed, Feb 08, 2017 at 11:20:19AM +0100, Thomas Gleixner wrote:
> > > On Mon, 6 Feb 2017, Ingo Molnar wrote:
> > > > * Peter Zijlstra <peterz@...radead.org> wrote:
> > > > >
> > > > > cpumasks are a pain, the above avoids allocating more of them.
> > > 
> > > Indeed.
> > > 
> > > > Yeah, so this could then be done by pointerifying ->cpus_allowed - more robust 
> > > > than the wrappery,
> > > 
> > > You mean:
> > > 
> > > struct task_struct {
> > >        cpumask_t	cpus_allowed;
> > >        cpumask_t	*effective_cpus_allowed;
> > > };
> 
> Yeah. I'd name it a bit differently and constify the pointer to get type 
> safety and to make sure the mask is never modified through the pointer:
> 
> 	struct task_struct {
> 		const cpumask_t		*cpus_ptr;
> 		cpumask_t		cpus_mask;
> 	};
> 
> ( I'd drop the 'allowed' part, it's obvious enough what task->cpus_mask does, 
>   right? )
> 
> and upstream would essentially just do:
> 
> 	t->cpus_allowed_ptr = &t->cpus_allowed;
> 
> And -rt, when it wants to pin a task, would do:
> 
> 	t->cpus_allowed_ptr = &cpumask_of(task_cpu(p));
> 
> The rules are:
> 
>  - Code that 'uses' ->cpus_allowed would use the pointer.
> 
>  - Code that 'modifies' ->cpus_allowed would use the direct mask.
> 
> The upstream advantages are:
> 
>  - The type separation of modifications from usage.
> 
>  - Removal of wrappery.
> 
>  - Maybe sometime in the future upstream would want to disable migration too ...
> 
> In fact -rt gains something too:
> 
>  - With this scheme we would AFAICS get slightly more optimal code on -rt.
>    (Because there's no __migration_disabled() branching anymore.)
> 
>  - Plus all new code is automatically -rt ready - no need to maintain the wrappery 
>    space. Much less code path forking.
> 
> So as I see it it's win-win for both upstream and for -rt!
> 
> > > and make the scheduler use effective_cpus_allowed instead of cpus_allowed? Or 
> > > what do you have in mind?
> > 
> > That scheme is weird for nr_cpus_allowed. Not to mention that the
> > pointer to the integer is larger than the integer itself.
> 
> So in the new scheme I don't think nr_cpus_allowed would have to be wrapped
> at all: whenever the pointer (or mask) is changed in set_cpus_allowed_common() 
> nr_cpus_allowed is recalculated as well - like today.
> 
> It should be self-maintaining. Am I missing something?

And -rt would do something like this in migration_disable()/enable():
 
	t->cpus_ptr = &cpumask_of(task_cpu(p));
	t->nr_cpus = 1;

	...

	t->cpus_ptr = &t->cpus_mask;
	t->nr_cpus = cpumask_weight(t->cpus_mask);

In addition to that we could cache the weight of the cpumask as an additional 
optimization:

	t->cpus_ptr = &t->cpus_mask;
	t->nr_cpus = t->cpus_mask_weight;

It all looks like a pretty natural construct to me. The migration_disabled() flag 
spreads almost a hundred branches all across the scheduler.

Thanks,

	Ingo