linux-kernel - Re: [patch V4 5/8] sched: highmem: Store local kmaps in task struct

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20201119121200.GG3121429@hirez.programming.kicks-ass.net>
Date:   Thu, 19 Nov 2020 13:12:00 +0100
From:   Peter Zijlstra <peterz@...radead.org>
To:     Thomas Gleixner <tglx@...utronix.de>
Cc:     LKML <linux-kernel@...r.kernel.org>, x86@...nel.org,
        Linus Torvalds <torvalds@...uxfoundation.org>,
        Christoph Hellwig <hch@....de>,
        Matthew Wilcox <willy@...radead.org>,
        Daniel Vetter <daniel@...ll.ch>,
        Andrew Morton <akpm@...ux-foundation.org>, linux-mm@...ck.org,
        Linus Torvalds <torvalds@...ux-foundation.org>,
        Ingo Molnar <mingo@...nel.org>,
        Juri Lelli <juri.lelli@...hat.com>,
        Vincent Guittot <vincent.guittot@...aro.org>,
        Dietmar Eggemann <dietmar.eggemann@....com>,
        Steven Rostedt <rostedt@...dmis.org>,
        Ben Segall <bsegall@...gle.com>, Mel Gorman <mgorman@...e.de>,
        Daniel Bristot de Oliveira <bristot@...hat.com>
Subject: Re: [patch V4 5/8] sched: highmem: Store local kmaps in task struct

On Thu, Nov 19, 2020 at 12:51:32PM +0100, Peter Zijlstra wrote:
> On Wed, Nov 18, 2020 at 08:48:43PM +0100, Thomas Gleixner wrote:
> 
> > @@ -4073,6 +4089,7 @@ prepare_task_switch(struct rq *rq, struc
> >  	perf_event_task_sched_out(prev, next);
> >  	rseq_preempt(prev);
> >  	fire_sched_out_preempt_notifiers(prev, next);
> > +	kmap_local_sched_out();
> >  	prepare_task(next);
> >  	prepare_arch_switch(next);
> >  }
> > @@ -4139,6 +4156,7 @@ static struct rq *finish_task_switch(str
> >  	finish_lock_switch(rq);
> >  	finish_arch_post_lock_switch();
> >  	kcov_finish_switch(current);
> > +	kmap_local_sched_in();
> >  
> >  	fire_sched_in_preempt_notifiers(current);
> >  	/*
> 
> > +void __kmap_local_sched_out(void)
> > +{
> > +	struct task_struct *tsk = current;
> > +	pte_t *kmap_pte = kmap_get_pte();
> > +	int i;
> > +
> > +	/* Clear kmaps */
> > +	for (i = 0; i < tsk->kmap_ctrl.idx; i++) {
> > +	}
> > +}
> > +
> > +void __kmap_local_sched_in(void)
> > +{
> > +	struct task_struct *tsk = current;
> > +	pte_t *kmap_pte = kmap_get_pte();
> > +	int i;
> > +
> > +	/* Restore kmaps */
> > +	for (i = 0; i < tsk->kmap_ctrl.idx; i++) {
> > +	}
> > +}
> 
> So even in the optimal case, this adds an unconditional load of
> tsk->kmap_ctrl.idx to schedule() (2 misses, one pre and one post).
> 
> Munging preempt-notifiers behind a static_branch, which in that same
> optimal case, avoided touching curr->preempt_notifier, resulted in a
> measurable performance improvement. See commit:
> 
>   1cde2930e154 ("sched/preempt: Add static_key() to preempt_notifiers")
> 
> Can we fudge some state in a cacheline we're already touching to avoid
> this?

The only state we seem to consistently look at after schedule() is
need_resched()'s TIF_NEED_RESCHED.

But adding a TIF_flag to all archs and setting/clearing it from
kmap_local might be a bit daft.. :/