linux-kernel - Re: [PATCH 2/2] sched/isolation: Add cpu_is

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <Y+JBL3HZ4ahRSEis@lothringen>
Date:   Tue, 7 Feb 2023 13:16:47 +0100
From:   Frederic Weisbecker <frederic@...nel.org>
To:     Waiman Long <longman@...hat.com>
Cc:     LKML <linux-kernel@...r.kernel.org>,
        Roman Gushchin <roman.gushchin@...ux.dev>,
        Michal Hocko <mhocko@...e.com>,
        Marcelo Tosatti <mtosatti@...hat.com>,
        Leonardo <leobras@...hat.com>,
        Johannes Weiner <hannes@...xchg.org>,
        Shakeel Butt <shakeelb@...gle.com>,
        Muchun Song <muchun.song@...ux.dev>,
        Andrew Morton <akpm@...ux-foundation.org>,
        Peter Zijlstra <peterz@...radead.org>,
        Thomas Gleixner <tglx@...utronix.de>, cgroups@...r.kernel.org,
        linux-mm@...ck.org
Subject: Re: [PATCH 2/2] sched/isolation: Add cpu_is_isolated() API

On Fri, Feb 03, 2023 at 10:53:46PM -0500, Waiman Long wrote:
> On 2/3/23 18:24, Frederic Weisbecker wrote:
> > Provide this new API to check if a CPU has been isolated either through
> > isolcpus= or nohz_full= kernel parameter.
> > 
> > It aims at avoiding kernel load deemed to be safely spared on CPUs
> > running sensitive workload that can't bear any disturbance, such as
> > pcp cache draining.
> > 
> > Suggested-by: Michal Hocko <mhocko@...e.com>
> > Signed-off-by: Frederic Weisbecker <frederic@...nel.org>
> > ---
> >   include/linux/sched/isolation.h | 6 ++++++
> >   1 file changed, 6 insertions(+)
> > 
> > diff --git a/include/linux/sched/isolation.h b/include/linux/sched/isolation.h
> > index b645cc81fe01..088672f08469 100644
> > --- a/include/linux/sched/isolation.h
> > +++ b/include/linux/sched/isolation.h
> > @@ -53,4 +53,10 @@ static inline bool housekeeping_cpu(int cpu, enum hk_type type)
> >   	return true;
> >   }
> > +static inline bool cpu_is_isolated(int cpu)
> > +{
> > +	return !housekeeping_test_cpu(cpu, HK_TYPE_DOMAIN) ||
> > +		 !housekeeping_test_cpu(cpu, HK_TYPE_KERNEL_NOISE);
> > +}
> > +
> >   #endif /* _LINUX_SCHED_ISOLATION_H */
> 
> CPUs in an isolated cpuset partition is similar to HK_TYPE_DOMAIN CPUs as
> load balancing is disabled. I can add an API to access the cpumask and add
> to this API. However, that list is dynamic as it can be changed at run time.
> Will that be a problem? Or should that be used separately?

So that's what I intended first but the dynamic part of cpuset made me
postpone that to better days.

But yes ideally it should look like:

static inline bool cpu_is_isolated(int cpu)
{
    return !housekeeping_test_cpu(cpu, HK_TYPE_KERNEL_NOISE) ||
           on_null_domain(cpu_rq(cpu));
}

And there should be a hook in something like detach_destroy_domains()
to flush the pcp cache when a CPU is attached to a NULL domain.

All that with proper RCU synchronization:

       UPDATE                                          READER
       ------                                          ------
       rcu_assign_pointer(cpu_rq(cpu)->sd, NULL);      rcu_read_lock();
       synchronize_rcu();                              if (!cpu_is_isolated(cpu))
       stock = &per_cpu(memcg_stock, cpu);                 schedule_work_on(cpu, &stock->work);
       flush_work(&stock->work);                       rcu_read_unlock()

Thanks.