[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <20211202182930.GA48887@fuller.cnet>
Date: Thu, 2 Dec 2021 15:29:30 -0300
From: Marcelo Tosatti <mtosatti@...hat.com>
To: Frederic Weisbecker <frederic@...nel.org>
Cc: linux-kernel@...r.kernel.org, Nitesh Lal <nilal@...hat.com>,
Nicolas Saenz Julienne <nsaenzju@...hat.com>,
Christoph Lameter <cl@...ux.com>,
Juri Lelli <juri.lelli@...hat.com>,
Peter Zijlstra <peterz@...radead.org>,
Alex Belits <abelits@...its.com>, Peter Xu <peterx@...hat.com>,
Thomas Gleixner <tglx@...utronix.de>,
Daniel Bristot de Oliveira <bristot@...hat.com>
Subject: Re: [patch v7 02/10] add prctl task isolation prctl docs and samples
On Thu, Dec 02, 2021 at 06:13:20PM +0100, Frederic Weisbecker wrote:
> On Mon, Nov 29, 2021 at 12:13:25PM -0300, Marcelo Tosatti wrote:
> > On Tue, Nov 23, 2021 at 01:36:20PM +0100, Frederic Weisbecker wrote:
> > > On Fri, Nov 12, 2021 at 09:35:33AM -0300, Marcelo Tosatti wrote:
> > > > +**PR_ISOL_CFG_SET**:
> > > > +
> > > > + Set task isolation configuration.
> > > > + The general format is::
> > > > +
> > > > + prctl(PR_ISOL_CFG_SET, what, arg3, arg4, arg5);
> > > > +
> > > > + The 'what' argument specifies what to configure. Possible values are:
> > > > +
> > > > + - ``I_CFG_FEAT``:
> > > > +
> > > > + Set configuration of task isolation features. 'arg3' specifies
> > > > + the feature. Possible values are:
> > > > +
> > > > + - ``ISOL_F_QUIESCE``:
> > > > +
> > > > + If arg4 is QUIESCE_CONTROL, set the control structure
> > > > + for quiescing of background kernel activities, from
> > > > + the location pointed to by ``(int *)arg5``::
> > > > +
> > > > + struct task_isol_quiesce_control {
> > > > + __u64 flags;
> > > > + __u64 quiesce_mask;
> > > > + __u64 quiesce_oneshot_mask;
> > > > + __u64 pad[5];
> > > > + };
> > > > +
> > > > + Where:
> > > > +
> > > > + *flags*: Additional flags (should be zero).
> > > > +
> > > > + *quiesce_mask*: A bitmask containing which kernel
> > > > + activities to quiesce.
> > > > +
> > > > + *quiesce_oneshot_mask*: A bitmask indicating which kernel
> > > > + activities should behave in oneshot mode, that is, quiescing
> > > > + will happen on return from prctl(PR_ISOL_ACTIVATE_SET), but not
> > > > + on return of subsequent system calls. The corresponding bit(s)
> > > > + must also be set at quiesce_mask.
> > > > +
> > > > + *pad*: Additional space for future enhancements.
> > > > +
> > > > + For quiesce_mask (and quiesce_oneshot_mask), possible bit sets are:
> > > > +
> > > > + - ``ISOL_F_QUIESCE_VMSTATS``
> > > > +
> > > > + VM statistics are maintained in per-CPU counters to
> > > > + improve performance. When a CPU modifies a VM statistic,
> > > > + this modification is kept in the per-CPU counter.
> > > > + Certain activities require a global count, which
> > > > + involves requesting each CPU to flush its local counters
> > > > + to the global VM counters.
> > > > +
> > > > + This flush is implemented via a workqueue item, which
> > > > + might schedule a workqueue on isolated CPUs.
> > > > +
> > > > + To avoid this interruption, task isolation can be
> > > > + configured to, upon return from system calls, synchronize
> > > > + the per-CPU counters to global counters, thus avoiding
> > > > + the interruption.
> > >
> > > Sorry I know this is already v7 but we really don't want to screw up this interface.
> >
> > No problem.
> >
> > > What would be more simple and flexible for individual features to quiesce:
> > >
> > > arg3 = ISOL_F_QUIESCE
> > > arg4 = which feature to quiesce (eg: ISOL_F_QUIESCE_VMSTATS)
> >
> > arg4 is QUIESCE_CONTROL today so one can expand the interface
> > (by adding new commands).
> >
> > > arg5 =
> > >
> > > struct task_isol_quiesce_control {
> > > __u64 flags; //with ONESHOT as the first and only possible flag for now
> > > __u64 pad[5];
> > > };
> >
> > So your idea is to allow expansion at this level ? Say while adding
> > a new
> >
> > ISOL_F_QUIESCE_NEWITEM
> >
> > one can add
> >
> > struct task_isol_quiesce_control_newitem {
> > __u64 flags;
> > __u64 pad[5];
> > };
> >
> > And add new fields to "struct task_isol_quiesce_control_newitem".
> >
> > One downside of this suggestion is that for use-cases of the task_isol_computation.c type,
> > (see patch 2 "add prctl task isolation prctl docs and samples"), this might need
> > multiple system calls for each enable/disable cycle. Is that OK?
> >
> > See more below.
> >
> > > This way we can really do a finegrained control over each feature to quiesce.
> >
> > With the patchset above, one can add new values to arg4
> > (at the same level of QUIESCE_CONTROL). Your suggestion does not save
> > room for that.
> >
> > One could add new values to the same space of I_CFG_FEAT:
> >
> > prctl(PR_ISOL_CFG_SET, I_CFG_FEAT_xxx, ...);
> >
> > But that sounds awkward.
> >
> > Or change the current ioctl to:
> >
> > prctl(PR_ISOL_CFG, I_CFG_FEAT_CONTROL, ...);
> >
> > Which makes it less awkward.
> >
> > What do you say?
> >
> > ---
> >
> > And then, what about keeping the bitmaps with enabled/one-shot mode
> > per feature per bit (to avoid multiple system calls)
> > but having (in the future) additional per-quiesce instance commands ?
>
> Ok got your points.
>
> I guess we can then simply rename ISOL_F_QUIESCE to ISOL_F_QUIESCE_MULTIPLE
> for simple all-in-one configuration. Then if the need ever arise in the future,
> we can always add ISOL_F_QUIESCE (or ISOL_F_QUIESCE_ONE) to do finegrained
> control over a single quiescing feature.
>
> Does that sound ok?
Yep, it does, will change that.
Powered by blists - more mailing lists