[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <ba97d52df60ac9c4a4afc2c03121a8c263aa5a15.camel@linux.ibm.com>
Date: Wed, 13 May 2020 01:31:30 -0300
From: Leonardo Bras <leonardo@...ux.ibm.com>
To: Nathan Lynch <nathanl@...ux.ibm.com>
Cc: linuxppc-dev@...ts.ozlabs.org, linux-kernel@...r.kernel.org,
Michael Ellerman <mpe@...erman.id.au>,
Nicholas Piggin <npiggin@...il.com>,
Benjamin Herrenschmidt <benh@...nel.crashing.org>,
Paul Mackerras <paulus@...ba.org>,
Greg Kroah-Hartman <gregkh@...uxfoundation.org>,
Thomas Gleixner <tglx@...utronix.de>,
Allison Randal <allison@...utok.net>,
"Gautham R. Shenoy" <ego@...ux.vnet.ibm.com>,
Nadav Amit <namit@...are.com>
Subject: Re: [PATCH 1/1] powerpc/rtas: Implement reentrant rtas call
Hello Nathan, thanks for the feedback!
On Fri, 2020-04-10 at 14:28 -0500, Nathan Lynch wrote:
> Leonardo Bras <leonardo@...ux.ibm.com> writes:
> > Implement rtas_call_reentrant() for reentrant rtas-calls:
> > "ibm,int-on", "ibm,int-off",ibm,get-xive" and "ibm,set-xive".
> >
> > On LoPAPR Version 1.1 (March 24, 2016), from 7.3.10.1 to 7.3.10.4,
> > items 2 and 3 say:
> >
> > 2 - For the PowerPC External Interrupt option: The * call must be
> > reentrant to the number of processors on the platform.
> > 3 - For the PowerPC External Interrupt option: The * argument call
> > buffer for each simultaneous call must be physically unique.
> >
> > So, these rtas-calls can be called in a lockless way, if using
> > a different buffer for each call.
> >
> From the language in the spec it's clear that these calls are intended
> to be reentrant with respect to themselves, but it's less clear to me
> that they are safe to call simultaneously with respect to each other or
> arbitrary other RTAS methods.
In my viewpoint, being reentrant to themselves, without being reentrant
to others would be very difficult to do, considering the way the
rtas_call is crafted to work.
I mean, I have no experience in rtas code, it's my viewpoint. In my
thoughts there is something like this:
common_path -> selects function by token -> reentrant function
|-> non-reentrant function
If there is one function that is reentrant, it means the common_path
and function selection by token would need to be reentrant too.
> > This can be useful to avoid deadlocks in crashing, where rtas-calls are
> > needed, but some other thread crashed holding the rtas.lock.
>
> Are these calls commonly used in the crash-handling path? Is this
> addressing a real issue you've seen?
>
Yes, I noticed deadlocks during crashes, like this one:
#0 arch_spin_lock
#1 lock_rtas ()
#2 rtas_call (token=8204, nargs=1, nret=1, outputs=0x0)
#3 ics_rtas_mask_real_irq (hw_irq=4100)
#4 machine_kexec_mask_interrupts
#5 default_machine_crash_shutdown
#6 machine_crash_shutdown
#7 __crash_kexec
#8 crash_kexec
#9 oops_end
On ics_rtas_mask_real_irq() we have both ibm_int_off and ibm_set_xive,
so it makes sense to also add ibm_int_on and ibm_get_xive as reentrant
too.
Full discussion available on this thread:
http://patchwork.ozlabs.org/project/linuxppc-dev/patch/20200401000020.590447-1-leonardo@linux.ibm.com/
>
> > +/*
> > + * Used for reentrant rtas calls.
> > + * According to LoPAR documentation, only "ibm,int-on", "ibm,int-off",
> > + * "ibm,get-xive" and "ibm,set-xive" are currently reentrant.
> > + * Reentrant calls need their own rtas_args buffer, so not using rtas.args.
> > + */
>
> Please use kernel-doc format in new code.
Sure, v2 is going to be fixed.
>
>
> > +int rtas_call_reentrant(int token, int nargs, int nret, int *outputs, ...)
> > +{
> > + va_list list;
> > + struct rtas_args rtas_args;
> > +
> > + if (!rtas.entry || token == RTAS_UNKNOWN_SERVICE)
> > + return -1;
> > +
> > + va_start(list, outputs);
> > + va_rtas_call_unlocked(&rtas_args, token, nargs, nret, list);
> > + va_end(list);
>
> No, I don't think you can place the RTAS argument buffer on the stack:
>
> 7.2.7, Software Implementation Note:
> | The OS must be aware that the effective address range for RTAS is 4
> | GB when instantiated in 32-bit mode and the OS should not pass RTAS
> | addresses or blocks of data which might fall outside of this range.
Agree, moved to PACA.
I will send a v2 soon, it will be a 2-patch patchset.
Best regards,
Leonardo Bras
Download attachment "signature.asc" of type "application/pgp-signature" (834 bytes)
Powered by blists - more mailing lists