Date:   Thu, 21 May 2020 17:11:44 +0100
From:   Valentin Schneider <valentin.schneider@....com>
To:     Russell King - ARM Linux admin <linux@...linux.org.uk>
Cc:     Marc Zyngier <maz@...nel.org>,
        linux-arm-kernel@...ts.infradead.org, linux-kernel@...r.kernel.org,
        Will Deacon <will@...nel.org>,
        Catalin Marinas <catalin.marinas@....com>,
        Thomas Gleixner <tglx@...utronix.de>,
        Jason Cooper <jason@...edaemon.net>,
        Sumit Garg <sumit.garg@...aro.org>, kernel-team@...roid.com
Subject: Re: [PATCH 04/11] ARM: Allow IPIs to be handled as normal interrupts


On 21/05/20 16:12, Russell King - ARM Linux admin wrote:
> On Thu, May 21, 2020 at 03:03:49PM +0100, Valentin Schneider wrote:
>>
>> On 19/05/20 23:24, Russell King - ARM Linux admin wrote:
>> > On Tue, May 19, 2020 at 05:17:48PM +0100, Marc Zyngier wrote:
>> >> In order to deal with IPIs as normal interrupts, let's add
>> >> a new way to register them with the architecture code.
>> >>
>> >> set_smp_ipi_range() takes a range of interrupts, and allows
>> >> the arch code to request them as if the were normal interrupts.
>> >> A standard handler is then called by the core IRQ code to deal
>> >> with the IPI.
>> >>
>> >> This means that we don't need to call irq_enter/irq_exit, and
>> >> that we don't need to deal with set_irq_regs either. So let's
>> >> move the dispatcher into its own function, and leave handle_IPI()
>> >> as a compatibility function.
>> >>
>> >> On the sending side, let's make use of ipi_send_mask, which
>> >> already exists for this purpose.
>> >
>> > You say nothing about the nesting of irq_enter() and irq_exit()
>> > for scheduler_ipi().
>> >
>> > Given that lockdep introduced the requirement that hard IRQs can't
>> > be nested, are we sure that calling irq_exit() twice is safe?
>> >
>> > Looking at irqtime_account_irq(), it seems that will cause double-
>> > accounting of in-interrupt time, since we will increment
>> > irq_start_time by just over twice the the period spent handling
>> > the IPI.
>> >
>> > I think the rest of irq_exit() should be safe, but still, this
>> > behaviour should be documented at the very least, if not avoided.
>> >
>>
>> x86 does the same (though IIUC only when tracing reschedule IPI's),
>
> Right, so when the system is operating normally, then the accounting is
> correct.  When the reschedule path is being explicitly traced, then
> the accounting will be doubled for it.
>

Right, it's true that they are only affected when tracing.


That said, AFAICT the accounting nests correctly. Consider:

  irq_enter() @t0
    irq_enter() @t1
    ...
    irq_exit() @t2
  irq_exit() @t3

Entering irqtime_account_irq() at time t, we get something like:

  delta = t - irq_start_time;
  irq_start_time = t;

  if (hardirq_count())
          total += delta;

Since we go through the accounting on both irq_enter() and irq_exit(), we'd
have something like:

  irq_enter() @t0
    irq_start_time = t0

  irq_enter() @t1
    delta = t1 - t0
    irq_start_time = t1
    total += t1 - t0

  irq_exit() @t2
    delta = t2 - t1
    irq_start_time = t2
    total += t2 - t1

  irq_exit() @t3
    delta = t3 - t2
    irq_start_time = t3
    total += t3 - t2


So at the end we have incremented the total by

  t1-t0 + t2-t1 + t3-t2 = t3 - t0

IOW the duration of the outermost pair (... Unless I goofed up).

> What's being proposed for ARM is to always have this mis-accounting,
> where no mis-accounting was present before - and some of us (me) /do/
> enable IRQ accounting in our kernels as standard. So, you can take
> this as a kernel regression report from a user.
>
>> and MIPS has the same issue as it also uses generic IRQ IPI's - so
>> although it's not ideal, I think we can live with it.
>
> Yes, but is there anyone who cares about this for MIPS?

Powered by blists - more mailing lists