linux-kernel - Re: [RFC patch] Use IPI_shortcut for lapic timer broadcast

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <3877989d0906290143y3b5306f4s4a990ecbfbfc49b6@mail.gmail.com>
Date:	Mon, 29 Jun 2009 16:43:16 +0800
From:	Luming Yu <luming.yu@...il.com>
To:	Ingo Molnar <mingo@...e.hu>
Cc:	Arjan van de Ven <arjan@...radead.org>,
	LKML <linux-kernel@...r.kernel.org>, suresh.b.siddha@...el.com,
	venkatesh.pallipadi@...el.com,
	Thomas Gleixner <tglx@...utronix.de>,
	"H. Peter Anvin" <hpa@...or.com>
Subject: Re: [RFC patch] Use IPI_shortcut for lapic timer broadcast

On Mon, Jun 29, 2009 at 4:30 PM, Ingo Molnar<mingo@...e.hu> wrote:
>
> * Luming Yu <luming.yu@...il.com> wrote:
>
>> On Mon, Jun 29, 2009 at 4:16 PM, Ingo Molnar<mingo@...e.hu> wrote:
>> >
>> > * Luming Yu <luming.yu@...il.com> wrote:
>> >
>> >> On Mon, Jun 29, 2009 at 3:20 PM, Ingo Molnar<mingo@...e.hu> wrote:
>> >> >
>> >> > * Luming Yu <luming.yu@...il.com> wrote:
>> >> >
>> >> >> Hello,
>> >> >>
>> >> >> We need to use IPI shortcut to send lapic timer broadcast
>> >> >> to avoid the latency of sending IPI one bye one on systems with many
>> >> >> logical processors when NO_HZ is disabled.
>> >> >> Without this patch,I have seen upstream kernel with RHEL 5 kernel
>> >> >> config boot hang .
>> >> >
>> >> > hm, that might be a valid optimization - but why does the lack of
>> >> > this optimization result in a hang?
>> >>
>> >> It is hang caused by kernel code for work around lapic-timer-stop
>> >> issue. With HZ=1000, and a lot of cpus (eg. 64 logical cpus), cpu
>> >> 0 will be busy working on send TIMER IPI instead of making
>> >> progress in boot (right after deep-C-state has been used).
>> >
>> > that's a bit weird. With HZ=1000 we have 1000 usecs between each
>> > timer tick. Assuming a CPU sends to a lot of CPUs (64 logical CPUs)
>> > that means that each IPI takes more than ~15 microseconds to
>> > process. On what hardware/platform can this happen realistically?
>>
>> https://bugzilla.redhat.com/show_bug.cgi?id=499271
>>
>> Someone has measured that it needs 50-100us latency to send one
>> IPI
>
> Ugh. What platform is it that takes this much time to pass an IPI?
>
> IPIs are the lifeline of process messaging under Linux. TLB flushes
> in threaded apps rely on it (heavily), the scheduler relies on it
> for wakeups (heavily) and a lot of other code relies on IPIs as
> well.
>
> Even a Pentium-5 100 MHz dual box was able to do cross-CPU IPIs
> within 10-20 microseconds more than a decade ago - so 50-100 usecs
> latency on a modern platform is totally out of this planet and will
> hurt Linux performance big time. And the worst thing about it is
> that none of the usual performance metrics will really show _why_
> performance is tanking ...
>


Please note this is deep-C-state related.
C state does add extra latency.. but I don't know how much...
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/