linux-kernel - Re: [PATCH 0/2] clocksource: Avoid incorrect hpet fallback

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:   Wed, 10 Nov 2021 18:25:14 -0500
From:   Waiman Long <longman@...hat.com>
To:     paulmck@...nel.org
Cc:     John Stultz <john.stultz@...aro.org>,
        Thomas Gleixner <tglx@...utronix.de>,
        Stephen Boyd <sboyd@...nel.org>, linux-kernel@...r.kernel.org,
        Peter Zijlstra <peterz@...radead.org>,
        Cassio Neri <cassio.neri@...il.com>,
        Linus Walleij <linus.walleij@...aro.org>,
        Frederic Weisbecker <frederic@...nel.org>,
        Feng Tang <feng.tang@...el.com>
Subject: Re: [PATCH 0/2] clocksource: Avoid incorrect hpet fallback

On 11/10/21 17:32, Paul E. McKenney wrote:
> On Wed, Nov 10, 2021 at 05:17:30PM -0500, Waiman Long wrote:
>> It was found that when an x86 system was being stressed by running
>> various different benchmark suites, the clocksource watchdog might
>> occasionally mark TSC as unstable and fall back to hpet which will
>> have a signficant impact on system performance.
>>
>> The current watchdog clocksource skew threshold of 50us is found to be
>> insufficient. So it is changed back to 100us before commit 2e27e793e280
>> ("clocksource: Reduce clocksource-skew threshold") in patch 1. Patch 2
>> adds a Kconfig option to allow kernel builder to control the actual
>> threshold to be used.
>>
>> Waiman Long (2):
>>    clocksource: Avoid accidental unstable marking of clocksources
>>    clocksource: Add a Kconfig option for WATCHDOG_MAX_SKEW
> The ability to control the fine-grained threshold seems useful, but is
> the TSC still marked unstable when this commit from -rcu is applied?
> It has passed significant testing on other workloads.
>
> 2a43fb0479aa ("clocksource: Forgive repeated long-latency watchdog clocksource reads")
>
> If the patch below takes care of your situation, my thought is to
> also take your second patch, which would allow people to set the
> cutoff more loosely or more tightly, as their situation dictates.
>
> Thoughts?

That is commit 14dbb29eda51 ("clocksource: Forgive repeated long-latency 
watchdog clocksource reads") in your linux-rcu git tree. From reading 
the patch, I believe it should be able to address the hpet fallback 
problem that Red Hat had encountered. Your patch said it was an 
out-of-tree patch. Are you planning to mainline it?

Patch 1 of this series contains some testing data that caused hpet 
fallback in our testing runs. In summary, a clock skew of 100us is found 
to be enough to avoid the problem with benchmark runs. However, we have 
some cases where TSC was marked unstable at bootup time with a skew of 
200us or more which, I believe, was caused by the thermal stress that 
the system was experiencing after running stressful benchmarks for hours.

At the end, we have to revert your clocksource patches before shipping 
RHEL9 beta last week.

Cheers,
Longman