linux-kernel - Re: [PATCH v3 0/4] Extending NMI watchdog during LPM

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:   Tue, 12 Jul 2022 11:32:07 +0200
From:   Laurent Dufour <ldufour@...ux.ibm.com>
To:     Nicholas Piggin <npiggin@...il.com>, benh@...nel.crashing.org,
        haren@...ux.vnet.ibm.com, linux@...ck-us.net, mpe@...erman.id.au,
        nathanl@...ux.ibm.com, paulus@...ba.org, wim@...ux-watchdog.org
Cc:     linux-kernel@...r.kernel.org, linuxppc-dev@...ts.ozlabs.org,
        linux-watchdog@...r.kernel.org
Subject: Re: [PATCH v3 0/4] Extending NMI watchdog during LPM

Le 12/07/2022 à 03:21, Nicholas Piggin a écrit :
> Excerpts from Laurent Dufour's message of June 27, 2022 11:53 pm:
>> When a partition is transferred, once it arrives at the destination node,
>> the partition is active but much of its memory must be transferred from the
>> start node.
>>
>> It depends on the activity in the partition, but the more CPU the partition
>> has, the more memory to be transferred is likely to be. This causes latency
>> when accessing pages that need to be transferred, and often, for large
>> partitions, it triggers the NMI watchdog.
> 
> Importantly, it can require page in of code that runs with irqs 
> disabled, which is unlike a guest normally runs under PowerVM
> (but it can under KVM) which is why we enabled the watchdog under
> PowerVM but not KVM. So, okay it makes sense to mak an exception
> for this case.

On PowerVM, irqs disabled code may trigger multiple page in of data, which
is generating latencies too. Furthermore, the hypervisor is currently not
prioritizing page in operation based on the faulting CPU state. So the
kernel may have to wait for user page in operations to last.

> Thanks,
> Nick
> 
>> The NMI watchdog causes the CPU stack to dump where it appears to be
>> stuck. In this case, it does not bring much information since it can happen
>> during any memory access of the kernel.
>>
>> In addition, the NMI interrupt mechanism is not secure and can generate a
>> dump system in the event that the interruption is taken while MSR[RI]=0.
>>
>> Depending on the LPAR size and load, it may be interesting to extend the
>> NMI watchdog timer during the LPM.
>>
>> That's configurable through sysctl with the new introduced variable
>> (specific to powerpc) nmi_watchdog_factor. This value represents the
>> percentage added to watchdog_tresh to set the NMI watchdog timeout during a
>> LPM.
>>
>> Changes in v3:
>>  - don't export watchdog_mutex
>>  - fix a comment in mobilty.c, wait_for_vasi_session_completed()
>>  - fix a build issue when !CONFIG_PPC_WATCHDOG
>>  - rework some printk and rename the sysctl variable.
>>
>> v2:
>> https://lore.kernel.org/all/20220614135414.37746-1-ldufour@linux.ibm.com/
>>
>> Laurent Dufour (4):
>>   powerpc/mobility: wait for memory transfer to complete
>>   watchdog: export lockup_detector_reconfigure
>>   powerpc/watchdog: introduce a NMI watchdog's factor
>>   pseries/mobility: set NMI watchdog factor during LPM
>>
>>  Documentation/admin-guide/sysctl/kernel.rst | 12 +++
>>  arch/powerpc/include/asm/nmi.h              |  2 +
>>  arch/powerpc/kernel/watchdog.c              | 21 ++++-
>>  arch/powerpc/platforms/pseries/mobility.c   | 85 ++++++++++++++++++++-
>>  include/linux/nmi.h                         |  2 +
>>  kernel/watchdog.c                           | 21 +++--
>>  6 files changed, 135 insertions(+), 8 deletions(-)
>>
>> -- 
>> 2.36.1
>>
>>