linux-kernel - Re: [RFC] sched,livepatch: call klp_try_switch_task in __cond

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <3a9bfb4a52b715bd8739d8834409c9549ec7f22f.camel@fb.com>
Date:   Tue, 10 May 2022 13:33:13 +0000
From:   Rik van Riel <riel@...com>
To:     "pmladek@...e.com" <pmladek@...e.com>,
        Song Liu <songliubraving@...com>
CC:     "joe.lawrence@...hat.com" <joe.lawrence@...hat.com>,
        "song@...nel.org" <song@...nel.org>,
        "jpoimboe@...hat.com" <jpoimboe@...hat.com>,
        "peterz@...radead.org" <peterz@...radead.org>,
        "mingo@...hat.com" <mingo@...hat.com>,
        "vincent.guittot@...aro.org" <vincent.guittot@...aro.org>,
        "live-patching@...r.kernel.org" <live-patching@...r.kernel.org>,
        "linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
        Kernel Team <Kernel-team@...com>
Subject: Re: [RFC] sched,livepatch: call klp_try_switch_task in __cond_resched

On Tue, 2022-05-10 at 09:56 +0200, Petr Mladek wrote:
> 
> IMHO, the problem is that klp_transition_work_fn() tries the
> transition "only" once per second, see
> 
> void klp_try_complete_transition(void)
> {
> [...]
>                 schedule_delayed_work(&klp_transition_work,
>                                       round_jiffies_relative(HZ));
> [...]
> }
> 
> It means that there are "only" 60 attempts to migrate the busy
> process.
> It fails when the process is in the running state or sleeping in a
> livepatched function. There is a _non-zero_ chance of a bad luck.
> 

We are definitely hitting that non-zero chance :)

> Anyway, the limit 60s looks like a bad idea to me. It is too low.

That has its own issues, though. System management software
tracks whether kpatch succeeds, and a run of the system
management software will not complete until all of the commands
it has run have completed.

One reason for this is that allowing system management software
to just fork more and more things that might potentially get
stuck is that you never want your system management software
to come even close to resembling a fork bomb :)

Rollout of the next config change to a system should not be
blocked on KLP completion.

I think the best approach for us might be to just track what
is causing the transition failures, and send in trivial patches
to make the outer loop in such kernel threads do the same KLP
transition the idle task already does.