linux-kernel - Re: [PATCH V4 1/3] livepatch: remove 'struct completion finish' from klp

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:   Wed, 10 Nov 2021 14:57:11 +0100
From:   Petr Mladek <pmladek@...e.com>
To:     Ming Lei <ming.lei@...hat.com>
Cc:     Josh Poimboeuf <jpoimboe@...hat.com>,
        Jiri Kosina <jikos@...nel.org>,
        Miroslav Benes <mbenes@...e.cz>, live-patching@...r.kernel.org,
        linux-kernel@...r.kernel.org,
        Greg Kroah-Hartman <gregkh@...uxfoundation.org>,
        Luis Chamberlain <mcgrof@...nel.org>,
        Joe Lawrence <joe.lawrence@...hat.com>
Subject: Re: [PATCH V4 1/3] livepatch: remove 'struct completion finish' from
 klp_patch

On Fri 2021-11-05 20:04:03, Ming Lei wrote:
> On Wed, Nov 03, 2021 at 01:52:29PM +0100, Petr Mladek wrote:
> > On Wed 2021-11-03 08:51:32, Ming Lei wrote:
> > > On Tue, Nov 02, 2021 at 04:56:10PM +0100, Petr Mladek wrote:
> > I see that you are aware of this behavior.
> > 
> > > >    kobject_put()
> > > >      kobject_release()
> > > >        INIT_DELAYED_WORK(&kobj->release, kobject_delayed_cleanup);
> > > >        schedule_delayed_work(&kobj->release, delay);
> > > > 
> > > >    asynchronously:
> > > > 
> > > >      kobject_delayed_cleanup()
> > > >       kobject_cleanup()
> > > > 	__kobject_del()
> > > 
> > > OK, this is one generic kobject release vs. module unloading issue to
> > > solve, not unique for klp module, and there should be lots of drivers
> > > suffering from it.
> > 
> > Yup, the problem is generic. It would be nice to have a generic
> > solution. For example, add kobject_release_sync() that would return
> > only when the object is really released.
> 
> The generic solution has been posted out:
> 
> https://lore.kernel.org/lkml/20211105063710.4092936-1-ming.lei@redhat.com/
> 
> which needn't any generic API change, just flushes all scheduled kobject
> cleanup work before freeing module, and the change is transparent for
> drivers.

No, it is not enough. The proposed "generic solution" solves only
the problem introduced by CONFIG_DEBUG_KOBJECT_RELEASE. It does not
prevent unloading the module from another reasons. I mean that it
does not help when the last reference is not released in time
or never from some reasons.


> IMO, kobject_release_sync() is one wrong direction for fixing this
> issue, since it is basically impossible to audit if one kobject_put()
> need to be replaced with kobject_release_sync().
> 
> > 
> > > > > --- a/kernel/livepatch/core.c
> > > > > +++ b/kernel/livepatch/core.c
> > > > > @@ -678,11 +678,6 @@ static void klp_free_patch_finish(struct klp_patch *patch)
> > > > >  	 * cannot get enabled again.
> > > > >  	 */
> > > > >  	kobject_put(&patch->kobj);
> > > > > -	wait_for_completion(&patch->finish);
> > > > > -
> > > > > -	/* Put the module after the last access to struct klp_patch. */
> > > > > -	if (!patch->forced)
> > > > > -		module_put(patch->mod);
> > > > 
> > > > klp_free_patch_finish() does not longer wait until the release
> > > > callbacks are called.
> > > > 
> > > > klp_free_patch_finish() is called also in klp_enable_patch() error
> > > > path.
> > > > 
> > > > klp_enable_patch() is called in module_init(). For example, see
> > > > samples/livepatch/livepatch-sample.c
> > > > 
> > > > The module must not get removed until the release callbacks are called.
> > > > Does the module loader check the module reference counter when
> > > > module_init() fails?
> > > 
> > > Good catch, that is really one corner case, in which the kobject has to
> > > be cleaned up before returning from mod->init(), cause there can't be
> > > module unloading in case of mod->init() failure.
> > 
> > Just to be sure. We want to keep the safe behavior in this case.
> > There are many situations when klp_enable() might fail. And the error
> > handling must be safe.
> > 
> > In general, livepatch developers are very conservative.
> > Livepatches are not easy to create. They are used only by people
> > who really want to avoid reboot. We want to keep the livepatch kernel
> > framework as safe as possible to avoid any potential damage.
> 
> The posted patch can cover this situation in which module_init() fails.

No, it works only when the last reference was dropped, the delayed
release queued, and the kobject added into kobj_cleanup_list.

The current livepatch code is much more safe. klp_free_patch_finish()
waits for the completion. It will allows to remove the module only
when the kobject was really released. It will block the module
removal as long as needed, even forever, if there is a bug somewhere.

If we use wait_for_completion_timeout() in a while cycle, we could
even report when it takes suspiciously long and something likely
went wrong. This is one way to "always" catch and report the problem.

I am not sure if similar approach is usable for device drivers. But I
had this in mind when I proposed the kobject_release_sync() API.

Best Regards,
Petr