Message-Id: <1504191233-2642-1-git-send-email-joe.lawrence@redhat.com>
Date:   Thu, 31 Aug 2017 10:53:50 -0400
From:   Joe Lawrence <joe.lawrence@...hat.com>
To:     live-patching@...r.kernel.org, linux-kernel@...r.kernel.org
Cc:     Josh Poimboeuf <jpoimboe@...hat.com>, Jessica Yu <jeyu@...nel.org>,
        Jiri Kosina <jikos@...nel.org>,
        Miroslav Benes <mbenes@...e.cz>,
        Petr Mladek <pmladek@...e.com>,
        Chris J Arges <chris.j.arges@...onical.com>
Subject: [PATCH v5 0/3] livepatch callbacks

v5:

- Move code comments from kernel/livepatch/core.h to
  include/linux/livepatch.h's struct klp_callbacks definition

- Change int klp_object.prepatch_callback_status to a bool
  klp_object.callbacks_enabled

  - fixes a hole where returned status=0 wasn't distinguished from
    "never-ran" status=0

- Removed a few extraneous checks in the execute callback routines

- In the module-coming case, be sure to execute the post-unpatch
  callback in the event that patching fails

- Moved my Signed-off-by line to the bottom in patches 2 and 3


Testing
=======

All test cases in the documentation match the previous patchset version,
except test6.  In this patchset version, we make sure that post-unpatch
callbacks only execute for those klp_objects who's pre-patch callbacks
ran (successfully), or would have run but omitted the optional callback.

For test6, this specifically means that the livepatch_callbacks_mod's
post-unpatch callback should *NOT* run:

  diff -Nupr v4_test/test6.out v5_test/test6.out
  --- v4_test/test6.out	2017-08-30 14:09:03.852440826 -0400
  +++ v5_test/test6.out	2017-08-30 14:59:33.967460088 -0400
  @@ -6,7 +6,6 @@ livepatch: enabling patch 'livepatch_cal
   livepatch_callbacks_demo: pre_patch_callback: vmlinux
   livepatch: pre-patch callback failed for object 'vmlinux'
   livepatch: failed to enable patch 'livepatch_callbacks_demo'
  -livepatch_callbacks_demo: post_unpatch_callback: livepatch_callbacks_mod -> [MODULE_STATE_LIVE] Normal state
   livepatch: 'livepatch_callbacks_demo': unpatching complete
   insmod: ERROR: could not insert module samples/livepatch/livepatch-callbacks-demo.ko: No such device
   % rmmod samples/livepatch/livepatch-callbacks-mod.ko


Two additional tests were run to verify the behavior if klp_object
patching fails.  (This required an instrumented kernel, so I didn't
bother including them in the documentation file):


Test 10
-------

Verify that when __klp_enable_patch() calls klp_patch_object() and
fails, that post-unpatch callbacks are properly executed.

- load busy target module (0s sleep)
- load livepatch (forced patching error)
- unload busy target module

Load the livepatch a target module first:

  % insmod samples/livepatch/livepatch-callbacks-busymod.ko sleep_secs=0
  [ 1494.116424] livepatch_callbacks_busymod: livepatch_callbacks_mod_init
  [ 1494.117527] livepatch_callbacks_busymod: busymod_work_func, sleeping 0 seconds ...
  [ 1494.121022] livepatch_callbacks_busymod: busymod_work_func exit

Then bring in a livepatch which will attempt (and fail) to patch the
target module.  Notice both vmlinux and livepatch_callbacks_busymod's
pre-patch callbacks and post-patch callbacks are executed:

  % insmod samples/livepatch/livepatch-callbacks-demo.ko 
  [ 1496.124475] livepatch: enabling patch 'livepatch_callbacks_demo'
  [ 1496.125155] livepatch_callbacks_demo: pre_patch_callback: vmlinux
  [ 1496.125784] livepatch_callbacks_demo: pre_patch_callback: livepatch_callbacks_busymod -> [MODULE_STATE_LIVE] Normal state
  [ 1496.131229] livepatch: *** forcing klp_patch_object() return of -EINVAL ***
  [ 1496.135648] livepatch: failed to patch object 'livepatch_callbacks_busymod'
  [ 1496.136279] livepatch: failed to enable patch 'livepatch_callbacks_demo'
  [ 1496.136859] livepatch_callbacks_demo: post_unpatch_callback: vmlinux
  [ 1496.137390] livepatch_callbacks_demo: post_unpatch_callback: livepatch_callbacks_busymod -> [MODULE_STATE_LIVE] Normal state
  [ 1496.138353] livepatch: 'livepatch_callbacks_demo': unpatching complete
  [ 1496.154692] insmod: ERROR: could not insert module samples/livepatch/livepatch-callbacks-demo.ko: Invalid parameters

  % rmmod samples/livepatch/livepatch-callbacks-busymod.ko
  [ 1498.161786] livepatch_callbacks_busymod: livepatch_callbacks_mod_exit


Test 11
-------

Verify that when klp_module_coming() calls klp_patch_object() and fails,
that post-unpatch callbacks are properly executed.

- load livepatch
- load busy target module (0s sleep) (forced patching error)
- disable livepatch
- unload livepatch

First load the livepatch (nothing to patch in vmlinux), pre and
post-patch vmlinux callbacks run:

  % insmod samples/livepatch/livepatch-callbacks-demo.ko 
  [ 1500.183685] livepatch: enabling patch 'livepatch_callbacks_demo'
  [ 1500.184703] livepatch_callbacks_demo: pre_patch_callback: vmlinux
  [ 1500.185538] livepatch: 'livepatch_callbacks_demo': starting patching transition
  [ 1501.727230] livepatch_callbacks_demo: post_patch_callback: vmlinux
  [ 1501.728336] livepatch: 'livepatch_callbacks_demo': patching complete

Then load a target module which the livepatch will attempt (and fail) to
patch. The livepatch_callbacks_busymod pre-patch callback runs, but then
klp_object patchingn fails.  It's post-unpatch callback is immediately
executed.

  % insmod samples/livepatch/livepatch-callbacks-busymod.ko sleep_secs=0
  [ 1502.191383] livepatch: applying patch 'livepatch_callbacks_demo' to loading module 'livepatch_callbacks_busymod'
  [ 1502.192664] livepatch_callbacks_demo: pre_patch_callback: livepatch_callbacks_busymod -> [MODULE_STATE_COMING] Full formed, running module_init
  [ 1502.197923] livepatch: *** forcing klp_patch_object() return of -EINVAL ***
  [ 1502.202389] livepatch: failed to apply patch 'livepatch_callbacks_demo' to module 'livepatch_callbacks_busymod' (-22)
  [ 1502.203727] livepatch_callbacks_demo: post_unpatch_callback: livepatch_callbacks_busymod -> [MODULE_STATE_COMING] Full formed, running module_init
  [ 1502.205356] livepatch: patch 'livepatch_callbacks_demo' failed for module 'livepatch_callbacks_busymod', refusing to load module 'livepatch_callbacks_busymod'
  [ 1502.222539] insmod: ERROR: could not insert module samples/livepatch/livepatch-callbacks-busymod.ko: Invalid parameters

When the livepatch is later disabled, the remaining klp_objects that are
loaded (vmlinux) will run their pre and post-unpatch callbacks:

  % echo 0 > /sys/kernel/livepatch/livepatch_callbacks_demo/enabled
  [ 1504.226238] livepatch_callbacks_demo: pre_unpatch_callback: vmlinux
  [ 1504.226836] livepatch: 'livepatch_callbacks_demo': starting unpatching transition
  [ 1505.759301] livepatch_callbacks_demo: post_unpatch_callback: vmlinux
  [ 1505.760111] livepatch: 'livepatch_callbacks_demo': unpatching complete

  % rmmod samples/livepatch/livepatch-callbacks-demo.ko


An additional test was run (modifying the sample livepatch module, so
again not included in the Documentation), where the livepatch did not
include a pre-patch callback, but did specify all the other callbacks.
This is a scenario Josh pointed out that v4's
klp_object.prepatch_callback_status did not fully cover.


Test 12
-------

- load target module
- load livepatch
- disable livepatch
- unload target module
- unload livepatch

  % insmod samples/livepatch/livepatch-callbacks-mod.ko 
  [   92.874499] livepatch_callbacks_mod: module verification failed: signature and/or required key missing - tainting kernel
  [   92.875802] livepatch_callbacks_mod: livepatch_callbacks_mod_init

Notice when the livepatch module is loaded, no pre-patch callbacks run.
All other callbacks do execute later though:

  % insmod samples/livepatch/livepatch-callbacks-demo.ko 
  [   94.958399] livepatch_callbacks_demo: tainting kernel with TAINT_LIVEPATCH
  [   94.960493] livepatch: enabling patch 'livepatch_callbacks_demo'
  [   94.961183] livepatch: 'livepatch_callbacks_demo': starting patching transition
  [   96.736263] livepatch_callbacks_demo: post_patch_callback: vmlinux
  [   96.736903] livepatch_callbacks_demo: post_patch_callback: livepatch_callbacks_mod -> [MODULE_STATE_LIVE] Normal state
  [   96.737819] livepatch: 'livepatch_callbacks_demo': patching complete

  % echo 0 > /sys/kernel/livepatch/livepatch_callbacks_demo/enabled
  [   96.965430] livepatch_callbacks_demo: pre_unpatch_callback: vmlinux
  [   96.966131] livepatch_callbacks_demo: pre_unpatch_callback: livepatch_callbacks_mod -> [MODULE_STATE_LIVE] Normal state
  [   96.967253] livepatch: 'livepatch_callbacks_demo': starting unpatching transition
  [   98.720218] livepatch_callbacks_demo: post_unpatch_callback: vmlinux
  [   98.721134] livepatch_callbacks_demo: post_unpatch_callback: livepatch_callbacks_mod -> [MODULE_STATE_LIVE] Normal state
  [   98.722217] livepatch: 'livepatch_callbacks_demo': unpatching complete

  % rmmod samples/livepatch/livepatch-callbacks-demo.ko
  % rmmod samples/livepatch/livepatch-callbacks-mod.ko
  [  100.986947] livepatch_callbacks_mod: livepatch_callbacks_mod_exit

-- Joe

Joe Lawrence (3):
  livepatch: add (un)patch callbacks
  livepatch: move transition "complete" notice into
    klp_complete_transition()
  livepatch: add transition notices

 Documentation/livepatch/callbacks.txt           | 594 ++++++++++++++++++++++++
 include/linux/livepatch.h                       |  25 +
 kernel/livepatch/core.c                         |  55 ++-
 kernel/livepatch/core.h                         |  37 ++
 kernel/livepatch/patch.c                        |   1 +
 kernel/livepatch/transition.c                   |  45 +-
 samples/livepatch/Makefile                      |   3 +
 samples/livepatch/livepatch-callbacks-busymod.c |  72 +++
 samples/livepatch/livepatch-callbacks-demo.c    | 234 ++++++++++
 samples/livepatch/livepatch-callbacks-mod.c     |  55 +++
 10 files changed, 1104 insertions(+), 17 deletions(-)
 create mode 100644 Documentation/livepatch/callbacks.txt
 create mode 100644 samples/livepatch/livepatch-callbacks-busymod.c
 create mode 100644 samples/livepatch/livepatch-callbacks-demo.c
 create mode 100644 samples/livepatch/livepatch-callbacks-mod.c

-- 
1.8.3.1

Powered by blists - more mailing lists