Date:   Thu,  8 Dec 2016 10:47:51 -0800
From:   "Luis R. Rodriguez" <mcgrof@...nel.org>
To:     shuah@...nel.org, jeyu@...hat.com, rusty@...tcorp.com.au,
        ebiederm@...ssion.com, dmitry.torokhov@...il.com, acme@...hat.com,
        corbet@....net
Cc:     martin.wilck@...e.com, mmarek@...e.com, pmladek@...e.com,
        hare@...e.com, rwright@....com, jeffm@...e.com, DSterba@...e.com,
        fdmanana@...e.com, neilb@...e.com, rgoldwyn@...e.com,
        subashab@...eaurora.org, xypron.glpk@....de, keescook@...omium.org,
        atomlin@...hat.com, mbenes@...e.cz, paulmck@...ux.vnet.ibm.com,
        dan.j.williams@...el.com, jpoimboe@...hat.com, davem@...emloft.net,
        mingo@...hat.com, akpm@...ux-foundation.org,
        torvalds@...ux-foundation.org, linux-kselftest@...r.kernel.org,
        linux-doc@...r.kernel.org, linux-kernel@...r.kernel.org,
        "Luis R. Rodriguez" <mcgrof@...nel.org>
Subject: [RFC 00/10] kmod: stress test driver, few fixes and enhancements

Upon running into an old kmod v19 issue with mount (get_fs_type()) a few of us
hunted for the cause of the issue. Although the issue ended up being a
userspace issue, a stress test driver was written to help reproduce the issue,
and along the way a few other fixes and sanity checks were implemented.

I've taken the time to generalize the stress test driver as a kselftest driver
with a 9 test cases. The last two test cases reveal an existing issue which
is not yet addressed upstream, even if you have kmod v19 present. A fix is
proposed in the last patch. Orignally we had discarded this patch as too
complex due to the alias handling, but upon further analysis of test cases
and memory pressure issues, it seems worth considering. Other than the
last patch I don't think much of the other patches are controversial, but
sending as RFC first just in case.

If its not clear, an end goal here is to make module loading a bit more
deterministic with stronger sanity checks and stress tests. Please note,
the stress test diver requires 4 GiB of RAM to run all tests without running
out of memory. A lot of this has to do with the memory requirements needed
for a dynamic test for multiple threads, but note that the final memory
pressure and OOMs actually don't come from this allocation, but instead
from many finit_module() calls, this consumes quite a bit of memory, specially
if you have a lot of dependencies which also need to be loaded prior to
your needed module -- as is the case for filesystem drivers.

These patches are available on my linux-next git-tree on my branch
20161208-kmod-test-driver-try2 [0], which is based on linux-next tag
next-20161208. Patches are also available based on v4.9-rc8 [1] for
those looking for a bit more stable tree given x86_64 on linux-next is
hosed at the moment.

Since kmod.c doesn't seem to get much love, and since I've been digging
quite a bit into it for other users (firmware) I suppose I could volunteer
myself to maintain this code as well, unless there are oppositions to this.

[0] https://git.kernel.org/cgit/linux/kernel/git/mcgrof/linux-next.git/log/?h=20161208-kmod-test-driver-try2
[1] https://git.kernel.org/cgit/linux/kernel/git/mcgrof/linux.git/log/?h=20161208-kmod-test-driver

Luis R. Rodriguez (10):
  kmod: add test driver to stress test the module loader
  module: fix memory leak on early load_module() failures
  kmod: add dynamic max concurrent thread count
  kmod: provide wrappers for kmod_concurrent inc/dec
  kmod: return -EBUSY if modprobe limit is reached
  kmod: provide sanity check on kmod_concurrent access
  kmod: use simplified rate limit printk
  sysctl: add support for unsigned int properly
  kmod: add helpers for getting kmod count and limit
  kmod: add a sanity check on module loading

 Documentation/admin-guide/kernel-parameters.txt |    7 +
 include/linux/kmod.h                            |    9 +
 include/linux/sysctl.h                          |    3 +
 init/Kconfig                                    |   23 +
 init/main.c                                     |    1 +
 kernel/kmod.c                                   |  244 ++++-
 kernel/module.c                                 |   12 +-
 kernel/sysctl.c                                 |  198 +++-
 lib/Kconfig.debug                               |   25 +
 lib/Makefile                                    |    1 +
 lib/test_kmod.c                                 | 1248 +++++++++++++++++++++++
 tools/testing/selftests/kmod/Makefile           |   11 +
 tools/testing/selftests/kmod/config             |    7 +
 tools/testing/selftests/kmod/kmod.sh            |  448 ++++++++
 14 files changed, 2199 insertions(+), 38 deletions(-)
 create mode 100644 lib/test_kmod.c
 create mode 100644 tools/testing/selftests/kmod/Makefile
 create mode 100644 tools/testing/selftests/kmod/config
 create mode 100755 tools/testing/selftests/kmod/kmod.sh

-- 
2.10.1

Powered by blists - more mailing lists