Message-ID: <20181204133817.GB8714@1wt.eu>
Date:   Tue, 4 Dec 2018 14:38:17 +0100
From:   Willy Tarreau <w@....eu>
To:     Ingo Molnar <mingo@...nel.org>
Cc:     "Paul E. McKenney" <paulmck@...ux.ibm.com>,
        Arnaldo Carvalho de Melo <acme@...radead.org>,
        Linus Torvalds <torvalds@...ux-foundation.org>,
        "H. Peter Anvin" <hpa@...or.com>, linux-kernel@...r.kernel.org,
        danymadden@...ibm.com, Dennis.Krein@...app.com, joe@...ches.com,
        joel@...lfernandes.org, ldr709@...il.com,
        pierceagriffiths@...il.com, rdunlap@...radead.org,
        zhouzhouyi@...il.com, connor.shu@...il.com
Subject: Re: [GIT PULL rcu/next] RCU commits for 4.21/5.0

Hi Ingo,

On Tue, Dec 04, 2018 at 09:08:37AM +0100, Ingo Molnar wrote:
> I noticed this bit from Willy:
> 
> >  tools/testing/selftests/rcutorture/bin/nolibc.h    | 2197 ++++++++++++++++++++
> 
> So <nolibc.h> is a rather large header and it comes with very little 
> documentation - but once you read through the header it's obvious what it 
> does, the code is clean and it's pretty cool all around, and in hindsight 
> the name is a strong hint about what the header does as well. ;)

Thanks for the positive comment, as it was initially not designed to be
merged into the kernel and was just a local home project. I figured it
could be a perfect solution to Paul's executable size issues and offered
some help to get it in relatively quickly, but surely we can do much better!

> Still it would be nice to at least add a top level description to the 
> header to make people (like me) who are reading the code before the 
> changelogs wonder less. For tooling headers we require a similar 
> self-explanatory, feel-fuzzy structure as for kernel headers.

I'm fine with doing this. I even wrote the very small header at the last
minute, without knowing if there was any chance it survives a review :-)

> Beyond adding a bit more documentation it would also be useful to factor 
> nolibc.h out into tools/include/nolibc/ or so, no reason to hide it in 
> rcutorture, I bet there's a number of other testcases and smaller 
> utilities in tools/ that could make good use of it.

Fine as well. It's important however to keep in mind that I only covered
the few architectures I could test (i386/x86_64/arm/arm64/mips), and even
there the coverage is still limited. I don't want it to become too much of
a pain to use for other utilities just by lack of initial coverage. However
I agree that better exposure will help contributions come in.

> My long term hope would be that eventually we could even create a minimal 
> klibc from it (a minimal libc provided by the kernel itself), giving 
> minimalist binaries a mechanism to link against klibc.so:
> 
> - klibc would be an explicit opt-in mechanism, i.e. binaries that are 
>   coupled with the kernel anyway (and initrd executables certainly are) 
>   could use this.

In fact it's very similar to my goal. I'm using it in initramfs and initrds
that do very little stuff and where it's acceptable to have a few #ifdef to
adapt to this or that libc. However I found it extremely convenient *not* to
require any external symbol, thus not to have to link against anything. But
I'm well aware that this position cannot last forever and that at some
point if we want to go further we'll possibly have a few layers (naked
syscalls returning -errno, decorated syscalls making use of an explicit
errno, libc-specific stuff like string functions). Possibly that in this
case only the naked version would remain in the .h and that the rest will
require linking with the .so/.a.

> - We could also add a way for the kernel to provide (non-swappable) 
>   binaries via an automatic /klib/ mount point or so. This would allow 
>   features like a minimal, console based rescue/debug shell that doesn't 
>   rely on any filesystem state or external library dependencies, other 
>   than the initial kernel+initrd image.

This could be convenient indeed, I never thought about this. I'm currently
doing something comparable using initramfs, so maybe in the end we don't
need the kernel to create anything beyond this, but instead just let the
user choose in the configuration what utilities should be added to the
initramfs sources.

(...)
> - klibc would also eventually allow deeper integration with the vDSO 
>   proper: for example on klibc based embedded systems we could link klibc
>   and the vDSO into a single vDSO library, further simplifying and 
>   optimizing it.

I already looked how to implement vDSO. I figured it was not very difficult
but will require that I maintain variables with the AUXV, then I thought
that it went beyond the scope of this minimalist implementaiton and
postponed this.

> - klibc would also allow faster feature propagation from kernel to libc 
>   as well, as we could prototype, test and benchmark new system calls and 
>   new features on klibc - i.e. klibc integration and testcases could be a
>   requirement for new system calls.

This actually is a good idea. There was already a discussion in another
thread about exposing syscalls better in the kernel for better interactions
with the libc, but it could start this way with test cases. It also increases
the likeliness that an awkward API is detected early when the person starts
to write his/her own part of the libc side.

> - There's no upper limit to how such a minimal kernel-shell (root only) 
>   environment would look like, beyond a minimal shell in principle it 
>   could include bits like:
(...)

That's more or less what the preinit present in my initramfs provides. I
just need a kernel and nothing else to start to manually find and mount
my rootfs while debugging, it's quite convenient. It's very limited
though. But I 100% agree with the benefits of such basic recovery
utilities shipped with the kernel images!

(...)
>     - a minimal version of 3D Tetris (just kidding)

I thought you were already kidding when talking about 3D in fact but
apparently not! I think you really mean GPU-based acceleration rather
than 3D since I hardly see how 3D stuff may improve my debugging
abilities :-)

> - ... all of which would allow a fully integrated, self-contained (!) 
>   solution with no external dependencies other than the build environment 
>   to build the binaries, that allows the debugging of a system and the 
>   eventual extraction of debug information, without having to interact 
>   with the local filesystem or even the local X-window state for these 
>   apps to run.

In my case I also ship the modules within the kernel image. It's extremely
convenient to have the choice of a number of kernels for a given rootfs.
You never have to wonder if the modules are present on the roofs or not,
you never have to prepare any initrd, you just select the kernel you want
and you're done. For development, when combined with the preinit I'm
talking about, it's awesome, because you just want something which barely
boots to the preinit prompt, then you can start to investigate.

> - I.e. a minimal Linux distribution done right, optimized for kernel 
>   development, system bootstrap, rescue enviroment, etc. - far more 
>   usable than a kdb session.

The distro I'm using was initially not made for this but to design
reliable minimalist systems, and it turned out extremely effective for
kernel development for these reasons. The whole rootfs is an initrd
which contains all the tools I need, so I can only confirm the comfort
it brings!

> Anyway, back to <nolibc.h>: wanted to ask Linus's and Arnaldo's opinion 
> about the merge of it, we can still re-base and re-try if there's any 
> objections.

I'm fine with this as well. I just want to be sure we don't postpone the
coverage of Paul's rcutorture needs because it started for this initially.
If we see the discussion going too far, we could also at least cover just
rcutorture first which would buy us time to decide how it should be done
better, then remove it once we can port rcutorture to the newly designed
solution.

Thanks,
Willy

Powered by blists - more mailing lists