Message-ID: <3663d244-627f-4928-ab2e-0328cd6a8fb1@t-8ch.de>
Date:   Wed, 30 Aug 2023 08:21:51 +0200
From:   Thomas Weißschuh <linux@...ssschuh.net>
To:     Willy Tarreau <w@....eu>
Cc:     Shuah Khan <shuah@...nel.org>, Zhangjin Wu <falcon@...ylab.org>,
        Yuan Tan <tanyuan@...ylab.org>, linux-kernel@...r.kernel.org,
        linux-kselftest@...r.kernel.org
Subject: Re: [PATCH 1/2] tools/nolibc: add stdarg.h header

On 2023-08-29 14:12:45+0200, Willy Tarreau wrote:
> On Tue, Aug 29, 2023 at 12:16:23PM +0200, Thomas Weißschuh wrote:
> > > OK. But then, doesn't it mean that if we don't provide our stdarg.h,
> > > the compilers' will be used ? I'm asking because we're already using
> > > va_list and va_args, for example in vfprintf() in stdio.h, which
> > > precisely includes <stdarg.h> so it must indeed come from the compiler.
> > 
> > It will be used *iff* -nostdinc is *not* passed.
> > 
> > I think we need to clarify the definition of the word "provided".
> > For me it means that the compiler ships an implementation of this header
> > file in the compiler-specific include directory.
> > 
> > If -nostdinc is passed this include directory is not actually usable.
> 
> OK I understand better now. I thought it was always usable.
> 
> > If a user wants to avoid the implicit usage of any system-provided
> > headers they need to pass -nostdinc, as far as I know there is no flag
> > to keep only the compiler-specific include directories.
> 
> So that means we may also have to implement our own stddef.h to move
> size_t there, and limits.h and move *MAX there as well if we want to
> support this. I'm not necessarily against this, it's just that we need
> to be consistent.

We would have to, *iff* the goal is to provide *all* headers in nolibc.
May goal was more limited:
nolibc should be self-contained, it should be able to work at all
with -nostdinc.
If users need more standard headers for their application they can add
those either as shim, custom implementation or from the compiler.

> Also something is puzzling me. If a normal program builds with -nostdinc,
> it means it does *not* want the libc's (nor the compiler's) headers to be
> included, probably because it comes with its own. In this case why would
> we impose ours ? For example, let's consider this tiny code snippet:
> 
>   $ cat arg.c
>   #include <stdarg.h>
>   va_list blah;
> 
>   $ gcc -c arg.c 
>   $ gcc -nostdinc -c arg.c
>   arg.c:1:20: error: no include path in which to search for stdarg.h
>       1 | #include <stdarg.h>
>         |                    ^
>   arg.c:2:1: error: unknown type name 'va_list'
>       2 | va_list blah;
>         | ^~~~~~~
>   arg.c:1:1: note: 'va_list' is defined in header '<stdarg.h>'; did you forget to '#include <stdarg.h>'?
>     +++ |+#include <stdarg.h>
>       1 | #include <stdarg.h>
>  
> You see, that's why I'm finding it confusing that we define headers that
> are supposed *not* to be defined with -nostdinc.

I'm confused.

If the user doesn't want to use nolibc they should not explicitly add it
to the include path.

> I think we need to carefully check what is supposed to be defined and
> what not when -nostdinc is used normally so that we defined what programs
> expect and not what they expect us *not* to define. Recently we've been
> empirically fixing nolibc-test build failures but it's just a test program
> that comes with its own biases. Maybe trying to build some portable libs
> that use very little from a libc (e.g. xxhash, liblzo etc) could give us
> some hints about certain basic assumptions that we do not fulfill.

It makes sense to figure out what is needed by larger projects from a
libc. But it feels to me like a bug vs. feature discussion.

Making larger real-world applications work is a feature while making the
following work is a bugfix:

$ cat nolibc.c
#include "nolibc.h"

int main(void)
{
	return 0;
}

$ gcc -nostdinc -Isysroot/x86/include -c nolibc.c
In file included from sysroot/x86/include/nolibc.h:98,
                 from nolibc-test.c:1:
sysroot/x86/include/sys.h:10:10: fatal error: stdarg.h: No such file or directory
   10 | #include <stdarg.h>
      |          ^~~~~~~~~~

> > One usecase is in nolibc-test itself, where Zhangjin ran into weird
> > and inconsistent behavior of system includes being pulled in.
> > By using -nostdinc we avoid this.
> 
> I see but a normal libc ought not to build with -nostdinc. I mean, we
> can define whatever we want once we know why we're doing it, but I think
> that as long as we find it confusing between those how are modifying this
> code, it will be very difficult to explain correctly to users. We're
> definitely missing some design rules I think. Maybe -nostdinc should be
> needed only when using -include nolibc.h for example, I don't know, but
> I still find that we're papering over a wider problem.
> 
> > I can also see this being useful for normal users.
> 
> I agree, that's also my concern actually.
> 
> > > > I could not find anybody doing this differently.
> > > > Using builtins seems to me to be the normal way to expose compiler
> > > > implementation specifics.
> > > 
> > > OK but it's already what the compiler does itself in its own stdarg that
> > > is provided. That's why I don't understand what specific case we're trying
> > > to cover here, I feel like we're providing an alternate stdarg in case the
> > > compiler doesn't provide one except that I've not seen a compiler not
> > > provide it (even tcc comes with it), it's like stddef.
> > 
> > It's all about supporting -nostdinc.
> 
> But unless I'm mistaken (and my example above seems to support this),
> a normal libc doesn't build with -nostdinc. That's the point I'd like
> us to clarify.

musl:

$ cat /usr/lib/musl/lib/musl-gcc.specs
...
*cc1:
%(cc1_cpu) -nostdinc -isystem /usr/lib/musl/include -isystem include%s
...


dietlibc:

$ cat Makefile
...
DEFAULTCFLAGS=-pipe -nostdinc -D_REENTRANT $(EXTRACFLAGS)
...


klibc re-adds the compilers include path,
This is an alternative we could also use:

$ cat Makefile
...
NOSTDINC_FLAGS := -nostdlib -nostdinc -isystem $(shell $(CC) -print-file-name=include)
...

(these are all I checked)

> > FYI stdint.h is also provided by nolibc, gcc and glibc.
> 
> True but that one didn't surprise me because it came with C99 and was
> usually shipped by the libc when compilers targetting previous versions
> were used, so I didn't see this as a replacement for the compiler's
> definition actually.
> 
> I don't know what dictates what goes in the compiler and what in the
> libc.  I'm fine with having to redefine everything that's missing if
> that's needed, but as indicated above, stddef.h and limits.h are
> missing despite being quite common.

I think it's not really clearly defined what goes where.

There was also a longer discussion on LKML about linux/stdarg.h [0]

The gcc authors argue that Linux should not ship a custom stdarg.h.
But in reality Linux, musl, dietlibc (and probably some more) today are
shipping their own stdarg.h.

> We have an interesting comment at the top of nolibc.h which says:
> 
>  * The available standard (but limited) include files are:
>  *   ctype.h, errno.h, signal.h, stdio.h, stdlib.h, string.h, time.h

This is out of date. It's missing signal.h, stdint.h, unistd.h.

>  *
>  * In addition, the following ones are expected to be provided by the compiler:
>  *   float.h, stdarg.h, stddef.h

What does "expected" mean here?
nolibc itself is perfectly fine without float.h and stddef.h.

>  *
>  * The following ones which are part to the C standard are not provided:
>  *   assert.h, locale.h, math.h, setjmp.h, limits.h

While true, a lot of other headers are also not provided.

> I think I draw the line based on what my compilers have always provided.
> That's definitely something we can redefine (and update the comment),
> I'm just seeking consistency, and I think you can understand :-/

I do understand.

To reiterate it here explicitly, in my opinion it's a worthwhile and
consistent goal to make "nolibc usable standalone with -nostdinc" for
maximal control by the user.

If not, I'd like to use the "-nostdinc -I$(cc -print-file-name=include)"
method to avoid dependencies on system header for nolibc-test
specifically.

Thomas

[0] https://lore.kernel.org/lkml/CAHk-=wgoX0pVqNMMOcrhq=nuOfoZB_3qihyHB3y1S8qo=MDs6w@mail.gmail.com/

Powered by blists - more mailing lists