linux-kernel - Re: [PATCH 0/5] cleanups, fixes, and progress towards avoiding "make headers"

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <CABi2SkXN1jsnrawCwADxWB0Whj+e-m7+9c=5Lo3gY5CYVrnpkQ@mail.gmail.com>
Date: Tue, 11 Jun 2024 07:13:44 -0700
From: Jeff Xu <jeffxu@...omium.org>
To: John Hubbard <jhubbard@...dia.com>
Cc: Andrew Morton <akpm@...ux-foundation.org>, Shuah Khan <shuah@...nel.org>, 
	Andrei Vagin <avagin@...gle.com>, Axel Rasmussen <axelrasmussen@...gle.com>, 
	Christian Brauner <brauner@...nel.org>, David Hildenbrand <david@...hat.com>, Kees Cook <kees@...nel.org>, 
	Kent Overstreet <kent.overstreet@...ux.dev>, "Liam R . Howlett" <Liam.Howlett@...cle.com>, 
	Muhammad Usama Anjum <usama.anjum@...labora.com>, Peter Xu <peterx@...hat.com>, 
	Rich Felker <dalias@...c.org>, linux-mm@...ck.org, linux-kselftest@...r.kernel.org, 
	LKML <linux-kernel@...r.kernel.org>
Subject: Re: [PATCH 0/5] cleanups, fixes, and progress towards avoiding "make headers"

On Mon, Jun 10, 2024 at 11:26 PM John Hubbard <jhubbard@...dia.com> wrote:
>
> On 6/10/24 9:45 PM, Jeff Xu wrote:
> > On Mon, Jun 10, 2024 at 9:34 PM John Hubbard <jhubbard@...dia.com> wrote:
> >> On 6/10/24 9:21 PM, Jeff Xu wrote:
> >>> Hi
> >>>
> >>>
> >>> On Fri, Jun 7, 2024 at 7:10 PM John Hubbard <jhubbard@...dia.com> wrote:
> >>>>
> >>>> Eventually, once the build succeeds on a sufficiently old distro, the
> >>>> idea is to delete $(KHDR_INCLUDES) from the selftests/mm build, and then
> >>>> after that, from selftests/lib.mk and all of the other selftest builds.
> >>>>
> >>>> For now, this series merely achieves a clean build of selftests/mm on a
> >>>> not-so-old distro: Ubuntu 23.04:
> >>>>
> >>>> 1. Add __NR_mseal.
> >>>>
> >>>> 2. Add fs.h, taken as usual from a snapshot of ./usr/include/linux/fs.h
> >>>> after running "make headers". This is how we have agreed to do this sort
> >>>> of thing, see [1].
> >>>>
> >>> What is the "official" way to build selftests/mm ?
> >>
> >>   From Documentation/dev-tools/kselftest.rst, it is:
> >>
> >>     $ make headers
> >>     $ make -C tools/testing/selftests
> >>
> >>> I tried a few ways, but it never worked, i.e. due to head missing.
> >>
> >> You are correct. Today's rules require "make headers" first. But
> >> I'm working on getting rid of that requirement, because it causes
> >> problems for some people and situations.
> >>
> >> (Even worse is the follow-up rule, in today's documentation,
> >> that tells us to *run* the selftests from within Make! This
> >> is just madness.
> >
> > That is hilarious! :)
>
> :)
>
> >
> >>   Because the tests need to run as root in
> >> many cases. And Make will try to rebuild if necessary...thus
> >> filling your tree full of root-owned files...but that's for
> >> another time.)
> >>
> >>>
> >>> 1>
> >>> cd tools/testing/selftests/mm
> >>> make
> >>>
> >>> migration.c:10:10: fatal error: numa.h: No such file or directory
> >>>      10 | #include <numa.h>
> >>>         |          ^~~~~~~~
> >>> compilation terminated.
> >>>
> >>> 2>
> >>> make headers
> >>> make -C tools/testing/selftests
> >>>
> >>> make[1]: Entering directory
> >>> '/usr/local/google/home/jeffxu/mm/tools/testing/selftests/mm'
> >>>     CC       migration
> >>> migration.c:10:10: fatal error: numa.h: No such file or directory
> >>>      10 | #include <numa.h>
> >>>
> >>
> >> Well, actually, for these, one should install libnuma-dev and
> >> numactl (those are Ubuntu package names. Arch Linux would be:
> >> numactl).
> >>
> >> I think. The idea is: use system headers if they are there, and
> >> local kernel tree header files if the items are so new that they
> >> haven't made it to $OLDEST_DISTO_REASONABLE.
> >>
> >> Something like that.
> >>
> > But I don't want to install random packages if possible.
>
> Well...keep in mind that it's not really random. If a test program
> requires numa.h, it's typically because it also links against libnuma,
> which *must* be supplied by the distro if you want to build. Because
> it doesn't come with the kernel, of course.
>
Agreed.

> So what you're really saying is that you'd like to build and run
> whatever you can at the moment--the build should soldier on past
> failures as much as possible.
>
Yes. That is what I meant. It would be a convenient feature.

> >
> > Can makefile rule continue to the next target in case of failure though ?
>
> That could be done, in general. The question is if that's really what
> we want, or should want. Maybe...
>
> > right now it stopped  at migration.c , if it continues to the next target, then
> > I don't  need to use gcc to manually build mseal_test.
>
> Let me take a peek at it in the morning.
>
>
>
> thanks,
> --
> John Hubbard
> NVIDIA
>