linux-kernel - Re: [TREE] "Fast Kernel Headers" Tree -v3

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [day] [month] [year] [list]
Message-ID: <db8f3d12-23d5-498b-ab09-61eede907575@linux.com>
Date:   Sat, 4 Nov 2023 09:07:44 +0000
From:   Lucas Tanure <tanure@...ux.com>
To:     Ingo Molnar <mingo@...nel.org>
Cc:     Kari Argillander <kari.argillander@...rgateuniverse.net>,
        Linux Kernel Mailing List <linux-kernel@...r.kernel.org>,
        linux-arch@...r.kernel.org,
        Linus Torvalds <torvalds@...ux-foundation.org>,
        Greg Kroah-Hartman <gregkh@...uxfoundation.org>,
        Randy Dunlap <rdunlap@...radead.org>,
        Nathan Chancellor <nathan@...nel.org>,
        Andrew Morton <akpm@...ux-foundation.org>,
        Peter Zijlstra <peterz@...radead.org>,
        Ard Biesheuvel <ardb@...nel.org>,
        Josh Poimboeuf <jpoimboe@...hat.com>,
        Jonathan Corbet <corbet@....net>,
        Kari Argillander <kari.argillander@...il.com>
Subject: Re: [TREE] "Fast Kernel Headers" Tree -v3

On 22-03-2022 19:03, Kari Argillander wrote:
> 22.03.2022 18.22 Kari Argillander (kari.argillander@...il.com) wrote:
>>
>> 22.03.2022 17.37 Randy Dunlap (rdunlap@...radead.org) wrote:
>>>
>>> Hi Kari,
>>>
>>> On 3/22/22 00:59, Kari Argillander wrote:
>>>> 15.3.2022 12.35 Ingo Molnar (mingo@...nel.org) wrote:
>>>>>
>>>>> This is -v3 of the "Fast Kernel Headers" tree, which is an ongoing rework
>>>>> of the Linux kernel's header hierarchy & header dependencies, with the dual
>>>>> goals of:
>>>>>
>>>>>   - speeding up the kernel build (both absolute and incremental build times)
>>>>>
>>>>>   - decoupling subsystem type & API definitions from each other
>>>>>
>>>>> The fast-headers tree consists of over 25 sub-trees internally, spanning
>>>>> over 2,300 commits, which can be found at:
>>>>>
>>>>>      git://git.kernel.org/pub/scm/linux/kernel/git/mingo/tip.git master
>>>>
>>>> I have had problems to build master branch (defconfig) with gcc9
>>>>      gcc (Ubuntu 9.4.0-1ubuntu1~20.04) 9.4.0
>>>>
>>>> I did also test v2 and problems where there too. I have no problem with gcc10 or
>>>> Clang11. Error I get is:
>>>>
>>>> In file included from ./include/linux/rcuwait_api.h:7,
>>>>                   from ./include/linux/rcuwait.h:6,
>>>>                   from ./include/linux/irq_work.h:7,
>>>>                   from ./include/linux/perf_event_types.h:44,
>>>>                   from ./include/linux/perf_event_api.h:17,
>>>>                   from arch/x86/kernel/kprobes/opt.c:8:
>>>> ./include/linux/rcuwait_api.h: In function ‘rcuwait_active’:
>>>> ./include/linux/rcupdate.h:328:9: error: dereferencing pointer to
>>>> incomplete type ‘struct task_struct’
>>>>    328 |  typeof(*p) *local = (typeof(*p) *__force)READ_ONCE(p); \
>>>>        |         ^
>>>> ./include/linux/rcupdate.h:439:31: note: in expansion of macro
>>>> ‘__rcu_access_pointer’
>>>>    439 | #define rcu_access_pointer(p) __rcu_access_pointer((p),
>>>> __UNIQUE_ID(rcu), __rcu)
>>>>        |                               ^~~~~~~~~~~~~~~~~~~~
>>>> ./include/linux/rcuwait_api.h:15:11: note: in expansion of macro
>>>> ‘rcu_access_pointer’
>>>>     15 |  return !!rcu_access_pointer(w->task);
>>>>
>>>>      Argillander
>>>
>>> You could try the patch here:
>>> https://lore.kernel.org/all/917e9ce0-c8cf-61b2-d1ba-ebf25bbd979d@infradead.org/
>>
>> I have to edit it to <linux/cgroup_types.h> as there is no <linux/cgroup-defs.h>
>> with fast headers. I also tried a couple other things but it didn't
>> seem to make a
>> difference.
>>
>>> although the build error that it fixes doesn't look exactly the same
>>> as yours.
>>
>> Quite close still. Maybe I should try to bisect this and I will also
>> see how bisectable
>> this branch is.
> 
> Ok. I have now bisect first bad to this commit.
> c4ad6fcb67c4 ("sched/headers: Reorganize, clean up and optimize
> kernel/sched/fair.c dependencies")
> Note that this has been also bisect by others.
> 
> With this I get little bit different error:
> 
> In file included from ./arch/x86/include/generated/asm/rwonce.h:1,
>                   from ./include/linux/compiler.h:255,
>                   from ./include/linux/export.h:43,
>                   from ./include/linux/linkage.h:7,
>                   from ./include/linux/kernel.h:17,
>                   from ./include/linux/cpumask.h:10,
>                   from ./include/linux/energy_model.h:4,
>                   from kernel/sched/fair.c:23:
> ./include/linux/psi.h: In function ‘cgroup_move_task’:
> ./include/linux/rcupdate.h:414:36: error: dereferencing pointer to
> incomplete type ‘struct css_set’
>    414 | #define RCU_INITIALIZER(v) (typeof(*(v)) __force __rcu *)(v)
>        |                                    ^~~~
> 
> Which is actually the same error that is in Randy's message. Patch of
> Randy works
> on top of this commit. But I cannot get this patch to work with HEAD,
> but probably
> I'm just missing something obvious. Still nice to see that probably a
> solution is near.
> 
>>>>> There's various changes in -v3, and it's now ported to the latest kernel
>>>>> (v5.17-rc8).
>>>>>
>>>>> Diffstat difference:
>>>>>
>>>>>   -v2: 25332 files changed, 178498 insertions(+), 74790 deletions(-)
>>>>>   -v3: 25513 files changed, 180947 insertions(+), 74572 deletions(-)
>>>
>>>
>>> --
>>> ~Randy
> 
Hi Ingo,

What is the fate of this patch series? Are you going to push this?
If not, can I? My approach would be take one by one, understand, test 
and push.
Would take years, but I have the time.

Thanks
Lucas Tanure