linux-kernel - Re: [PATCH 01/50] x86/boot/e820: Introduce arch/x86/include/asm/e820/types.h

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <20170201085647.GA21885@gmail.com>
Date:   Wed, 1 Feb 2017 09:56:47 +0100
From:   Ingo Molnar <mingo@...nel.org>
To:     Sam Ravnborg <sam@...nborg.org>
Cc:     linux-kernel@...r.kernel.org,
        Andrew Morton <akpm@...ux-foundation.org>,
        Andy Lutomirski <luto@...capital.net>,
        Borislav Petkov <bp@...en8.de>,
        "H . Peter Anvin" <hpa@...or.com>,
        Linus Torvalds <torvalds@...ux-foundation.org>,
        Peter Zijlstra <peterz@...radead.org>,
        Thomas Gleixner <tglx@...utronix.de>,
        Yinghai Lu <yinghai@...nel.org>,
        Arnaldo Carvalho de Melo <acme@...radead.org>
Subject: Re: [PATCH 01/50] x86/boot/e820: Introduce
 arch/x86/include/asm/e820/types.h

* Sam Ravnborg <sam@...nborg.org> wrote:

> > Firsty, the headers are not maintained by the user-space project, 99.999% of 
> > the maintenance is done by the kernel developers.
> 
> In the inital mail triggering this plan was that the kernel is moving away from 
> having uapi headers what-so-ever.

No, that is a misunderstanding:

> Quoting the original mail:
> "
> The plan is to keep the old UAPI header in place but the kernel won't
> use it anymore - and after some time we'll try to remove it. 
> "

You misunderstood my mail and you misunderstood the patch: we transition from the 
old UAPI header to a new one, but the exported data structures are still kept!

If you check the patches you'll see that bootparam.h still exports the e820_entry 
data structure. The 'old' header is simply one that is being phased out (if we 
can) - but the information is still exported.

> Translated:
> The plan is that the kernel will stop using headers from uapi/*
> The headers will be left for a while and then they will be deleted.

No, not at all.

> Perf being intimidate with the kernel is not the best example to come up with. 

No, that's wrong too, most larger tooling projects that care about feature 
propagation latency in fact already do something quite similar to what perf does.

For example the tooling side of GPU drivers (libdrm) has a copy of all the 
relevant UAPI headers:

triton:~/libdrm/include/drm> ls -l
total 316
-rw-rw-r-- 1 mingo mingo 19119 Feb  1 09:47 amdgpu_drm.h
-rw-rw-r-- 1 mingo mingo 11850 Feb  1 09:47 drm_fourcc.h
-rw-rw-r-- 1 mingo mingo 27613 Feb  1 09:47 drm.h
-rw-rw-r-- 1 mingo mingo 18313 Feb  1 09:47 drm_mode.h
-rw-rw-r-- 1 mingo mingo  2701 Feb  1 09:47 drm_sarea.h
-rw-rw-r-- 1 mingo mingo 46684 Feb  1 09:47 i915_drm.h
-rw-rw-r-- 1 mingo mingo  7895 Feb  1 09:47 mach64_drm.h
-rw-rw-r-- 1 mingo mingo 12923 Feb  1 09:47 mga_drm.h
-rw-rw-r-- 1 mingo mingo  5662 Feb  1 09:47 nouveau_drm.h
-rw-rw-r-- 1 mingo mingo  4217 Feb  1 09:47 qxl_drm.h
-rw-rw-r-- 1 mingo mingo  9901 Feb  1 09:47 r128_drm.h
-rw-rw-r-- 1 mingo mingo 38509 Feb  1 09:47 radeon_drm.h
-rw-rw-r-- 1 mingo mingo  5201 Feb  1 09:47 README
-rw-rw-r-- 1 mingo mingo  7054 Feb  1 09:47 savage_drm.h
-rw-rw-r-- 1 mingo mingo  2534 Feb  1 09:47 sis_drm.h
-rw-rw-r-- 1 mingo mingo  5526 Feb  1 09:47 tegra_drm.h
-rw-rw-r-- 1 mingo mingo  9534 Feb  1 09:47 vc4_drm.h
-rw-rw-r-- 1 mingo mingo  8291 Feb  1 09:47 via_drm.h
-rw-rw-r-- 1 mingo mingo  4704 Feb  1 09:47 virtgpu_drm.h
-rw-rw-r-- 1 mingo mingo 31225 Feb  1 09:47 vmwgfx_drm.h

For example i915_drm.h is a copy of include/uapi/drm/i915_drm.h, which is being 
synched between the two projects regularly.

> Think about to 100's of program that uses a few ioclt to talk with drivers etc.

Those can use distro UAPI headers just fine, if they don't care about the 6-12 
months delay it takes to get updated kernel headers. I.e. what you propose works 
for well-established ABIs that have been around for years.

To actually _progress_ with a tooling project, in close cooperation with the 
kernel side, the UAPI method of sharing via distro headers as-is hinders 
development agility big time...

Distro UAPI headers work fine in a world where the kernel is a static entity and 
does not update its ABIs. I.e. it only works if there's no actual kernel side 
extensions to the ABI. The whole UAPI distro headers approach is designed for the 
case where the style of sharing the headers matters the least: for a stagnant 
kernel or a stagnant tooling project ...

Btw., his kind of rigid, suboptimal, latency laden method of sharing information 
between the kernel and tooling might be one of the reasons why in general the 
Linux tooling landscape sucks, compared to other OSs...

Thanks,

	Ingo