lists.openwall.net   lists  /  announce  owl-users  owl-dev  john-users  john-dev  passwdqc-users  yescrypt  popa3d-users  /  oss-security  kernel-hardening  musl  sabotage  tlsify  passwords  /  crypt-dev  xvendor  /  Bugtraq  Full-Disclosure  linux-kernel  linux-netdev  linux-ext4  linux-hardening  linux-cve-announce  PHC 
Open Source and information security mailing list archives
 
Hash Suite for Android: free password hash cracker in your pocket
[<prev] [next>] [<thread-prev] [day] [month] [year] [list]
Date:   Fri, 20 Aug 2021 18:02:36 -0300
From:   Arnaldo Carvalho de Melo <acme@...nel.org>
To:     Jiri Olsa <jolsa@...nel.org>
Cc:     Andrii Nakryiko <andriin@...com>, dwarves@...r.kernel.org,
        netdev@...r.kernel.org, bpf@...r.kernel.org,
        Alexei Starovoitov <ast@...nel.org>,
        Yonghong Song <yhs@...com>, Hao Luo <haoluo@...gle.com>,
        Martin KaFai Lau <kafai@...com>,
        Song Liu <songliubraving@...com>,
        John Fastabend <john.fastabend@...il.com>,
        KP Singh <kpsingh@...omium.org>,
        Nathan Chancellor <nathan@...nel.org>,
        Sedat Dilek <sedat.dilek@...il.com>
Subject: Re: [RFC] dwarves/pahole: Add test scripts

Em Fri, Aug 20, 2021 at 05:39:22PM -0300, Arnaldo Carvalho de Melo escreveu:
> So, now looking at:
 
> test_btfdiff    on /var/home/acme/.pahole_test_objects/x86-gcc/vmlinux ... FAIL

Minor stuff, why would BTF get that const and DWARF not?

⬢[acme@...lbox pahole]$ btfdiff /var/home/acme/.pahole_test_objects/x86-gcc/vmlinux
--- /tmp/btfdiff.dwarf.j8SZhr	2021-08-20 17:40:52.052186096 -0300
+++ /tmp/btfdiff.btf.5QMi8C	2021-08-20 17:40:52.298191569 -0300
@@ -33719,7 +33719,7 @@ struct e1000_option {

 			/* XXX 4 bytes hole, try to pack */

-			struct e1000_opt_list * p;       /*    40     8 */
+			const struct e1000_opt_list  * p; /*    40     8 */
 		} l;                                     /*    32    16 */
 	} arg;                                           /*    32    16 */

⬢[acme@...lbox pahole]$ btfdiff /var/home/acme/.pahole_test_objects/x86-clang/vmlinux
--- /tmp/btfdiff.dwarf.1TAWtj	2021-08-20 17:41:18.821781610 -0300
+++ /tmp/btfdiff.btf.FMLZRt	2021-08-20 17:41:19.058786883 -0300
@@ -21584,7 +21584,7 @@ struct cpu_rmap {
 	struct {
 		u16                index;                /*    16     2 */
 		u16                dist;                 /*    18     2 */
-	} near[0]; /*    16     0 */
+	} near[]; /*    16     0 */

 	/* size: 16, cachelines: 1, members: 5 */
 	/* last cacheline: 16 bytes */
@@ -65743,7 +65743,7 @@ struct linux_efi_memreserve {
 	struct {
 		phys_addr_t        base;                 /*    16     8 */
 		phys_addr_t        size;                 /*    24     8 */
-	} entry[0]; /*    16     0 */
+	} entry[]; /*    16     0 */

 	/* size: 16, cachelines: 1, members: 4 */
 	/* last cacheline: 16 bytes */
@@ -72814,7 +72814,7 @@ struct netlink_policy_dump_state {
 	struct {
 		const struct nla_policy  * policy;       /*    16     8 */
 		unsigned int       maxtype;              /*    24     4 */
-	} policies[0]; /*    16     0 */
+	} policies[]; /*    16     0 */

 	/* size: 16, cachelines: 1, members: 4 */
 	/* sum members: 12, holes: 1, sum holes: 4 */
⬢[acme@...lbox pahole]$o

I see, there are two declarations for this struct, one at:

drivers/net/ethernet/intel/e1000/e1000_param.c

166 struct e1000_option {
167         enum { enable_option, range_option, list_option } type;
168         const char *name;
169         const char *err;
170         int def;
171         union {
172                 struct { /* range_option info */
173                         int min;
174                         int max;
175                 } r;
176                 struct { /* list_option info */
177                         int nr;
178                         const struct e1000_opt_list { int i; char *str; } *p;
179                 } l;
180         } arg;
181 };

and the other at (note that extra 'e' after 1000 :) ):

drivers/net/ethernet/intel/e1000e/param.c

141 struct e1000_option {
142         enum { enable_option, range_option, list_option } type;
143         const char *name;
144         const char *err;
145         int def;
146         union {
147                 /* range_option info */
148                 struct {
149                         int min;
150                         int max;
151                 } r;
152                 /* list_option info */
153                 struct {
154                         int nr;
155                         struct e1000_opt_list {
156                                 int i;
157                                 char *str;
158                         } *p;
159                 } l;
160         } arg;
161 };

The only difference is that 'const'.

I _guess_ that the BTF deduplicator is picking the one with the 'const'
while pahole's (which is just used when pretty printing, not when BTF
encoding) with the other (it shouldn't as per my intention, I'm not
arguing about if both should appear as different types).

I tested it with:

  pahole -j -F dwarf -C e1000_option /var/home/acme/.pahole_test_objects/x86-gcc/vmlinux

to use the new DWARF multithreaded loading, like 'btfdiff' now does, and
its related to which of these types get loaded first, but its just
related to pahole pretty printing, not for the BTF encoding, where all
types will be passed to BTF for deduplication.

Same thing happens with clang, its just that when pretty printing in
this test run the clang multithreaded DWARF loading saw the type that
BTF picked (the one with 'const').

When I re-run it with:

⬢[acme@...lbox pahole]$ git diff
diff --git a/btfdiff b/btfdiff
index 77543630d1965b5e..71d3dbf5fc861e16 100755
--- a/btfdiff
+++ b/btfdiff
@@ -26,7 +26,6 @@ pahole_bin=${PAHOLE-"pahole"}
 ${pahole_bin} -F dwarf \
              --flat_arrays \
              --sort \
-             --jobs \
              --suppress_aligned_attribute \
              --suppress_force_paddings \
              --suppress_packed \
⬢[acme@...lbox pahole]$ git diff

I get:

test_btfdiff    on /var/home/acme/.pahole_test_objects/aarch64-clang/vmlinux ... OK
test_btfdiff    on /var/home/acme/.pahole_test_objects/x86-clang/vmlinux ... FAIL

clang's is just that zero entries array diff again.

I'll get 1.22 out of the door tomorrow as it passed these tests and also
the ones at libbpf github CI and my local ones and get these scripts merged
in 1.23, will check the clang output as well.

Thanks for the scripts!

- Arnaldo

Powered by blists - more mailing lists

Powered by Openwall GNU/*/Linux Powered by OpenVZ