lists.openwall.net   lists  /  announce  owl-users  owl-dev  john-users  john-dev  passwdqc-users  yescrypt  popa3d-users  /  oss-security  kernel-hardening  musl  sabotage  tlsify  passwords  /  crypt-dev  xvendor  /  Bugtraq  Full-Disclosure  linux-kernel  linux-netdev  linux-ext4  linux-hardening  linux-cve-announce  PHC 
Open Source and information security mailing list archives
 
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <Y7XP9924wTevHcBT@maniforge.lan>
Date:   Wed, 4 Jan 2023 13:13:59 -0600
From:   David Vernet <void@...ifault.com>
To:     Stanislav Fomichev <sdf@...gle.com>
Cc:     Magnus Karlsson <magnus.karlsson@...il.com>,
        magnus.karlsson@...el.com, bjorn@...nel.org, ast@...nel.org,
        daniel@...earbox.net, netdev@...r.kernel.org,
        maciej.fijalkowski@...el.com, bpf@...r.kernel.org, yhs@...com,
        andrii@...nel.org, martin.lau@...ux.dev, song@...nel.org,
        john.fastabend@...il.com, kpsingh@...nel.org, haoluo@...gle.com,
        jolsa@...nel.org, tirthendu.sarkar@...el.com,
        jonathan.lemon@...il.com
Subject: Re: [PATCH bpf-next v2 11/15] selftests/xsk: get rid of built-in XDP
 program

On Wed, Jan 04, 2023 at 10:19:37AM -0800, Stanislav Fomichev wrote:
> On Wed, Jan 4, 2023 at 10:14 AM David Vernet <void@...ifault.com> wrote:
> >
> > On Wed, Jan 04, 2023 at 01:17:40PM +0100, Magnus Karlsson wrote:
> > > From: Magnus Karlsson <magnus.karlsson@...el.com>
> > >
> > > Get rid of the built-in XDP program that was part of the old libbpf
> > > code in xsk.c and replace it with an eBPF program build using the
> > > framework by all the other bpf selftests. This will form the base for
> > > adding more programs in later commits.
> > >
> > > Signed-off-by: Magnus Karlsson <magnus.karlsson@...el.com>
> > > ---
> > >  tools/testing/selftests/bpf/Makefile          |  2 +-
> > >  .../selftests/bpf/progs/xsk_xdp_progs.c       | 19 ++++
> > >  tools/testing/selftests/bpf/xsk.c             | 88 ++++---------------
> > >  tools/testing/selftests/bpf/xsk.h             |  6 +-
> > >  tools/testing/selftests/bpf/xskxceiver.c      | 72 ++++++++-------
> > >  tools/testing/selftests/bpf/xskxceiver.h      |  7 +-
> > >  6 files changed, 88 insertions(+), 106 deletions(-)
> > >  create mode 100644 tools/testing/selftests/bpf/progs/xsk_xdp_progs.c
> > >
> > > diff --git a/tools/testing/selftests/bpf/Makefile b/tools/testing/selftests/bpf/Makefile
> > > index 205e8c3c346a..a0193a8f9da6 100644
> > > --- a/tools/testing/selftests/bpf/Makefile
> > > +++ b/tools/testing/selftests/bpf/Makefile
> > > @@ -240,7 +240,7 @@ $(OUTPUT)/flow_dissector_load: $(TESTING_HELPERS)
> > >  $(OUTPUT)/test_maps: $(TESTING_HELPERS)
> > >  $(OUTPUT)/test_verifier: $(TESTING_HELPERS) $(CAP_HELPERS)
> > >  $(OUTPUT)/xsk.o: $(BPFOBJ)
> > > -$(OUTPUT)/xskxceiver: $(OUTPUT)/xsk.o
> > > +$(OUTPUT)/xskxceiver: $(OUTPUT)/xsk.o $(OUTPUT)/xsk_xdp_progs.skel.h
> >
> > Hi Magnus,
> >
> > This seems to break the selftests build for clang:
> >
> > $ pwd
> > <redacted>/bpf-next/tools/testing/selftests/bpf
> >
> > $ make LLVM=1 CC=clang
> >   MKDIR    libbpf
> >   HOSTCC  /home/void/upstream/bpf-next/tools/testing/selftests/bpf/tools/build/libbpf/fixdep.o
> >   HOSTLD  /home/void/upstream/bpf-next/tools/testing/selftests/bpf/tools/build/libbpf/fixdep-in.o
> >   LINK    /home/void/upstream/bpf-next/tools/testing/selftests/bpf/tools/build/libbpf/fixdep
> >
> > ...
> >
> >   GEN-SKEL [test_progs-no_alu32] test_static_linked.skel.h
> >   LINK-BPF [test_progs-no_alu32] test_usdt.bpf.o
> >   GEN-SKEL [test_progs-no_alu32] linked_vars.skel.h
> >   GEN-SKEL [test_progs-no_alu32] linked_funcs.skel.h
> >   EXT-COPY [test_progs-no_alu32] urandom_read bpf_testmod.ko liburandom_read.so xdp_synproxy sign-file ima_setup.sh verify_sig_setup.sh btf_dump_test_case_bitfields.c btf_dump_test_case_multidim.c btf_dump_test_case_namespacing.c btf_dump_test_case_ordering.c btf_dump_test_case_packing.c btf_dump_test_case_padding.c btf_dump_test_case_syntax.c
> >   GEN-SKEL [test_progs-no_alu32] linked_maps.skel.h
> >   GEN-SKEL [test_progs-no_alu32] test_subskeleton.skel.h
> >   BINARY   xskxceiver
> >   BINARY   bench
> >   GEN-SKEL [test_progs-no_alu32] test_subskeleton_lib.skel.h
> >   GEN-SKEL [test_progs-no_alu32] test_usdt.skel.h
> > clang-15: error: cannot specify -o when generating multiple output files
> > make: *** [Makefile:171: /home/void/upstream/bpf-next/tools/testing/selftests/bpf/xskxceiver] Error 1
> > make: *** Waiting for unfinished jobs....
> > make[1]: Nothing to be done for 'docs'.
> > $
> >
> > It's also broken on CI: https://github.com/kernel-patches/bpf/actions/runs/3837984934/jobs/6533917001
> >
> > Could you please look into this?
> 
> Ugh, that's the same issue I'm getting for my xdp_hw_metadata binary.
> And I'm still at loss on how to reproduce it locally. (I've tried 'apt
> install clang-16 in ubuntu latest docker container' and it's still
> fine).

I was able to reproduce this issue locally:

[void@...iforge bpf]$ make -j LLVM=1 CC=clang
  GEN-SKEL [test_progs] pyperf600.skel.h
  GEN-SKEL [test_progs] test_verif_scale2.skel.h
  LINK-BPF [test_progs] test_static_linked.bpf.o
  LINK-BPF [test_progs] linked_funcs.bpf.o

...

  LINK-BPF [test_progs-no_alu32] test_usdt.bpf.o
  EXT-COPY [test_progs-no_alu32] urandom_read bpf_testmod.ko liburandom_read.so xdp_synproxy sign-file ima_setup.sh verify_sig_setup.sh btf_dump_test_case_bitfields.c btf_dump_test_case_multidim.c btf_dump_test_case_namespacing.c btf_dump_test_case_ordering.c btf_dump_test_case_packing.c btf_dump_test_case_padding.c btf_dump_test_case_syntax.c
  GEN-SKEL [test_progs-no_alu32] linked_funcs.skel.h
  BINARY   bench
  GEN-SKEL [test_progs-no_alu32] test_subskeleton.skel.h
  BINARY   xdp_hw_metadata
  GEN-SKEL [test_progs-no_alu32] test_subskeleton_lib.skel.h
  GEN-SKEL [test_progs-no_alu32] test_usdt.skel.h
clang-15: error: cannot specify -o when generating multiple output files
make: *** [Makefile:171: /home/void/upstream/bpf-next/tools/testing/selftests/bpf/xdp_hw_metadata] Error 1
make: *** Waiting for unfinished jobs....
make[1]: Nothing to be done for 'docs'.

Here's the actual clang command being executed:

[void@...iforge bpf]$ make LLVM=1 --dry-run xdp_hw_metadata
printf '  %-8s%s %s%s\n' "BINARY" "" "xdp_hw_metadata" "";
clang --target=x86_64-linux-gnu -fintegrated-as -g -O0 -rdynamic -Wall -Werror -DHAVE_GENHDR  -I/home/void/upstream/bpf-next/tools/testing/selftests/bpf -I/home/void/upstream/bpf-next/tools/testing/selftests/bpf/tools/include -I/home/void/upstream/bpf-next/include/generated -I/home/void/upstream/bpf-next/tools/lib -I/home/void/upstream/bpf-next/tools/include -I/home/void/upstream/bpf-next/tools/include/uapi -I/home/void/upstream/bpf-next/tools/testing/selftests/bpf -Wno-unused-command-line-argument     -static  xdp_hw_metadata.c /home/void/upstream/bpf-next/tools/testing/selftests/bpf/tools/build/libbpf/libbpf.a /home/void/upstream/bpf-next/tools/testing/selftests/bpf/xsk.o /home/void/upstream/bpf-next/tools/testing/selftests/bpf/xdp_hw_metadata.skel.h /home/void/upstream/bpf-next/tools/testing/selftests/bpf/network_helpers.o -lelf -lz -lrt -lpthread -o /home/void/upstream/bpf-next/tools/testing/selftests/bpf/xdp_hw_metadata

and the output using --debug=j

[void@...iforge bpf]$ make LLVM=1 --debug=j xdp_hw_metadata
Putting child 0x55cc78cd6670 (/home/void/upstream/bpf-next/tools/testing/selftests/bpf/xdp_hw_metadata) PID 693804 on the chain.
Live child 0x55cc78cd6670 (/home/void/upstream/bpf-next/tools/testing/selftests/bpf/xdp_hw_metadata) PID 693804
  BINARY   xdp_hw_metadata
Reaping winning child 0x55cc78cd6670 PID 693804
Live child 0x55cc78cd6670 (/home/void/upstream/bpf-next/tools/testing/selftests/bpf/xdp_hw_metadata) PID 693805
clang-15: error: cannot specify -o when generating multiple output files
Reaping losing child 0x55cc78cd6670 PID 693805
make: *** [Makefile:171: /home/void/upstream/bpf-next/tools/testing/selftests/bpf/xdp_hw_metadata] Error 1
Removing child 0x55cc78cd6670 PID 693805 from chain.

make is taking the xdp_hw_metadata.skel.h file and providing it as an
input to clang. So I believe what's going on here is that the clang
command above is actually creating two output files:

1. xdp_hw_metadata
2. The precompiled header generated from xdp_hw_metadata.skel.h

and the error is clang reasonably saying: "I don't know which output
file you're referring to with -o". I'm surprised that gcc doesn't
complain about this, but I assume that it's doing the far more
non-intuitive thing of first outputting the precompiled header as
xdp_hw_metadata, and then immediately overwriting it with the actual
xdp_hw_metadata binary.

> Any pointers on how to debug those github actions locally?
> 
> > Thanks,
> > David

Powered by blists - more mailing lists

Powered by Openwall GNU/*/Linux Powered by OpenVZ