netdev - Re: [PATCH bpf-next 0/5] Support local vmtest for riscv64

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <52117f9c-b691-409f-ad2a-a25f53a9433d@huaweicloud.com>
Date: Sat, 30 Mar 2024 18:12:57 +0800
From: Pu Lehui <pulehui@...weicloud.com>
To: Eduard Zingerman <eddyz87@...il.com>, bpf@...r.kernel.org,
 linux-riscv@...ts.infradead.org, netdev@...r.kernel.org
Cc: Björn Töpel <bjorn@...nel.org>,
 Alexei Starovoitov <ast@...nel.org>, Daniel Borkmann <daniel@...earbox.net>,
 Andrii Nakryiko <andrii@...nel.org>, Martin KaFai Lau
 <martin.lau@...ux.dev>, Song Liu <song@...nel.org>,
 Yonghong Song <yhs@...com>, John Fastabend <john.fastabend@...il.com>,
 KP Singh <kpsingh@...nel.org>, Stanislav Fomichev <sdf@...gle.com>,
 Hao Luo <haoluo@...gle.com>, Jiri Olsa <jolsa@...nel.org>,
 Mykola Lysenko <mykolal@...com>, Manu Bretelle <chantr4@...il.com>,
 Pu Lehui <pulehui@...wei.com>
Subject: Re: [PATCH bpf-next 0/5] Support local vmtest for riscv64


On 2024/3/30 3:46, Eduard Zingerman wrote:
> On Fri, 2024-03-29 at 18:10 +0800, Pu Lehui wrote:
> [...]
> 
>>> Apparently jammy does not have binaries built for riscv64, or I'm failing to find correct mirror.
>>> Could you please provide some instructions on how to prepare rootfs?
>>
>> Hi Eduard, We need the mirror repository of ubuntu-ports, you could try
>> http://de.ports.ubuntu.com/.
> 
> Hi Pu, thank you this mirrorm it works.
> 
> Unfortunately my local setup is still not good enough.
> I've installed cross-riscv64-gcc14 but it seems that a few more
> libraries are necessary, as I get the following compilation errors: >
>    $ PLATFORM=riscv64 CROSS_COMPILE=riscv64-suse-linux- ./vmtest.sh -- ./test_verifier
>    ... kernel compiles ok ...
>    ../../../../scripts/sign-file.c:25:10: fatal error: openssl/opensslv.h: No such file or directory
>       25 | #include <openssl/opensslv.h>
>          |          ^~~~~~~~~~~~~~~~~~~~
>    compilation terminated.
>      CC      /home/eddy/work/bpf-next/tools/testing/selftests/bpf/tools/build/host/libbpf/sharedobjs/bpf.o
>    In file included from nlattr.c:14:
>    libbpf_internal.h:19:10: fatal error: libelf.h: No such file or directory
>       19 | #include <libelf.h>
>    ...
> 
> Looks like I won't be able to test this patch-set, unless you have
> some writeup on how to create a riscv64 dev environment at hand.
> Sorry for the noise

Yeah, environmental issues are indeed a developer's nightmare. I will 
try to do something for the newcomers of riscv64 bpf. At present, I have 
simply built a docker local vmtest environment [0] based on Bjorn's 
riscv-cross-builder. We can directly run vmtest within this environment. 
Hopefully it will help.

Link: https://github.com/pulehui/riscv-cross-builder/tree/vmtest [0]

PS: Since the current rootfs of riscv64 is not in the INDEX, I simply 
modified vmtest.sh to support local rootfs. And we can use it by:
```
PLATFORM=riscv64 CROSS_COMPILE=riscv64-linux-gnu- \
     tools/testing/selftests/bpf/vmtest.sh -l /rootfs -- \
         ./test_progs -d \
             \"$(cat tools/testing/selftests/bpf/DENYLIST.riscv64 \
                 | cut -d'#' -f1 \
                 | sed -e 's/^[[:space:]]*//' \
                       -e 's/[[:space:]]*$//' \
                 | tr -s '\n' ','\
             )\"
```

diff --git a/tools/testing/selftests/bpf/vmtest.sh 
b/tools/testing/selftests/bpf/vmtest.sh
index f6889de9b498..17aff708c416 100755
--- a/tools/testing/selftests/bpf/vmtest.sh
+++ b/tools/testing/selftests/bpf/vmtest.sh
@@ -148,6 +148,21 @@ download_rootfs()
  		zstd -d | sudo tar -C "$dir" -x
  }

+load_rootfs()
+{
+	local image_dir="$1"
+	local rootfsversion="$2"
+	local dir="$3"
+
+	if ! which zstd &> /dev/null; then
+		echo 'Could not find "zstd" on the system, please install zstd'
+		exit 1
+	fi
+
+	cat "${image_dir}/libbpf-vmtest-rootfs-$rootfsversion.tar.zst" |
+		zstd -d | sudo tar -C "$dir" -x
+}
+
  recompile_kernel()
  {
  	local kernel_checkout="$1"
@@ -234,6 +249,7 @@ EOF

  create_vm_image()
  {
+	local local_image_dir="$1"
  	local rootfs_img="${OUTPUT_DIR}/${ROOTFS_IMAGE}"
  	local mount_dir="${OUTPUT_DIR}/${MOUNT_DIR}"

@@ -245,7 +261,11 @@ create_vm_image()
  	mkfs.ext4 -q "${rootfs_img}"

  	mount_image
-	download_rootfs "$(newest_rootfs_version)" "${mount_dir}"
+	if [[ "${local_image_dir}" == "" ]]; then
+		download_rootfs "$(newest_rootfs_version)" "${mount_dir}"
+	else
+		load_rootfs "${local_image_dir}" "$(newest_rootfs_version)" 
"${mount_dir}"
+	fi
  	unmount_image
  }

@@ -363,12 +383,16 @@ main()
  	local update_image="no"
  	local exit_command="poweroff -f"
  	local debug_shell="no"
+	local local_image_dir=""

-	while getopts ':hskid:j:' opt; do
+	while getopts ':hskil:d:j:' opt; do
  		case ${opt} in
  		i)
  			update_image="yes"
  			;;
+		l)
+			local_image_dir="$OPTARG"
+			;;
  		d)
  			OUTPUT_DIR="$OPTARG"
  			;;
@@ -445,7 +469,7 @@ main()
  	fi

  	if [[ "${update_image}" == "yes" ]]; then
-		create_vm_image
+		create_vm_image "${local_image_dir}"
  	fi

  	update_selftests "${kernel_checkout}" "${make_command}"