netdev - Re: [PATCH bpf] libbpf: fix version identification on busybox

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [day] [month] [year] [list]
Message-ID: <20190920221427.GD2760@khorivan>
Date:   Sat, 21 Sep 2019 01:14:28 +0300
From:   Ivan Khoronzhuk <ivan.khoronzhuk@...aro.org>
To:     Andrii Nakryiko <andrii.nakryiko@...il.com>
Cc:     Alexei Starovoitov <ast@...nel.org>,
        Daniel Borkmann <daniel@...earbox.net>,
        Martin Lau <kafai@...com>, Andrii Nakryiko <andriin@...com>,
        Yonghong Song <yhs@...com>,
        Networking <netdev@...r.kernel.org>, bpf <bpf@...r.kernel.org>,
        open list <linux-kernel@...r.kernel.org>
Subject: Re: [PATCH bpf] libbpf: fix version identification on busybox

On Fri, Sep 20, 2019 at 02:51:14PM -0700, Andrii Nakryiko wrote:
>On Fri, Sep 20, 2019 at 12:19 PM Ivan Khoronzhuk
><ivan.khoronzhuk@...aro.org> wrote:
>>
>> On Fri, Sep 20, 2019 at 09:34:51PM +0300, Ivan Khoronzhuk wrote:
>> >On Fri, Sep 20, 2019 at 09:41:54AM -0700, Andrii Nakryiko wrote:
>> >>On Fri, Sep 20, 2019 at 1:22 AM Ivan Khoronzhuk
>> >><ivan.khoronzhuk@...aro.org> wrote:
>> >>>
>> >>>On Thu, Sep 19, 2019 at 01:02:40PM -0700, Andrii Nakryiko wrote:
>> >>>>On Thu, Sep 19, 2019 at 11:22 AM Ivan Khoronzhuk
>> >>>><ivan.khoronzhuk@...aro.org> wrote:
>> >>>>>
>> >>>>> It's very often for embedded to have stripped version of sort in
>> >>>>> busybox, when no -V option present. It breaks build natively on target
>> >>>>> board causing recursive loop.
>> >>>>>
>> >>>>> BusyBox v1.24.1 (2019-04-06 04:09:16 UTC) multi-call binary. \
>> >>>>> Usage: sort [-nrugMcszbdfimSTokt] [-o FILE] [-k \
>> >>>>> start[.offset][opts][,end[.offset][opts]] [-t CHAR] [FILE]...
>> >>>>>
>> >>>>> Lets modify command a little to avoid -V option.
>> >>>>>
>> >>>>> Fixes: dadb81d0afe732 ("libbpf: make libbpf.map source of truth for libbpf version")
>> >>>>>
>> >>>>> Signed-off-by: Ivan Khoronzhuk <ivan.khoronzhuk@...aro.org>
>> >>>>> ---
>> >>>>>
>> >>>>> Based on bpf/master
>> >>>>>
>> >>>>>  tools/lib/bpf/Makefile | 2 +-
>> >>>>>  1 file changed, 1 insertion(+), 1 deletion(-)
>> >>>>>
>> >>>>> diff --git a/tools/lib/bpf/Makefile b/tools/lib/bpf/Makefile
>> >>>>> index c6f94cffe06e..a12490ad6215 100644
>> >>>>> --- a/tools/lib/bpf/Makefile
>> >>>>> +++ b/tools/lib/bpf/Makefile
>> >>>>> @@ -3,7 +3,7 @@
>> >>>>>
>> >>>>>  LIBBPF_VERSION := $(shell \
>> >>>>>         grep -oE '^LIBBPF_([0-9.]+)' libbpf.map | \
>> >>>>> -       sort -rV | head -n1 | cut -d'_' -f2)
>> >>>>> +       cut -d'_' -f2 | sort -r | head -n1)
>> >>>>
>> >>>>You can't just sort alphabetically, because:
>> >>>>
>> >>>>1.2
>> >>>>1.11
>> >>>>
>> >>>>should be in that order. See discussion on mailing thread for original commit.
>> >>>
>> >>>if X1.X2.X3, where X = {0,1,....99999}
>> >>>Then it can be:
>> >>>
>> >>>-LIBBPF_VERSION := $(shell \
>> >>>-       grep -oE '^LIBBPF_([0-9.]+)' libbpf.map | \
>> >>>-       sort -rV | head -n1 | cut -d'_' -f2)
>> >>>+_LBPFLIST := $(patsubst %;,%,$(patsubst LIBBPF_%,%,$(filter LIBBPF_%, \
>> >>>+           $(shell cat libbpf.map))))
>> >>>+_LBPFLIST2 := $(foreach v,$(_LBPFLIST), \
>> >>>+               $(subst $() $(),,$(foreach n,$(subst .,$() $(),$(v)), \
>> >>>+                       $(shell printf "%05d" $(n)))))
>> >>>+_LBPF_VER := $(word $(words $(sort $(_LBPFLIST2))), $(sort $(_LBPFLIST2)))
>> >>>+LIBBPF_VERSION := $(patsubst %_$(_LBPF_VER),%,$(filter %_$(_LBPF_VER), \
>> >>>+        $(join $(addsuffix _, $(_LBPFLIST)),$(_LBPFLIST2))))
>> >>>
>> >>>It's bigger but avoids invocations of grep/sort/cut/head, only cat/printf
>> >>>, thus -V option also.
>> >>>
>> >>
>> >>No way, this is way too ugly (and still unreliable, if we ever have
>> >>X.Y.Z.W or something). I'd rather go with my original approach of
>> >Yes, forgot to add
>> >X1,X2,X3,...XN, where X = {0,1,....99999} and N = const for all versions.
>> >But frankly, 1.0.0 looks too far.
>>
>> It actually works for any numbs of X1.X2...X100
>> but not when you have couple kindof:
>> X1.X2.X3
>> and
>> X1.X2.X3.X4
>>
>> But, no absolutely any problem to extend this solution to handle all cases,
>> by just adding leading 0 to every "transformed version", say limit it to 10
>> possible 'dots' (%5*10d) and it will work as clocks. Advantage - mostly make
>> functions.
>>
>> Here can be couple more solutions with sed, not sure it can look less maniac.
>>
>> >
>> >>fetching the last version in libbpf.map file. See
>> >>https://www.spinics.net/lists/netdev/msg592703.html.
>>
>> Yes it's nice but, no sort, no X1.X2.X3....XN
>>
>> Main is to solve it for a long time.
>
>Thinking a bit more about this, I'm even more convinced that we should
>just go with my original approach: find last section in libbpf.map and
>extract LIBBPF version from that. That will handle whatever crazy
>version format we might decide to use (e.g., 1.2.3-experimental).
>We'll just need to make sure that latest version is the last in
>libbpf.map, which will just happen naturally. So instead of this
>Makefile complexity, please can you port back my original approach?
>Thanks!

I don't insist, placed it for history and to show it can be sorted
alphabetically, I can live with cross-compilation that I hope goes soon,
on host no need to worry about this at all. So I better leave this change
up to you.

-- 
Regards,
Ivan Khoronzhuk