linux-kernel - Re: [PATCH v3] kheaders: prevent `find` from seeing perl temp files

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <CAK7LNARNDt=BFsZdHWc=amQw-HhD4q-Ri8012t3xELkhcFObVQ@mail.gmail.com>
Date: Sat, 21 Dec 2024 13:15:41 +0900
From: Masahiro Yamada <masahiroy@...nel.org>
To: HONG Yifan <elsk@...gle.com>
Cc: Miguel Ojeda <ojeda@...nel.org>, Matthias Maennich <maennich@...gle.com>, kernel-team@...roid.com, 
	linux-kernel@...r.kernel.org
Subject: Re: [PATCH v3] kheaders: prevent `find` from seeing perl temp files

On Thu, Dec 19, 2024 at 5:21 AM HONG Yifan <elsk@...gle.com> wrote:
>
> Symptom:
>
> The command
>
>     find ... | xargs ... perl -i
>
> occasionally triggers error messages like the following, with the build
> still succeeding:
>
>     Can't open <redacted>/kernel/.tmp_cpio_dir/include/dt-bindings/clock/XXNX4nW9: No such file or directory.
>
> Analysis:
>
> With strace, the root cause has been identified to be `perl -i` creating
> temporary files inside $cpio_dir, which causes `find` to see the
> temporary files and emit the names. `find` is likely implemented with
> readdir. POSIX `readdir` says:
>
>     If a file is removed from or added to the directory after the most
>     recent call to opendir() or rewinddir(), whether a subsequent call
>     to readdir() returns an entry for that file is unspecified.
>
> So if the libc that `find` links against choose to return that entry
> in readdir(), a possible sequence of events is the following:
>
> 1. find emits foo.h
> 2. xargs executes `perl -i foo.h`
> 3. perl (pid=100) creates temporary file `XXXXXXXX`
> 4. find sees file `XXXXXXXX` and emit it
> 5. PID 100 exits, cleaning up the temporary file `XXXXXXXX`
> 6. xargs executes `perl -i XXXXXXXX`
> 7. perl (pid=200) tries to read the file, but it doesn't exist any more.
>
> ... triggering the error message.
>
> One can reproduce the bug with the following command (assuming PWD
> contains the list of headers in kheaders.tar.xz)
>
>     for i in $(seq 100); do
>         find -type f -print0 |
>             xargs -0 -P8 -n1 perl -pi -e 'BEGIN {undef $/;}; s/\/\*((?!SPDX).)*?\*\///smg;';
>     done
>
> With a `find` linking against musl libc, the error message is emitted
> 6/100 times.
>
> The fix:
>
> This change stores the results of `find` before feeding them into xargs.
> find and xargs will no longer be able to see temporary files that perl
> creates after this change.
>
> Signed-off-by: HONG Yifan <elsk@...gle.com>
> ---
> v3: (this patch)
>     Change from `cat contents.txt | xargs` to `xargs < contents.txt` to pass
>     shellcheck. Fix typo in commit message.

Applied to linux-kbuild (after resolving conflicts)
Thanks.



-- 
Best Regards
Masahiro Yamada