Message-ID: <f114ff4a-6612-0935-12ac-0e2ac18d896c@samsung.com>
Date:   Wed, 21 Apr 2021 12:57:45 +0200
From:   Marek Szyprowski <m.szyprowski@...sung.com>
To:     Marco Elver <elver@...gle.com>
Cc:     Peter Zijlstra <peterz@...radead.org>,
        Alexander Shishkin <alexander.shishkin@...ux.intel.com>,
        Arnaldo Carvalho de Melo <acme@...nel.org>,
        Ingo Molnar <mingo@...hat.com>, Jiri Olsa <jolsa@...hat.com>,
        Mark Rutland <mark.rutland@....com>,
        Namhyung Kim <namhyung@...nel.org>,
        Thomas Gleixner <tglx@...utronix.de>,
        Alexander Potapenko <glider@...gle.com>,
        Al Viro <viro@...iv.linux.org.uk>,
        Arnd Bergmann <arnd@...db.de>,
        Christian Brauner <christian@...uner.io>,
        Dmitry Vyukov <dvyukov@...gle.com>,
        Jann Horn <jannh@...gle.com>, Jens Axboe <axboe@...nel.dk>,
        Matt Morehouse <mascasa@...gle.com>,
        Peter Collingbourne <pcc@...gle.com>,
        Ian Rogers <irogers@...gle.com>,
        Oleg Nesterov <oleg@...hat.com>,
        kasan-dev <kasan-dev@...glegroups.com>,
        linux-arch <linux-arch@...r.kernel.org>,
        linux-fsdevel <linux-fsdevel@...r.kernel.org>,
        LKML <linux-kernel@...r.kernel.org>,
        the arch/x86 maintainers <x86@...nel.org>,
        "open list:KERNEL SELFTEST FRAMEWORK" 
        <linux-kselftest@...r.kernel.org>,
        Geert Uytterhoeven <geert@...ux-m68k.org>,
        Bartlomiej Zolnierkiewicz <b.zolnierkie@...sung.com>
Subject: Re: [PATCH v4 05/10] signal: Introduce TRAP_PERF si_code and
 si_perf to siginfo

On 21.04.2021 11:35, Marek Szyprowski wrote:
> On 21.04.2021 10:11, Marco Elver wrote:
>> On Wed, 21 Apr 2021 at 09:35, Marek Szyprowski 
>> <m.szyprowski@...sung.com> wrote:
>>> On 21.04.2021 08:21, Marek Szyprowski wrote:
>>>> On 21.04.2021 00:42, Marco Elver wrote:
>>>>> On Tue, 20 Apr 2021 at 23:26, Marek Szyprowski
>>>>> <m.szyprowski@...sung.com> wrote:
>>>>>> On 08.04.2021 12:36, Marco Elver wrote:
>>>>>>> Introduces the TRAP_PERF si_code, and associated siginfo_t field
>>>>>>> si_perf. These will be used by the perf event subsystem to send
>>>>>>> signals
>>>>>>> (if requested) to the task where an event occurred.
>>>>>>>
>>>>>>> Acked-by: Geert Uytterhoeven <geert@...ux-m68k.org> # m68k
>>>>>>> Acked-by: Arnd Bergmann <arnd@...db.de> # asm-generic
>>>>>>> Signed-off-by: Marco Elver <elver@...gle.com>
>>>>>> This patch landed in linux-next as commit fb6cc127e0b6 ("signal:
>>>>>> Introduce TRAP_PERF si_code and si_perf to siginfo"). It causes
>>>>>> regression on my test systems (arm 32bit and 64bit). Most systems 
>>>>>> fails
>>>>>> to boot in the given time frame. I've observed that there is a 
>>>>>> timeout
>>>>>> waiting for udev to populate /dev and then also during the network
>>>>>> interfaces configuration. Reverting this commit, together with
>>>>>> 97ba62b27867 ("perf: Add support for SIGTRAP on perf events") to 
>>>>>> let it
>>>>>> compile, on top of next-20210420 fixes the issue.
>>>>> Thanks, this is weird for sure and nothing in particular stands out.
>>>>>
>>>>> I have questions:
>>>>> -- Can you please share your config?
>>>> This happens with standard multi_v7_defconfig (arm) or just defconfig
>>>> for arm64.
>>>>
>>>>> -- Also, can you share how you run this? Can it be reproduced in 
>>>>> qemu?
>>>> Nothing special. I just boot my test systems and see that they are
>>>> waiting lots of time during the udev populating /dev and network
>>>> interfaces configuration. I didn't try with qemu yet.
>>>>> -- How did you derive this patch to be at fault? Why not just
>>>>> 97ba62b27867, given you also need to revert it?
>>>> Well, I've just run my boot tests with automated 'git bisect' and that
>>>> was its result. It was a bit late in the evening, so I didn't analyze
>>>> it further, I've just posted a report about the issue I've found. It
>>>> looks that bisecting pointed to a wrong commit somehow.
>>>>> If you are unsure which patch exactly it is, can you try just
>>>>> reverting 97ba62b27867 and see what happens?
>>>> Indeed, this is a real faulty commit. Initially I've decided to revert
>>>> it to let kernel compile (it uses some symbols introduced by this
>>>> commit). Reverting only it on top of linux-next 20210420 also fixes
>>>> the issue. I'm sorry for the noise in this thread. I hope we will find
>>>> what really causes the issue.
>>> This was a premature conclusion. It looks that during the test I've did
>>> while writing that reply, the modules were not deployed properly and a
>>> test board (RPi4) booted without modules. In that case the board booted
>>> fine and there was no udev timeout. After deploying kernel modules, the
>>> udev timeout is back.
>> I'm confused now. Can you confirm that the problem is due to your
>> kernel modules, or do you think it's still due to 97ba62b27867? Or
>> fb6cc127e0b6 (this patch)?
>
> I don't use any custom kernel modules. I just deploy all modules that 
> are being built from the given kernel defconfig (arm 
> multi_v7_defconfig or arm64 default) and they are automatically loaded 
> during the boot by udev. I've checked again and bisect was right. The 
> kernel built from fb6cc127e0b6 suffers from the described issue, while 
> the one build from the previous commit (2e498d0a74e5) works fine.

I've managed to reproduce this issue with qemu. I've compiled the kernel 
for arm 32bit with multi_v7_defconfig and used some older Debian rootfs 
image. The log and qemu parameters are here: 
https://paste.debian.net/1194526/

Check the timestamp for the 'EXT4-fs (vda): re-mounted' message and 
'done (timeout)' status for the 'Waiting for /dev to be fully populated' 
message. This happens only when kernel modules build from the 
multi_v7_defconfig are deployed on the rootfs.

Best regards
-- 
Marek Szyprowski, PhD
Samsung R&D Institute Poland

Powered by blists - more mailing lists