Date:   Wed, 22 Sep 2021 13:28:37 +0200
From:   Maxime Ripard <maxime@...no.tech>
To:     Sudip Mukherjee <sudipm.mukherjee@...il.com>
Cc:     Emma Anholt <emma@...olt.net>, David Airlie <airlied@...ux.ie>,
        Daniel Vetter <daniel@...ll.ch>,
        Philipp Zabel <p.zabel@...gutronix.de>,
        dri-devel <dri-devel@...ts.freedesktop.org>,
        linux-kernel <linux-kernel@...r.kernel.org>,
        Linus Torvalds <torvalds@...ux-foundation.org>
Subject: Re: Regression with mainline kernel on rpi4

On Wed, Sep 22, 2021 at 11:10:34AM +0100, Sudip Mukherjee wrote:
> On Wed, Sep 22, 2021 at 10:57 AM Maxime Ripard <maxime@...no.tech> wrote:
> >
> > On Mon, Sep 20, 2021 at 06:21:42PM +0100, Sudip Mukherjee wrote:
> > > On Mon, Sep 20, 2021 at 6:10 PM Maxime Ripard <maxime@...no.tech> wrote:
> > > >
> > > > On Mon, Sep 20, 2021 at 05:35:00PM +0100, Sudip Mukherjee wrote:
> > > > > On Mon, Sep 20, 2021 at 4:53 PM Maxime Ripard <maxime@...no.tech> wrote:
> > > > > >
> > > > > > On Mon, Sep 20, 2021 at 05:43:33PM +0200, Maxime Ripard wrote:
> > > > > > > On Mon, Sep 20, 2021 at 04:47:31PM +0200, Maxime Ripard wrote:
> > > > > > > > On Sat, Sep 04, 2021 at 10:40:29AM +0100, Sudip Mukherjee wrote:
> > > > > > > > > Hi Maxime,
> > > > > > > > >
> > > > > > > > > On Sat, Sep 4, 2021 at 10:10 AM Maxime Ripard <maxime@...no.tech> wrote:
> > > > > > > > > >
> > > > > > > > > > On Fri, Sep 03, 2021 at 09:09:50PM +0100, Sudip Mukherjee wrote:
> > > > > > > > > > > Hi Maxime,
> > > > > > > > > > >
> > > > > > > > > > > On Fri, Sep 3, 2021 at 5:03 PM Maxime Ripard <maxime@...no.tech> wrote:
> > > > > > > > > > > >
> > > > > > > > > > > > Hi Sudip,
> > > > > > > > > > > >
> > > > > > > > > > > > On Thu, Sep 02, 2021 at 10:08:19AM +0100, Sudip Mukherjee wrote:
> > > > > > > > > > > > > Hi All,
> > > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > <snip>
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > You can see the complete dmesg at
> > > > > > > > > > > > > https://openqa.qa.codethink.co.uk/tests/76#step/dmesg/8
> > > > > > > > > > > >
> > > > > > > > > > > > What test were you running?
> > > > > > > > > > >
> > > > > > > > > > > Nothing particular, its just a boot test that we do every night after
> > > > > > > > > > > pulling from torvalds/linux.git
> > > > > > > > > >
> > > > > > > > > > What are you booting to then?
> > > > > > > > >
> > > > > > > > > I am not sure I understood the question.
> > > > > > > > > Its an Ubuntu image. The desktop environment is gnome. And as
> > > > > > > > > mentioned earlier, we use the HEAD of linus tree every night to boot
> > > > > > > > > the rpi4 and test that we can login via desktop environment and that
> > > > > > > > > there is no regression in dmesg.
> > > > > > > >
> > > > > > > > Looking at the CI, this isn't from a RPi but from qemu?
> > > > >
> > > > > No, this is from rpi4 board (4GB), not qemu. The CI is a little
> > > > > complicated here, lava boots the board with the new kernel and will
> > > > > then trigger the openQA job. openQA will then connect to the board
> > > > > using vnc to test the desktop. This is the last link that I sent to
> > > > > Linus when he asked for it.
> > > > > https://lava.qa.codethink.co.uk/scheduler/job/164#L1356
> > > > >
> > > > > And this is the lava job for today -
> > > > > https://lava.qa.codethink.co.uk/scheduler/job/173
> > > >
> > > > Is it connected to a monitor then?
> > >
> > > Missed replying to this one earlier. I have a hdmi lilliput monitor
> > > connected to it.
> > >
> > > >
> > > > > > > >
> > > > > > > > What defconfig are you using? How did you generate the Ubuntu image?
> > > > > > > > Through debootstrap? Any additional package?
> > > > >
> > > > > Its the default ubuntu config. I can send you the config if you want.
> > > >
> > > > Yes, please.
> > >
> > > Attached.
> > > My build script will copy this config as .config,  do olddefconfig
> > > and then build.
> >
> > I still can't reproduce it.
> >
> > What other customisations did you do to that image? It looks like
> > there's some customs scripts in there (test-mainline.sh), what are they
> > doing?
> 
> That test script is triggering the openqa job, but its running only
> after lava is able to login. The trace is appearing before the login
> prompt even, so test_mainline.sh should not matter here.
> The only customization done to the default ubuntu image is the initrd.
> Can you please try with the initrd from
> https://elisa-builder-00.iol.unh.edu/kernel/mainline/rpi4/initrd.gz..
> I will check with another board also.

Still works fine (and it required some mangling of the kernel command line).

If we summarize:

  - You initially just dumped a panic and a link to your QA, without any
    more context:

  - Then stating that you're not doing any test, really;

  - Well, except for booting Ubuntu, but no other modification

  - But you're not booting the standard image

  - And with a custom initrd

  - And that QA link states that you're booting from QEMU, but you're
    not.

Please provide a full documentation on what you're doing to generate
that image, from scratch, in order to get that panic you reported
previously.

I've spent an entire day trying to make sense of what you're doing
exactly to get into that situation. I have other things to work on and I
don't plan on figuring out any random CI system.

Maxime

Download attachment "signature.asc" of type "application/pgp-signature" (229 bytes)

Powered by blists - more mailing lists