linux-kernel - Re: [RFC PATCH v1 00/25] printk: new implementation

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <20200127134535.GA4439@lxhi-065.adit-jv.com>
Date:   Mon, 27 Jan 2020 14:45:35 +0100
From:   Eugeniu Rosca <erosca@...adit-jv.com>
To:     Petr Mladek <pmladek@...e.com>
CC:     Eugeniu Rosca <erosca@...adit-jv.com>,
        Geert Uytterhoeven <geert@...ux-m68k.org>,
        John Ogness <john.ogness@...utronix.de>,
        Linux Kernel Mailing List <linux-kernel@...r.kernel.org>,
        Peter Zijlstra <peterz@...radead.org>,
        Geert Uytterhoeven <geert+renesas@...der.be>,
        Kuninori Morimoto <kuninori.morimoto.gx@...esas.com>,
        Andrew Gabbasov <andrew_gabbasov@...tor.com>,
        Sanjeev Chugh <sanjeev_chugh@...tor.com>,
        Sergey Senozhatsky <sergey.senozhatsky.work@...il.com>,
        Steven Rostedt <rostedt@...dmis.org>,
        Daniel Wang <wonderfly@...gle.com>,
        Dean Jenkins <dean_jenkins@...tor.com>,
        Andrew Morton <akpm@...ux-foundation.org>,
        Linus Torvalds <torvalds@...ux-foundation.org>,
        Greg Kroah-Hartman <gregkh@...uxfoundation.org>,
        Dirk Behme <dirk.behme@...bosch.com>,
        Alan Cox <gnomes@...rguk.ukuu.org.uk>,
        Jiri Slaby <jslaby@...e.com>,
        Peter Feiner <pfeiner@...gle.com>,
        "open list:SERIAL DRIVERS" <linux-serial@...r.kernel.org>,
        Sergey Senozhatsky <sergey.senozhatsky@...il.com>,
        Eugeniu Rosca <roscaeugeniu@...il.com>
Subject: Re: [RFC PATCH v1 00/25] printk: new implementation

Hi Petr,

Thank you for your insights.

On Mon, Jan 27, 2020 at 01:32:49PM +0100, Petr Mladek wrote:
> On Fri 2020-01-24 17:09:29, Eugeniu Rosca wrote:
> > On Wed, Jan 22, 2020 at 08:48:12PM +0100, Geert Uytterhoeven wrote:
> > > On Wed, Jan 22, 2020 at 5:59 PM Eugeniu Rosca <erosca@...adit-jv.com> wrote:
> > > > On Wed, Jan 22, 2020 at 08:31:44AM +0100, Geert Uytterhoeven wrote:
> > > > > On Wed, Jan 22, 2020 at 3:34 AM Eugeniu Rosca <erosca@...adit-jv.com> wrote:
> > > > > > So, what's specific to R-Car3, based on my testing, is that the issue
> > > > > > can only be reproduced if the printk storm originates on CPU0
> > 
> > Slight amendment the above statement. Below results are got on R-Car
> > H3ULCB running renesas-drivers-2020-01-14-v5.5-rc6 (cX stands for CPUx,
> > whitespace stands for clean audio, '!' stands for distorted audio):
> > 
> >                   printk @:
> >                   c0 c1 c2 c3 c4 c5 c6 c7
> > speaker-test @ c0 !
> >                c1 !  !
> >                c2 !     !
> >                c3 !        !
> >                c4 !           !
> >                c5 !              !
> >                c6 !                 !
> >                c7 !                    !
> > 
> > One can see two patterns in the chart. The audio has glitches whenever:
> >  - printk and the audio application run on the same CPU, or:
> >  - printk runs on CPU0
> 
> The proper longterm solution seems to be offloading printk console
> handling to a kthread so that it can be bound to a particular CPU
> and does not block audio.

Same understanding here. I don't think this is possible w/o the full
switch to the kthread-based concept proposed in this series (I sought
an easier way out, but failed).

Even after pinning the printk kthread to CPUn, we still accept living
with the huge latencies of the console drivers on CPUn. To avoid audio
glitches being caused by the serial console, userspace would need to
additionally blacklist CPUn from any RT workloads by setting the core
affinity of audio applications appropriately.

The above imposes certain constraints on the CPU partitioning in the
system, but that's the most mainline-friendly solution I can think of.
Any alternative views would be appreciated.

> 
> Anyway, there is a question whether you really need to send all messages
> via the serial console. It might make sense to filter less important
> messages using "loglevel=" or "quiet" kernel parameters. The full
> log can be read later from userspace (dmesg, syslog, /dev/kmsg).
> Filtering can get disabled when debugging non-booting kernel.
> In this case, a distorted audio is the least problem.

This has been discussed in detail with the reporters of the issue.
Yes, it might be an infrequent requirement in general, but the goal
is to achieve clean audio even (and specifically) with verbose
serial console output.

-- 
Best Regards
Eugeniu Rosca