linux-kernel - Re: output: was: Re: [PATCH v4] printk: Userspace format enumeration support

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <YC02GBghuhmlvrXk@chrisdown.name>
Date:   Wed, 17 Feb 2021 15:28:24 +0000
From:   Chris Down <chris@...isdown.name>
To:     Petr Mladek <pmladek@...e.com>
Cc:     linux-kernel@...r.kernel.org,
        Sergey Senozhatsky <sergey.senozhatsky@...il.com>,
        John Ogness <john.ogness@...utronix.de>,
        Johannes Weiner <hannes@...xchg.org>,
        Andrew Morton <akpm@...ux-foundation.org>,
        Steven Rostedt <rostedt@...dmis.org>,
        Greg Kroah-Hartman <gregkh@...uxfoundation.org>,
        Kees Cook <keescook@...omium.org>, kernel-team@...com
Subject: Re: output: was: Re: [PATCH v4] printk: Userspace format enumeration
 support

Petr Mladek writes:
>I guess that you already use it internally and have its own tooling
>around it.

Hmm, we're actually not using it yet widely because I don't want to backport it 
to our prod kernel until we're reasonably agreed on the format :-)

My main concern is making sure that parsing is reliable, and there's as
little "well, there shouldn't ever be a <char> here" as possible. That's why I
preferred originally to use the already well established char array/printk
rules where possible, since they're mature and well tested.

I'm not against some other solution though, as long as it meets these
requirements. It looks like your proposed format with escaping prior to sending 
to userspace also meets that requirement.

>    $ cat pf.py
>    #!/usr/bin/env python
>    with open("/sys/kernel/debug/printk/formats/vmlinux") as f:
>        raw_fmts = f.read().split("\x00")[:-1]
>        for raw_fmt in raw_fmts:
>            level, fmt = raw_fmt[1], raw_fmt[2:]
>            print(f"Level {level}: {fmt!r}")
>
>I wonder how it would look in another scripting languages, like
>bash or perl. Any non-printable character is tricky and would
>complicate it.

It's really not that complicated there, either. Since you bring up bash, it's
even less work than the Python solution:

     while IFS= read -r -d $'\0' fmt; do
         printf 'Level %s: %q\n' "${fmt:1:1}" "${fmt:2}"
     done < /sys/kernel/debug/printk/formats/vmlinux

The changelog describes the use case: automated detection of printk fmts
changing. For that reason, I don't understand why there's a desire to produce a
human readable format by default when the only known consumer of this is going
to be automation anyway.

If a use case for that comes up, we can always have a directory producing human 
readable versions. I personally don't see the need though.

As long as it's reliably parseable though, I won't die on this hill, I just 
don't understand the rationale :-)

>> Re: not being not safe for machine processing because it only works for a
>> single digit, I'm a little confused. KERN_* levels are, as far as I know,
>> only a single byte wide, and we rely on that already (eg. in
>> printk_skip_header()).
>
>It is great that you mentioned it. KERN_ levels are implementation
>detail.
>
>It used to be "<level>". The binary KERN_SOH has been introduced
>in v3.6-rc1 by the commit 04d2c8c83d0e3ac5f ("printk: convert
>the format for KERN_<LEVEL> to a 2 byte pattern").
>
>In v4.9-rc1, the commit 4bcc595ccd80decb4 ("printk: reinstate KERN_CONT
>for printing continuation lines") added the possibility to define
>both KERN_LEVEL + KERN_CONT at the same time. It is not handled
>by your python snippet above.

Thanks, this is a good callout. I will make sure v5 handles that.

In a hypothetical scenario where we do go towards something human-readable, how 
do you perceive that should look? Something like this?

     % less ...
     <c, 5> 'Some fmt with cont + level 5\n'
     <5> 'Some fmt with only level 5\n'
     <c> 'Some fmt with only LOG_CONT\n'

>> We also already have precedent for
>> null-separation/control characters in (for example) /proc/pid/cmdline.
>
>Yes, there is a precedent. But it does not mean that we should not
>try to do it better.

To be clear, I'm actually asserting that I believe the machine-readable version 
_is_ better, not that it's simply unnecessary to produce a human-readable 
version :-)

As mentioned earlier in this e-mail though, it's not a hill I want to die on.  
If you believe it should be human-readable, as long as its reliably parseable, 
I'm happy to do that.