linux-kernel - Re: [PATCH] printk: Add best-effort printk() buffering.

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-Id: <201705082205.DAE12471.FOHJSFOQFtOVML@I-love.SAKURA.ne.jp>
Date:   Mon, 8 May 2017 22:05:13 +0900
From:   Tetsuo Handa <penguin-kernel@...ove.SAKURA.ne.jp>
To:     sergey.senozhatsky.work@...il.com
Cc:     linux-kernel@...r.kernel.org, mingo@...hat.com,
        peterz@...radead.org, pmladek@...e.com,
        sergey.senozhatsky@...il.com, rostedt@...dmis.org
Subject: Re: [PATCH] printk: Add best-effort printk() buffering.

Sergey Senozhatsky wrote:
> sorry for the delay.

No problem.

> 
> On (04/30/17 22:54), Tetsuo Handa wrote:
> > Sometimes we want to printk() multiple lines in a group without being
> > disturbed by concurrent printk() from interrupts and/or other threads.
> > For example, mixed printk() output of multiple thread's dump makes it
> > hard to interpret.
> 
> hm, it's very close to what printk-safe does [and printk-nmi, of course].
> the difference is that buffered-printk does not disable local IRQs,
> unlike printk-safe, which has to do it by design. so the question is,
> can buffered-printk impose atomicity requirements? it seems that it can
> (am I wrong?). and, if so, then can we use printk-safe instead? we can
> add a new printk_buffered_begin/printk_buffered_end API, for example,
> (or enter/exit) for that purpose, that would set a buffered-printk
> `printk_context' bit so we can flush buffers in a "special way", not via IRQ
> work, and may be avoid message loss (printk-safe buffers are bigger in size
> than proposed PAGE_SIZE buffers).

printk_buffered_begin()/printk_buffered_end() corresponds to
get_printk_buffer()/put_printk_buffer().
printk_context() distinguishes atomic contexts.
flush_printk_buffer() flushes from non-NMI context.

What does atomicity requirements mean?

> 
> 
> > This patch introduces fixed-sized statically allocated buffers for
> > buffering printk() output for each thread/context in best effort
> > (i.e. up to PAGE_SIZE bytes, up to 16 concurrent printk() users).
> 
> hm, 16 is rather random, it's too much for UP and probably not enough for
> a 240 CPUs system. for the time being there are 3 buffered-printk users
> (as far as I can see), but who knows how more will be added in the future.
> each CPU can have overlapping printks from process, IRQ and NMI contexts.
> for NMI we use printk-nmi buffers, so it's out of the list; but, in general,
> *it seems* that we better depend on the number of CPUs the system has.
> which, once again, returns us back to printk-safe...
> 
> thoughts?

I can make 16 a CONFIG_ option. Would you read 201705031521.EIJ39594.MFtOVOHSFLFOJQ@...ove.SAKURA.ne.jp ?
But as long as actually writing to console devices is slow, message loss
is inevitable no matter how big buffer is used. Rather, I'd expect an API
which allows printk() users in schedulable context (e.g. kmallocwd and/or
warn_alloc() for reporting allocation stalls) to wait until written to
console devices. That will more likely to reduce message loss.

> > +	while (1) {
> > +		char *text = ptr->buf;
> > +		unsigned int text_len = ptr->used;
> > +		char *cp = memchr(text, '\n', text_len);
> > +		char c;
> 
> what guarantees that there'll always be a terminating newline?

Nothing guarantees. Why need such a guarantee?