linux-kernel - Re: Instrumentation and RCU

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:   Tue, 10 Mar 2020 13:40:03 -0400 (EDT)
From:   Mathieu Desnoyers <mathieu.desnoyers@...icios.com>
To:     Thomas Gleixner <tglx@...utronix.de>
Cc:     linux-kernel <linux-kernel@...r.kernel.org>,
        Peter Zijlstra <peterz@...radead.org>,
        rostedt <rostedt@...dmis.org>,
        Masami Hiramatsu <mhiramat@...nel.org>,
        Alexei Starovoitov <ast@...nel.org>,
        paulmck <paulmck@...nel.org>,
        "Joel Fernandes, Google" <joel@...lfernandes.org>,
        Frederic Weisbecker <frederic@...nel.org>
Subject: Re: Instrumentation and RCU

----- On Mar 10, 2020, at 12:48 PM, Thomas Gleixner tglx@...utronix.de wrote:

> Mathieu Desnoyers <mathieu.desnoyers@...icios.com> writes:
[...]
> See the entry series I'm working on. Aside of moving all this nonsense
> into C-code it splits lockdep and tracing so it looks like this:
> 
>            lockdep_hardirqs_off();
>            user_exit_irqsoff();
>            __trace_hardirqs_off();
> 
> The latter uses regular RCU and not the scru/rcu_irq dance.
> 

Awesome :)

> 
>>>> If a tracer recurses, or if a tracer attempts to trace another tracer, the
>>>> instrumentation would break the recursion chain by preventing instrumentation
>>>> from firing. If we end up caring about tracers tracing other tracers, we could
>>>> have one distinct flag per tracer and let each tracer break the recursion chain.
>>>>
>>>> Having this flag per kernel stack rather than per CPU or per thread would
>>>> allow tracing of nested interrupt handlers (and NMIs), but would break
>>>> call chains both within the same stack or going through a trap. I think
>>>> it could be a nice complementary safety net to handle mishaps in a non-fatal
>>>> way.
>>> 
>>> That works as long as none of this uses breakpoint based patching to
>>> dynamically disable/enable stuff.
>>
>> I'm clearly missing something here. I was expecting the "in_tracing" flag trick
>> to be able to fix the breakpoint recursion issue. What is the problem I'm
>> missing
>> here ?
> 
> How do you "fix" that when you can't reach the tracepoint because you
> trip over a breakpoint and then while trying to fixup that stuff you hit
> another one?

I may still be missing something, but if the fixup code (AFAIU the code performing
the out-of-line single-stepping of the original instruction) belongs to a section
hidden from instrumentation, it should not be an issue.

The basic idea would be, e.g. pseudo-code for int3:

<int3>  <---- in section which cannot be instrumented
if (recursion_ctx->in_tracer) {
   single-step original instruction
   iret
}
[...] prepare stuff
recursion_ctx->in_tracer = true;
instr_allowed()

call external kernel functions (which can be instrumented)

instr_disallowed()
recursion_ctx->in_tracer = false;
single-step original instruction
iret

The purpose of the "in_tracer" flag is to protect whatever is done within external
kernel functions (which can be instrumented) from triggering tracer recursion. It
needs to be combined with hiding of early/late low-level entry/exit functions from
instrumentation (as you propose) to work.

Thanks,

Mathieu

-- 
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com