Message-ID: <39cf3aab-7073-443b-8876-9de65f4c315e@efficios.com>
Date: Wed, 9 Jul 2025 09:46:14 -0400
From: Mathieu Desnoyers <mathieu.desnoyers@...icios.com>
To: Jens Remus <jremus@...ux.ibm.com>, Steven Rostedt <rostedt@...dmis.org>
Cc: Steven Rostedt <rostedt@...nel.org>, linux-kernel@...r.kernel.org,
 linux-trace-kernel@...r.kernel.org, bpf@...r.kernel.org, x86@...nel.org,
 Masami Hiramatsu <mhiramat@...nel.org>, Josh Poimboeuf
 <jpoimboe@...nel.org>, Peter Zijlstra <peterz@...radead.org>,
 Ingo Molnar <mingo@...nel.org>, Jiri Olsa <jolsa@...nel.org>,
 Namhyung Kim <namhyung@...nel.org>, Thomas Gleixner <tglx@...utronix.de>,
 Andrii Nakryiko <andrii@...nel.org>, Indu Bhagat <indu.bhagat@...cle.com>,
 "Jose E. Marchesi" <jemarch@....org>,
 Beau Belgrave <beaub@...ux.microsoft.com>,
 Linus Torvalds <torvalds@...ux-foundation.org>,
 Andrew Morton <akpm@...ux-foundation.org>, Jens Axboe <axboe@...nel.dk>,
 Florian Weimer <fweimer@...hat.com>, Sam James <sam@...too.org>,
 Heiko Carstens <hca@...ux.ibm.com>, Vasily Gorbik <gor@...ux.ibm.com>
Subject: Re: [PATCH v8 06/12] unwind_user/sframe: Wire up unwind_user to
 sframe

On 2025-07-09 03:58, Jens Remus wrote:
> On 08.07.2025 22:11, Steven Rostedt wrote:
>> On Tue, 8 Jul 2025 15:58:56 -0400
>> Mathieu Desnoyers <mathieu.desnoyers@...icios.com> wrote:
>>
>>>> @@ -111,6 +128,8 @@ static int unwind_user_start(struct unwind_user_state *state)
>>>>    
>>>>    	if (IS_ENABLED(CONFIG_HAVE_UNWIND_USER_COMPAT_FP) && in_compat_mode(regs))
>>>>    		state->type = UNWIND_USER_TYPE_COMPAT_FP;
>>>> +	else if (current_has_sframe())
>>>> +		state->type = UNWIND_USER_TYPE_SFRAME;
>>>
>>> I think you'll want to update the state->type during the
>>> traversal (in next()), because depending on whether
>>> sframe is available for a given memory area of code
>>> or not, the next() function can use either frame pointers
>>> or sframe during the same traversal. It would be good
>>> to know which is used after each specific call to next().
>>
>>  From my understanding this sets up what is available for the task at the
>> beginning.
>>
>> So once we say "this task has sframes" it will try to use it every time. In
>> next we have:
>>
>> 	if (compat_fp_state(state)) {
>> 		frame = &compat_fp_frame;
>> 	} else if (sframe_state(state)) {
>> 		/* sframe expects the frame to be local storage */
>> 		frame = &_frame;
>> 		if (sframe_find(state->ip, frame)) {
>> 			if (!IS_ENABLED(CONFIG_HAVE_UNWIND_USER_FP))
>> 				goto done;
>> 			frame = &fp_frame;
>> 		}
>> 	} else if (fp_state(state)) {
>> 		frame = &fp_frame;
>> 	} else {
>> 		goto done;
>> 	}
>>
>> Where if sframe_find() fails and we switch over to frame pointers, if frame
>> pointers works, we can continue. But the next iteration, where the frame
>> pointer finds the previous ip, that ip may be in the sframe section again.
>>
>> I've seen this work with my trace_printk()s. A function from code that is
>> running sframes calls into a library function that has frame pointers. The
>> walk walks through the frame pointers in the library, and when it hits the
>> code that has sframes, it starts using that again.
> 
> I think Mathieu has a point, as unwind_user_next() calls the optional
> architecture-specific arch_unwind_user_next() at the end.  The x86
> implementation does state->type specific processing (for
> UNWIND_USER_TYPE_COMPAT_FP).
> 
>> If we switched the state to just FP, it will never try to use sframes.
>>
>> So this state is more about "what does this task have" than what was used
>> per iteration.
> 
> While there is currently no fallback to UNWIND_USER_TYPE_COMPAT_FP that
> would strictly require this, it could be useful to have both information.
> 
> Or the logic in unwind_user_start(), unwind_user_next(), and *_state()
> may need to be adjusted so that state->type reflects the currently used
> method, which unwind_user_next() determines and sets anew for every step.

I concur with Jens. I think we should keep track of both:

1) available unwind methods,

2) unwind method used for the current frame.

E.g.:

/*
  * unwind types, listed in priority order: lower numbers are
  * attempted first if available.
  */
enum unwind_user_type_bits {
         UNWIND_USER_TYPE_SFRAME_BIT = 0,
         UNWIND_USER_TYPE_FP_BIT = 1,
         UNWIND_USER_TYPE_COMPAT_FP_BIT = 2,

	_NR_UNWIND_USER_TYPE_BITS,
};

enum unwind_user_type {
         UNWIND_USER_TYPE_NONE = 0,
         UNWIND_USER_TYPE_SFRAME = (1U << UNWIND_USER_TYPE_SFRAME_BIT),
         UNWIND_USER_TYPE_FP = (1U << UNWIND_USER_TYPE_FP_BIT),
         UNWIND_USER_TYPE_COMPAT_FP = (1U <<  UNWIND_USER_TYPE_COMPAT_FP_BIT),
};

And have the following fields in struct unwind_user_state:

/* Unwind time used for the most recent unwind traversal iteration. */
enum unwind_user_type current_type;

/* Unwind types available in the current context. Bitmask of enum unwind_user_type. */
unsigned int available_types;

So as we end up adding stuff like registered JIT unwind info, we will
want to expand the "available types". And it makes sense to both keep
track of all available types (as a way to quickly know which mechanisms
we need to query for the current task) *and* to let the caller know
which unwind type was used for the current frame.

And AFAIU we'd be inserting a "jit unwind info" type between SFRAME and FP in
the future, because the jit unwind info would be more reliable than FP. This
would require that we bump the number for FP and COMPAT_FP, but that would
be OK because this is not ABI.

Thoughts ?

Thanks,

Mathieu

> 
> Regards,
> Jens


-- 
Mathieu Desnoyers
EfficiOS Inc.
https://www.efficios.com

Powered by blists - more mailing lists