Message-ID: <cf26c916-74a0-4945-981f-0c3c9d9bfd40@oracle.com>
Date: Wed, 23 Jul 2025 01:16:47 -0700
From: Indu Bhagat <indu.bhagat@...cle.com>
To: Mathieu Desnoyers <mathieu.desnoyers@...icios.com>,
        rostedt <rostedt@...dmis.org>
Cc: linux-kernel@...r.kernel.org, linux-trace-kernel@...r.kernel.org,
        bpf@...r.kernel.org, x86@...nel.org,
        Masami Hiramatsu <mhiramat@...nel.org>,
        Josh Poimboeuf <jpoimboe@...nel.org>,
        Peter Zijlstra <peterz@...radead.org>, Ingo Molnar <mingo@...nel.org>,
        Jiri Olsa <jolsa@...nel.org>, Namhyung Kim <namhyung@...nel.org>,
        Thomas Gleixner <tglx@...utronix.de>,
        Andrii Nakryiko <andrii@...nel.org>,
        "Jose E. Marchesi" <jemarch@....org>,
        Beau Belgrave <beaub@...ux.microsoft.com>,
        Jens Remus
 <jremus@...ux.ibm.com>,
        Linus Torvalds <torvalds@...ux-foundation.org>,
        Andrew Morton <akpm@...ux-foundation.org>,
        Jens Axboe <axboe@...nel.dk>, Florian Weimer <fweimer@...hat.com>,
        Sam James <sam@...too.org>, Brian Robbins <brianrob@...rosoft.com>,
        Elena Zannoni <elena.zannoni@...cle.com>
Subject: Re: [RFC] New codectl(2) system call for sframe registration

On 7/22/25 11:49 AM, Mathieu Desnoyers wrote:
> On 2025-07-22 14:21, Indu Bhagat wrote:
>> On 7/21/25 8:20 AM, Mathieu Desnoyers wrote:
>>> Hi!
>>>
>>> I've written up an RFC for a new system call to handle sframe 
>>> registration
>>> for shared libraries. There has been interest to cover both sframe in
>>> the short term, but also JIT use-cases in the long term, so I'm
>>> covering both here in this RFC to provide the full context. 
>>> Implementation
>>> wise we could start by only covering the sframe use-case.
>>>
>>> I've called it "codectl(2)" for now, but I'm of course open to feedback.
>>>
>>> For ELF, I'm including the optional pathname, build id, and debug link
>>> information which are really useful to translate from instruction 
>>> pointers
>>> to executable/library name, symbol, offset, source file, line number.
>>> This is what we are using in LTTng-UST and Babeltrace debug-info filter
>>> plugin [1], and I think this would be relevant for kernel tracers as 
>>> well
>>> so they can make the resulting stack traces meaningful to users.
>>>
>>> sys_codectl(2)
>>> =================
>>>
>>> * arg0: unsigned int @option:
>>>
>>> /* Additional labels can be added to enum code_opt, for 
>>> extensibility. */
>>>
>>> enum code_opt {
>>>      CODE_REGISTER_ELF,
>>>      CODE_REGISTER_JIT,
>>>      CODE_UNREGISTER,
>>> };
>>>
>>> * arg1: void * @info
>>>
>>> /* if (@option == CODE_REGISTER_ELF) */
>>>
>>> /*
>>>   * text_start, text_end, sframe_start, sframe_end allow unwinding of 
>>> the
>>>   * call stack.
>>>   *
>>>   * elf_start, elf_end, pathname, and either build_id or debug_link 
>>> allows
>>>   * mapping instruction pointers to file, symbol, offset, and source 
>>> file
>>>   * location.
>>>   */
>>> struct code_elf_info {
>>> :   __u64 elf_start;
>>>      __u64 elf_end;
>>
>> What are the elf_start , elf_end intended for ?
> 
> The intent is to know at which address the first loadable segment of
> the shared object is mapped (elf_start), and the size of the shared
> object mapping, which is the sum of the size of its PT_LOAD segments.
> 
> This allows tooling to easily lookup which addresses belong to that
> shared object, for any loaded segment, whether it's code or data.
> 
>>
>>>      __u64 text_start;
>>>      __u64 text_end;
>>>      __u64 sframe_start;
>>>      __u64 sframe_end;
>>>      __u64 pathname;              /* char *, NULL if unavailable. */
>>>
>>>      __u64 build_id;              /* char *, NULL if unavailable. */
>>>      __u64 debug_link_pathname;   /* char *, NULL if unavailable. */
>>>      __u32 build_id_len;
>>>      __u32 debug_link_crc;
>>> };
>>>
>>>
>>> /* if (@option == CODE_REGISTER_JIT) */
>>>
>>> /*
>>>   * Registration of sorted JIT unwind table: The reserved memory area is
>>>   * of size reserved_len. Userspace increases used_len as new code is
>>>   * populated between text_start and text_end. This area is populated in
>>>   * increasing address order, and its ABI requires to have no 
>>> overlapping
>>>   * fre. This fits the common use-case where JITs populate code into
>>>   * a given memory area by increasing address order. The sorted unwind
>>>   * tables can be chained with a singly-linked list as they become full.
>>>   * Consecutive chained tables are also in sorted text address order.
>>>   *
>>>   * Note: if there is an eventual use-case for unsorted jit unwind 
>>> table,
>>>   * this would be introduced as a new "code option".
>>>   */
>>>
>>> struct code_jit_info {
>>>      __u64 text_start;      /* text_start >= addr */
>>>      __u64 text_end;        /* addr < text_end */
>>>      __u64 unwind_head;     /* struct code_jit_unwind_table * */
>>> };
>>>
>>
>> I see the discussion has evolved here with the general sentiment that 
>> the JIT part needs to be kept in mind for a rough sketch but cannot be 
>> designed at this time. But two comments (if we keep JIT part in the 
>> discussion):
>>    - I think we need to keep __u64 unwind_head not a pointer to a 
>> defined structure (struct code_jit_unwind_table * above), but some 
>> opaque type like we have for SFrame case.
> 
> What is the reason for making this an opaque type for sframe ?
> 

So that the system call only does the work of registering the memory of 
specific size as stack trace data for addr range (text_start, text_end). 
  IIUC, in the current proposal, the format of the stack trace 
information is exposed in the arg. So when the format evolves, this will 
mean additional management via some flags?

> 
>>    - The reserved_len should ideally be a part of code_jit_info, so 
>> the length can be known without parsing the contents.
> 
> I've placed reserved_len within the unwind table because I planned to
> have the jit information for a given range of text be a linked list of
> tables. Therefore, if one table fills up, then another table can be
> chained at the tail. Having the reserved_len part of each table makes
> things easier to combine into a linked list.
> 
> Thanks for your feedback !
> 
> Mathieu
> 
>>
>>> struct code_jit_unwind_fre {
>>>      /*
>>>       * Contains info similar to sframe, allowing unwind for a given
>>>       * code address range.
>>>       */
>>>      __u32 size;
>>>      __u32 ip_off;  /* offset from text_start */
>>>      __s32 cfa_off;
>>>      __s32 ra_off;
>>>      __s32 fp_off;
>>>      __u8 info;
>>> };
>>>
>>> struct code_jit_unwind_table {
>>>      __u64 reserved_len;
>>>      __u64 used_len; /*
>>>                       * Incremented by userspace (store-release), 
>>> read by
>>>                       * the kernel (load-acquire).
>>>                       */
>>>      __u64 next;     /* Chain with next struct code_jit_unwind_table. */
>>>      struct code_jit_unwind_fre fre[];
>>> };
>>>
>>> /* if (@option == CODE_UNREGISTER) */
>>>
>>> void *info
>>>
>>> * arg2: size_t info_size
>>>
>>> /*
>>>   * Size of @info structure, allowing extensibility. See
>>>   * copy_struct_from_user().
>>>   */
>>>
>>> * arg3: unsigned int flags (0)
>>>
>>> /* Flags for extensibility. */
>>>
>>> Your feedback is welcome,
>>>
>>> Thanks,
>>>
>>> Mathieu
>>>
>>> [1] https://babeltrace.org/docs/v2.0/man7/babeltrace2-filter.lttng- 
>>> utils.debug-info.7/
>>>
>>
> 
> 

Powered by blists - more mailing lists