[<prev] [next>] [thread-next>] [day] [month] [year] [list]
Message-ID: <CAPB3MF5x5rSsYCKutpo1f=1DaQbz30QM6ny7fnB9hMGmwfkdbA@mail.gmail.com>
Date: Sat, 30 Aug 2025 22:39:42 +0800
From: cam enih <nanericwang@...il.com>
To: Qingfang Deng <dqfext@...il.com>
Cc: andrew+netdev@...n.ch, davem@...emloft.net, edumazet@...gle.com,
kuba@...nel.org, kuniyu@...gle.com, linux-kernel@...r.kernel.org,
linux-ppp@...r.kernel.org, nbd@....name, netdev@...r.kernel.org,
pabeni@...hat.com, pablo@...filter.org
Subject: Re: [Regression Bug] Re: [PATCH net v3 2/2] ppp: fix race conditions
in ppp_fill_forward_path
Here comes more details, and it might not be your commit that causes
the panic. sorry but I don't know where to go from here.
```
Aug 30 22:00:20 localhost systemd-networkd[281]: Failed to parse
hostname, ignoring: Invalid argument
Aug 30 22:00:20 localhost systemd-networkd[281]: br0: DHCPv4 server:
DISCOVER (0xf1ac0b63)
Aug 30 22:00:20 localhost kernel: BUG: kernel NULL pointer
dereference, address: 0000000000000058
Aug 30 22:00:20 localhost kernel: #PF: supervisor read access in kernel mode
Aug 30 22:00:20 localhost kernel: #PF: error_code(0x0000) - not-present page
Aug 30 22:00:20 localhost kernel: PGD 0 P4D 0
Aug 30 22:00:20 localhost kernel: Oops: Oops: 0000 [#1] PREEMPT_RT SMP
Aug 30 22:00:20 localhost kernel: CPU: 1 UID: 981 PID: 281 Comm:
systemd-network Not tainted 6.12.44-xanmod1-1-lts #1
Aug 30 22:00:20 localhost kernel: Hardware name: Default string
Default string/Default string, BIOS 5.19 03/30/2021
Aug 30 22:00:20 localhost kernel: RIP:
0010:ip_route_output_key_hash_rcu+0x778/0x930
Aug 30 22:00:20 localhost kernel: Code: 8b 45 c0 48 8b 40 30 65 48 03
05 33 69 be 76 e9 eb fd ff ff 4c 89 45 c8 e8 e5 22 aa ff 4c 8b 45 c8
e9 ba fc ff ff 49 8b 45 18 <48> 8b 40 58 48 3d a0 b7 87 89 0f 84 f7 00
00 00 48 89 c2 48 8d 78
Aug 30 22:00:20 localhost kernel: RSP: 0018:ffffb0eec0a979c8 EFLAGS: 00010246
Aug 30 22:00:20 localhost kernel: RAX: 0000000000000000 RBX:
ffffb0eec0a97b00 RCX: 0000000000000005
Aug 30 22:00:20 localhost kernel: RDX: 0000000000000000 RSI:
000000000002a424 RDI: ffffffff89ab8c40
Aug 30 22:00:20 localhost kernel: RBP: ffffb0eec0a97a28 R08:
0000000000000020 R09: 0000000000000000
Aug 30 22:00:20 localhost kernel: R10: ffff89d68577bc00 R11:
0000000000000003 R12: ffff89d683779000
Aug 30 22:00:20 localhost kernel: R13: ffffb0eec0a97a40 R14:
0000000000000008 R15: 0000000090000000
Aug 30 22:00:20 localhost kernel: FS: 00007fed08af58c0(0000)
GS:ffff89d7f7d00000(0000) knlGS:0000000000000000
Aug 30 22:00:20 localhost kernel: CS: 0010 DS: 0000 ES: 0000 CR0:
0000000080050033
Aug 30 22:00:20 localhost kernel: CR2: 0000000000000058 CR3:
0000000116589001 CR4: 0000000000b70ef0
Aug 30 22:00:20 localhost kernel: Call Trace:
Aug 30 22:00:20 localhost kernel: <TASK>
Aug 30 22:00:20 localhost kernel: ? autoremove_wake_function+0x70/0x70
Aug 30 22:00:20 localhost kernel: ip_route_output_flow+0x73/0xc0
Aug 30 22:00:20 localhost kernel: udp_sendmsg+0x47c/0xd50
Aug 30 22:00:20 localhost kernel: ? ip_frag_init+0x60/0x60
Aug 30 22:00:20 localhost kernel: inet_sendmsg+0x37/0x50
Aug 30 22:00:20 localhost kernel: ? inet_sendmsg+0x37/0x50
Aug 30 22:00:20 localhost kernel: ____sys_sendmsg+0x235/0x250
Aug 30 22:00:20 localhost kernel: ___sys_sendmsg+0x1ca/0x210
Aug 30 22:00:20 localhost kernel: __sys_sendmsg+0xcf/0x120
Aug 30 22:00:20 localhost kernel: __x64_sys_sendmsg+0x1c/0x20
Aug 30 22:00:20 localhost kernel: x64_sys_call+0x719/0x1780
Aug 30 22:00:20 localhost kernel: do_syscall_64+0x79/0x150
Aug 30 22:00:20 localhost kernel: ? syscall_exit_to_user_mode+0x15/0xf0
Aug 30 22:00:20 localhost kernel: ? do_syscall_64+0x85/0x150
Aug 30 22:00:20 localhost kernel: ? do_syscall_64+0x85/0x150
Aug 30 22:00:20 localhost kernel: entry_SYSCALL_64_after_hwframe+0x4b/0x53
Aug 30 22:00:20 localhost kernel: RIP: 0033:0x7fed0933f1ce
Aug 30 22:00:20 localhost kernel: Code: 4d 89 d8 e8 64 be 00 00 4c 8b
5d f8 41 8b 93 08 03 00 00 59 5e 48 83 f8 fc 74 11 c9 c3 0f 1f 80 00
00 00 00 48 8b 45 10 0f 05 <c9> c3 83 e2 39 83 fa 08 75 e7 e8 13 ff ff
ff 0f 1f 00 f3 0f 1e fa
Aug 30 22:00:20 localhost kernel: RSP: 002b:00007ffc62fe8b20 EFLAGS:
00000202 ORIG_RAX: 000000000000002e
Aug 30 22:00:20 localhost kernel: RAX: ffffffffffffffda RBX:
0000053b981d1d80 RCX: 00007fed0933f1ce
Aug 30 22:00:20 localhost kernel: RDX: 0000000000000000 RSI:
00007ffc62fe8b90 RDI: 0000000000000017
Aug 30 22:00:20 localhost kernel: RBP: 00007ffc62fe8b30 R08:
0000000000000000 R09: 0000000000000000
Aug 30 22:00:20 localhost kernel: R10: 0000000000000000 R11:
0000000000000202 R12: 0000053b981d1d80
Aug 30 22:00:20 localhost kernel: R13: 0000053b981d0b80 R14:
0000053b98034800 R15: 0000053b9803481c
Aug 30 22:00:20 localhost kernel: </TASK>
Aug 30 22:00:20 localhost kernel: Modules linked in: xt_mark xt_owner
xt_TPROXY nf_tproxy_ipv4 pppoe pppox af_packet sch_cake bridge stp llc
xt_DSCP xt_set xt_TCPMSS xt_tcpudp iptable_mangle xt_connlimit
nf_conncount xt_conntrack iptable_filter xt_MASQUERADE iptable_nat
nf_nat msr ip_set_hash_net ip_set hid_generic intel_rapl_msr evdev
nls_ascii coretemp intel_tcc_cooling x86_pkg_temp_thermal
intel_powerclamp nls_cp437 sha256_ssse3 vfat rapl fat intel_cstate
intel_uncore i2c_i801 usbhid spi_intel_pci spi_intel i2c_smbus hid igb
ptp pps_core i2c_algo_bit processor_thermal_device_pci_legacy
intel_soc_dts_iosf fan processor_thermal_device hwmon
processor_thermal_wt_hint thermal processor_thermal_rfim i2c_core
processor_thermal_rapl intel_rapl_common iosf_mbi
processor_thermal_wt_req processor_thermal_power_floor
processor_thermal_mbox int340x_thermal_zone acpi_pad button
ppp_generic slhc nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 nfnetlink
ip_tables x_tables ipv6 xhci_pci xhci_hcd usbcore usb_common
Aug 30 22:00:20 localhost kernel: CR2: 0000000000000058
Aug 30 22:00:20 localhost kernel: ---[ end trace 0000000000000000 ]---
Aug 30 22:00:20 localhost kernel: RIP:
0010:ip_route_output_key_hash_rcu+0x778/0x930
Aug 30 22:00:20 localhost kernel: Code: 8b 45 c0 48 8b 40 30 65 48 03
05 33 69 be 76 e9 eb fd ff ff 4c 89 45 c8 e8 e5 22 aa ff 4c 8b 45 c8
e9 ba fc ff ff 49 8b 45 18 <48> 8b 40 58 48 3d a0 b7 87 89 0f 84 f7 00
00 00 48 89 c2 48 8d 78
Aug 30 22:00:20 localhost kernel: RSP: 0018:ffffb0eec0a979c8 EFLAGS: 00010246
Aug 30 22:00:20 localhost kernel: RAX: 0000000000000000 RBX:
ffffb0eec0a97b00 RCX: 0000000000000005
Aug 30 22:00:20 localhost kernel: RDX: 0000000000000000 RSI:
000000000002a424 RDI: ffffffff89ab8c40
Aug 30 22:00:20 localhost kernel: RBP: ffffb0eec0a97a28 R08:
0000000000000020 R09: 0000000000000000
Aug 30 22:00:20 localhost kernel: R10: ffff89d68577bc00 R11:
0000000000000003 R12: ffff89d683779000
Aug 30 22:00:20 localhost kernel: R13: ffffb0eec0a97a40 R14:
0000000000000008 R15: 0000000090000000
Aug 30 22:00:20 localhost kernel: FS: 00007fed08af58c0(0000)
GS:ffff89d7f7d00000(0000) knlGS:0000000000000000
Aug 30 22:00:20 localhost kernel: CS: 0010 DS: 0000 ES: 0000 CR0:
0000000080050033
Aug 30 22:00:20 localhost kernel: CR2: 0000000000000058 CR3:
0000000116589001 CR4: 0000000000b70ef0
Aug 30 22:00:20 localhost kernel: note: systemd-network[281] exited
with irqs disabled
Aug 30 22:00:20 localhost kernel: ------------[ cut here ]------------
Aug 30 22:00:20 localhost kernel: Voluntary context switch within RCU
read-side critical section!
Aug 30 22:00:20 localhost kernel: WARNING: CPU: 1 PID: 281 at
rcu_note_context_switch+0x3f7/0x550
Aug 30 22:00:20 localhost kernel: Modules linked in: xt_mark xt_owner
xt_TPROXY nf_tproxy_ipv4 pppoe pppox af_packet sch_cake bridge stp llc
xt_DSCP xt_set xt_TCPMSS xt_tcpudp iptable_mangle xt_connlimit
nf_conncount xt_conntrack iptable_filter xt_MASQUERADE iptable_nat
nf_nat msr ip_set_hash_net ip_set hid_generic intel_rapl_msr evdev
nls_ascii coretemp intel_tcc_cooling x86_pkg_temp_thermal
intel_powerclamp nls_cp437 sha256_ssse3 vfat rapl fat intel_cstate
intel_uncore i2c_i801 usbhid spi_intel_pci spi_intel i2c_smbus hid igb
ptp pps_core i2c_algo_bit processor_thermal_device_pci_legacy
intel_soc_dts_iosf fan processor_thermal_device hwmon
processor_thermal_wt_hint thermal processor_thermal_rfim i2c_core
processor_thermal_rapl intel_rapl_common iosf_mbi
processor_thermal_wt_req processor_thermal_power_floor
processor_thermal_mbox int340x_thermal_zone acpi_pad button
ppp_generic slhc nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 nfnetlink
ip_tables x_tables ipv6 xhci_pci xhci_hcd usbcore usb_common
Aug 30 22:00:20 localhost kernel: CPU: 1 UID: 981 PID: 281 Comm:
systemd-network Tainted: G D 6.12.44-xanmod1-1-lts #1
Aug 30 22:00:20 localhost kernel: Tainted: [D]=DIE
Aug 30 22:00:20 localhost kernel: Hardware name: Default string
Default string/Default string, BIOS 5.19 03/30/2021
Aug 30 22:00:20 localhost kernel: RIP: 0010:rcu_note_context_switch+0x3f7/0x550
Aug 30 22:00:20 localhost kernel: Code: ff ff 45 85 c9 0f 84 06 fd ff
ff 48 89 b9 a8 00 00 00 e9 fa fc ff ff 48 c7 c7 20 76 98 89 c6 05 df
8d be 00 01 e8 09 2a f8 ff <0f> 0b e9 48 fc ff ff 44 89 4d d0 44 89 45
d4 48 89 4d d8 48 89 7d
Aug 30 22:00:20 localhost kernel: RSP: 0018:ffffb0eec0a97b70 EFLAGS: 00010046
Aug 30 22:00:20 localhost kernel: RAX: 0000000000000000 RBX:
ffff89d7f7d258c0 RCX: 0000000000000027
Aug 30 22:00:20 localhost kernel: RDX: ffff89d7f7d1c748 RSI:
0000000000000001 RDI: ffff89d7f7d1c740
Aug 30 22:00:20 localhost kernel: RBP: ffffb0eec0a97ba0 R08:
0000000000000e43 R09: ffffffff8998765e
Aug 30 22:00:20 localhost kernel: R10: ffffffff8998765f R11:
0000000000000000 R12: ffff89d68d46a900
Aug 30 22:00:20 localhost kernel: R13: 0000000000000000 R14:
ffff89d68d46a900 R15: ffff89d68d46a900
Aug 30 22:00:20 localhost kernel: FS: 0000000000000000(0000)
GS:ffff89d7f7d00000(0000) knlGS:0000000000000000
Aug 30 22:00:20 localhost kernel: CS: 0010 DS: 0000 ES: 0000 CR0:
0000000080050033
Aug 30 22:00:20 localhost kernel: CR2: 0000000000000058 CR3:
0000000190e18003 CR4: 0000000000b70ef0
Aug 30 22:00:20 localhost kernel: Call Trace:
Aug 30 22:00:20 localhost kernel: <TASK>
Aug 30 22:00:20 localhost kernel: __schedule+0x73/0x720
Aug 30 22:00:20 localhost kernel: ? swake_up_one+0x69/0x80
Aug 30 22:00:20 localhost kernel: schedule+0x26/0xe0
Aug 30 22:00:20 localhost kernel: schedule_timeout+0x106/0x120
Aug 30 22:00:20 localhost kernel: wait_for_completion_state+0x12a/0x200
Aug 30 22:00:20 localhost kernel: __wait_rcu_gp+0x11f/0x1b0
Aug 30 22:00:20 localhost kernel: synchronize_rcu_normal.part.0+0x68/0x80
Aug 30 22:00:20 localhost kernel: ? __call_rcu_common+0x6e0/0x6e0
Aug 30 22:00:20 localhost kernel: ? get_completed_synchronize_rcu+0x10/0x10
Aug 30 22:00:20 localhost kernel: synchronize_rcu_normal+0xc6/0xd0
Aug 30 22:00:20 localhost kernel: synchronize_rcu_expedited+0x1bb/0x210
Aug 30 22:00:20 localhost kernel: ? change_mnt_propagation+0x32b/0x350
Aug 30 22:00:20 localhost kernel: ? __put_mountpoint.part.0+0x83/0x90
Aug 30 22:00:20 localhost kernel: namespace_unlock+0xc2/0x180
Aug 30 22:00:20 localhost kernel: put_mnt_ns+0x70/0xa0
Aug 30 22:00:20 localhost kernel: free_nsproxy+0x1a/0x190
Aug 30 22:00:20 localhost kernel: exit_task_namespaces+0x62/0x80
Aug 30 22:00:20 localhost kernel: do_exit+0x2ad/0xa10
Aug 30 22:00:20 localhost kernel: make_task_dead+0x88/0x190
Aug 30 22:00:20 localhost kernel: rewind_stack_and_make_dead+0x16/0x20
Aug 30 22:00:20 localhost kernel: RIP: 0033:0x7fed0933f1ce
Aug 30 22:00:20 localhost kernel: Code: Unable to access opcode bytes
at 0x7fed0933f1a4.
Aug 30 22:00:20 localhost kernel: RSP: 002b:00007ffc62fe8b20 EFLAGS:
00000202 ORIG_RAX: 000000000000002e
Aug 30 22:00:20 localhost kernel: RAX: ffffffffffffffda RBX:
0000053b981d1d80 RCX: 00007fed0933f1ce
Aug 30 22:00:20 localhost kernel: RDX: 0000000000000000 RSI:
00007ffc62fe8b90 RDI: 0000000000000017
Aug 30 22:00:20 localhost kernel: RBP: 00007ffc62fe8b30 R08:
0000000000000000 R09: 0000000000000000
Aug 30 22:00:20 localhost kernel: R10: 0000000000000000 R11:
0000000000000202 R12: 0000053b981d1d80
Aug 30 22:00:20 localhost kernel: R13: 0000053b981d0b80 R14:
0000053b98034800 R15: 0000053b9803481c
Aug 30 22:00:20 localhost kernel: </TASK>
Aug 30 22:00:20 localhost kernel: ---[ end trace 0000000000000000 ]---
```
Due to my lack of knowledge in kernel source code, I relied on the
explanation from LLM. FYI:
```
Your system experienced a Kernel NULL Pointer Dereference, which is a
type of crash where the kernel tries to read memory from address
0x0000000000000058 (which is invalid/NULL) because it expected a valid
data structure to be there.
Here's a step-by-step breakdown:
The Trigger: The systemd-networkd service (PID 281) was handling a
DHCP discovery event on the br0 bridge interface.
The Code Path: To send a network packet (likely a DHCP response), it
went through the kernel's networking stack:
udp_sendmsg -> ip_route_output_flow -> ip_route_output_key_hash_rcu
The Crash: Inside the function ip_route_output_key_hash_rcu, the
kernel tried to access a member of a data structure (a pointer to a
struct fib_info or similar) at offset 0x58 (88 bytes into the
structure).
The assembly instruction was: 49 8b 45 18 <48> 8b 40 58
This means: "Take the value at register %r13 + 0x18, then from that
address, read the value at offset 0x58."
The value at %r13 + 0x18 was NULL (0x0000000000000000). Adding 0x58 to
it resulted in the invalid address 0x58, causing the crash.
The Cause: This strongly indicates a race condition or a
use-after-free bug. A likely scenario is:
The kernel correctly found a route to use and obtained a pointer to
the associated data structure (e.g., a struct rtable).
After the pointer was obtained but before it was used, the structure
was freed (likely by another CPU core or process), setting the pointer
to NULL.
When the kernel finally went to use it, it dereferenced a NULL
pointer, causing the panic.
In simple terms: A part of the kernel's networking code was too slow
to grab a piece of critical routing information before another part of
the kernel cleaned it up and threw it away.
```
-Eric
On Sat, Aug 30, 2025 at 1:04 AM cam enih <nanericwang@...il.com> wrote:
>
> > Does this happen only if you set up a PPPoE connection?
> Yes, I have another server running without PPPoE and it works fine with 6.12.44-xanmod1.
>
> > Do they have a debug kernel image with KALLSYMS, so I won't be looking
> at some random hex?
> Ok, I'll try to find one.
>
> -Eric
>
> On Fri, Aug 29, 2025 at 11:31 PM Qingfang Deng <dqfext@...il.com> wrote:
>>
>> Hi Eric,
>>
>> On Fri, Aug 29, 2025 at 9:05 PM cam enih <nanericwang@...il.com> wrote:
>> >
>> > Hi all,
>> > Having upgraded from 6.12.43 to 6.12.44, my kernel crashed at early boot. The root cause is most likely related to the commit 94731cc551e29511d85aa8dec61a6c071b1f2430 (Fixes: f6efc675c9dd ("net: ppp: resolve forwarding path for bridge pppoe devices")). Please confirm, thanks.
>>
>> Does this happen only if you set up a PPPoE connection?
>>
>> >
>> > -Eric
>> >
>> > ```
>> > Aug 29 20:36:16 localhost kernel: NET: Registered PF_PPPOX protocol family
>> > Aug 29 20:36:17 localhost systemd-networkd[266]: Failed to parse hostname, ignoring: Invalid argument
>> > Aug 29 20:36:17 localhost systemd-networkd[266]: br0: DHCPv4 server: DISCOVER (0xebeec00c)
>> > Aug 29 20:36:17 localhost kernel: BUG: kernel NULL pointer dereference, address: 0000000000000058
>> > Aug 29 20:36:17 localhost kernel: #PF: supervisor read access in kernel mode
>> > Aug 29 20:36:17 localhost kernel: #PF: error_code(0x0000) - not-present page
>> > Aug 29 20:36:17 localhost kernel: PGD 0 P4D 0
>> > Aug 29 20:36:17 localhost kernel: Oops: Oops: 0000 [#1] PREEMPT_RT SMP
>> > Aug 29 20:36:17 localhost kernel: CPU: 1 UID: 981 PID: 266 Comm: systemd-network Not tainted 6.12.44-xanmod1-1-lts #1
>>
>> Looks like it's a downstream fork:
>> https://gitlab.com/xanmod/linux/-/releases/6.12.44-xanmod1
>> Have you reported to them too?
>>
>> > Aug 29 20:36:17 localhost kernel: Hardware name: Default string Default string/Default string, BIOS 5.19 03/30/2021
>> > Aug 29 20:36:17 localhost kernel: RIP: 0010:0xffffffffb32b2f6c
>> > Aug 29 20:36:17 localhost kernel: Code: 85 8e 01 00 00 48 8b 44 24 08 48 8b 40 30 65 48 03 05 48 26 d6 4c e9 f0 fd ff ff e8 5e 9c c1 ff e9 ca fc ff ff 49 8b 44 24 18 <48> 8b 40 58 48 3d 00 e3 65 b3 0f 84 0f 01 00 00 48 89 c2 48 8d 78
>> > Aug 29 20:36:17 localhost kernel: RSP: 0018:ffff9bd080c778d8 EFLAGS: 00010246
>> > Aug 29 20:36:17 localhost kernel: RAX: 0000000000000000 RBX: ffff9bd080c77a00 RCX: 0000000000000001
>> > Aug 29 20:36:17 localhost kernel: RDX: 0000000000000000 RSI: 000000000002a424 RDI: ffffffffb38b6040
>> > Aug 29 20:36:17 localhost kernel: RBP: ffff999345ad1000 R08: 0000000000000003 R09: 0000000000000000
>> > Aug 29 20:36:17 localhost kernel: R10: ffff999342eb7900 R11: 0000000000000000 R12: ffff9bd080c77948
>> > Aug 29 20:36:17 localhost kernel: R13: 0000000000000008 R14: 0000000000000000 R15: 0000000090000000
>> > Aug 29 20:36:17 localhost kernel: FS: 00007fc0bab148c0(0000) GS:ffff9994b7d00000(0000) knlGS:0000000000000000
>> > Aug 29 20:36:17 localhost kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
>> > Aug 29 20:36:17 localhost kernel: CR2: 0000000000000058 CR3: 000000010b438006 CR4: 0000000000b70ef0
>> > Aug 29 20:36:17 localhost kernel: Call Trace:
>> > Aug 29 20:36:17 localhost kernel: <TASK>
>> > Aug 29 20:36:17 localhost kernel: ? 0xffffffffb321d725
>> > Aug 29 20:36:17 localhost kernel: 0xffffffffb32b4197
>> > Aug 29 20:36:17 localhost kernel: 0xffffffffb32f5d6c
>> > Aug 29 20:36:17 localhost kernel: ? 0xffffffffb32b8c40
>> > Aug 29 20:36:17 localhost kernel: ? 0xffffffffb3212da5
>> > Aug 29 20:36:17 localhost kernel: 0xffffffffb3212da5
>> > Aug 29 20:36:17 localhost kernel: 0xffffffffb32131ea
>> > Aug 29 20:36:17 localhost kernel: 0xffffffffb32152ea
>> > Aug 29 20:36:17 localhost kernel: 0xffffffffb3364479
>> > Aug 29 20:36:17 localhost kernel: ? 0xffffffffb3364485
>> > Aug 29 20:36:17 localhost kernel: ? 0xffffffffb3215617
>> > Aug 29 20:36:17 localhost kernel: ? 0xffffffffb2f2cd31
>> > Aug 29 20:36:17 localhost kernel: ? 0xffffffffb33684f7
>> > Aug 29 20:36:17 localhost kernel: 0xffffffffb34000b0
>>
>> Do they have a debug kernel image with KALLSYMS, so I won't be looking
>> at some random hex?
>>
>> Thanks
>> - Qingfang
Powered by blists - more mailing lists