Message-ID: <c2524c96-d71c-d7db-22ec-12da905dc180@fb.com>
Date:   Fri, 26 Jul 2019 15:45:39 +0000
From:   Yonghong Song <yhs@...com>
To:     "sedat.dilek@...il.com" <sedat.dilek@...il.com>,
        Alexei Starovoitov <ast@...nel.org>,
        Daniel Borkmann <daniel@...earbox.net>,
        Martin Lau <kafai@...com>, Song Liu <songliubraving@...com>
CC:     "netdev@...r.kernel.org" <netdev@...r.kernel.org>,
        "bpf@...r.kernel.org" <bpf@...r.kernel.org>,
        Clang-Built-Linux ML <clang-built-linux@...glegroups.com>,
        Kees Cook <keescook@...omium.org>,
        "Nick Desaulniers" <ndesaulniers@...gle.com>,
        Nathan Chancellor <natechancellor@...il.com>
Subject: Re: next-20190723: bpf/seccomp - systemd/journald issue?



On 7/26/19 1:26 AM, Sedat Dilek wrote:
> Hi,
> 
> I have opened a new issue in the ClangBuiltLinux issue tracker.

Glad to know clang 9 has asm goto support and now It can compile
kernel again.

> 
> I am seeing a problem in the area bpf/seccomp causing
> systemd/journald/udevd services to fail.
> 
> [Fri Jul 26 08:08:43 2019] systemd[453]: systemd-udevd.service: Failed
> to connect stdout to the journal socket, ignoring: Connection refused
> 
> This happens when I use the (LLVM) LLD ld.lld-9 linker but not with
> BFD linker ld.bfd on Debian/buster AMD64.
> In both cases I use clang-9 (prerelease).

Looks like it is a lld bug.

I see the stack trace has __bpf_prog_run32() which is used by
kernel bpf interpreter. Could you try to enable bpf jit
   sysctl net.core.bpf_jit_enable = 1
If this passed, it will prove it is interpreter related.

> 
> Base for testing: next-20190723.
> 
> The call-trace looks like this:
> 
> [Fri Jul 26 08:08:42 2019] BUG: unable to handle page fault for
> address: ffffffff85403370
> [Fri Jul 26 08:08:42 2019] #PF: supervisor read access in kernel mode
> [Fri Jul 26 08:08:42 2019] #PF: error_code(0x0000) - not-present page
> [Fri Jul 26 08:08:42 2019] PGD 7620e067 P4D 7620e067 PUD 7620f063 PMD
> 44fe85063 PTE 800fffff8a3fc062
> [Fri Jul 26 08:08:42 2019] Oops: 0000 [#1] SMP PTI
> [Fri Jul 26 08:08:42 2019] CPU: 2 PID: 417 Comm: (journald) Not
> tainted 5.3.0-rc1-5-amd64-cbl-asmgoto #5~buster+dileks1
> [Fri Jul 26 08:08:42 2019] Hardware name: LENOVO
> 20HDCTO1WW/20HDCTO1WW, BIOS N1QET83W (1.58 ) 04/18/2019
> [Fri Jul 26 08:08:42 2019] RIP: 0010:___bpf_prog_run+0x40/0x14f0
> [Fri Jul 26 08:08:42 2019] Code: f3 eb 24 48 83 f8 38 0f 84 a9 0c 00
> 00 48 83 f8 39 0f 85 8a 14 00 00 0f 1f 00 48 0f bf 43 02 48 8d 1c c3
> 48 83 c3 08 0f b6 33 <48> 8b 04 f5 10 2e 40 85 48 83 f8 3b 7f 62 48 83
> f8 1e 0f 8f c8 00
> [Fri Jul 26 08:08:42 2019] RSP: 0018:ffff992ec028fcb8 EFLAGS: 00010246
> [Fri Jul 26 08:08:42 2019] RAX: ffff992ec028fd60 RBX: ffff992ec00e9038
> RCX: 0000000000000002
> [Fri Jul 26 08:08:42 2019] RDX: ffff992ec028fd40 RSI: 00000000000000ac
> RDI: ffff992ec028fce0
> [Fri Jul 26 08:08:42 2019] RBP: ffff992ec028fcd0 R08: 0000000000000000
> R09: ffff992ec028ff58
> [Fri Jul 26 08:08:42 2019] R10: 0000000000000000 R11: ffffffff849b8210
> R12: 000000007fff0000
> [Fri Jul 26 08:08:42 2019] R13: ffff992ec028feb8 R14: 0000000000000000
> R15: ffff992ec028fce0
> [Fri Jul 26 08:08:42 2019] FS:  00007f5d20f1d940(0000)
> GS:ffff8ba3d2500000(0000) knlGS:0000000000000000
> [Fri Jul 26 08:08:42 2019] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [Fri Jul 26 08:08:42 2019] CR2: ffffffff85403370 CR3: 0000000445b3e001
> CR4: 00000000003606e0
> [Fri Jul 26 08:08:42 2019] Call Trace:
> [Fri Jul 26 08:08:42 2019]  __bpf_prog_run32+0x44/0x70
> [Fri Jul 26 08:08:42 2019]  ? flush_tlb_func_common+0xd8/0x230
> [Fri Jul 26 08:08:42 2019]  ? mem_cgroup_commit_charge+0x8c/0x120
> [Fri Jul 26 08:08:42 2019]  ? wp_page_copy+0x464/0x7a0
> [Fri Jul 26 08:08:42 2019]  seccomp_run_filters+0x54/0x110
> [Fri Jul 26 08:08:42 2019]  __seccomp_filter+0xf7/0x6e0
> [Fri Jul 26 08:08:42 2019]  ? do_wp_page+0x32b/0x5d0
> [Fri Jul 26 08:08:42 2019]  ? handle_mm_fault+0x90d/0xbf0
> [Fri Jul 26 08:08:42 2019]  syscall_trace_enter+0x182/0x290
> [Fri Jul 26 08:08:42 2019]  do_syscall_64+0x30/0x90
> [Fri Jul 26 08:08:42 2019]  entry_SYSCALL_64_after_hwframe+0x44/0xa9
> [Fri Jul 26 08:08:42 2019] RIP: 0033:0x7f5d220d7f59
> [Fri Jul 26 08:08:42 2019] Code: 00 c3 66 2e 0f 1f 84 00 00 00 00 00
> 0f 1f 44 00 00 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8
> 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 07 6f 0c 00
> f7 d8 64 89 01 48
> [Fri Jul 26 08:08:42 2019] RSP: 002b:00007ffd11332b48 EFLAGS: 00000246
> ORIG_RAX: 000000000000013d
> [Fri Jul 26 08:08:42 2019] RAX: ffffffffffffffda RBX: 000055bf8ab34010
> RCX: 00007f5d220d7f59
> [Fri Jul 26 08:08:42 2019] RDX: 000055bf8ab34010 RSI: 0000000000000000
> RDI: 0000000000000001
> [Fri Jul 26 08:08:42 2019] RBP: 000055bf8ab97fb0 R08: 000055bf8abbe180
> R09: 00000000c000003e
> [Fri Jul 26 08:08:42 2019] R10: 000055bf8abbe1e0 R11: 0000000000000246
> R12: 00007ffd11332ba0
> [Fri Jul 26 08:08:42 2019] R13: 00007ffd11332b98 R14: 00007f5d21f087f8
> R15: 000000000000002c
> [Fri Jul 26 08:08:42 2019] Modules linked in: i2c_dev parport_pc
> sunrpc ppdev lp parport efivarfs ip_tables x_tables autofs4 ext4
> crc32c_generic mbcache crc16 jbd2 btrfs zstd_decompress zstd_compress
> algif_skcipher af_alg sd_mod dm_crypt dm_mod raid10 raid456
> async_raid6_recov async_memcpy async_pq async_xor async_tx xor
> raid6_pq libcrc32c raid1 uas raid0 usb_storage multipath linear
> scsi_mod md_mod hid_cherry hid_generic usbhid hid crct10dif_pclmul
> crc32_pclmul crc32c_intel ghash_clmulni_intel aesni_intel aes_x86_64
> i915 glue_helper crypto_simd nvme i2c_algo_bit cryptd psmouse xhci_pci
> drm_kms_helper e1000e i2c_i801 xhci_hcd intel_lpss_pci nvme_core
> intel_lpss drm usbcore thermal wmi video button
> [Fri Jul 26 08:08:42 2019] CR2: ffffffff85403370
> [Fri Jul 26 08:08:42 2019] ---[ end trace 867b35c7d6c6705a ]---
> [Fri Jul 26 08:08:42 2019] RIP: 0010:___bpf_prog_run+0x40/0x14f0
> [Fri Jul 26 08:08:42 2019] Code: f3 eb 24 48 83 f8 38 0f 84 a9 0c 00
> 00 48 83 f8 39 0f 85 8a 14 00 00 0f 1f 00 48 0f bf 43 02 48 8d 1c c3
> 48 83 c3 08 0f b6 33 <48> 8b 04 f5 10 2e 40 85 48 83 f8 3b 7f 62 48 83
> f8 1e 0f 8f c8 00
> [Fri Jul 26 08:08:42 2019] RSP: 0018:ffff992ec028fcb8 EFLAGS: 00010246
> [Fri Jul 26 08:08:42 2019] RAX: ffff992ec028fd60 RBX: ffff992ec00e9038
> RCX: 0000000000000002
> [Fri Jul 26 08:08:42 2019] RDX: ffff992ec028fd40 RSI: 00000000000000ac
> RDI: ffff992ec028fce0
> [Fri Jul 26 08:08:42 2019] RBP: ffff992ec028fcd0 R08: 0000000000000000
> R09: ffff992ec028ff58
> [Fri Jul 26 08:08:42 2019] R10: 0000000000000000 R11: ffffffff849b8210
> R12: 000000007fff0000
> [Fri Jul 26 08:08:42 2019] R13: ffff992ec028feb8 R14: 0000000000000000
> R15: ffff992ec028fce0
> [Fri Jul 26 08:08:42 2019] FS:  00007f5d20f1d940(0000)
> GS:ffff8ba3d2500000(0000) knlGS:0000000000000000
> [Fri Jul 26 08:08:42 2019] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [Fri Jul 26 08:08:42 2019] CR2: ffffffff85403370 CR3: 0000000445b3e001
> CR4: 00000000003606e0
> 
> More details in [1] and what I tried (for example CONFIG_SECCOMP=n)
> 
> I have no clue about BPF or SECCOMP.
> 
> Can you comment on this?
> 
> If this touches BPF: Can you give me some hints and instructions in debugging?
> 
> My kernel-config and dmesg-log are attached.
> 
> Thanks.
> 
> Regards,
> - Sedat -
> 
> [1] https://github.com/ClangBuiltLinux/linux/issues/619
> 

Powered by blists - more mailing lists