| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Message-ID: <202210241614.2ae4c1f5-yujie.liu@intel.com>
Date: Wed, 26 Oct 2022 17:10:28 +0800
From: kernel test robot <yujie.liu@...el.com>
To: Peter Zijlstra <peterz@...radead.org>
CC: <oe-lkp@...ts.linux.dev>, <lkp@...el.com>,
Thomas Gleixner <tglx@...utronix.de>,
<linux-kernel@...r.kernel.org>, <x86@...nel.org>,
<fengwei.yin@...el.com>, <ying.huang@...el.com>
Subject: [tip:x86/core] [kallsyms] f138918162:
WARNING:CPU:#PID:#at_fs/xfs/xfs_message.c:#xfs_buf_alert_ratelimited.cold-#[xfs]
Hi Peter,
For below patch, we couldn't find any connection between the code
change of kallsyms and xfstests testcase, but we got very stable test
results. We tested commit f1389181622a and its parent commit
7825451fa4dc for 12 runs each, parent was 100% good while this commit
was 100% bad, so we still send out this report FYI though we don't have
clear clue of the root cause. Please check the details below if
interested in further investigation. Thanks.
Greeting,
FYI, we noticed WARNING:CPU:#PID:#at_fs/xfs/xfs_message.c:#xfs_buf_alert_ratelimited.cold-#[xfs] due to commit (built with gcc-11):
commit: f1389181622a08d6f1c71407a64df36b809d632b ("kallsyms: Take callthunks into account")
https://git.kernel.org/cgit/linux/kernel/git/tip/tip.git x86/core
[test failed on linux-next/master 4d48f589d294ddc5e01d3b0dc7cecc55324c05ca]
in testcase: xfstests
version: xfstests-x86_64-5a5e419-1_20221017
with following parameters:
disk: 4HDD
fs: xfs
test: xfs-no-bug-assert
test-description: xfstests is a regression test suite for xfs and other files ystems.
test-url: git://git.kernel.org/pub/scm/fs/xfs/xfstests-dev.git
on test machine: 4 threads 1 sockets Intel(R) Core(TM) i3-3220 CPU @ 3.30GHz (Ivy Bridge) with 8G memory
caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace):
== dmesg log ==
[ 75.708690][ T3668] ------------[ cut here ]------------
[ 75.714041][ T3668] WARNING: CPU: 1 PID: 3668 at fs/xfs/xfs_message.c:104 xfs_buf_alert_ratelimited.cold-0x7/0x45 [xfs]
[ 75.725013][ T3668] Modules linked in: xfs dm_mod netconsole btrfs blake2b_generic xor raid6_pq zstd_compress libcrc32c intel_rapl_msr intel_rapl_common i915 sd_mod t10_pi crc64_rocksoft_generic crc64_rocksoft crc64 sg x86_pkg_temp_thermal intel_powerclamp drm_buddy coretemp ipmi_devintf ipmi_msghandler kvm_intel intel_gtt kvm drm_display_helper irqbypass crct10dif_pclmul ttm crc32_pclmul crc32c_intel ghash_clmulni_intel sha512_ssse3 ahci drm_kms_helper wmi_bmof rapl libahci intel_cstate syscopyarea intel_uncore libata sysfillrect mei_me i2c_i801 i2c_smbus sysimgblt lpc_ich mei fb_sys_fops video wmi drm fuse ip_tables
[ 75.779828][ T3668] CPU: 1 PID: 3668 Comm: mount Not tainted 6.1.0-rc1-00053-gf1389181622a #1
[ 75.788389][ T3668] Hardware name: Hewlett-Packard HP Pro 3340 MT/17A1, BIOS 8.07 01/24/2013
[ 75.796864][ T3668] RIP: 0010:xfs_buf_alert_ratelimited.cold-0x7/0x45 [xfs]
[ 75.804035][ T3668] Code: 2a 83 e0 07 48 c1 e9 03 8a 14 11 38 c2 7f 10 84 d2 74 0c 48 c7 c7 b0 d5 1f a2 e8 fe 06 80 df 80 3d 17 7d 1e 00 00 74 02 0f 0b <0f> 0b c3 cc cc cc cc 48 8d 45 10 48 8d 54 24 28 4c 89 f6 48 c7 c7
All code
========
0: 2a 83 e0 07 48 c1 sub -0x3eb7f820(%rbx),%al
6: e9 03 8a 14 11 jmpq 0x11148a0e
b: 38 c2 cmp %al,%dl
d: 7f 10 jg 0x1f
f: 84 d2 test %dl,%dl
11: 74 0c je 0x1f
13: 48 c7 c7 b0 d5 1f a2 mov $0xffffffffa21fd5b0,%rdi
1a: e8 fe 06 80 df callq 0xffffffffdf80071d
1f: 80 3d 17 7d 1e 00 00 cmpb $0x0,0x1e7d17(%rip) # 0x1e7d3d
26: 74 02 je 0x2a
28: 0f 0b ud2
2a:* 0f 0b ud2 <-- trapping instruction
2c: c3 retq
2d: cc int3
2e: cc int3
2f: cc int3
30: cc int3
31: 48 8d 45 10 lea 0x10(%rbp),%rax
35: 48 8d 54 24 28 lea 0x28(%rsp),%rdx
3a: 4c 89 f6 mov %r14,%rsi
3d: 48 rex.W
3e: c7 .byte 0xc7
3f: c7 .byte 0xc7
Code starting with the faulting instruction
===========================================
0: 0f 0b ud2
2: c3 retq
3: cc int3
4: cc int3
5: cc int3
6: cc int3
7: 48 8d 45 10 lea 0x10(%rbp),%rax
b: 48 8d 54 24 28 lea 0x28(%rsp),%rdx
10: 4c 89 f6 mov %r14,%rsi
13: 48 rex.W
14: c7 .byte 0xc7
15: c7 .byte 0xc7
[ 75.823537][ T3668] RSP: 0018:ffffc9000365f710 EFLAGS: 00010246
[ 75.829497][ T3668] RAX: 0000000000000000 RBX: 000000000004adff RCX: 1ffffffff443fab6
[ 75.837369][ T3668] RDX: dffffc0000000004 RSI: ffffc9000365f478 RDI: fffff520006cbed3
[ 75.845239][ T3668] RBP: 000000000004ae00 R08: 00000000ffffffea R09: ffffc9000365f437
[ 75.853107][ T3668] R10: fffff520006cbe86 R11: 0000000000000001 R12: 00000000ffffff8b
[ 75.860971][ T3668] R13: ffffc9000365f868 R14: ffff8881fe5aa000 R15: ffff888205400000
[ 75.868844][ T3668] FS: 00007fda5f7a6840(0000) GS:ffff8881a4c80000(0000) knlGS:0000000000000000
[ 75.877679][ T3668] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 75.884170][ T3668] CR2: 00007ffe57523908 CR3: 000000010baa2003 CR4: 00000000001706e0
[ 75.892047][ T3668] Call Trace:
[ 75.895237][ T3668] <TASK>
[ 75.898077][ T3668] xlog_find_verify_log_record (fs/xfs/xfs_log_recover.c:431) xfs
[ 75.904461][ T3668] ? xlog_find_verify_log_record (fs/xfs/xfs_log_recover.c:399) xfs
[ 75.910856][ T3668] xlog_find_head (fs/xfs/xfs_log_recover.c:705) xfs
[ 75.916097][ T3668] ? xlog_find_head (fs/xfs/xfs_log_recover.c:501) xfs
[ 75.921358][ T3668] ? update_curr (kernel/sched/fair.c:911)
[ 75.925847][ T3668] xlog_find_tail (fs/xfs/xfs_log_recover.c:1256) xfs
[ 75.930974][ T3668] ? check_preempt_wakeup (kernel/sched/fair.c:7463)
[ 75.936246][ T3668] ? xlog_find_tail (fs/xfs/xfs_log_recover.c:1243) xfs
[ 75.941480][ T3668] ? check_preempt_curr (kernel/sched/core.c:2177)
[ 75.946586][ T3668] ? ttwu_do_wakeup (kernel/sched/core.c:3612)
[ 75.951249][ T3668] xlog_recover (fs/xfs/xfs_log_recover.c:3355) xfs
[ 75.956249][ T3668] ? xlog_recover (fs/xfs/xfs_log_recover.c:3350) xfs
[ 75.961324][ T3668] xfs_log_mount (fs/xfs/xfs_log.c:739) xfs
[ 75.966486][ T3668] xfs_mountfs (fs/xfs/xfs_mount.c:805 (discriminator 4)) xfs
[ 75.971557][ T3668] ? kmem_alloc (fs/xfs/kmem.c:21) xfs
[ 75.976512][ T3668] ? xfs_mountfs (fs/xfs/xfs_mount.c:621) xfs
[ 75.981563][ T3668] ? kmem_alloc (fs/xfs/kmem.c:21) xfs
[ 75.986508][ T3668] ? xfs_fstrm_free_func (fs/xfs/xfs_filestream.c:82) xfs
[ 75.992172][ T3668] ? xfs_mru_cache_create (fs/xfs/xfs_mru_cache.c:360) xfs
[ 75.998070][ T3668] xfs_fs_fill_super (fs/xfs/xfs_super.c:1666) xfs
[ 76.003647][ T3668] get_tree_bdev (fs/super.c:1324)
[ 76.008140][ T3668] ? xfs_fs_fill_super (fs/xfs/xfs_super.c:1451) xfs
[ 76.013707][ T3668] vfs_get_tree (fs/super.c:1531)
[ 76.018021][ T3668] do_new_mount (fs/namespace.c:3041)
[ 76.022421][ T3668] ? do_new_mount (fs/namespace.c:3001)
[ 76.026823][ T3668] ? security_capable (security/security.c:809 (discriminator 13))
[ 76.031577][ T3668] path_mount (fs/namespace.c:3370)
[ 76.035901][ T3668] ? kasan_set_track (mm/kasan/common.c:52)
[ 76.040570][ T3668] ? kasan_save_free_info (mm/kasan/generic.c:513)
[ 76.045660][ T3668] ? path_mount (fs/namespace.c:3297)
[ 76.049976][ T3668] ? kmem_cache_free (mm/slub.c:1750 mm/slub.c:3661 mm/slub.c:3683)
[ 76.054820][ T3668] ? getname_flags (fs/namei.c:205)
[ 76.060008][ T3668] __x64_sys_mount (fs/namespace.c:3384 fs/namespace.c:3591 fs/namespace.c:3568 fs/namespace.c:3568)
[ 76.064670][ T3668] ? __x64_sys_mount (fs/namespace.c:3568)
[ 76.069334][ T3668] ? from_kgid_munged (kernel/user_namespace.c:520)
[ 76.074089][ T3668] ? getname_flags (fs/namei.c:205)
[ 76.079276][ T3668] do_syscall_64 (arch/x86/entry/common.c:50 arch/x86/entry/common.c:80)
[ 76.083595][ T3668] entry_SYSCALL_64_after_hwframe (arch/x86/entry/entry_64.S:120)
[ 76.089382][ T3668] RIP: 0033:0x7fda5f9e56ea
[ 76.093705][ T3668] Code: 48 8b 0d a9 17 0d 00 f7 d8 64 89 01 48 83 c8 ff c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 49 89 ca b8 a5 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 76 17 0d 00 f7 d8 64 89 01 48
All code
========
0: 48 8b 0d a9 17 0d 00 mov 0xd17a9(%rip),%rcx # 0xd17b0
7: f7 d8 neg %eax
9: 64 89 01 mov %eax,%fs:(%rcx)
c: 48 83 c8 ff or $0xffffffffffffffff,%rax
10: c3 retq
11: 66 2e 0f 1f 84 00 00 nopw %cs:0x0(%rax,%rax,1)
18: 00 00 00
1b: 0f 1f 44 00 00 nopl 0x0(%rax,%rax,1)
20: 49 89 ca mov %rcx,%r10
23: b8 a5 00 00 00 mov $0xa5,%eax
28: 0f 05 syscall
2a:* 48 3d 01 f0 ff ff cmp $0xfffffffffffff001,%rax <-- trapping instruction
30: 73 01 jae 0x33
32: c3 retq
33: 48 8b 0d 76 17 0d 00 mov 0xd1776(%rip),%rcx # 0xd17b0
3a: f7 d8 neg %eax
3c: 64 89 01 mov %eax,%fs:(%rcx)
3f: 48 rex.W
Code starting with the faulting instruction
===========================================
0: 48 3d 01 f0 ff ff cmp $0xfffffffffffff001,%rax
6: 73 01 jae 0x9
8: c3 retq
9: 48 8b 0d 76 17 0d 00 mov 0xd1776(%rip),%rcx # 0xd1786
10: f7 d8 neg %eax
12: 64 89 01 mov %eax,%fs:(%rcx)
15: 48 rex.W
== xfstests log ==
2022-10-26 03:45:10 export TEST_DIR=/fs/sda2
2022-10-26 03:45:10 export TEST_DEV=/dev/sda2
2022-10-26 03:45:10 export FSTYP=xfs
2022-10-26 03:45:10 export SCRATCH_MNT=/fs/scratch
2022-10-26 03:45:10 mkdir /fs/scratch -p
2022-10-26 03:45:10 export SCRATCH_DEV=/dev/sda5
2022-10-26 03:45:10 export SCRATCH_LOGDEV=/dev/sda3
2022-10-26 03:45:10 export SCRATCH_XFS_LIST_METADATA_FIELDS=u3.sfdir3.hdr.parent.i4
2022-10-26 03:45:10 export SCRATCH_XFS_LIST_FUZZ_VERBS=random
2022-10-26 03:45:10 sed "s:^:xfs/:" //lkp/benchmarks/xfstests/tests/xfs-no-bug-assert
2022-10-26 03:45:10 ./check xfs/098 xfs/115 xfs/439
FSTYP -- xfs (debug)
PLATFORM -- Linux/x86_64 lkp-ivb-d04 6.1.0-rc1-00053-gf1389181622a #1 SMP Fri Oct 21 15:14:38 CST 2022
MKFS_OPTIONS -- -f /dev/sda5
MOUNT_OPTIONS -- /dev/sda5 /fs/scratch
xfs/098 - output mismatch (see /lkp/benchmarks/xfstests/results//xfs/098.out.bad)
--- tests/xfs/098.out 2022-10-17 16:29:33.000000000 +0000
+++ /lkp/benchmarks/xfstests/results//xfs/098.out.bad 2022-10-26 03:45:38.477659002 +0000
@@ -6,6 +6,7 @@
+ corrupt image
+ mount image
+ repair fs
+_check_dmesg: something found in dmesg (see /lkp/benchmarks/xfstests/results//xfs/098.dmesg)
+ mount image (2)
+ chattr -R -i
+ modify files (2)
...
(Run 'diff -u /lkp/benchmarks/xfstests/tests/xfs/098.out /lkp/benchmarks/xfstests/results//xfs/098.out.bad' to see the entire diff)
xfs/115 12s
xfs/439 - output mismatch (see /lkp/benchmarks/xfstests/results//xfs/439.out.bad)
--- tests/xfs/439.out 2022-10-17 16:29:33.000000000 +0000
+++ /lkp/benchmarks/xfstests/results//xfs/439.out.bad 2022-10-26 03:45:53.942658319 +0000
@@ -1,2 +1,3 @@
QA output created by 439
+_check_dmesg: something found in dmesg (see /lkp/benchmarks/xfstests/results//xfs/439.dmesg)
Silence is golden
...
(Run 'diff -u /lkp/benchmarks/xfstests/tests/xfs/439.out /lkp/benchmarks/xfstests/results//xfs/439.out.bad' to see the entire diff)
Ran: xfs/098 xfs/115 xfs/439
Failures: xfs/098 xfs/439
Failed 2 of 3 tests
== Comparison of test results ==
=========================================================================================
tbox_group/testcase/rootfs/kconfig/compiler/disk/fs/test:
lkp-ivb-d04/xfstests/debian-11.1-x86_64-20220510.cgz/x86_64-rhel-8.3-func/gcc-11/4HDD/xfs/xfs-no-bug-assert
commit:
7825451fa4dc0 ("static_call: Add call depth tracking support")
f1389181622a0 ("kallsyms: Take callthunks into account")
7825451fa4dc0466 f1389181622a08d6f1c71407a64
---------------- ---------------------------
fail:runs %reproduction fail:runs
| | |
:12 100% 12:12 xfstests.xfs.098.fail
:12 100% 12:12 xfstests.xfs.439.fail
12:12 0% 12:12 dmesg.Assertion_failed
12:12 -100% :12 dmesg.RIP:assfail[xfs]
:12 100% 12:12 dmesg.WARNING:CPU:#PID:#at_fs/xfs/xfs_message.c:#xfs_buf_alert_ratelimited.cold-#[xfs]
12:12 -100% :12 dmesg.WARNING:at_fs/xfs/xfs_message.c:#assfail[xfs]
If you fix the issue, kindly add following tag
| Reported-by: kernel test robot <yujie.liu@...el.com>
| Link: https://lore.kernel.org/r/202210241614.2ae4c1f5-yujie.liu@intel.com
To reproduce:
git clone https://github.com/intel/lkp-tests.git
cd lkp-tests
sudo bin/lkp install job.yaml # job file is attached in this email
bin/lkp split-job --compatible job.yaml # generate the yaml file for lkp run
sudo bin/lkp run generated-yaml-file
# if come across any failure that blocks the test,
# please remove ~/.lkp and /lkp dir to run from a clean state.
--
0-DAY CI Kernel Test Service
https://01.org/lkp
View attachment "config-6.1.0-rc1-00053-gf1389181622a" of type "text/plain" (170257 bytes)
View attachment "job-script" of type "text/plain" (5847 bytes)
Download attachment "dmesg.xz" of type "application/x-xz" (13268 bytes)
View attachment "xfstests" of type "text/plain" (2098 bytes)
View attachment "job.yaml" of type "text/plain" (4952 bytes)
View attachment "reproduce" of type "text/plain" (922 bytes)
Powered by blists - more mailing lists