| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Message-ID: <CACsaVZJ748KRdMxv=kikeYo7v7urim_bsz8U-YuYNBC58FeuJA@mail.gmail.com> Date: Mon, 16 Nov 2015 21:22:37 -0800 From: Kyle Sanderson <kyle.leet@...il.com> To: Thomas Gleixner <tglx@...utronix.de> Cc: Linux-Kernal <linux-kernel@...r.kernel.org>, Linus Torvalds <torvalds@...ux-foundation.org> Subject: Re: BUG: unable to handle kernel paging request at ffffe8ff7fc00001 Looks like massive corruption, so the oops above probably isn't anywhere near the cause. NFS was bouncing a bit, dmesg below... I've downgraded back to 3.14.56, will see if it continues. [523529.636418] nfs: server ftpback-bhs1-9.ip-198-100-151.net OK [532541.586437] ntfs: driver 2.1.32 [Flags: R/O MODULE]. [532541.657558] REISERFS warning (device sda4): sh-2006 read_super_block: bread failed (dev sda4, block 8, size 1024) [532541.657621] REISERFS warning (device sda4): sh-2006 read_super_block: bread failed (dev sda4, block 64, size 1024) [532541.657682] REISERFS warning (device sda4): sh-2021 reiserfs_fill_super: can not find reiserfs on sda4 [532541.658379] EXT3-fs (sda4): error: unable to read superblock [532541.659020] EXT2-fs (sda4): error: unable to read superblock [532541.659664] EXT4-fs (sda4): unable to read superblock [532541.660423] squashfs: SQUASHFS error: Can't find a SQUASHFS superblock on sda4 [532541.661194] FAT-fs (sda4): bogus number of reserved sectors [532541.661226] FAT-fs (sda4): Can't find a valid FAT filesystem [532541.661901] isofs_fill_super: bread failed, dev=sda4, iso_blknum=16, block=32 [532541.662588] UDF-fs: error (device sda4): udf_read_tagged: read failed, block=256, location=256 [532541.662819] UDF-fs: error (device sda4): udf_read_tagged: tag version 0x0000 != 0x0002 || 0x0003, block 0 [532541.662879] UDF-fs: error (device sda4): udf_read_tagged: read failed, block=512, location=512 [532541.662935] UDF-fs: error (device sda4): udf_read_tagged: tag version 0x0000 != 0x0002 || 0x0003, block 0 [532541.662994] UDF-fs: error (device sda4): udf_read_tagged: tag version 0x0000 != 0x0002 || 0x0003, block 0 [532541.663052] UDF-fs: warning (device sda4): udf_load_vrs: No anchor found [532541.663086] UDF-fs: Rescanning with blocksize 2048 [532541.663124] UDF-fs: error (device sda4): udf_read_tagged: read failed, block=256, location=256 [532541.663181] UDF-fs: error (device sda4): udf_read_tagged: read failed, block=512, location=512 [532541.663238] UDF-fs: warning (device sda4): udf_load_vrs: No anchor found [532541.663271] UDF-fs: warning (device sda4): udf_fill_super: No partition found (1) [532541.664182] XFS (sda4): Invalid superblock magic number [532541.665014] (mount,17550,6):ocfs2_get_sector:1822 ERROR: status = -12 [532541.665048] (mount,17550,6):ocfs2_sb_probe:821 ERROR: status = -12 [532541.665081] (mount,17550,6):ocfs2_fill_super:1026 ERROR: superblock probe failed! [532541.665117] (mount,17550,6):ocfs2_fill_super:1217 ERROR: status = -12 [532541.665775] attempt to access beyond end of device [532541.665806] sda4: rw=48, want=136, limit=2 [532541.665833] gfs2: error -5 reading superblock [532541.666457] gfs2: gfs2 mount does not exist [532541.667821] FAT-fs (sda4): bogus number of reserved sectors [532541.667853] FAT-fs (sda4): Can't find a valid FAT filesystem [532541.668621] ntfs: (device sda4): read_ntfs_boot_sector(): Primary boot sector is invalid. [532541.672594] ntfs: (device sda4): read_ntfs_boot_sector(): Mount option errors=recover not used. Aborting without trying to recover. [532541.672657] ntfs: (device sda4): ntfs_fill_super(): Not an NTFS volume. [532603.588489] REISERFS warning (device sda4): sh-2006 read_super_block: bread failed (dev sda4, block 8, size 1024) [532603.588552] REISERFS warning (device sda4): sh-2006 read_super_block: bread failed (dev sda4, block 64, size 1024) [532603.588613] REISERFS warning (device sda4): sh-2021 reiserfs_fill_super: can not find reiserfs on sda4 [532603.589374] EXT3-fs (sda4): error: unable to read superblock [532603.590054] EXT2-fs (sda4): error: unable to read superblock [532603.590738] EXT4-fs (sda4): unable to read superblock [532603.591524] squashfs: SQUASHFS error: Can't find a SQUASHFS superblock on sda4 [532603.592590] FAT-fs (sda4): bogus number of reserved sectors [532603.592622] FAT-fs (sda4): Can't find a valid FAT filesystem [532603.593273] isofs_fill_super: bread failed, dev=sda4, iso_blknum=16, block=32 [532603.593945] UDF-fs: error (device sda4): udf_read_tagged: read failed, block=256, location=256 [532603.594247] UDF-fs: error (device sda4): udf_read_tagged: tag version 0x0000 != 0x0002 || 0x0003, block 0 [532603.594312] UDF-fs: error (device sda4): udf_read_tagged: read failed, block=512, location=512 [532603.594371] UDF-fs: error (device sda4): udf_read_tagged: tag version 0x0000 != 0x0002 || 0x0003, block 0 [532603.594432] UDF-fs: error (device sda4): udf_read_tagged: tag version 0x0000 != 0x0002 || 0x0003, block 0 [532603.594492] UDF-fs: warning (device sda4): udf_load_vrs: No anchor found [532603.594527] UDF-fs: Rescanning with blocksize 2048 [532603.594582] UDF-fs: error (device sda4): udf_read_tagged: read failed, block=256, location=256 [532603.594642] UDF-fs: error (device sda4): udf_read_tagged: read failed, block=512, location=512 [532603.594700] UDF-fs: warning (device sda4): udf_load_vrs: No anchor found [532603.594746] UDF-fs: warning (device sda4): udf_fill_super: No partition found (1) [532603.595665] XFS (sda4): Invalid superblock magic number [532603.596525] (mount,18660,7):ocfs2_get_sector:1822 ERROR: status = -12 [532603.596559] (mount,18660,7):ocfs2_sb_probe:821 ERROR: status = -12 [532603.596592] (mount,18660,7):ocfs2_fill_super:1026 ERROR: superblock probe failed! [532603.596627] (mount,18660,7):ocfs2_fill_super:1217 ERROR: status = -12 [532603.597271] attempt to access beyond end of device [532603.597302] sda4: rw=48, want=136, limit=2 [532603.597329] gfs2: error -5 reading superblock [532603.597944] gfs2: gfs2 mount does not exist [532603.599274] FAT-fs (sda4): bogus number of reserved sectors [532603.599306] FAT-fs (sda4): Can't find a valid FAT filesystem [532603.600068] ntfs: (device sda4): read_ntfs_boot_sector(): Primary boot sector is invalid. [532603.600106] ntfs: (device sda4): read_ntfs_boot_sector(): Mount option errors=recover not used. Aborting without trying to recover. [532603.600169] ntfs: (device sda4): ntfs_fill_super(): Not an NTFS volume. [532611.781265] REISERFS warning (device sda4): sh-2006 read_super_block: bread failed (dev sda4, block 8, size 1024) [532611.781329] REISERFS warning (device sda4): sh-2006 read_super_block: bread failed (dev sda4, block 64, size 1024) [532611.781389] REISERFS warning (device sda4): sh-2021 reiserfs_fill_super: can not find reiserfs on sda4 [532611.782171] EXT3-fs (sda4): error: unable to read superblock [532611.782989] EXT2-fs (sda4): error: unable to read superblock [532611.783768] EXT4-fs (sda4): unable to read superblock [532611.784548] squashfs: SQUASHFS error: Can't find a SQUASHFS superblock on sda4 [532611.785311] FAT-fs (sda4): bogus number of reserved sectors [532611.785343] FAT-fs (sda4): Can't find a valid FAT filesystem [532611.786003] isofs_fill_super: bread failed, dev=sda4, iso_blknum=16, block=32 [532611.786658] UDF-fs: error (device sda4): udf_read_tagged: read failed, block=256, location=256 [532611.786888] UDF-fs: error (device sda4): udf_read_tagged: tag version 0x0000 != 0x0002 || 0x0003, block 0 [532611.786957] UDF-fs: error (device sda4): udf_read_tagged: read failed, block=512, location=512 [532611.787014] UDF-fs: error (device sda4): udf_read_tagged: tag version 0x0000 != 0x0002 || 0x0003, block 0 [532611.787072] UDF-fs: error (device sda4): udf_read_tagged: tag version 0x0000 != 0x0002 || 0x0003, block 0 [532611.787131] UDF-fs: warning (device sda4): udf_load_vrs: No anchor found [532611.787164] UDF-fs: Rescanning with blocksize 2048 [532611.787202] UDF-fs: error (device sda4): udf_read_tagged: read failed, block=256, location=256 [532611.787259] UDF-fs: error (device sda4): udf_read_tagged: read failed, block=512, location=512 [532611.787316] UDF-fs: warning (device sda4): udf_load_vrs: No anchor found [532611.787349] UDF-fs: warning (device sda4): udf_fill_super: No partition found (1) [532611.788237] XFS (sda4): Invalid superblock magic number [532611.789060] (mount,19654,5):ocfs2_get_sector:1822 ERROR: status = -12 [532611.789094] (mount,19654,5):ocfs2_sb_probe:821 ERROR: status = -12 [532611.789127] (mount,19654,5):ocfs2_fill_super:1026 ERROR: superblock probe failed! [532611.789162] (mount,19654,5):ocfs2_fill_super:1217 ERROR: status = -12 [532611.789800] attempt to access beyond end of device [532611.789831] sda4: rw=48, want=136, limit=2 [532611.789858] gfs2: error -5 reading superblock [532611.790487] gfs2: gfs2 mount does not exist [532611.791860] FAT-fs (sda4): bogus number of reserved sectors [532611.791892] FAT-fs (sda4): Can't find a valid FAT filesystem [532611.792679] ntfs: (device sda4): read_ntfs_boot_sector(): Primary boot sector is invalid. [532611.792716] ntfs: (device sda4): read_ntfs_boot_sector(): Mount option errors=recover not used. Aborting without trying to recover. [532611.792780] ntfs: (device sda4): ntfs_fill_super(): Not an NTFS volume. Thanks, Kyle. On Mon, Nov 16, 2015 at 5:53 AM, Kyle Sanderson <kyle.leet@...il.com> wrote: > I'll reboot the box tonight. > > I've been unable to run newer kernels (> 4.0) (gentoo/hardened-sources) on a > variety different systems (amd64: atom, ivy bridge, haswell), they usually > panic after a few hours. Primary issue being they're remote in a DC so I > can't hook up a serial console. > > Thanks guys, > Kyle. > > On 16 Nov 2015 12:54 a.m., "Thomas Gleixner" <tglx@...utronix.de> wrote: >> >> On Sun, 15 Nov 2015, Linus Torvalds wrote: >> > On Sun, Nov 15, 2015 at 2:28 PM, Kyle Sanderson <kyle.leet@...il.com> >> > wrote: >> > > [] BUG: unable to handle kernel paging request at ffffe8ff7fc00001 >> > > [] IP: [<ffffffff810a174f>] kstat_irqs+0x4f/0x90 >> > > [] CPU: 2 PID: 1078 Comm: usage.pl Not tainted 4.1.7-hardened-r1 #1 >> > > [] Hardware name: Supermicro Super Server/X10SRi-F, BIOS 1.0b >> > > 04/21/2015 >> > RSI: 000060f700000001 >> > > [] Call Trace: >> > > [] [<>] kstat_irqs_usr+0x1e/0x40 >> >> > The code ends up being >> > >> > mov 0x48(%r13),%rsi >> > mov __per_cpu_offset(,%rcx,8),%rcx >> > add (%rsi,%rcx,1),%ebx <-- trapping instruction >> > >> > which is just the >> > >> > sum += *per_cpu_ptr(desc->kstat_irqs, cpu); >> > >> > part of kstat_irqs(). >> > >> > Your registers being >> > >> > RSI: 000060f700000001 >> > RCX: ffff88087fc00000 >> > >> > and it's RSI that makes no sense - RCX looks like a real kernel >> > pointer. So it looks like it's the "desc->kstat_irqs" thing that is >> > for some reason garbage. >> > >> > I don't see any sane possible reason this would happen, though. >> > Thomas, does this look like anything you've seen before? >> >> No. What's strange is that this does explode while reading >> /proc/interrupts and it did not happen when interrupt accounting took >> place. >> >> Though this looks like memory corruption and it might be an interrupt >> which fired only at boot time, i.e. before the corruption happened. >> >> No idea how to decode that. Kyle, is that reproducible? >> >> Thanks, >> >> tglx >> > -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@...r.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Powered by blists - more mailing lists