linux-ext4 - Re: ext4 file system corruption with v4.19.3 / v4.19.4

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20181127212255.GA2987@roeck-us.net>
Date:   Tue, 27 Nov 2018 13:22:55 -0800
From:   Guenter Roeck <linux@...ck-us.net>
To:     Rainer Fiebig <jrf@...lbox.org>
Cc:     linux-kernel@...r.kernel.org, grendel@...stedcode.net,
        Theodore Ts'o <tytso@....edu>,
        Andreas Dilger <adilger.kernel@...ger.ca>,
        linux-ext4@...r.kernel.org
Subject: Re: ext4 file system corruption with v4.19.3 / v4.19.4

On Tue, Nov 27, 2018 at 07:55:01PM +0100, Rainer Fiebig wrote:
> Am Dienstag, 27. November 2018, 15:48:19 schrieb Marek Habersack:
> > On 27/11/2018 15:32, Guenter Roeck wrote:
> > Hi,
> > 
> > You might try to see if you have CONFIG_SCSI_MQ_DEFAULT=yes in your kernel
> > config. Starting with 4.19.1 it somehow interferes with ext4 and causes
> > problems similar to the ones you list below. Ever since I disabled MQ
> > (either recompile your kernel or add `scsi_mod.use_blk_mq=0` to the kernel
> > command line) none of those errors came back.
> > 
> > hope it helps,
> > 
> > marek
> 
> Unfortunately, this doesn't seem to work in every case: 
> https://bugzilla.kernel.org/show_bug.cgi?id=201685#c54
> 
> And I'm using a defconfig-4.19.3 (meaning: CONFIG_SCSI_MQ_DEFAULT=yes) in a VM 
> and I'm not seeing those errors there. OK, it's a VM - but anyway.
> 

Agreed. I disabled CONFIG_SCSI_MQ_DEFAULT, but the problem is still seen
at least on one of my servers, so disabling it does not help, at least not
in my case.

If the problem is somehow related to CONFIG_SCSI_MQ_DEFAULT, you might
have to explicitly use a scsi drive (virtio-scsi-pci or similar) to
trigger its use in a VM.

Guenter

> The definite cause of this can only be found by bisecting, IMO. And it needs 
> to be pinned down because else some feeling of insecurity will remain.
> 
> So long!
> 
> Rainer Fiebig
> 
> > 
> > > [trying again, this time with correct kernel.org address]
> > > 
> > > Hi,
> > > 
> > > I have seen the following and similar problems several times,
> > > with both v4.19.3 and v4.19.4:
> > > 
> > > Nov 23 04:32:25 mars kernel: [112668.673671] EXT4-fs error (device sdb1):
> > > ext4_iget:4831: inode #12602889: comm git: bad extra_isize 33661 (inode
> > > size 256)
> > > Nov 23 04:32:25 mars kernel: [112668.675217] Aborting journal on device
> > > sdb1-8. Nov 23 04:32:25 mars kernel: [112668.676681] EXT4-fs (sdb1):
> > > Remounting filesystem read-only Nov 23 04:32:25 mars kernel:
> > > [112668.808886] EXT4-fs error (device sdb1): ext4_iget:4831: inode
> > > #12602881: comm rm: bad extra_isize 33685 (inode size 256)
> > > ...
> > > 
> > > Nov 25 00:12:43 saturn kernel: [59377.725984] EXT4-fs error (device sda1):
> > > ext4_lookup:1578: inode #238034131: comm updatedb.mlocat: deleted inode
> > > referenced: 238160407
> > > Nov 25 00:12:43 saturn kernel: [59377.766638] Aborting journal on device
> > > sda1-8. Nov 25 00:12:43 saturn kernel: [59377.779372] EXT4-fs (sda1):
> > > Remounting filesystem read-only ...
> > > 
> > > Nov 24 01:52:31 saturn kernel: [189085.240016] EXT4-fs error (device
> > > sda1): ext4_lookup:1578: inode #52038457: comm nfsd: deleted inode
> > > referenced: 52043796
> > > Nov 24 01:52:31 saturn kernel: [189085.263427] Aborting journal on device
> > > sda1-8. Nov 24 01:52:31 saturn kernel: [189085.275313] EXT4-fs (sda1):
> > > Remounting filesystem read-only
> > > 
> > > 
> > > The same systems running v4.18.6 never experienced a problem.
> > > 
> > > Has anyone else seen similar problems ? Is there anything I can do
> > > to help tracking down the problem ?
> > > 
> > > Thanks,
> > > Guenter
> 
> -- 
> The truth always turns out to be simpler than you thought.
> Richard Feynman