linux-ext4 - Re: DMAR regression in 2.6.31 leads to ext4 corruption?

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:	Fri, 9 Oct 2009 18:47:14 -0700
From:	Andy Isaacson <adi@...apodia.org>
To:	Chris Wright <chrisw@...s-sol.org>
Cc:	linux-kernel@...r.kernel.org, linux-ext4@...r.kernel.org,
	iommu@...ts.linux-foundation.org
Subject: Re: DMAR regression in 2.6.31 leads to ext4 corruption?

On Fri, Oct 09, 2009 at 05:09:26PM -0700, Chris Wright wrote:
> There's some timing coincidence there, but it's a full 1/2 second between
> the ext4 error and the DMAR fault (and there's various DMAR faults
> along the way for the same buffer before and after the ext4 error).
> That fault is quite typical of a driver bug, and it's the VGA device
> (rather its driver) that is culpable.  The IOMMU caught the VGA device
> trying to do a DMA write to a buffer mapped r/o.

Yeah, this timing coincidence isn't very compelling to me, but
*something* sure is hosing my filesystems.  (Note that I've had ext3 and
ext4 both fail with what look like missed, misdirected, or corrupted
writes.)

> > The full output of fsck and full dmesg are at the URL below.
> > 
> > I don't know that DMAR is resulting in my repeated filesystem
> > corruption, but it does seem like a potential cause (and would explain
> > why I'm seeing this whereas most people aren't, since few people are
> > using VT-d *and* i915).
> 
> I do use it every day on my primary workstation (x200), and haven't
> had any issue (I'm using ext3).

What BIOS version are you using?  The X200 I'm testing on has "BIOS
Version 2.02 (6DET38WW) 2008-12-19".  Hmm, I see there's a 3.08
2009-09-07 available...

Could you provide a dmesg from a working machine?

> > I see that the BROKEN_GFX_WA code has been removed; do we actually
> > believe that the relevant code is working?  Could it be corrupting my
> > AHCI DMAs if not?
> 
> It should be for your adapter (after 66a4fe0c merged in agp fixes).
> While it could still be broken (aside of the initial faults before the
> device is even initialized in Linux -- I'm not seeing any faults, btw),
> iommu=pt will put all devices in a 1:1 mapped domain and would suppress
> the DMAR faults you see (similar to intel_iommu=off, but allowing the
> iommu to still be used for pci device assignment).  However, doing that
> or enabling the gfx workaround would allow the device to generate invalid
> DMA requests since if effectively disables the IOMMU for the gfx device,
> which would leave a better opportunity for DMA related corruption.

OK, thanks for the confirmation that the BROKEN_GFX_WA issues should be
fixed in current linus kernels, I'm certainly running with 66a4fe0c.

> The earlier fs issues we saw w/ the IOMMU were when it was actively
> blocking disk DMA requests, but that's not happening here.

Well, we don't know for sure what happened on the previous boot where
the filesystem corruption occurred.  I'm imagining a nightmare scenario
where GPU erroneous writes cause DMAR faults and handling them somehow
causes AHCI DMA requests to get lost.

I'm going to go ahead on the theory that the BIOS needs an update.

-andy
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html