lists.openwall.net   lists  /  announce  owl-users  owl-dev  john-users  john-dev  passwdqc-users  popa3d-users  /  oss-security  kernel-hardening  musl  sabotage  tlsify  passwords  /  crypt-dev  xvendor  /  Bugtraq  Full-Disclosure  linux-kernel  linux-netdev  linux-ext4  PHC 
Open Source and information security mailing list archives
 
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Date:	Fri, 10 Dec 2010 20:14:56 -0600
From:	Jon Nelson <jnelson@...poni.net>
To:	"Ted Ts'o" <tytso@....edu>, Matt <jackdachef@...il.com>,
	Chris Mason <chris.mason@...cle.com>,
	Andi Kleen <andi@...stfloor.org>,
	Jon Nelson <jnelson@...poni.net>,
	Mike Snitzer <snitzer@...hat.com>,
	Milan Broz <mbroz@...hat.com>,
	linux-btrfs <linux-btrfs@...r.kernel.org>,
	dm-devel <dm-devel@...hat.com>,
	Linux Kernel <linux-kernel@...r.kernel.org>,
	htd <htd@...cy-poultry.org>, htejun <htejun@...il.com>,
	linux-ext4 <linux-ext4@...r.kernel.org>
Subject: Re: hunt for 2.6.37 dm-crypt+ext4 corruption? (was: Re: dm-crypt
 barrier support is effective)

On Fri, Dec 10, 2010 at 10:54 AM, Jon Nelson <jnelson@...poni.net> wrote:
> On Fri, Dec 10, 2010 at 8:58 AM, Jon Nelson <jnelson@...poni.net> wrote:
>> On Fri, Dec 10, 2010 at 12:52 AM, Jon Nelson <jnelson@...poni.net> wrote:
>>> On Thu, Dec 9, 2010 at 8:38 PM, Ted Ts'o <tytso@....edu> wrote:
>>>> On Fri, Dec 10, 2010 at 02:53:30AM +0100, Matt wrote:
>>>>>
>>>>> Try a kernel before 5a87b7a5da250c9be6d757758425dfeaf8ed3179
>>>>>
>>>>> from the tests I've done that one showed the least or no corruption if
>>>>> you count the empty /etc/env.d/03opengl as an artefact
>>>>
>>>> Yes, that's a good test.  Also try commit bd2d0210cf.  The patch
>>>> series that is most likely to be at fault if there is a regression in
>>>> between 5a87b7a5d and bd2d0210cf inclusive.
>>>>
>>>> I did a lot of testing before submitting it, but that wa a tricky
>>>> rewrite.  If you can reproduce the problem reliably, it might be good
>>>> to try commit 16828088f9 (the commit before 5a87b7a5d) and commit
>>>> bd2d0210cf.  If it reliably reproduces on bd2d0210cf, but is clean on
>>>> 16828088f9, then it's my ext4 block i/o submission patches, and we'll
>>>> need to either figure out what's going on or back out that set of
>>>> changes.
>>>>
>>>> If that's the case, a bisect of those changes (it's only 6 commits, so
>>>> it shouldn't take long) would be most appreciated.
>>>
>>> I observed the behavior on bd2d0210cf in a qemu-kvm install of
>>> openSUSE 11.3 (x86_64) on *totally* different host - an AMD quad-core.
>>>
>>> I did /not/ observe the behavior on 16828088f9 (yet). I'll run the
>>> test a few more times on 1682..
>>>
>>> Additionally, I am building a bisected kernel now (
>>> cb20d5188366f04d96d2e07b1240cc92170ade40 ), but won't be able to get
>>> back at it for a while.
>>
>> cb20d5188366f04d96d2e07b1240cc92170ade40 seems OK so far. I'm going to
>> try 1de3e3df917459422cb2aecac440febc8879d410 soon.
>
> Barring false negatives, bd2d0210cf22f2bd0cef72eb97cf94fc7d31d8cc
> appears to be the culprit (according to git bisect).
> I will test bd2d0210cf22f2bd0cef72eb97cf94fc7d31d8cc again, confirm
> the behavior, and work backwards to try to reduce the possibility of
> false negatives.

A few additional notes, in no particular order:

- For me, triggering the problem is fairly easy when encryption is involved.
- I'm now 81 iterations into testing
bd2d0210cf22f2bd0cef72eb97cf94fc7d31d8cc *without* encryption.  Out of
81 iterations, I have 4 failures: #16, 40, 62, and 64.

I will now try 1de3e3df917459422cb2aecac440febc8879d410 much more extensively.

Is this useful information?

-- 
Jon
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Powered by blists - more mailing lists

Powered by Openwall GNU/*/Linux - Powered by OpenVZ