linux-ext4 - Re: [patch] ext2/3: document conditions when reliable operation is possible

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:	Wed, 26 Aug 2009 09:32:32 +1000 (EST)
From:	"NeilBrown" <neilb@...e.de>
To:	"Ric Wheeler" <rwheeler@...hat.com>
Cc:	"Theodore Tso" <tytso@....edu>, "Pavel Machek" <pavel@....cz>,
	"Florian Weimer" <fweimer@....de>,
	"Goswin von Brederlow" <goswin-v-b@....de>,
	"Rob Landley" <rob@...dley.net>,
	"kernel list" <linux-kernel@...r.kernel.org>,
	"Andrew Morton" <akpm@...l.org>, mtk.manpages@...il.com,
	rdunlap@...otime.net, linux-doc@...r.kernel.org,
	linux-ext4@...r.kernel.org
Subject: Re: [patch] ext2/3: document conditions when reliable operation is 
     possible

On Wed, August 26, 2009 9:10 am, Ric Wheeler wrote:
> On 08/25/2009 06:58 PM, Neil Brown wrote:
>> On Monday August 24, tytso@....edu wrote:
>>> On Mon, Aug 24, 2009 at 11:25:19PM +0200, Pavel Machek wrote:
>>>>> I have to admit that I have not paid enough attention to this
>>>>> specifics
>>>>> of your ext3 + flash card issue - is it the ftl stuff doing out of
>>>>> order
>>>>> IO's?
>>>>
>>>> The problem is that flash cards destroy whole erase block on unplug,
>>>> and ext3 can't cope with that.
>>>
>>> Sure --- but name **any** filesystem that can deal with the fact that
>>> 128k or 256k worth of data might disappear when you pull out the flash
>>> card while it is writing a single sector?
>>
>> A Log structured filesystem could certainly be written to deal with
>> such a situation, providing by 'deal with' you mean 'only loses data
>> that has not yet been acknowledged to the application'.  Of course the
>> filesystem would need clear visibility into exactly how these blocks
>> are positioned.
>>
>> I've been playing with just such a filesystem for some time (never
>> really finding enough time) with the goal of making it work over RAID5
>> with no data risk due to power loss.  One day it will be functional
>> enough for others to try....
>>
>> It is entirely possible that NILFS could be made to meet that
>> requirement, but I haven't made time to explore NILFS so I cannot be
>> sure.
>>
>> NeilBrown
>>
>
> I am not sure that log structure will protect you from this scenario since
> once
> you clean the log, the non-logged data is assumed to be correct.
>
> If your cheap flash storage device can nuke random regions of that clean
> storage, you will lose data....

Hence my observation that "the filesystem would need clear visibility into
exactly how these blocks are positioned".
If there is an FTL in the way that randomly relocates blocks, and a
power fail during write could corrupt data that appears to be
megabytes away in some unpredictable location, then yes: a log structure
won't help.

However I would like to imagine that even a cheep flash device, if it
only ever got writes that were exactly the size of the erase-block, would
not break those writes over multiple erase blocks, so some degree of
integrity and predictability could be preserved.  Even more so, I would
love to  be able to disable the FTL, or at least have clear and correct
documentation about how it works.

So yes, not a panacea.  But an avenue with real possibilities.

NeilBrown

--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html