linux-ext4 - Re: [patch] document flash/RAID dangers

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:	Tue, 25 Aug 2009 19:48:09 -0400
From:	Ric Wheeler <rwheeler@...hat.com>
To:	Pavel Machek <pavel@....cz>
CC:	david@...g.hm, Theodore Tso <tytso@....edu>,
	Florian Weimer <fweimer@....de>,
	Goswin von Brederlow <goswin-v-b@....de>,
	Rob Landley <rob@...dley.net>,
	kernel list <linux-kernel@...r.kernel.org>,
	Andrew Morton <akpm@...l.org>, mtk.manpages@...il.com,
	rdunlap@...otime.net, linux-doc@...r.kernel.org,
	linux-ext4@...r.kernel.org, corbet@....net
Subject: Re: [patch] document flash/RAID dangers

> ---
> There are storage devices that high highly undesirable properties
> when they are disconnected or suffer power failures while writes are
> in progress; such devices include flash devices and MD RAID 4/5/6
> arrays.  These devices have the property of potentially
> corrupting blocks being written at the time of the power failure, and
> worse yet, amplifying the region where blocks are corrupted such that
> additional sectors are also damaged during the power failure.

I would strike the entire mention of MD devices since it is your assertion, not 
a proven fact. You will cause more data loss from common events (single sector 
errors, complete drive failure) by steering people away from more reliable 
storage configurations because of a really rare edge case (power failure during 
split write to two raid members while doing a RAID rebuild).

>
> Users who use such storage devices are well advised take
> countermeasures, such as the use of Uninterruptible Power Supplies,
> and making sure the flash device is not hot-unplugged while the device
> is being used.  Regular backups when using these devices is also a
> Very Good Idea.

All users who care about data integrity - including those who do not use MD5 but 
just regular single S-ATA disks - will get better reliability from a UPS.

>
> Otherwise, file systems placed on these devices can suffer silent data
> and file system corruption.  An forced use of fsck may detect metadata
> corruption resulting in file system corruption, but will not suffice
> to detect data corruption.
>

This is very misleading. All storage "can" have silent data loss, you are making 
a statement without specifics about frequency.

FSCK can repair the file system metadata, but will not detect any data loss or 
corruption in the data blocks allocated to user files. To detect data loss 
properly, you need to checksum (or digitally sign) all objects stored in a file 
system and verify them on a regular basis.

Also helps to keep a separate list of those objects on another device so that 
when the metadata does take a hit, you can enumerate your objects and verify 
that you have not lost anything.

ric

ric

--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html