linux-kernel - Re: Need help increasing raid scan efficiency.

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20250603160415.61c9ca7c@Zen-II-x12.niklas.com>
Date: Tue, 3 Jun 2025 16:04:40 -0400
From: David Niklas <simd@...mail.net>
To: Linux RAID <linux-raid@...r.kernel.org>
Cc: linux-kernel@...r.kernel.org
Subject: Re: Need help increasing raid scan efficiency.

On Tue, 3 Jun 2025 17:46:01 +0100
Wol <antlists@...ngman.org.uk> wrote:
> On 03/06/2025 02:05, David Niklas wrote:
> > So I setup the array into read-only mode and started the array with
> > only two of the drives. Drives 0 and 1. Then I proceeded to try and
> > start a second pair, drives 2 and 3, so that I could scan them
> > simultaneously. With the intent of then switching it over to 0 and 2
> > and 1 and 3, then 0 and 3 and 1 and 2.  
> 
> BACKUP! BACKUP!! BACKUP!!!

It's when I was trying to make my yearly backup that I found out it was
corrupting. I have HDDs I backup to. When I backup, I erase the
previous year (as I don't have enough room otherwise), then backup the new
year. As a system, it worked up until now.

> Is your array that messed up that it won't assemble? If you can just
> get it to assemble normally that's your best bet by far. Trying to
> assemble it as two pairs is throwing away the whole point of a raid 6!

It assembles fine with all the disks, the problem is the data corruption
that has occurred across the members.

> And make sure you know the order of the drives in the array! I hope you
> haven't lost that infof.

Everything is written down on paper.

> If your event counts are all similar, then you'll hopefully recover
> most of your data. Your biggest worry will be the mobo and ram having
> trashing an in-flight write that corrupts the disk.

Yes, that's my problem. I wanted to try and isolate the disk pairs so
that I could try and figure out if there is any pattern or differing
copies that would allow me to restore the corrupted data.

> Then once you've got the array assembled, I can't remember the command,
> but there is a command that will read the entire stripe, check the
> paritIES - both of them, and recreate the data. If that fails, your
> data is probably toast, and nothing you can do will be able to retrieve
> much :-(
> 
> Cheers,
> Wol
> 

Searching online turned up raid6check.
https://unix.stackexchange.com/questions/137384/raid6-scrubbing-mismatch-repair

But the people there also pointed out that Linux's raid repair operation
only recalculates the parity. I would have thought that it did a best of
3 option. I mean, that's a big part of why we have RAID6 instead of RAID5,
right?

I think you misunderstood my original question, how do I assemble the
RAID6 pairs (RO mode) into two different arrays such that I can read from
them simultaneously?

If I have to do some coding with respect to the mdadm utility, I'm
willing. But for all I know, the Linux Kernel might destroy all of my data
if I try something like that.

Thanks,
David