linux-ext4 - Re: ext4 recovery

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:   Mon, 20 Jan 2020 19:20:43 -0700
From:   Andreas Dilger <adilger@...ger.ca>
To:     James Courtier-Dutton <james.dutton@...il.com>
Cc:     "Theodore Y. Ts'o" <tytso@....edu>,
        linux-ext4 <linux-ext4@...r.kernel.org>
Subject: Re: ext4 recovery

You can use "filefrag -k" to get units of KB instead of blocks for the physical offsets of the files, and "lvdisplay -m /dev/XXX" to show the mapping of LVs to disks (not sure if you can get it to print different units, or just LE number and you gave to multiply by the LE size).  Hopefully the LV was layer out linearly and is just a matter of computing "> X and < Y" to determine the affected range. 

Note that it is likely that some files which were on the second disk will not show anything from filefrag since the inodes were zeroed out by dd_rescue. Some files on the second disk that _do_ show blocks that were allocated on that disk may just have zeroes for the data. 

Cheers, Andreas

> On Jan 20, 2020, at 15:07, James Courtier-Dutton <james.dutton@...il.com> wrote:
> 
> On Tue, 14 Jan 2020 at 21:25, Theodore Y. Ts'o <tytso@....edu> wrote:
>> 
>>> On Tue, Jan 14, 2020 at 04:03:53PM +0000, James Courtier-Dutton wrote:
>>> 
>>> Say I started with 1 disk using LVM with an ext4 partition.
>>> I then added another disk. Added it to the LVM group, expanded the
>>> ext4 partition to then fill 2 disks.
>>> I then added another disk. Added it to the LVM group, expanded the
>>> ext4 partition to then fill 3 disks.
>> 
>> Where you using RAID 0, or some more advanced RAID level?
>> 
>>> One of the disk has now failed.
>> 
>> How has it failed?  It is dead dead dead?  Or are there a large number
>> of sector errors?
>> 
>>> Are there any tools available for ext4 that could help recover this?
>>> Note, I am a single user, not a company, so there is no money to give
>>> to a data recovery company, so I wish to try myself.
>> 
>> How valuable is your data?  The first thing I would recommend, if your
>> data is worth it (and only you can make that decision) is to create a
>> new RAID set (using larger disks if that helps reduce the price) so
>> you can make an block-level image backup using the dd_rescue program.
>> 
>> If you can, then run e2fsck on the backup copy, and then see what you
>> can recover from that image set.  This will save time (how much is
>> your time worth?) and perhaps increase the amount of data you can
>> recover (how much is your data worth?).
>> 
>>                                        - Ted
> 
> Hi,
> 
> Thank you all for the help.
> I have made some progress.
> Disk 1 is 100% OK.
> Disk 2 did dd_rescue and recovered about 30% of the sectors.
> Disk 3 is 100% OK.
> 
> I have made images of all of that. LVM worked out what order they went
> in, and ext4 even let me mount the LVM volume.
> I have managed to recover a lot of the data, so thank you for the hints.
> What I would like to do now is find out which files were on Disk 2.
> I have found the fiemap  IOCTL that gives me the logical sectors for
> the file itself.
> How do I convert the value from the IOCTL into a physical sector, or
> at least an offset into the LVM volume?
> I can then work at that sector 0 to A is on Disk 1,  A to B is on Disk
> 2, and B to C is on Disk 3.
> 
> For example, this is how it is done on btrfs. What is the equivalent on ext4?
> 
> $ sudo filefrag -v UEFI_Spec_2_8_final.pdf
> Filesystem type is: 9123683e
> File size of UEFI_Spec_2_8_final.pdf is 18586279 (4538 blocks of 4096 bytes)
> ext:     logical_offset:        physical_offset: length:   expected: flags:
>   0:        0..    4537:    9798022..   9802559:   4538:             last,eof
> UEFI_Spec_2_8_final.pdf: 1 extent found
> $ sudo btrfs-map-logical -l $[9798022*4096] /dev/nvme0n1p7
> mirror 1 logical 40132698112 physical 31475654656 device /dev/nvme0n1p7
> $ sudo dd if=/dev/nvme0n1p7 bs=1
> skip=31475654656 count=64 2>/dev/null | hexdump -C
> 00000000  25 50 44 46 2d 31 2e 36  0d 25 e2 e3 cf d3 0d 0a  |%PDF-1.6.%......|
> 00000010  33 30 35 35 31 39 20 30  20 6f 62 6a 0d 3c 3c 2f  |305519 0 obj.<</|
> 00000020  46 69 6c 74 65 72 2f 46  6c 61 74 65 44 65 63 6f  |Filter/FlateDeco|
> 00000030  64 65 2f 46 69 72 73 74  20 31 31 37 39 2f 4c 65  |de/First 1179/Le|
> 00000040
> 
> 
> Kind Regards
> 
> James