linux-ext4 - Re: [PATCH 2/2] ext4: fix bug in ext4_mb_normalize

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:	Tue, 3 Jun 2014 16:36:39 -0400
From:	Theodore Ts'o <tytso@....edu>
To:	Lukáš Czerner <lczerner@...hat.com>
Cc:	Maurizio Lombardi <mlombard@...hat.com>, adilger.kernel@...ger.ca,
	linux-ext4@...r.kernel.org, linux-fsdevel@...r.kernel.org
Subject: Re: [PATCH 2/2] ext4: fix bug in ext4_mb_normalize_request()

On Tue, Jun 03, 2014 at 08:43:40PM +0200, Lukáš Czerner wrote:
> 
> I think that leaving the alignment of the start offset for the small
> files/allocation is not good idea. We might end up with suboptimal
> file layout for smaller files. While this is not a big deal for
> bigger files, with smaller ones it might cause some troubles.

I thought we were only aligning the start offset for files > 2MB?

> Also I started looking into normalize_request and hopefully I'll
> have a patch soon. Ted, do you have any suggestion for a test to
> make sure that I do not make things worse ? You mentioned something
> simple on LSF, but I do not remember what it was.

The two mechanisms which we have currently are:

1) e2freefrag to measure free space fragmentation

2) e2fsck -fE fragcheck /dev/sdXX

The latter will give you a reports such as this:

142618(f): expecting 950605 actual extent phys 950800 log 89 len 26
142618(f): expecting 950826 actual extent phys 950550 log 282 len 10

Which would correspond to the following:

debugfs:  stat <142618>
Inode: 142618   Type: regular    Mode:  0666   Flags: 0x80000
Generation: 2697220261    Version: 0x00000000:00000001
User:     0   Group:     0   Size: 1194112
File ACL: 0    Directory ACL: 0
Links: 1   Blockcount: 360
Fragment:  Address: 0    Number: 0    Size: 0
 ctime: 0x53495656:aa328a78 -- Sat Apr 12 11:05:58 2014
 atime: 0x53495643:c20a0ea4 -- Sat Apr 12 11:05:39 2014
 mtime: 0x53495653:43ad6c78 -- Sat Apr 12 11:05:55 2014
crtime: 0x53495641:1332f128 -- Sat Apr 12 11:05:37 2014
Size of extra inode fields: 28
EXTENTS:
(68-76):950596-950604, (89-114):950800-950825, (282-291):950550-950559

This is admittedly imperfect.  First of all, it gives false positives
when the file has holes (as in the above case).  And even if it didn't
what we should do is to print the previous extent before the
contiunity, since what's interesting is how big was the previous
extent before we had a discontinuity, and because the length of the
current extent isn't all that interesting in the case of last extent
(the "tail") of the file.

Hmm... if I have time I might look into patching e2fsck so that e2fsck
-E fragcheck is more useful.

What's also missing is some way of taking all of this fine-grained
information is turning it into one to three figures of merit that
could be used to compare different tweaks to the block allocation
algorithm.

Cheers,

					- Ted
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html