linux-ext4 - Re: [PATCH 2/2] ext4: fix bug in ext4_mb_normalize

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:	Fri, 6 Jun 2014 09:09:38 +0200 (CEST)
From:	Lukáš Czerner <lczerner@...hat.com>
To:	"Theodore Ts'o" <tytso@....edu>
cc:	adilger.kernel@...ger.ca, linux-ext4@...r.kernel.org,
	linux-fsdevel@...r.kernel.org
Subject: Re: [PATCH 2/2] ext4: fix bug in ext4_mb_normalize_request()

On Tue, 3 Jun 2014, Theodore Ts'o wrote:

> Date: Tue, 3 Jun 2014 16:36:39 -0400
> From: Theodore Ts'o <tytso@....edu>
> To: Lukáš Czerner <lczerner@...hat.com>
> Cc: Maurizio Lombardi <mlombard@...hat.com>, adilger.kernel@...ger.ca,
>     linux-ext4@...r.kernel.org, linux-fsdevel@...r.kernel.org
> Subject: Re: [PATCH 2/2] ext4: fix bug in ext4_mb_normalize_request()
> 
> On Tue, Jun 03, 2014 at 08:43:40PM +0200, Lukáš Czerner wrote:
> > 
> > I think that leaving the alignment of the start offset for the small
> > files/allocation is not good idea. We might end up with suboptimal
> > file layout for smaller files. While this is not a big deal for
> > bigger files, with smaller ones it might cause some troubles.
> 
> I thought we were only aligning the start offset for files > 2MB?
> 
> > Also I started looking into normalize_request and hopefully I'll
> > have a patch soon. Ted, do you have any suggestion for a test to
> > make sure that I do not make things worse ? You mentioned something
> > simple on LSF, but I do not remember what it was.
> 
> The two mechanisms which we have currently are:
> 
> 1) e2freefrag to measure free space fragmentation

Yes, that's what I am using along with e4defrag and debugfs -R
"dump_extents" to figure out what changed.

> 
> 2) e2fsck -fE fragcheck /dev/sdXX

I was not aware of that, thanks. However I was more interested in
workload to test on. Right now I have a simple script which is
doing:

        # Test fallocate time
        echo "[+] Fallocate test"
        time fallocate -l70G ${MNT}/file
        do_freefrag
        do_defrag ${MNT}/file
        do_debugfs /file
        rm -f ${MNT}/file
        sync

        # Copy linux source
        echo "[+] Copy linux source"
        cp -r $linux_source ${MNT}/linux1 &
        cp -r $linux_source ${MNT}/linux2 &
        time wait
        do_freefrag

        # Tar linux source
        echo "[+] Tar linux source"
        tar -cf ${MNT}/linux1_1.tar ${MNT}/linux1 &
        tar -cf ${MNT}/linux1_2.tar ${MNT}/linux1 &
        time wait
        do_freefrag
        do_defrag ${MNT}/linux1_1.tar
        do_debugfs /linux1_1.tar
        do_defrag ${MNT}/linux1_2.tar
        do_debugfs /linux1_2.tar

        # Untar linux source
        echo "[+] Untar linux source"
        mkdir ${MNT}/tt1
        mkdir ${MNT}/tt2
        tar -xf ${MNT}/linux1_1.tar -C ${MNT}/tt1 &
        tar -xf ${MNT}/linux1_2.tar -C ${MNT}/tt2 &
        time wait
        do_freefrag

        # Singe dd
        echo "[+] Single dd"
        time dd if=/dev/zero of=${MNT}/file bs=512 count=4194304
        do_freefrag
        do_defrag ${MNT}/file
        do_debugfs /file

        # Multiple dd test
        echo "[+] Multiple dd"
        # 2G
        dd if=/dev/zero of=${MNT}/file0 bs=4096 count=524288 &
        # 2G
        dd if=/dev/urandom of=${MNT}/file1 bs=512 count=4194304 &
        # 2G
        dd if=/dev/zero of=${MNT}/file2 bs=1M count=2048 &
        # 4M
        dd if=/dev/zero of=${MNT}/file3 bs=4096 count=1024 &
        # 16M
        dd if=/dev/zero of=${MNT}/file4 bs=4096 count=4096 &
        time wait
        do_freefrag
        do_defrag ${MNT}/file0
        do_debugfs /file0
        do_defrag ${MNT}/file1
        do_debugfs /file1
        do_defrag ${MNT}/file2
        do_debugfs /file2
        do_defrag ${MNT}/file3
        do_debugfs /file3
        do_defrag ${MNT}/file4
        do_debugfs /file4

        # Run fsstress
        echo "[+] Run fsstress"
        time $fsstress -s 123456 -p24 -n 999 -d $MNT
        do_freefrag
        do_defrag ${MNT}

And then do the same thing on the loop device and investigate the
underlying image file.

Thanks!
-Lukas