linux-ext4 - Re: Repeatable block allocation problem.

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <CA+1E3rLM6kfw+gZxLwjwcFXr-xCn96b4AxyQTrZN6v6Lh2_nGw@mail.gmail.com>
Date:   Mon, 17 Sep 2018 22:10:25 +0530
From:   Joshi <joshiiitr@...il.com>
To:     sandeen@...hat.com
Cc:     linux-ext4@...r.kernel.org
Subject: Re: Repeatable block allocation problem.

It's not that I need specific values for extents/blocks for a file. It
is just about being able to create/prepare a file of given size with
same (repeatable) number of extents and block numbers, in a controlled
environment.
I thought it would happen naturally, if I keep preconditions same
(i.e. post format with lazy initialization disabled. I had not
disabled lazy-allocation though).
But extent count differed, at times.
And I wondered whether Ext4 code makes any conscious attempt to
randomize the extent/block allocation?

End objective of above was to create fragmented file, with repeatable
extent-count/block-numbers.
I'll check xfstests as well.

Thanks,

On Mon, Sep 17, 2018 at 9:31 PM Eric Sandeen <sandeen@...hat.com> wrote:
>
> On 9/17/18 10:59 AM, Joshi wrote:
> > Sorry if i gave the impression that i needed something in user-mode only. I am fine even if change involves modifying ext4 source.
>
> Perhaps you can describe what defrag behavior you are trying to test,
> and we can offer some suggestions for good ways to construct that test?
>
> There are already a handful of defrag tests in xfstests, fwiw.
>
> Thanks,
> -Eric
>
> > On Mon, 17 Sep 2018, 21:14 Eric Sandeen, <esandeen@...hat.com <mailto:esandeen@...hat.com>> wrote:
> >
> >     On 9/17/18 6:22 AM, Joshi wrote:
> >     > I've been doing some defrag related tests, and for that I needed to be
> >     > able to create file with same set of block numbers (i.e. extents), for
> >     > at least two times.
> >
> >     Userspace cannot control that.
> >
> >     > May I know if there is any randomness in Ext4 allocator, and if there
> >     > is any, can I disable it for the purpose of getting repeatable
> >     > block-allocation patterns.
> >     >
> >     > Here are experiment details -
> >     > For a 100K file (created using dd if=/dev/zero of=/mnt/f100k bs=4K
> >     > count=100, oflag=direct) I got extent info in one run as this -
> >     >
> >     > File size of /mnt/file400k is 409600 (100 blocks of 4096 bytes)
> >     >  ext:     logical_offset:        physical_offset: length:   expected: flags:
> >     >    0:        0..      15:      34816..     34831:     16:
> >     >    1:       16..      99:      33824..     33907:     84:      34832: last,eof
> >     >
> >     > while in the the second run I got somewhat different runs -
> >     > File size of /mnt/file400k is 409600 (100 blocks of 4096 bytes)
> >     >  ext:     logical_offset:        physical_offset: length:   expected: flags:
> >     >    0:        0..       0:      34816..     34816:      1:
> >     >    1:        1..      15:      34320..     34334:     15:      34817:
> >     >    2:       16..      99:      33824..     33907:     84:      34335: last,eof
> >     >
> >     > Each run beings with a mkfs.ext4 with lazy inode/journal
> >     > initialization disabled.
> >
> >     Userspace doesn't get to pick physical locations for allocations, you cannot
> >     write a test which depends on doing so.
> >
> >     -Eric
> >
>


-- 
Joshi