linux-ext4 - Re: question about ext4 block allocation

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20170209153009.GB3009@quack2.suse.cz>
Date:   Thu, 9 Feb 2017 16:30:09 +0100
From:   Jan Kara <jack@...e.cz>
To:     Ross Zwisler <ross.zwisler@...ux.intel.com>
Cc:     Jan Kara <jack@...e.cz>, Theodore Ts'o <tytso@....edu>,
        linux-ext4@...r.kernel.org, Xiong Zhou <xzhou@...hat.com>,
        linux-nvdimm@...ts.01.org
Subject: Re: question about ext4 block allocation

Hi Ross,

On Mon 06-02-17 16:14:09, Ross Zwisler wrote:
> I recently hit an issue in my DAX testing where I was unable to get ext4 to
> give me 2 MiB sized and aligned block allocations in a situation where I
> thought I should be able to.  I'm using a PMEM ramdisk of size 16 GiB, created
> using the memmap kernel command line parameter.
> 
>   # fdisk -l /dev/pmem0
>   Disk /dev/pmem0: 16 GiB, 17179869184 bytes, 33554432 sectors
>   Units: sectors of 1 * 512 = 512 bytes
>   Sector size (logical/physical): 512 bytes / 4096 bytes
>   I/O size (minimum/optimal): 4096 bytes / 4096 bytes
> 
> The very simple test program I used to reproduce this can be found at the
> bottom of this mail.  Here is the quick function that I used to recreate my
> filesystem each run:
> 
>   # type go_ext4
>   go_ext4 is a function
>   go_ext4 () 
>   { 
>       umount /dev/pmem0 2> /dev/null;
>       mkfs.ext4 -b 4096 -E stride=512 -F /dev/pmem0;
>       mount -o dax /dev/pmem0 ~/dax;
>       cd ~/fsync
>   }

...

> Great.  That's what I want.  But, if I create the filesystem and use the test
> to create a file that is 64 MiB in size, the PMD fault fails because the PFN I
> get from the filesystem isn't 2MiB aligned:
> 
> test-1475  [006] .... 11809.982188: dax_pmd_fault: dev 259:0 ino 0xc shared
> WRITE|ALLOW_RETRY|KILLABLE|USER address 0x40280000 vm_start 0x40000000 vm_end
> 0x40400000 pgoff 0x280 max_pgoff 0x3fff 
> 
> test-1475  [006] .... 11809.982398: dax_pmd_insert_mapping_fallback: dev 259:0
> ino 0xc shared write address 0x40280000 length 0x200000 pfn 0x108601 DEV|MAP
> radix_entry 0x0
> 
> test-1475  [006] .... 11809.982399: dax_pmd_fault_done: dev 259:0 ino 0xc
> shared WRITE|ALLOW_RETRY|KILLABLE|USER address 0x40280000 vm_start 0x40000000
> vm_end 0x40400000 pgoff 0x280 max_pgoff 0x3fff FALLBACK
> 
> The PFN for the block allocation I get from ext4 is 0x108601, which isn't
> aligned, so we fail the PG_PMD_COLOUR alignment check in
> dax_iomap_pmd_fault(), and use PTEs instead.

Yeah, it's a bug in ext4 allocator. Requests for 128MB are exactly a group
size so we find completely empty group and satisfy the request. Even larger
requests will get split into 128MB chunks. 32MB requests are small enough
that they go via a special path for power-of-two sized requests. However
64MB allocation request can be satisfied from somewhat filled group (there
are sb backup blocks in group 1 in your case) and we screw up when deciding
whether to treat such request as power-of-two or not and don't align it at
all in the end.

Another problem is that the stride size ends up unused due to another bug
in ext4. The second attached patch fixes that issue.

With these two patches applied I get file blocks aligned. That being said
the stripe-aligned allocator does a poor job of creating large extents
(larger than stripe-width) however that is more difficult to fix.

								Honza
-- 
Jan Kara <jack@...e.com>
SUSE Labs, CR

View attachment "0001-ext4-Fix-stripe-unaligned-allocations.patch" of type "text/x-patch" (2095 bytes)

View attachment "0002-ext4-Do-not-use-stripe_with-if-it-is-not-set.patch" of type "text/x-patch" (1057 bytes)