linux-ext4 - Re: [PATCH] e2fsck: do not allow initialized blocks pass i

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:   Fri, 29 Jun 2018 10:46:34 +0200
From:   Lukas Czerner <lczerner@...hat.com>
To:     Andreas Dilger <adilger@...ger.ca>
Cc:     "Theodore Y. Ts'o" <tytso@....EDU>, linux-ext4@...r.kernel.org
Subject: Re: [PATCH] e2fsck: do not allow initialized blocks pass i_size

On Thu, Jun 28, 2018 at 09:05:04PM -0600, Andreas Dilger wrote:
> 
> > On Jun 28, 2018, at 7:45 PM, Theodore Y. Ts'o <tytso@....EDU> wrote:
> > 
> > On Thu, Jun 28, 2018 at 06:07:47PM -0600, Andreas Dilger wrote:
> >>> But does an RDMA operation actually do a block allocation?  Really?
> >>> And if it is willing to do a block allocation, why is it not willing
> >>> to bump i_size?
> >> 
> >> It's not that the RDMA does block allocation, but rather that the RDMA
> >> always transfers and writes the full PAGE_SIZE of data, even if i_size
> >> is less than the end of the last block.  This simplifies the RDMA code
> >> so that it can always write the data instead of having to stop at i_size.

Every time we write past i_size we need to extend the i_size, why is
this case different ?

> > 
> > Right.  So there are two choices:
> > 
> > 1) Keep the blocks beyond i_size marked as uninitialized.  You
> > transfer and write the full PAGE_SIZE of data, but it simply will
> > never be available to the user.

Yes, that's for extent mapped files.

> > 
> > 2)  Zero the page, write it out to the file, and then extend i_size and
> > mark the extents as uninitialized.

Except at that point you do not really need to mark the extent as
unitialized, the blocks are allocated and written to and i_size is
extended. That's how it needs to be done for indirect block mapped
files.

> 
> The end of the page would already be zeroed before write.
> 
> > Why is it that Lustre is choosing to keep i_size where it is, but to
> > mark the blocks beyond it as initialized?
> 
> This isn't about initialized vs. uninitialized extents.  It is only about
> allocated vs. unallocated blocks, possibly with block-mapped files.  There
> is no way to have uninitialized blocks with a block-mapped file.
> 
> The code is checking whether there are any blocks allocated beyond i_size,
> and if there are, without the patch it considers i_size broken and extends
> it to the end of the last allocated block.  The patch allows a small number
> of blocks to be allocated beyond i_size without triggering this heuristic.

I do not think that ext4 has any capacity to allocate initialized blocks
beyond i_size without actually writing to it and if we write beyong i_size
then we need to extend i_size so if we see anything like that it's a problem
that needs fixing. I think it's as simple as that.

Unless you can show me where in upstream ext4 we can get to this
situation I am strongly for fixing e2fsck. But maybe I am missing
something.

> 
> The only difference vs. the previous code is that it correctly calculates
> what the PAGE_SIZE aligned block number is (the old code assumed that lblock
> was the base-1 block number instead of the base-0 block number that it is).

Right, but the test is broken when PAGE_SIZE > 4k that's how I found
out in the first place. See me previous email about f_eofblocks and
f_pgsize_gt_blksize failures.

Thanks!
-Lukas