linux-ext4 - Re: [LSF/MM/BPF TOPIC] extsize and forcealign design in filesystems for atomic writes

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <e86ed85f-6941-44ef-96a5-0ca15faaec1d@oracle.com>
Date: Fri, 7 Feb 2025 12:01:32 +0000
From: John Garry <john.g.garry@...cle.com>
To: Ojaswin Mujoo <ojaswin@...ux.ibm.com>
Cc: lsf-pc@...ts.linux-foundation.org, linux-xfs@...r.kernel.org,
        linux-fsdevel@...r.kernel.org, djwong@...nel.org, dchinner@...hat.com,
        hch@....de, ritesh.list@...il.com, jack@...e.cz, tytso@....edu,
        linux-ext4@...r.kernel.org
Subject: Re: [LSF/MM/BPF TOPIC] extsize and forcealign design in filesystems
 for atomic writes


> Yes, bigalloc is indeed good enough as a start but yes eventually
> something like forcealign will be beneficial as not everyone prefers an
> FS-wide cluster-size allocation granularity.
> 
> We do have a patch for atomic writes with bigalloc that was sent way
> back in mid 2024 but then we went into the same discussion of mixed
> mapping[1].
> 
> Hmm I think it might be time to revisit that and see if we can do
> something better there.
> 
> [1] https://urldefense.com/v3/__https://lore.kernel.org/linux-ext4/37baa9f4c6c2994df7383d8b719078a527e521b9.1729825985.git.ritesh.list@gmail.com/__;!!ACWV5N9M2RV99hQ!OJKieZJEIvc-M87u_dxAxiEGC4zN0PQmfdLT6k73Y7_Lvr9m-iodyrytRCFxDPbVzsOlk-1kuXXvaKLA-y9kCQ$

Feel free to pick up the iomap patches I had for zeroing when trying to 
atomic write mixed mappings - that's in my v3 series IIRC.

But you might still get some push back on them...

>>
>>>
>>>>> I agree that forcealign is not the only way we can have atomic writes
>>>>> work but I do feel there is value in having forcealign for FSes and
>>>>> hence we should have a discussion around it so we can get the interface
>>>>> right.
>>>>>
>>>> I thought that the interface for forcealign according to the candidate xfs
>>>> implementation was quite straightforward. no?
>>> As mentioned in the original proposal, there are still a open problems
>>> around extsize and forcealign.
>>>
>>> - The allocation and deallocation semantics are not completely clear to
>>> 	me for example we allow operations like unaligned punch_hole but not
>>> 	unaligned insert and collapse range, and I couldn't see that
>>> 	documented anywhere.
>>
>> For xfs, we were imposing the same restrictions as which we have for
>> rtextsize > 1.
>>
>> If you check the following:
>> https://urldefense.com/v3/__https://lore.kernel.org/linux-xfs/20240813163638.3751939-9-john.g.garry@oracle.com/__;!!ACWV5N9M2RV99hQ!OJKieZJEIvc-M87u_dxAxiEGC4zN0PQmfdLT6k73Y7_Lvr9m-iodyrytRCFxDPbVzsOlk-1kuXXvaKLSPqPbqA$
>>
>> You can see how the large allocunit value is affected by forcealign, and
>> then check callers of xfs_is_falloc_aligned() -> xfs_inode_alloc_unitsize()
>> to see how this affects some fallocate modes.
> 
> True, but it's something that just implicitly happens when we use
> forcealign. I eventually found out while testing forcealign with
> different operations but such things can come as a surprise to users
> especially when we support some operations to be unaligned and then
> reject some other similar ones.
> 
> punch_hole/collapse_range is just an example and yes it might not be
> very important to support unaligned collapse range but in the long run
> it would be good to have these things documented/discussed.

Maybe the man pages can be documented for forcealign/rtextsize > 1 punch 
holes/collapse behaviour - at a quick glance, I could not see anything. 
Indeed, I am not sure how bigalloc affects punch holes/collapse range 
either.

Thanks,
John