linux-ext4 - Re: [PATCH 1/4] vfs: vfs-level fiemap interface

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-Id: <29D7C671-F1D5-48BE-AEB4-07669FC3B405@cam.ac.uk>
Date:	Sat, 13 Sep 2008 22:59:40 +0100
From:	Anton Altaparmakov <aia21@....ac.uk>
To:	Theodore Tso <tytso@....EDU>
Cc:	linux-fsdevel@...r.kernel.org, linux-ext4@...r.kernel.org,
	akpm@...uxfoundation.org, Mark Fasheh <mfasheh@...e.com>
Subject: Re: [PATCH 1/4] vfs: vfs-level fiemap interface

Hi,

On 13 Sep 2008, at 22:29, Theodore Tso wrote:
> On Sat, Sep 13, 2008 at 10:17:07PM +0100, Anton Altaparmakov wrote:
>> Just a quick question (not a criticism of the patches in any way):
>>
>> Is there a specific reason that the flag FIEMAP_EXTENT_DATA_ENCRYPTED
>> exists?  I am only asking because there isn't a
>> FIEMAP_EXTENT_DATA_COMPRESSED flag yet the effect on the data is the
>> same.  I suppose one could argue that compression is a form of
>> encryption (as one cannot read the data on disk without decompressing
>> it) and thus for compressed files I can just set
>> FIEMAP_EXTENT_DATA_ENCRYPTED, too?
>
> You could, but actually, the interface isn't really set up well to
> deal with compressed files.  In order to deal with compressed data you
> really would need two fields in struct fiemap_extent:
> fe_length_physical, fe_length_logical, like this:
>
> struct fiemap_extent {
> 	__u64 fe_logical;  /* logical offset in bytes for the start of
> 			    * the extent from the beginning of the file */
> 	__u64 fe_physical; /* physical offset in bytes for the start
> 			    * of the extent from the beginning of the disk */
> 	__u64 fe_length;   /* length in bytes for this extent */
> 	__u64 fe_length_physical;   /* length in bytes for this extent as
> 	      			     * encoded/compressed on disk */
> 	__u32 fe_flags;    /* FIEMAP_EXTENT_* flags for this extent */
> 	__u32 fe_reserved;
> };

Yes, that could make sense for compressed files although for NTFS at  
least it does not matter because the logical regions which do not  
exist on disk (due to compression having made the corresponding  
physical regions smaller by at least one block) are present in the  
extent list as sparse extents, so the extents for a compressed file  
could look like this for example (lengths are in blocks):

logical ofs	physical ofs	length
0		X		14
14		SPARSE		2
16		Y		4
20		SPARSE		12
32		Z		9
41		SPARSE		7		
...		...		...

As you may have noticed from the above example in NTFS each block of  
16 blocks is compressed independently, hence the sparse extents  
interspersed with the compressed extents to bring the logical offset  
in line with the physical offset.

> Part of the problem/frustration as we go around and round fs-devel is
> that some people would request new features to support new
> filesystems, and other people have been rejecting new features because
> it makes the interface more complicated.
>
> In fact, the previous version of the patch had a flag for compression,
> but Cristoph Hellwig objected because he claimed there were no users
> for it.

He is plain wrong then!  NTFS supports compressed files and always has  
done!  We only support them read-only at present but that does not  
matter for the purposes of FIEMAP.  And in any case one day we will  
have write support for compressed files, too...

> I don't care how this interface ends up, at least as far as this point
> is concerned, but it's really annoying when some people want to add
> features, and other people are objecting and trying to remove
> features, and months and months and months go by while the patch
> languishes while people play tug of war on the d*mned thing.
>
> Probably the best thing to do is to add a much larger number of
> reserved fields, of size __u32 and __u64, appropriately aligned, and
> then when someone adds a compressed filesystem, we can add the
> structure definition without changing the size of the structure.
> Otherwise, the thou-shalt-not-anticipate-the-needs-of-any-future-
> users-of-the-interface nazi's will strike again, and more months will
> go by with absolutely no progress....


We already have two file systems in the mainline kernel tree that  
support transparent compression/decompression: ntfs and zisofs so  
there is no "future" anywhere in sight...  (-;

Best regards,

	Anton
-- 
Anton Altaparmakov <aia21 at cam.ac.uk> (replace at with @)
Unix Support, Computing Service, University of Cambridge, CB2 3QH, UK
Linux NTFS maintainer, http://www.linux-ntfs.org/

--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html