linux-ext4 - Re: [Patch 0/4] RFC : Support for data gradation of a single file.

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <CAExFE6=z_2syxpr+yiRnFvSg658HVB4-S6NRcv7WdsmDt4J=eQ@mail.gmail.com>
Date:   Tue, 10 Apr 2018 15:22:38 +0530
From:   Sayan Ghosh <sgdgp.2014@...il.com>
To:     "Theodore Y. Ts'o" <tytso@....edu>
Cc:     linux-ext4@...r.kernel.org, linux-fsdevel@...r.kernel.org,
        "Bhattacharya, Suparna" <suparna.bhattacharya@....com>,
        niloy ganguly <ganguly.niloy@...il.com>,
        Madhumita Mallick <madhu.cse.ju@...il.com>,
        "Bharde, Madhumita" <madhumita.bharde@....com>
Subject: Re: [Patch 0/4] RFC : Support for data gradation of a single file.

Hi Theodore,

> It wasn't clear what was your purpose in posting these patches.  There
> are a large number of ways in which they simply aren't ready for
> upstream merging.  As a short list:
>
> 1)  They are against an ancient version of the kernel (4.7.2).
>
> 2)  There are a large number of TODO's in it in the code
>
> 3) The boundary between the two different tiers of storage is
> currently harded in a header file using a #define (!).
>
>
> If the goal is to gather comments about the design, I wish you had
> presented the problem statement to the ext4 mailig list much earlier.

Yes, we want to get an early feedback of the problem statement as well
as the patchset in general. The next task is to modify the codes
against the current kernel version. Also as mentioned in the TO-DOs,
we are looking for better ideas on 1) finding a way to not hard code
and automatically finding the boundaries of the storage tiers. 2) to
automatically detect what the faster tier is for block allocation and
view. However solving the 1st TO-DO about boundaries is more important
for making the system robust.


> The other thing to consider is whether it makes any sense at all to
> solve this problem by haing a single file system where part of the
> storage is DAX, and part is not.  Why not just have two file systems,
> one which is 100% DAX, and another which is 100% HDD/SSD, and store
> the data in two files in two different file sytsems?

As we mentioned in the problem statement, we are interested in
providing a reduced view of a single file where important and
unimportant portions are interspersed - hence splitting it in two
filesystems with important and unimportant parts would not serve our
objective. Let’s say in the example, an user wants the full view of
the video. In this case splitting the video in two filesystems would
not be ideal, as the user needs to be provided with both important and
unimportant blocks. Creating a sparse layout to overlay two files will
unnecessarily be complicated. It’ll hence be ideal if a file has those
graded information as a metadata (extended attributes in our case),
and use those information to properly place and fetch when necessary.


Regards,
Sayan Ghosh
‌On Sat, Apr 7, 2018 at 3:57 AM, Theodore Y. Ts'o <tytso@....edu> wrote:
> Hi Sayan,
>
> It wasn't clear what was your purpose in posting these patches.  There
> are a large number of ways in which they simply aren't ready for
> upstream merging.  As a short list:
>
> 1)  They are against an ancient version of the kernel (4.7.2).
>
> 2)  There are a large number of TODO's in it in the code
>
> 3) The boundary between the two different tiers of storage is
> currently harded in a header file using a #define (!).
>
>
> If the goal is to gather comments about the design, I wish you had
> presented the problem statement to the ext4 mailig list much earlier.
> It might have saved you time in terms since we could have given you
> feedback before you had done all of this work on this patch set.
>
> Andreas' comments about making the allocation hints persistent not
> making any sense are very much on target.  Once the file is written,
> the hints won't be needed at all.
>
> In addition, you should strongly think about some way propagating the
> fact that some blocks in device-mapper device are backed by DAX, and
> others are not, as a device-mapper interface.  And it might not
> necessarily a single break point where below a block number is SSD or
> HDD storage, and above a block number it's DAX storage.
>
> The other thing to consider is whether it makes any sense at all to
> solve this problem by haing a single file system where part of the
> storage is DAX, and part is not.  Why not just have two file systems,
> one which is 100% DAX, and another which is 100% HDD/SSD, and store
> the data in two files in two different file sytsems?
>
>                                                 - Ted

</tytso@....edu>