linux-kernel - Re: [RFC PATCH 0/2] apply write hints to select the type of segments

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:   Mon, 13 Nov 2017 09:26:32 +0800
From:   Chao Yu <yuchao0@...wei.com>
To:     Hyunchul Lee <hyc.lee@...il.com>
CC:     Jaegeuk Kim <jaegeuk@...nel.org>,
        <linux-f2fs-devel@...ts.sourceforge.net>,
        <linux-kernel@...r.kernel.org>, <kernel-team@....com>,
        Hyunchul Lee <cheol.lee@....com>
Subject: Re: [RFC PATCH 0/2] apply write hints to select the type of segments

On 2017/11/13 8:24, Hyunchul Lee wrote:
> On 11/10/2017 03:42 PM, Chao Yu wrote:
>> On 2017/11/10 8:23, Hyunchul Lee wrote:
>>> Hello, Chao
>>>
>>> On 11/09/2017 06:12 PM, Chao Yu wrote:
>>>> On 2017/11/9 13:51, Hyunchul Lee wrote:
>>>>> From: Hyunchul Lee <cheol.lee@....com>
>>>>>
>>>>> Using write hints[1], applications can inform the life time of the data
>>>>> written to devices. and this[2] reported that the write hints patch
>>>>> decreased writes in NAND by 25%.
>>>>>
>>>>> This hints help F2FS to determine the followings.
>>>>>   1) the segment types where the data will be written.
>>>>>   2) the hints that will be passed down to devices with the data of segments.
>>>>>
>>>>> This patch set implements the first mapping from write hints to segment types
>>>>> as shown below.
>>>>>
>>>>>   hints                     segment type
>>>>>   -----                     ------------
>>>>>   WRITE_LIFE_SHORT          CURSEG_COLD_DATA
>>>>>   WRITE_LIFE_EXTREME        CURSEG_HOT_DATA
>>>>>   others                    CURSEG_WARM_DATA
>>>>>
>>>>> The F2FS poliy for hot/cold seperation has precedence over this hints, And
>>>>> hints are not applied in in-place update.
>>>>
>>>> Could we change to disable IPU if file/inode write hint is existing?
>>>>
>>>
>>> I am afraid that this makes side effects. for example, this could cause
>>> out-of-place updates even when there are not enough free segments. 
>>> I can write the patch that handles these situations. But I wonder 
>>> that this is required, and I am not sure which IPU polices can be disabled.
>>
>> Oh, As I replied in another thread, I think IPU just affects filesystem
>> hot/cold separating, rather than this feature. So I think it will be okay
>> to not consider it.
>>
>>>
>>>>>
>>>>> Before the second mapping is implemented, write hints are not passed down
>>>>> to devices. Because it is better that the data of a segment have the same 
>>>>> hint.
>>>>>
>>>>> [1]: c75b1d9421f80f4143e389d2d50ddfc8a28c8c35
>>>>> [2]: https://lwn.net/Articles/726477/
>>>>
>>>> Could you write a patch to support passing write hint to block layer for
>>>> buffered writes as below commit:
>>>> 0127251c45ae ("ext4: add support for passing in write hints for buffered writes")
>>>>
>>>
>>> Sure I will. I wrote it already ;)
>>
>> Cool, ;)
>>
>>> I think that datas from the same segment should be passed down with the same
>>> hint, and the following mapping is reasonable. I wonder what is your opinion
>>> about it.
>>>
>>>   segment type               hints
>>>   ------------               -----
>>>   CURSEG_COLD_DATA           WRITE_LIFE_EXTREME
>>>   CURSEG_HOT_DATA            WRITE_LIFE_SHORT
>>>   CURSEG_COLD_NODE           WRITE_LIFE_NORMAL
>>
>> We have WRITE_LIFE_LONG defined rather than WRITE_LIFE_NORMAL in fs.h?
>>
>>>   CURSEG_HOT_NODE            WRITE_LIFE_MEDIUM
>>
>> As I know, in scenario of cell phone, data of meta_inode is hottest, then hot
>> data, warm node, and cold node should be coldest. So I suggested we can define
>> as below:
>>
>> META_DATA			WRITE_LIFE_SHORT
>> HOT_DATA & WARM_NODE		WRITE_LIFE_MEDIUM
>> HOT_NODE & WARM_DATA		WRITE_LIFE_LONG
>> COLD_NODE & COLD_DATA		WRITE_LIFE_EXTREME
>>
> 
> I agree, But I am not sure that assigning the same hint to a node and data
> segment is good. Because NVMe is likely to write them in the same erase 
> block if they have the same hint.

If we do not give the hint, they can still be written to the same erase block,
right? it will not be worse?

Thanks,

> 
> Thanks.
> 
>> Thanks,
>>
>>>   others                     WRITE_LIFE_NONE
>>>  
>>>> Thanks,
>>>>
>>>>>
>>>>> Hyunchul Lee (2):
>>>>>   f2fs: apply write hints to select the type of segments for buffered
>>>>>     write
>>>>>   f2fs: apply write hints to select the type of segment for direct write
>>>>>
>>>>>  fs/f2fs/data.c    | 101 ++++++++++++++++++++++++++++++++----------------------
>>>>>  fs/f2fs/f2fs.h    |   1 +
>>>>>  fs/f2fs/segment.c |  14 +++++++-
>>>>>  3 files changed, 74 insertions(+), 42 deletions(-)
>>>>>
>>>>
>>>>
>>>
>>> Thanks
>>>
>>> .
>>>
>>
>>
> 
> .
>