linux-kernel - Re: [PATCH] f2fs: set file as cold when file defragmentation

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <771a05fe-e26f-d635-5f8d-5be72f82345f@huawei.com>
Date:   Tue, 11 May 2021 15:23:06 +0800
From:   Chao Yu <yuchao0@...wei.com>
To:     <daejun7.park@...sung.com>, Jaegeuk Kim <jaegeuk@...nel.org>
CC:     "chao@...nel.org" <chao@...nel.org>,
        "linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
        "linux-f2fs-devel@...ts.sourceforge.net" 
        <linux-f2fs-devel@...ts.sourceforge.net>
Subject: Re: [PATCH] f2fs: set file as cold when file defragmentation

On 2021/5/11 14:41, Daejun Park wrote:
>> On 2021/5/11 13:09, Jaegeuk Kim wrote:
>>> On 05/11, Chao Yu wrote:
>>>> On 2021/5/10 22:47, Jaegeuk Kim wrote:
>>>>> On 05/06, Chao Yu wrote:
>>>>>> On 2021/5/6 12:46, Jaegeuk Kim wrote:
>>>>>>> On 05/06, Chao Yu wrote:
>>>>>>>> On 2021/4/29 14:20, Daejun Park wrote:
>>>>>>>>> In file defragmentation by ioctl, all data blocks in the file are
>>>>>>>>> re-written out-of-place. File defragmentation implies user will not update
>>>>>>>>> and mostly read the file. So before the defragmentation, we set file
>>>>>>>>> temperature as cold for better block allocation.
>>>>>>>>
>>>>>>>> I don't think all fragmented files are cold, e.g. db file.
>>>>>>>
>>>>>>> I have a bit different opinion. I think one example would be users want to
>>>>>>> defragment a file, when the they want to get higher read bandwidth for
>>>>>>
>>>>>> Multimedia file was already defined as cold file now via default extension
>>>>>> list?
>>>>>
>>>>> I just gave an example. And default is default.
>>>>>
>>>>>>
>>>>>>> usually multimedia files. That's likely to be cold files. Moreover, I don't
>>>>>>> think they want to defragment db files which will be fragmented soon?
>>>>>>
>>>>>> I guess like db files have less update but more access?
>>>>>
>>>>> I think both, and we set it as hot.
>>>>
>>>> Then hot and cold bit will set to the same db file after defragmentation?
>>>
>>> Do you set cold bit to db files? I mean, generally db is not cold, but hot.
>>
>> I never set cold bit to db files, I mean if we defragment db file which
>> has less update and more access, db file may have bot hot and cold bit.
>>
>> To Daejun, may I ask that is Samsung planning to use this defragment ioctl
>> in products? what's the user scenario?
> 
> It is just my idea for defragmentation, not Samsung.

Alright,

> I think the user will call the defrag ioctl for the files that have been updated.

Sadly, I don't see there is any user of this defragment interface since it was
been introduced... so I really don't know the real use scenario of this interface
now.

> 
> On the other hand, I think FS should be able to support defrag file even
> when in-place update is applied. What do you think?

bool f2fs_should_update_inplace(struct inode *inode, struct f2fs_io_info *fio)
{
	if (f2fs_is_pinned_file(inode))
		return true;

	/* if this is cold file, we should overwrite to avoid fragmentation */
	if (file_is_cold(inode))
		return true;

If cold bit was set, later rewrite in defragment interface can only trigger
IPU due to above IPU policy check, so after this interface, file is still
fragmented... what's the difference compared to just setting cold bit via
setxattr?

And if user know that he will trigger less update and more read in the file,
why not just calling setxattr("system.advise", cold_bit) to set the file as
cold before it becomes fragmented, e.g. at the time of file creation?

Thanks,

> 
> Thanks,
> Daejun
>>
>> Thanks,
>>
>>>
>>>>
>>>> Thanks,
>>>>
>>>>>
>>>>>>
>>>>>> Thanks,
>>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>> We have separated interface (via f2fs_xattr_advise_handler, e.g. setfattr -n
>>>>>>>> "system.advise" -v value) to indicate this file is a hot/cold file, so my
>>>>>>>> suggestion is after file defragmentation, if you think this file is cold, and
>>>>>>>> use setxattr() to set it as cold.
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>>
>>>>>>>>>
>>>>>>>>> Signed-off-by: Daejun Park <daejun7.park@...sung.com>
>>>>>>>>> ---
>>>>>>>>>       fs/f2fs/file.c | 3 +++
>>>>>>>>>       1 file changed, 3 insertions(+)
>>>>>>>>>
>>>>>>>>> diff --git a/fs/f2fs/file.c b/fs/f2fs/file.c
>>>>>>>>> index d697c8900fa7..dcac965a05fe 100644
>>>>>>>>> --- a/fs/f2fs/file.c
>>>>>>>>> +++ b/fs/f2fs/file.c
>>>>>>>>> @@ -2669,6 +2669,9 @@ static int f2fs_defragment_range(struct f2fs_sb_info *sbi,
>>>>>>>>>               map.m_len = pg_end - pg_start;
>>>>>>>>>               total = 0;
>>>>>>>>> +        if (!file_is_cold(inode))
>>>>>>>>> +                file_set_cold(inode);
>>>>>>>>> +
>>>>>>>>>               while (map.m_lblk < pg_end) {
>>>>>>>>>                       pgoff_t idx;
>>>>>>>>>                       int cnt = 0;
>>>>>>>>>
>>>>>>> .
>>>>>>>
>>>>> .
>>>>>
>>> .
>>>
>>
>>
>>   
> .
>