linux-kernel - Re: [PATCH 1/2] f2fs: enable to gc page whose inode already atomic commit

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <3182ade9-4153-9e47-f8a5-5c87371a3900@huawei.com>
Date:   Mon, 5 Feb 2018 17:37:40 +0800
From:   Yunlong Song <yunlong.song@...wei.com>
To:     Chao Yu <yuchao0@...wei.com>, Chao Yu <chao@...nel.org>,
        <jaegeuk@...nel.org>, <yunlong.song@...oud.com>
CC:     <miaoxie@...wei.com>, <bintian.wang@...wei.com>,
        <shengyong1@...wei.com>, <heyunlei@...wei.com>,
        <linux-f2fs-devel@...ts.sourceforge.net>,
        <linux-kernel@...r.kernel.org>
Subject: Re: [PATCH 1/2] f2fs: enable to gc page whose inode already atomic
 commit


> OK, details as I explained before:
> 
> atomic_commit				GC
> - file_write_and_wait_range
> 					- move_data_block
> 					 - f2fs_submit_page_write
> 					  - f2fs_update_data_blkaddr
> 					   - set_page_dirty
>   - fsync_node_pages
> 
> 1. atomic writes data page #1 & update node #1
> 2. GC data page #2 & update node #2
> 3. page #1 & node #1 & #2 can be committed into nand flash before page #2 be
> committed.
> 
> After a sudden pow-cut, database transaction will be inconsistent. So I think
> there will be better to exclude gc/atomic_write to each other, with a lock
> instead of flag checking.
> 

I do not understand why this transaction is inconsistent, is it a
problem that page #2 is not committed into nand flash? Since normal
gc also has this problem:

Suppose that there is db file A, f2fs_gc moves data page #1 of db file
A. But if write checkpoint only commit node page #1 and then a sudden
power-cut happens. Data page #1 is not committed to nand flash, but
node page #1 is committed. Is the db transaction broken and
inconsistent?

Come back to your example, I think data page 2 of atomic file does not
belong to this transaction, so even node page 2 is committed, it is just
the same problem as what I have listed above(db file A), and it does not
break this transaction.

> Thanks,
> 
>>>>>
>>>>> So how about just using dio_rwsem[WRITE] during atomic committing to exclude
>>>>> GCing data block of atomic opened file?
>>>>>
>>>>> Thanks,
>>>>>
>>>>>>
>>>>>> Signed-off-by: Yunlong Song <yunlong.song@...wei.com>
>>>>>> ---
>>>>>>     fs/f2fs/data.c | 5 ++---
>>>>>>     fs/f2fs/gc.c   | 6 ++++--
>>>>>>     2 files changed, 6 insertions(+), 5 deletions(-)
>>>>>>
>>>>>> diff --git a/fs/f2fs/data.c b/fs/f2fs/data.c
>>>>>> index 7435830..edafcb6 100644
>>>>>> --- a/fs/f2fs/data.c
>>>>>> +++ b/fs/f2fs/data.c
>>>>>> @@ -1580,14 +1580,13 @@ bool should_update_outplace(struct inode *inode, struct f2fs_io_info *fio)
>>>>>>     		return true;
>>>>>>     	if (S_ISDIR(inode->i_mode))
>>>>>>     		return true;
>>>>>> -	if (f2fs_is_atomic_file(inode))
>>>>>> -		return true;
>>>>>>     	if (fio) {
>>>>>>     		if (is_cold_data(fio->page))
>>>>>>     			return true;
>>>>>>     		if (IS_ATOMIC_WRITTEN_PAGE(fio->page))
>>>>>>     			return true;
>>>>>> -	}
>>>>>> +	} else if (f2fs_is_atomic_file(inode))
>>>>>> +		return true;
>>>>>>     	return false;
>>>>>>     }
>>>>>>     
>>>>>> diff --git a/fs/f2fs/gc.c b/fs/f2fs/gc.c
>>>>>> index b9d93fd..84ab3ff 100644
>>>>>> --- a/fs/f2fs/gc.c
>>>>>> +++ b/fs/f2fs/gc.c
>>>>>> @@ -622,7 +622,8 @@ static void move_data_block(struct inode *inode, block_t bidx,
>>>>>>     	if (!check_valid_map(F2FS_I_SB(inode), segno, off))
>>>>>>     		goto out;
>>>>>>     
>>>>>> -	if (f2fs_is_atomic_file(inode))
>>>>>> +	if (f2fs_is_atomic_file(inode) &&
>>>>>> +		!f2fs_is_commit_atomic_write(inode))
>>>>>>     		goto out;
>>>>>>     
>>>>>>     	if (f2fs_is_pinned_file(inode)) {
>>>>>> @@ -729,7 +730,8 @@ static void move_data_page(struct inode *inode, block_t bidx, int gc_type,
>>>>>>     	if (!check_valid_map(F2FS_I_SB(inode), segno, off))
>>>>>>     		goto out;
>>>>>>     
>>>>>> -	if (f2fs_is_atomic_file(inode))
>>>>>> +	if (f2fs_is_atomic_file(inode) &&
>>>>>> +		!f2fs_is_commit_atomic_write(inode))
>>>>>>     		goto out;
>>>>>>     	if (f2fs_is_pinned_file(inode)) {
>>>>>>     		if (gc_type == FG_GC)
>>>>>>
>>>>>
>>>>> .
>>>>>
>>>>
>>>
>>>
>>> .
>>>
>>
> 
> 
> .
> 

-- 
Thanks,
Yunlong Song