linux-kernel - Re: [RFC PATCH v5 06/10] ovl: implement overlayfs' ->write

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <17d31bf3d62.1119ad4be10313.6832593367889908304@mykernel.net>
Date:   Thu, 18 Nov 2021 14:32:36 +0800
From:   Chengguang Xu <cgxu519@...ernel.net>
To:     "Miklos Szeredi" <miklos@...redi.hu>
Cc:     "Jan Kara" <jack@...e.cz>, "Amir Goldstein" <amir73il@...il.com>,
        "linux-fsdevel" <linux-fsdevel@...r.kernel.org>,
        "overlayfs" <linux-unionfs@...r.kernel.org>,
        "linux-kernel" <linux-kernel@...r.kernel.org>
Subject: Re: [RFC PATCH v5 06/10] ovl: implement overlayfs' ->write_inode
 operation


 ---- 在 星期三, 2021-11-17 14:11:29 Chengguang Xu <cgxu519@...ernel.net> 撰写 ----
 >  ---- 在 星期二, 2021-11-16 20:35:55 Miklos Szeredi <miklos@...redi.hu> 撰写 ----
 >  > On Tue, 16 Nov 2021 at 03:20, Chengguang Xu <cgxu519@...ernel.net> wrote:
 >  > >
 >  > >  ---- 在 星期四, 2021-10-07 21:34:19 Miklos Szeredi <miklos@...redi.hu> 撰写 ----
 >  > >  > On Thu, 7 Oct 2021 at 15:10, Chengguang Xu <cgxu519@...ernel.net> wrote:
 >  > >  > >  > However that wasn't what I was asking about.  AFAICS ->write_inode()
 >  > >  > >  > won't start write back for dirty pages.   Maybe I'm missing something,
 >  > >  > >  > but there it looks as if nothing will actually trigger writeback for
 >  > >  > >  > dirty pages in upper inode.
 >  > >  > >  >
 >  > >  > >
 >  > >  > > Actually, page writeback on upper inode will be triggered by overlayfs ->writepages and
 >  > >  > > overlayfs' ->writepages will be called by vfs writeback function (i.e writeback_sb_inodes).
 >  > >  >
 >  > >  > Right.
 >  > >  >
 >  > >  > But wouldn't it be simpler to do this from ->write_inode()?
 >  > >  >
 >  > >  > I.e. call write_inode_now() as suggested by Jan.
 >  > >  >
 >  > >  > Also could just call mark_inode_dirty() on the overlay inode
 >  > >  > regardless of the dirty flags on the upper inode since it shouldn't
 >  > >  > matter and results in simpler logic.
 >  > >  >
 >  > >
 >  > > Hi Miklos，
 >  > >
 >  > > Sorry for delayed response for this, I've been busy with another project.
 >  > >
 >  > > I agree with your suggesion above and further more how about just mark overlay inode dirty
 >  > > when it has upper inode? This approach will make marking dirtiness simple enough.
 >  > 
 >  > Are you suggesting that all non-lower overlay inodes should always be dirty?
 >  > 
 >  > The logic would be simple, no doubt, but there's the cost to walking
 >  > those overlay inodes which don't have a dirty upper inode, right?  
 > 
 > That's true.
 > 
 >  > Can you quantify this cost with a benchmark?  Can be totally synthetic,
 >  > e.g. lookup a million upper files without modifying them, then call
 >  > syncfs.
 >  > 
 > 
 > No problem, I'll do some tests for the performance.
 > 

Hi Miklos,

I did some rough tests and the results like below.
In practice,  I don't think that 1.3s extra time of syncfs will cause significant problem.
What do you think?



Test bed: kvm vm 
2.50GHz cpu 32core
64GB mem
vm kernel  5.15.0-rc1+ (with ovl syncfs patch V6)

one millon files spread to 2 level of dir hierarchy.
test step:
1) create testfiles in ovl upper dir
2) mount overlayfs
3) excute ls -lR to lookup all file in overlay merge dir
4) excute slabtop to make sure overlay inode number
5) call syncfs to the file in merge dir

Tested five times and the reusults are in 1.310s ~ 1.326s

root@...144-4-centos test]# time ./syncfs ovl-merge/create-file.sh 
syncfs success

real    0m1.310s
user    0m0.000s
sys     0m0.001s
[root@...144-4-centos test]# time ./syncfs ovl-merge/create-file.sh 
syncfs success

real    0m1.326s
user    0m0.001s
sys     0m0.000s
[root@...144-4-centos test]# time ./syncfs ovl-merge/create-file.sh 
syncfs success

real    0m1.321s
user    0m0.000s
sys     0m0.001s
[root@...144-4-centos test]# time ./syncfs ovl-merge/create-file.sh 
syncfs success

real    0m1.316s
user    0m0.000s
sys     0m0.001s
[root@...144-4-centos test]# time ./syncfs ovl-merge/create-file.sh 
syncfs success

real    0m1.314s
user    0m0.001s
sys     0m0.001s


Directly run syncfs to the file in ovl-upper dir.
Tested five times and the reusults are in 0.001s ~ 0.003s

[root@...144-4-centos test]# time ./syncfs a
syncfs success

real    0m0.002s
user    0m0.001s
sys     0m0.000s
[root@...144-4-centos test]# time ./syncfs ovl-upper/create-file.sh 
syncfs success

real    0m0.003s
user    0m0.001s
sys     0m0.000s
[root@...144-4-centos test]# time ./syncfs ovl-upper/create-file.sh 
syncfs success

real    0m0.001s
user    0m0.000s
sys     0m0.001s
[root@...144-4-centos test]# time ./syncfs ovl-upper/create-file.sh 
syncfs success

real    0m0.001s
user    0m0.000s
sys     0m0.001s
[root@...144-4-centos test]# time ./syncfs ovl-upper/create-file.sh 
syncfs success

real    0m0.001s
user    0m0.000s
sys     0m0.001s
[root@...144-4-centos test]# time ./syncfs ovl-upper/create-file.sh 
syncfs success

real    0m0.001s
user    0m0.000s
sys     0m0.001