lists.openwall.net   lists  /  announce  owl-users  owl-dev  john-users  john-dev  passwdqc-users  yescrypt  popa3d-users  /  oss-security  kernel-hardening  musl  sabotage  tlsify  passwords  /  crypt-dev  xvendor  /  Bugtraq  Full-Disclosure  linux-kernel  linux-netdev  linux-ext4  linux-hardening  linux-cve-announce  PHC 
Open Source and information security mailing list archives
 
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Date:   Fri, 13 Dec 2019 16:59:12 +0100
From:   Jan Kara <jack@...e.cz>
To:     Paul Richards <paul.richards@...il.com>
Cc:     linux-ext4@...r.kernel.org
Subject: Re: Query about ext4 commit interval vs dirty_expire_centisecs

Hello!

On Tue 19-11-19 08:47:31, Paul Richards wrote:
> I'm trying to understand the interaction between the ext4 `commit`
> interval option, and the `vm.dirty_expire_centisecs` tuneable.
> 
> The ext4 `commit` documentation says:
> 
> > Ext4 can be told to sync all its data and metadata every 'nrsec' seconds. The default value is 5 seconds. This means that if you lose your power, you will lose as much as the latest 5 seconds of work (your filesystem will not be damaged though, thanks to the journaling).
> 
> The `dirty_expire_centisecs` documentation says:
> 
> > This tunable is used to define when dirty data is old enough to be eligible for writeout by the kernel flusher threads. It is expressed in 100'ths of a second. Data which has been dirty in-memory for longer than this interval will be written out next time a flusher thread wakes up.
> 
> 
> Superficially these sound like they have a very similar effect.  They
> periodically flush out data that hasn't been explicitly fsync'd by the
> application.  I'd like to understand a bit more the interaction
> between these.

Yes, the effect is rather similar but not quite the same. The first thing
to observe is kind of obvious fact that ext4 commit interval influences
just the particular filesystem while dirty_expire_centisecs influences
behavior of global writeback over all filesystems.

Secondly, commit interval is really the maximum age of ext4 transation.  So
if there is metadata change pending in the journal, it will become
persistent at latest after this time. So for say 'mkdir' that will be
persistent at latest after this time. For data operations things are more
complex. E.g. when delayed allocation is used (which is the default), the
change gets logged in the journal only during writeback. So it can take up
to dirty_expire_centisecs for data to be written back from page cache, that
results in filesystem journalling block allocations etc. and then it can
take upto commit interval for these changes to become persistent. So in
this case the intervals add up. There are also other special cases
somewhere in between but generally it is reasonable to assume that data gets
automatically persistent in dirty_expire_centisecs + commit_interval time.
Note both these times are actually times when writeback is triggered so
if the disk gets too busy, the actual time when data is completely on disk
may be much higher.

								Honza

-- 
Jan Kara <jack@...e.com>
SUSE Labs, CR

Powered by blists - more mailing lists