linux-ext4 - Re: [PATCH v5 0/3] ext4, jbd2: journal cycled record transactions between each mount

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <62be1bf9-293a-7833-d13d-e7dee7d04742@huawei.com>
Date:   Thu, 23 Mar 2023 16:20:06 +0800
From:   Zhang Yi <yi.zhang@...wei.com>
To:     Andreas Dilger <adilger@...ger.ca>,
        Zhang Yi <yi.zhang@...weicloud.com>
CC:     Ext4 Developers List <linux-ext4@...r.kernel.org>,
        Theodore Ts'o <tytso@....edu>, Jan Kara <jack@...e.cz>,
        <yukuai3@...wei.com>
Subject: Re: [PATCH v5 0/3] ext4, jbd2: journal cycled record transactions
 between each mount

On 2023/3/23 5:34, Andreas Dilger wrote:
> On Mar 21, 2023, at 7:33 PM, Zhang Yi <yi.zhang@...weicloud.com> wrote:
>> This patch set add a new journal option 'JBD2_CYCLE_RECORD' and always
>> enable on ext4. It saves journal head for a clean unmounted file system
>> in the journal super block, which could let us record journal
>> transactions between each mount continuously. It could help us to do
>> journal backtrack and find root cause from a corrupted filesystem.
>> Current filesystem's corruption analysis is difficult and less useful
>> information, especially on the real products. It is useful to some
>> extent, especially for the cases of doing fuzzy tests and deploy in some
>> shout-runing products.
> 
> Another interesting side benefit of this change is that it gets a step
> closer to the "lazy ext4" (log-structured optimization) that had been
> described some time ago at FAST:
> 
> https://lwn.net/Articles/720226/
> https://www.usenix.org/system/files/conference/fast17/fast17-aghayev.pdf
> https://lists.openwall.net/linux-ext4/2017/04/11/1
> 
> Essentially, free space in the filesystem (or a large external device)
> could be used as a continuous journal, and metadata would only rarely
> be checkpointed to the actual filesystem.  If the "journal" is close to
> wrapping to the start, either the meta/data is checkpointed (if it is
> no longer actively used or can make a large write), or re-journaled to
> the end of the journal.  At remount time, the full journal is read into
> memory (discarding old copies of blocks) and this is used to identify
> the current metadata rather than reading from the filesystem itself.
> 
> This would allow e.g. very efficient flash caching of metadata (and also
> journaled data for small writes) for an HDD (or QLC) device.
> 

This is interesting, but current change looks like is just one small step.
It's been almost 6 years after the last talk I can found[1]. Is there
anyone still working on it?

[1] https://lore.kernel.org/linux-ext4/6B0F0C59-6930-41B3-8EE4-EA5BEECEB9F9@dilger.ca/

Thanks,
Yi.