linux-kernel - Re: [PATCH RFC] mm: implement write-behind policy for sequential file writes

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <CA+55aFyXrxN8Dqw9QK9NPWk+ZD52fT=q2y7ByPt9pooOrio3Nw@mail.gmail.com>
Date:   Mon, 2 Oct 2017 12:54:53 -0700
From:   Linus Torvalds <torvalds@...ux-foundation.org>
To:     Konstantin Khlebnikov <khlebnikov@...dex-team.ru>
Cc:     linux-fsdevel <linux-fsdevel@...r.kernel.org>,
        linux-mm <linux-mm@...ck.org>,
        Linux Kernel Mailing List <linux-kernel@...r.kernel.org>,
        Jens Axboe <axboe@...nel.dk>, Michal Hocko <mhocko@...e.com>,
        Mel Gorman <mgorman@...e.de>,
        Johannes Weiner <hannes@...xchg.org>,
        Tejun Heo <tj@...nel.org>,
        Andrew Morton <akpm@...ux-foundation.org>
Subject: Re: [PATCH RFC] mm: implement write-behind policy for sequential file writes

On Mon, Oct 2, 2017 at 2:54 AM, Konstantin Khlebnikov
<khlebnikov@...dex-team.ru> wrote:
>
> This patch implements write-behind policy which tracks sequential writes
> and starts background writeback when have enough dirty pages in a row.

This looks lovely to me.

I do wonder if you also looked at finishing the background
write-behind at close() time, because it strikes me that once you
start doing that async writeout, it would probably be good to make
sure you try to do the whole file.

I'm thinking of filesystems that do delayed allocation etc - I'd
expect that you'd want the whole file to get allocated on disk
together, rather than have the "first 256kB aligned chunks" allocated
thanks to write-behind, and then the final part allocated much later
(after other files may have triggered their own write-behind). Think
loads like copying lots of pictures around, for example.

I don't have any particularly strong feelings about this, but I do
suspect that once you have started that IO, you do want to finish it
all up as the file write is done. No?

It would also be really nice to see some numbers. Perhaps a comparison
of "vmstat 1" or similar when writing a big file to some slow medium
like a USB stick (which is something we've done very very badly at,
and this should help smooth out)?

                Linus