linux-ext4 - RE: Re: [PATCH] ext4: hand over jobs handling discard commands on commit complete phase to kworkers

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:   Wed, 17 May 2017 01:24:06 +0000
From:   Daeho Jeong <daeho.jeong@...sung.com>
To:     Jan Kara <jack@...e.cz>, Daeho Jeong <daeho.jeong@...sung.com>
CC:     "jack@...e.com" <jack@...e.com>, "tytso@....edu" <tytso@....edu>,
        "linux-ext4@...r.kernel.org" <linux-ext4@...r.kernel.org>
Subject: RE: Re: [PATCH] ext4: hand over jobs handling discard commands on
 commit complete phase to kworkers

> So I see several problems with this. Firstly, it breaks the ENOSPC handling
> logic which relies on the fact that after forcing a transaction commit all
> blocks held by the transaction are released - now they will be released
> only after the work is completed and thus we can prematurely report ENOSPC.

Hi, Jan.

We already freed all the blocks in the block bitmap and increased
sbi->s_freeclusters_counter in ext4_free_blocks() in advance of
ext4_free_data_callback() which is handling discard commands and releasing
blocks in the buddy cache. So, I think that it's ok about ENOSPC, because
we are ckecking whether we can allocate the blocks or not using
ext4_claim_free_clusters(), which is just seeing sbi->s_freeclusters_counter,
and the blocks were already freed from on-disk block bitmap.

> Secondly, offloading the discard work doesn't change the fundamental fact
> that discard is slow (for some devices) and this change just hides this
> fact at the possible cost of for example higher file fragmentation as a
> result of delayed block freeing. Also the outstanding queue of discard
> requests isn't limited in any way again leading to possible strange
> allocation / ENOSPC issues.

Yes, I agree with you about that the discard handling will be still slow.
However, by hiding this, we can get a better responsiveness of fsync() from
30s to 0.255s in the worst case and this is very important to mobile environments
where fsync() delay means the users have to wait to do the next action in a while.
For the higher file fragmentation, even now, we cannot free the blocks fastly
in the buddy cache because we have to handle all the discard commands before
freeing blocks in the buddy. So, we already have the same problem now. :-)

Thank you.