linux-ext4 - RE: [PATCH] jbd2: wake up j_wait_done

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <3ED34739A4E85E4F894367D57617CDEF9ED90708@LAX-EX-MB2.datadirect.datadirectnet.com>
Date:	Wed, 22 Jun 2016 11:04:27 +0000
From:	Wang Shilong <wshilong@....com>
To:	Wang Shilong <wangshilong1991@...il.com>,
	Ext4 Developers List <linux-ext4@...r.kernel.org>
Subject: RE: [PATCH] jbd2: wake up j_wait_done_commit before commit callback

Please ignore this version of patch, patch has problems. will send a v2.

Thanks,
Shilong

________________________________________
From: Wang Shilong [wangshilong1991@...il.com]
Sent: Friday, June 17, 2016 18:05
To: Ext4 Developers List; Wang Shilong
Subject: Re: [PATCH] jbd2: wake up j_wait_done_commit before commit callback

On Thu, Jun 16, 2016 at 12:45 PM, Wang Shilong
<wangshilong1991@...il.com> wrote:
> From: Wang Shilong <wshilong@....com>
>
> Lustre register some callbacks in ext4 to keep data consistency,
> we see millons callbacks possibility on MDS for metadata operations.
>
> If there are many commit callbacks which might take some cpu
> time, we can mark commit done before we call commit callback.
> this can relax some thread waiting for commit finished, for example
> transaction throttle in end transaction.

This patch might not be correct before patch:
[PATCH -v2] ext4: optimize ext4_should_retry_alloc() to improve ENOSPC
performance

Because some thread might assume transaction commit can free some
metadata space, and in that case transaction callback need done before
we mark transaction done.

But Lustre just use commit callback to confirm transaction is done and data
is in a consistent state.

After Ted's patch, Lustre is only user for commit callback, it is fine to
apply this optimizations.

Regards,
Shilong
>
> Signed-off-by: Wang Shilong <wshilong@....com>
> ---
>  fs/jbd2/commit.c |   12 ++++++++----
>  1 files changed, 8 insertions(+), 4 deletions(-)
>
> diff --git a/fs/jbd2/commit.c b/fs/jbd2/commit.c
> index 7007809..a9cf5a4 100644
> --- a/fs/jbd2/commit.c
> +++ b/fs/jbd2/commit.c
> @@ -379,6 +379,7 @@ void jbd2_journal_commit_transaction(journal_t *journal)
>         tid_t first_tid;
>         int update_tail;
>         int csum_size = 0;
> +       int need_free = 0;
>         LIST_HEAD(io_bufs);
>         LIST_HEAD(log_bufs);
>
> @@ -1100,9 +1101,6 @@ restart_loop:
>
>         write_unlock(&journal->j_state_lock);
>
> -       if (journal->j_commit_callback)
> -               journal->j_commit_callback(journal, commit_transaction);
> -
>         trace_jbd2_end_commit(journal, commit_transaction);
>         jbd_debug(1, "JBD2: commit %d complete, head %d\n",
>                   journal->j_commit_sequence, journal->j_tail_sequence);
> @@ -1114,12 +1112,18 @@ restart_loop:
>         if (commit_transaction->t_checkpoint_list == NULL &&
>             commit_transaction->t_checkpoint_io_list == NULL) {
>                 __jbd2_journal_drop_transaction(journal, commit_transaction);
> -               jbd2_journal_free_transaction(commit_transaction);
> +               need_free = 1;
>         }
>         spin_unlock(&journal->j_list_lock);
>         write_unlock(&journal->j_state_lock);
>         wake_up(&journal->j_wait_done_commit);
>
> +       if (journal->j_commit_callback)
> +               journal->j_commit_callback(journal, commit_transaction);
> +
> +       if (need_free)
> +               jbd2_journal_free_transaction(commit_transaction);
> +
>         /*
>          * Calculate overall stats
>          */
> --
> 1.7.1
>
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html