linux-ext4 - RE: fsync stuck at jbd2_log_wait

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <COL130-W95F37DFE0CBD0ED0680AABD67E0@phx.gbl>
Date:	Wed, 12 Aug 2015 12:28:10 -0700
From:	Roy Yang <bsdnet@...look.com>
To:	Theodore Ts'o <tytso@....edu>
CC:	"linux-ext4@...r.kernel.org" <linux-ext4@...r.kernel.org>
Subject: RE: fsync stuck at jbd2_log_wait_commit on NVMe devices

Hi Ted, 

     I will collect some info about jbd2 and get back to this list.
     
     To answer your questions,  our workload is mostly sequential  writes from a couple of daemons.
     Every second, we will create 6 to 10 files, and write around 8M bytes into them. We only
     do extend or truncate when needed, but I am sure we rarely do this on NVMe devices.

     Reading the kernel document,  I also see ext4 ordered option will combine metadata and data blocks
     for journaling. If the  data block is big, and the workload is heavy, will JBD2 be soaked?

     Thanks,
     
     Roy

----------------------------------------
> Date: Wed, 12 Aug 2015 14:37:50 -0400
> From: tytso@....edu
> To: bsdnet@...look.com
> CC: linux-ext4@...r.kernel.org
> Subject: Re: fsync stuck at jbd2_log_wait_commit on NVMe devices
>
> Hi Roy,
>
> My suggestion is to collect information from the jbd2_run_stats and
> jbd2_checkpoint_stats tracepoints.
>
> cd /sys/kernel/debug/tracing
> echo 1> events/jbd2/jbd2_run_stats/enable
> echo 1> events/jbd2/jbd2_checkpoint_stats/enable
> cat trace_pipe> /tmp/traces &
> tail -f /tmp/traces
>
> The jbd2_handle_stats tracepoint can be informative, but it's also far
> more voluminous.
>
> That will give us a hint where things are getting bottlenecked.
>
> What sort of workload is your application doing? Is it just primarily
> doing random writes into a preallocated file? Is it creating or
> deleting files? Extending or truncating files? etc.
>
> - Ted
> --
> To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
> the body of a message to majordomo@...r.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
 		 	   		  --
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html