linux-ext4 - Re: [PATCHv2 1/1] jbd2: Fix I/O hang in jbd2_journal_release_jbd

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <4C7811C7.2050506@linux.vnet.ibm.com>
Date:	Fri, 27 Aug 2010 14:28:07 -0500
From:	Brian King <brking@...ux.vnet.ibm.com>
To:	"Ted Ts'o" <tytso@....edu>
CC:	Josef Bacik <josef@...hat.com>, linux-ext4@...r.kernel.org,
	cmm@...ux.vnet.ibm.com, pmac@....ibm.com
Subject: Re: [PATCHv2 1/1] jbd2: Fix I/O hang in jbd2_journal_release_jbd_inode

On 08/27/2010 02:10 PM, Ted Ts'o wrote:
> On Wed, Jul 14, 2010 at 01:58:46PM -0500, Brian King wrote:
>>
>> I've been debugging a hang in jbd2_journal_release_jbd_inode
>> which is being seen on Power 6 systems quite a lot. When we get
>> in the hung state, all I/O to the disk in question gets blocked
>> where we stay indefinitely. Looking at the task list, I can see
>> we are stuck in jbd2_journal_release_jbd_inode waiting on a
>> wake up. I added some debug code to detect this scenario and
>> dump additional data if we were stuck in jbd2_journal_release_jbd_inode
>> for longer than 30 minutes. When it hit, I was able to see that
>> i_flags was 0, suggesting we missed the wake up.
>>
>> This patch changes i_flags to be an unsigned long, uses bit operators
>> to access it, and adds barriers around the accesses. Prior to applying
>> this patch, we were regularly hitting this hang on numerous systems
>> in our test environment. After applying the patch, the hangs no longer
>> occur. Its still not clear to me why the j_list_lock doesn't protect us
>> in this path. It also appears a hang very similar to this was seen
>> in the past and then was no longer recreatable:
> 
> I've been look at this patch, and I can see how converting to bitops
> definitely makes sense.  I can also see how adding
> smp_mb__after_clear_bit() makes sense.  However, it's not clear the
> smp_mb() call here helps?

It may not be necessary. I originally added it in order to balance
the test_bit with the clear_bit. I'll check with the folks hitting this
in test and see if I can get access to the failing machine. If so,
I'll pull this out and see if we actually need it or not.

Thanks,

Brian

-- 
Brian King
Linux on Power Virtualization
IBM Linux Technology Center


--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html