linux-ext4 - a potential deadlock?

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Date:	Fri, 26 Oct 2007 14:52:29 +0800
From:	"LIOU Payphone" <lioupayphone@...il.com>
To:	<linux-ext4@...r.kernel.org>
Subject: a potential deadlock?

Hi All

Here is a question to be confirmed.

In ext3_ioctl() with "cmd == EXT3_IOC_SETFLAGS", we firstly lock
"inode->i_mutex", start a handle with 1 journal-block by calling
ext3_journal_start(). In ext3_new_blocks(), say QUOTA was enabled with vfsv0
format, we will call the function "DQUOT_ALLOC_BLOCK()". The handle in
ext3_new_blocks()  was started by high-level functions, and
DQUOT_ALLOC_BLOCK() will finally calles ext3_quota_write() in which it try
to lock the "i_mutex" of the inode of a quota-file. 

At it happens, when we want to modify the inodes of quota-files via
ext3_ioctl(cmd = EXT3_IOC_SETFLAGS) (say process-A), another guy try to
execute ext3_quota_write() by calling DQUOT_ALLOC_BLOCK() (say process-B). I
guess a potential deadlock between process-A and process-B would happen in
such a executing sequence:

(1) process-B got many journal-blocks, then came into ext3_new_blocks(),
hung up
(2) process-A locked i_mutex of the inode of a quota-file, then try to
starts a handle. Unfortunately, there are no enough journal-blocks left for
process-A.
(3) process-B awakened, and came into DQUOT_ALLOC_BLOCK(), finally came into
the function ext3_quota_write() who also wants to lock the i_mutex of the
inode of a quota-file. But the i_mutex was locked by process-A. so process-B
has no choice but to wait.
(4) if the ext3-filesystem was  too busy to release jounal-blocks for
process-A, or a unexpected incident happened. Both  the two situations would
result in no journal-blocks for any other processes. Apparently, process-A
have to wait for available journal-blocks. so process-A was hung-up with
i_mutex of the inode of a quota-file locked.
(5) process-B was blocked by the "inode->i_mutex" subsequently.

a deadlock happened?

is such a suppose reasonable?

Payphone

-
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html