linux-kernel - Re: [RFC][PATCH] fs: Prevent syncing frozen file system

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20150709234012.GI3902@dastard>
Date:	Fri, 10 Jul 2015 09:40:12 +1000
From:	Dave Chinner <david@...morbit.com>
To:	Lukas Czerner <lczerner@...hat.com>
Cc:	viro@...IV.linux.org.uk, bfields@...ldses.org,
	linux-kernel@...r.kernel.org, linux-fsdevel@...r.kernel.org
Subject: Re: [RFC][PATCH] fs: Prevent syncing frozen file system

On Thu, Jul 09, 2015 at 07:45:45PM +0200, Lukas Czerner wrote:
> Currently we can end up in a deadlock because of broken
> sb_start_write -> s_umount ordering.
> 
> The race goes like this:
> 
>  - write the file
>  - unlink the file - final_iput will not be calles as file is opened
>  - freeze the file system
>  - Now simultaneously close the file and call sync (or syncfs on that
>    particular file system). Sync will get to wait_sb_inodes() where it will
>    grab the referece to the inode (__iget()) and later to call iput().

This problem goes away with the sync scalability patchset that josef
has been trying to get merged:

git://git.kernel.org/pub/scm/linux/kernel/git/josef/btrfs-next.git superblock-scaling

That patchset removes the full sb inodes list walk in
wait_sb_inodes() and replaces it with a walk of inodes cleaned
during the sync, which will be an empty list in the case of sync
running on an empty filesystem. This commit does the work:

https://git.kernel.org/cgit/linux/kernel/git/josef/btrfs-next.git/commit/?h=superblock-scaling&id=9bea30d5f4521db674203f365b1e0970588b2650

<As a separate note, can we *please* get that patchset merged given
that there are now several outstanding issues that it fixes in one
go?>

>    If we manage to close the file and drop the reference in between those
>    calls sync will attempt to do a iput_final() because the inode is now
>    unlinked and we're holding the last reference to it. This will
>    however block on a frozen file system (ext4_delete_inode for
>    example).
> 
> Note that I've not been able to reproduce the issue, I've only seen this
> happen once. However with some instrumentation (like msleep() in the
> wait_sb_inodes() it can be achieved.
> 
> Fix this by properly doing sb_start_write/sb_end_write to prevent us
> from fsfreeze.
> 
> Note that with this patch syncfs will block on the frozen file system
> which is probably ok, but sync will block if any file system happens to
> be frozen - not sure if that's a problem, but it's certainly different
> from what we've been used to.

sync should not block on frozen fileystems. By definition, a frozen
filesystem is a clean filesystem, and so sync should really just be
skipping over them.

> +++ b/fs/super.c
> @@ -514,10 +514,17 @@ void iterate_supers(void (*f)(struct super_block *, void *), void *arg)
>  		sb->s_count++;
>  		spin_unlock(&sb_lock);
>  
> +		/*
> +		 * Whatever we're going to do to the file system we have to
> +		 * make sure that we'll not end up blocking on frozen file
> +		 * system.
> +		 */
> +		sb_start_write(sb);
>  		down_read(&sb->s_umount);
>  		if (sb->s_root && (sb->s_flags & MS_BORN))
>  			f(sb, arg);
>  		up_read(&sb->s_umount);
> +		sb_end_write(sb);
>  
>  		spin_lock(&sb_lock);
>  		if (p)

That deadlocks sysrq-j (emergency thaw)...

Cheers,

Dave.
-- 
Dave Chinner
david@...morbit.com
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/