linux-kernel - Re: [PATCH] backing_dev: Fix hung task on sync

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20140219092731.GA4849@quack.suse.cz>
Date:	Wed, 19 Feb 2014 10:27:31 +0100
From:	Jan Kara <jack@...e.cz>
To:	Tejun Heo <tj@...nel.org>
Cc:	Derek Basehore <dbasehore@...omium.org>,
	Alexander Viro <viro@...to.linux.org.uk>,
	Jan Kara <jack@...e.cz>,
	Andrew Morton <akpm@...ux-foundation.org>,
	Greg Kroah-Hartman <gregkh@...uxfoundation.org>,
	"Darrick J. Wong" <darrick.wong@...cle.com>,
	Kees Cook <keescook@...omium.org>,
	linux-fsdevel@...r.kernel.org, linux-kernel@...r.kernel.org,
	linux-mm@...ck.org, bleung@...omium.org, sonnyrao@...omium.org,
	semenzato@...omium.org
Subject: Re: [PATCH] backing_dev: Fix hung task on sync

On Tue 18-02-14 17:55:48, Tejun Heo wrote:
> Hello,
> 
> On Fri, Feb 14, 2014 at 08:12:17PM -0800, Derek Basehore wrote:
> > bdi_wakeup_thread_delayed used the mod_delayed_work function to schedule work
> > to writeback dirty inodes. The problem with this is that it can delay work that
> > is scheduled for immediate execution, such as the work from sync_inodes_sb.
> > This can happen since mod_delayed_work can now steal work from a work_queue.
> > This fixes the problem by using queue_delayed_work instead. This is a
> > regression from the move to the bdi workqueue design.
> > 
> > The reason that this causes a problem is that laptop-mode will change the
> > delay, dirty_writeback_centisecs, to 60000 (10 minutes) by default. In the case
> > that bdi_wakeup_thread_delayed races with sync_inodes_sb, sync will be stopped
> > for 10 minutes and trigger a hung task. Even if dirty_writeback_centisecs is
> > not long enough to cause a hung task, we still don't want to delay sync for
> > that long.
> 
> Oops.
> 
> > For the same reason, this also changes bdi_writeback_workfn to immediately
> > queue the work again in the case that the work_list is not empty. The same
> > problem can happen if the sync work is run on the rescue worker.
> > 
> > Signed-off-by: Derek Basehore <dbasehore@...omium.org>
> > ---
> >  fs/fs-writeback.c | 5 +++--
> >  mm/backing-dev.c  | 2 +-
> >  2 files changed, 4 insertions(+), 3 deletions(-)
> > 
> > diff --git a/fs/fs-writeback.c b/fs/fs-writeback.c
> > index e0259a1..95b7b8c 100644
> > --- a/fs/fs-writeback.c
> > +++ b/fs/fs-writeback.c
> > @@ -1047,8 +1047,9 @@ void bdi_writeback_workfn(struct work_struct *work)
> >  		trace_writeback_pages_written(pages_written);
> >  	}
> >  
> > -	if (!list_empty(&bdi->work_list) ||
> > -	    (wb_has_dirty_io(wb) && dirty_writeback_interval))
> > +	if (!list_empty(&bdi->work_list))
> > +		mod_delayed_work(bdi_wq, &wb->dwork, 0);
> > +	else if (wb_has_dirty_io(wb) && dirty_writeback_interval)
> >  		queue_delayed_work(bdi_wq, &wb->dwork,
> >  			msecs_to_jiffies(dirty_writeback_interval * 10));
> 
> Can you please add some comments explaining why the specific variants
> are being used here?
> 
> > diff --git a/mm/backing-dev.c b/mm/backing-dev.c
> > index ce682f7..3fde024 100644
> > --- a/mm/backing-dev.c
> > +++ b/mm/backing-dev.c
> > @@ -294,7 +294,7 @@ void bdi_wakeup_thread_delayed(struct backing_dev_info *bdi)
> >  	unsigned long timeout;
> >  
> >  	timeout = msecs_to_jiffies(dirty_writeback_interval * 10);
> > -	mod_delayed_work(bdi_wq, &bdi->wb.dwork, timeout);
> > +	queue_delayed_work(bdi_wq, &bdi->wb.dwork, timeout);
> 
> and here?
> 
> Hmmm.... but doesn't this create an opposite problem?  Now a flush
> queued for an earlier time may be overridden by something scheduled
> later, no?
  You are the workqueue expert so you may know better ;) But the way I
understand it is that queue_delayed_work() does nothing if the timer is
already running. Since we queue flusher work to run either immediately or
after dirty_writeback_interval we are safe to run queue_delayed_work()
whenever we want it to run after dirty_writeback_interval and
mod_delayed_work() whenever we want to run it immediately.

But it's subtle and some interface where we could say queue delayed work
after no later than X would be easier to grasp.

								Honza
-- 
Jan Kara <jack@...e.cz>
SUSE Labs, CR
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/