linux-kernel - Re: INFO: task hung in pipe

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:   Sat, 8 Aug 2020 10:01:21 +0900
From:   Tetsuo Handa <penguin-kernel@...ove.sakura.ne.jp>
To:     Andrea Arcangeli <aarcange@...hat.com>,
        Linus Torvalds <torvalds@...ux-foundation.org>,
        Al Viro <viro@...iv.linux.org.uk>
Cc:     syzbot <syzbot+96cc7aba7e969b1d305c@...kaller.appspotmail.com>,
        syzkaller-bugs <syzkaller-bugs@...glegroups.com>,
        linux-fsdevel <linux-fsdevel@...r.kernel.org>,
        Linux Kernel Mailing List <linux-kernel@...r.kernel.org>,
        Dmitry Vyukov <dvyukov@...gle.com>,
        Andrew Morton <akpm@...ux-foundation.org>
Subject: Re: INFO: task hung in pipe_read (2)

On 2020/08/07 14:31, Andrea Arcangeli wrote:
>> Andrea? Comments? As mentioned, this is probably much too aggressive,
>> but I do think we need to limit the time that the kernel will wait for
>> page faults.
> 
> Why is pipe preventing to SIGKILL the task that is blocked on the
> mutex_lock? Is there any good reason for it or it simply has margin
> for improvement regardless of the hangcheck report? It'd be great if
> we can look into that before looking into the uffd specific bits.

It would be possible to use _killable version for this specific function, but

> 
> The hangcheck timer would have zero issues with tasks that can be
> killed, if only the pipe code could be improved to use mutex_lock_killable.
> 
> 		/* use "==" to skip the TASK_KILLABLE tasks waiting on NFS */
> 		if (t->state == TASK_UNINTERRUPTIBLE)
> 			check_hung_task(t, timeout);
> 
> The hangcheck report is just telling us one task was in D state a
> little too long, but it wasn't fatal error and the kernel wasn't
> actually destabilized and the only malfunction reported is that a task
> was unkillable for too long.

use of killable waits disables ability to detect possibility of deadlock (because
lockdep can't check possibility of deadlock which involves actions in userspace), for
syzkaller process is SIGKILLed after 5 seconds while khungtaskd's timeout is 140 seconds.

If we encounter a deadlock in an unattended operation (e.g. some server process),
we don't have a method for resolving the deadlock. Therefore, I consider that
t->state == TASK_UNINTERRUPTIBLE check is a bad choice. Unless a sleep is neutral
(e.g. no lock is held, or obviously safe to sleep with that specific lock held),
sleeping for 140 seconds inside the kernel is a bad sign even if interruptible/killable.

> 
> Now if it's impossible to improve the pipe code so it works better not
> just for uffd, there's still no reason to worry: we could disable uffd
> in the pipe context. For example ptrace opts-out of uffds, so that gdb
> doesn't get stuck if you read a pointer that should be handled by the
> process that is under debug. I hope it won't be necessary but it
> wouldn't be a major issue, certainly it wouldn't risk breaking qemu
> (and non-cooperative APIs are privileged so it could still skip the
> timeout).

Can we do something like this?

  bool retried = false;
retry:
  lock();
  disable_fault();
  ret = access_memory_that_might_fault();
  enable_fault();
  if (ret == -EWOULDFAULT && !retried)
    goto retry_without_lock;
  if (ret == 0)
    ret = do_something();
  unlock();
  return ret;
retry_without_lock:
  unlock();
  ret = access_memory_that_might_fault();
  retried = true;
  goto retry;