linux-kernel - Re: [PATCH] sched: Avoid that __wait_on_bit

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <ccb421f9-9d0a-6bb6-cd52-8ada311bbe9b@sandisk.com>
Date:	Fri, 12 Aug 2016 15:47:00 -0700
From:	Bart Van Assche <bart.vanassche@...disk.com>
To:	Oleg Nesterov <oleg@...hat.com>
CC:	Peter Zijlstra <peterz@...radead.org>,
	"mingo@...nel.org" <mingo@...nel.org>,
	Andrew Morton <akpm@...ux-foundation.org>,
	"Johannes Weiner" <hannes@...xchg.org>, Neil Brown <neilb@...e.de>,
	Michael Shaver <jmshaver@...il.com>,
	"linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>
Subject: Re: [PATCH] sched: Avoid that __wait_on_bit_lock() hangs

On 08/12/2016 09:16 AM, Oleg Nesterov wrote:
> Please drop two patches I sent before and try the new one below.

Hello Oleg,

Thanks for the patch. In addition to your patch I also applied the
attached two patches before I started testing. It took some time
before I could reproduce the hang in truncate_inode_pages_range().
To my surprise the following appeared in the system log instead of
a list of waiting tasks when I succeeded to reproduce this hang:

Aug 12 14:48:06 ion-dev-ib-ini systemd-udevd[500]: seq 11210 '/devices/virtual/block/dm-0' is taking a long time
Aug 12 14:48:07 ion-dev-ib-ini systemd-udevd[500]: seq 11227 '/devices/virtual/block/dm-1' is taking a long time
Aug 12 14:50:06 ion-dev-ib-ini systemd-udevd[500]: seq 11210 '/devices/virtual/block/dm-0' killed
Aug 12 14:50:06 ion-dev-ib-ini kernel: do_generic_file_read / pid 17232: killed
Aug 12 14:50:06 ion-dev-ib-ini systemd[1]: Started Cleanup of Temporary Directories.
Aug 12 14:50:36 ion-dev-ib-ini kernel: __lock_page_impl / pid 17224 / m 0x2: timeout - continuing to wait for 17224
Aug 12 14:50:36 ion-dev-ib-ini kernel: __lock_page_impl / pid 17232 / m 0x2: timeout - continuing to wait for 17232
Aug 12 14:51:06 ion-dev-ib-ini kernel: __lock_page_impl / pid 17224 / m 0x2: timeout - continuing to wait for 17224
Aug 12 14:51:06 ion-dev-ib-ini kernel: __lock_page_impl / pid 17232 / m 0x2: timeout - continuing to wait for 17232
[ ... ]

Running echo w > /proc/sysrq-trigger learned me that both pid 17224 and
17232 were hanging in truncate_inode_pages_range(). Does this mean that
some code in mm or in the filesystem I was using for this test (ext4) does
not unlock all pages it should unlock if a fatal signal is received?

Please let me know if you would like me to repost this message on an
mm-related mailing list.

Thanks,

Bart.

The echo w > /proc/sysrq-trigger output:

sysrq: SysRq : Show Blocked State
  task                        PC stack   pid father
systemd-udevd   D ffff88039870b7e8     0 17224    500 0x00000006
Call Trace:
 [<ffffffff816219f7>] schedule+0x37/0x90
 [<ffffffff81626019>] schedule_timeout+0x249/0x470
 [<ffffffff81620dcf>] io_schedule_timeout+0x9f/0x110
 [<ffffffff81622204>] bit_wait_io_timeout+0x24/0x70
 [<ffffffff81621f89>] __wait_on_bit_lock+0x49/0xa0
 [<ffffffff81152be5>] __lock_page_impl+0xe5/0x160
 [<ffffffff81152c6e>] __lock_page+0xe/0x10
 [<ffffffff811666a6>] truncate_inode_pages_range+0x416/0x7c0
 [<ffffffff81166a60>] truncate_inode_pages+0x10/0x20
 [<ffffffff81214200>] kill_bdev+0x30/0x40
 [<ffffffff81215521>] __blkdev_put+0x71/0x360
 [<ffffffff81215859>] blkdev_put+0x49/0x170
 [<ffffffff812159a0>] blkdev_close+0x20/0x30
 [<ffffffff811d6058>] __fput+0xe8/0x1f0
 [<ffffffff811d6199>] ____fput+0x9/0x10
 [<ffffffff81084453>] task_work_run+0x83/0xb0
 [<ffffffff810661ee>] do_exit+0x3ee/0xc40
 [<ffffffff81066acb>] do_group_exit+0x4b/0xc0
 [<ffffffff81073f1a>] get_signal+0x2ca/0x940
 [<ffffffff8101bf43>] do_signal+0x23/0x660
 [<ffffffff810022b3>] exit_to_usermode_loop+0x73/0xb0
 [<ffffffff81002cb0>] syscall_return_slowpath+0xb0/0xc0
 [<ffffffff816274b3>] entry_SYSCALL_64_fastpath+0xa6/0xa8
systemd-udevd   D ffff88006ce6f7e8     0 17232    500 0x00000006
Call Trace:
 [<ffffffff816219f7>] schedule+0x37/0x90
 [<ffffffff81626019>] schedule_timeout+0x249/0x470
 [<ffffffff81620dcf>] io_schedule_timeout+0x9f/0x110
 [<ffffffff81622204>] bit_wait_io_timeout+0x24/0x70
 [<ffffffff81621f89>] __wait_on_bit_lock+0x49/0xa0
 [<ffffffff81152be5>] __lock_page_impl+0xe5/0x160
 [<ffffffff81152c6e>] __lock_page+0xe/0x10
 [<ffffffff811666a6>] truncate_inode_pages_range+0x416/0x7c0
 [<ffffffff81166a60>] truncate_inode_pages+0x10/0x20
 [<ffffffff81214200>] kill_bdev+0x30/0x40
 [<ffffffff81215521>] __blkdev_put+0x71/0x360
 [<ffffffff81215859>] blkdev_put+0x49/0x170
 [<ffffffff812159a0>] blkdev_close+0x20/0x30
 [<ffffffff811d6058>] __fput+0xe8/0x1f0
 [<ffffffff811d6199>] ____fput+0x9/0x10
 [<ffffffff81084453>] task_work_run+0x83/0xb0
 [<ffffffff810661ee>] do_exit+0x3ee/0xc40
 [<ffffffff81066acb>] do_group_exit+0x4b/0xc0
 [<ffffffff81073f1a>] get_signal+0x2ca/0x940
 [<ffffffff8101bf43>] do_signal+0x23/0x660
 [<ffffffff810022b3>] exit_to_usermode_loop+0x73/0xb0
 [<ffffffff81002cb0>] syscall_return_slowpath+0xb0/0xc0
 [<ffffffff816274b3>] entry_SYSCALL_64_fastpath+0xa6/0xa8


View attachment "0001-mm-__lock_page-dbg.patch" of type "text/x-patch" (7101 bytes)

View attachment "0001-do_generic_file_read-Fail-immediately-if-killed.patch" of type "text/x-patch" (883 bytes)