| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Message-ID: <CACT4Y+Y=stiPSxgT0HrT-xc4+79130GwL1PxkqiWx0vO4WTLsQ@mail.gmail.com>
Date: Tue, 15 Mar 2016 14:10:11 +0100
From: Dmitry Vyukov <dvyukov@...gle.com>
To: Doug Gilbert <dgilbert@...erlog.com>,
"Martin K. Petersen" <martin.petersen@...cle.com>,
linux-scsi <linux-scsi@...r.kernel.org>,
LKML <linux-kernel@...r.kernel.org>
Cc: syzkaller <syzkaller@...glegroups.com>,
Kostya Serebryany <kcc@...gle.com>,
Alexander Potapenko <glider@...gle.com>,
Sasha Levin <sasha.levin@...cle.com>
Subject: scsi: machine hang due to write to /dev/sg0
Hello,
I am not sure whether there is some to fix or not, since it can be
triggered only by root. But still if I run the following program
several times in a row, whole machine becomes unusable for several
minutes (ssh, ps and pretty much everything hangs):
// autogenerated by syzkaller (http://github.com/google/syzkaller)
#include <pthread.h>
#include <stdint.h>
#include <string.h>
#include <sys/syscall.h>
#include <unistd.h>
int main()
{
syscall(SYS_mmap, 0x20000000ul, 0x25000ul, 0x3ul, 0x32ul, -1, 0x0ul);
int fd = syscall(SYS_open, "/dev/sg0", 0x20202ul);
*(uint32_t*)0x2001f639 = (uint32_t)0x50;
*(uint32_t*)0x2001f63d = (uint32_t)0x0;
*(uint64_t*)0x2001f641 = (uint64_t)0xffff;
*(uint32_t*)0x2001f649 = (uint32_t)0x8688;
*(uint32_t*)0x2001f64d = (uint32_t)0x8;
*(uint32_t*)0x2001f651 = (uint32_t)0x2;
*(uint32_t*)0x2001f655 = (uint32_t)0x8;
*(uint16_t*)0x2001f659 = (uint16_t)0xfffffffffffff801;
*(uint16_t*)0x2001f65b = (uint16_t)0x1f;
*(uint32_t*)0x2001f65d = (uint32_t)0x8;
*(uint32_t*)0x2001f661 = (uint32_t)0x7fbe57c5;
*(uint32_t*)0x2001f665 = (uint32_t)0x0;
*(uint32_t*)0x2001f669 = (uint32_t)0x0;
*(uint32_t*)0x2001f66d = (uint32_t)0x0;
*(uint32_t*)0x2001f671 = (uint32_t)0x0;
*(uint32_t*)0x2001f675 = (uint32_t)0x0;
*(uint32_t*)0x2001f679 = (uint32_t)0x0;
*(uint32_t*)0x2001f67d = (uint32_t)0x0;
*(uint32_t*)0x2001f681 = (uint32_t)0x0;
*(uint32_t*)0x2001f685 = (uint32_t)0x0;
syscall(SYS_write, fd, 0x2001f639ul, 0x50ul, 0, 0, 0);
return 0;
}
Sysrq d shows the following during the hang:
[ 131.732528] Showing all locks held in the system:
[ 131.732995] 3 locks held by kworker/u9:2/151:
[ 131.733321] #0: ("writeback"){.+.+.+}, at: [<ffffffff813afd89>]
process_one_work+0x699/0x15c0
[ 131.734064] #1: ((&(&wb->dwork)->work)){+.+.+.}, at:
[<ffffffff813afdb9>] process_one_work+0x6c9/0x15c0
[ 131.734834] #2: (&type->s_umount_key#29){.+.+..}, at:
[<ffffffff817d7c00>] trylock_super+0x20/0x100
[ 131.735638] 4 locks held by rs:main Q:Reg/6703:
[ 131.736051] #0: (&f->f_pos_lock){+.+.+.}, at:
[<ffffffff8182ee46>] __fdget_pos+0xd6/0x100
[ 131.736707] #1: (sb_writers#3){.+.+.+}, at: [<ffffffff817d6c3c>]
__sb_start_write+0xec/0x130
[ 131.737760] #2: (&sb->s_type->i_mutex_key#12){+.+.+.}, at:
[<ffffffff81a5b40a>] ext4_file_write_iter+0x15a/0xc80
[ 131.738543] #3: (jbd2_handle){+.+...}, at: [<ffffffff81bbb6c9>]
start_this_handle+0x5f9/0x1420
[ 131.754552] 3 locks held by bash/6809:
[ 131.754804] #0: (sb_writers#6){.+.+.+}, at: [<ffffffff817d6c3c>]
__sb_start_write+0xec/0x130
[ 131.755504] #1: (rcu_read_lock){......}, at: [<ffffffff82fe63f0>]
__handle_sysrq+0x0/0x4d0
[ 131.756528] #2: (tasklist_lock){.+.+..}, at: [<ffffffff8145c350>]
debug_show_all_locks+0x70/0x280
[ 131.757329] 3 locks held by sshd/6823:
[ 131.757603] #0: (sb_writers#3){.+.+.+}, at: [<ffffffff817d6c3c>]
__sb_start_write+0xec/0x130
[ 131.758269] #1: (&sb->s_type->i_mutex_key#12){+.+.+.}, at:
[<ffffffff81a5b40a>] ext4_file_write_iter+0x15a/0xc80
[ 131.759149] #2: (jbd2_handle){+.+...}, at: [<ffffffff81bbb6c9>]
start_this_handle+0x5f9/0x1420
The problem seems to be with task 151 which holds some critical mutex
and then blocks in bt_get for minutes:
root@...ukov-z840:~# cat /proc/151/stack
[<ffffffff82bc5f92>] bt_get+0x2d2/0x700
[<ffffffff82bc6cab>] blk_mq_get_tag+0x11b/0x360
[<ffffffff82bb5534>] __blk_mq_alloc_request+0x24/0xa20
[<ffffffff82bbd426>] blk_mq_map_request+0x786/0xc10
[<ffffffff82bc116e>] blk_sq_make_request+0x18e/0xdb0
[<ffffffff82b92fa0>] generic_make_request+0x310/0x800
[<ffffffff82b935ee>] submit_bio+0x15e/0x4c0
[<ffffffff81885100>] submit_bh_wbc+0x400/0x560
[<ffffffff8188a9a6>] __block_write_full_page.constprop.52+0x386/0x8c0
[<ffffffff8188b0d9>] block_write_full_page+0x1f9/0x2a0
[<ffffffff8188d524>] blkdev_writepage+0x24/0x30
[<ffffffff8167d091>] __writepage+0x61/0xc0
[<ffffffff8167ff9a>] write_cache_pages+0x4ca/0xf20
[<ffffffff81680abc>] generic_writepages+0xcc/0x120
[<ffffffff8188d48d>] blkdev_writepages+0x7d/0xd0
[<ffffffff81688f47>] do_writepages+0x97/0x100
[<ffffffff818655e8>] __writeback_single_inode+0xe8/0x1190
[<ffffffff81867a7d>] writeback_sb_inodes+0x44d/0x11f0
[<ffffffff81868924>] __writeback_inodes_wb+0x104/0x1e0
[<ffffffff8186932e>] wb_writeback+0x7ce/0xc90
[<ffffffff8186b2ee>] wb_workfn+0xa0e/0x1000
[<ffffffff813afe7e>] process_one_work+0x78e/0x15c0
[<ffffffff813b0d8b>] worker_thread+0xdb/0xfc0
[<ffffffff813c195f>] kthread+0x23f/0x2d0
[<ffffffff866d1b2f>] ret_from_fork+0x3f/0x70
[<ffffffffffffffff>] 0xffffffffffffffff
root@...ukov-z840:~# cat /proc/6823/stack
[<ffffffff81bc0f30>] do_get_write_access+0x6f0/0x10e0
[<ffffffff81bc196d>] jbd2_journal_get_write_access+0x4d/0xa0
[<ffffffff81b35821>] __ext4_journal_get_write_access+0x51/0x90
[<ffffffff81a7f3d3>] ext4_reserve_inode_write+0xe3/0x180
[<ffffffff81a7f5a3>] ext4_mark_inode_dirty+0x133/0xb20
[<ffffffff81a8cb11>] ext4_dirty_inode+0x71/0xa0
[<ffffffff81864915>] __mark_inode_dirty+0x575/0x1160
[<ffffffff8182442a>] generic_update_time+0x1aa/0x270
[<ffffffff8182482c>] file_update_time+0x25c/0x3e0
[<ffffffff81660e11>] __generic_file_write_iter+0x1d1/0x5a0
[<ffffffff81a5b597>] ext4_file_write_iter+0x2e7/0xc80
[<ffffffff817ccd22>] __vfs_write+0x302/0x4b0
[<ffffffff817ce607>] vfs_write+0x167/0x4a0
[<ffffffff817d18f1>] SyS_write+0x111/0x220
[<ffffffff866d1776>] entry_SYSCALL_64_fastpath+0x16/0x7a
[<ffffffffffffffff>] 0xffffffffffffffff
root@...ukov-z840:~# cat /proc/6703/stack
[<ffffffff81bc0f30>] do_get_write_access+0x6f0/0x10e0
[<ffffffff81bc196d>] jbd2_journal_get_write_access+0x4d/0xa0
[<ffffffff81b35821>] __ext4_journal_get_write_access+0x51/0x90
[<ffffffff81a7f3d3>] ext4_reserve_inode_write+0xe3/0x180
[<ffffffff81a7f5a3>] ext4_mark_inode_dirty+0x133/0xb20
[<ffffffff81a8cb11>] ext4_dirty_inode+0x71/0xa0
[<ffffffff81864915>] __mark_inode_dirty+0x575/0x1160
[<ffffffff8182442a>] generic_update_time+0x1aa/0x270
[<ffffffff8182482c>] file_update_time+0x25c/0x3e0
[<ffffffff81660e11>] __generic_file_write_iter+0x1d1/0x5a0
[<ffffffff81a5b597>] ext4_file_write_iter+0x2e7/0xc80
[<ffffffff817ccd22>] __vfs_write+0x302/0x4b0
[<ffffffff817ce607>] vfs_write+0x167/0x4a0
[<ffffffff817d18f1>] SyS_write+0x111/0x220
[<ffffffff866d1776>] entry_SYSCALL_64_fastpath+0x16/0x7a
[<ffffffffffffffff>] 0xffffffffffffffff
Is this hang in bt_get legitimate? If yes, is it possible to block in
bt_get without holds the other mutexes?
This happens in qemu and I don't have any actual scsi devices afaict.
I am on commit 8e0f93cda48ed054e1216bab5c60017e1a5fc1e8.
Powered by blists - more mailing lists