| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Message-ID: <622177618.727.1393062606061.JavaMail.zimbra@rapitasystems.com>
Date: Sat, 22 Feb 2014 09:50:06 +0000 (GMT)
From: Matthew Rahtz <mrahtz@...itasystems.com>
To: Jan Kara <jack@...e.cz>
Cc: linux-ext4@...r.kernel.org
Subject: Re: warning in ext4_journal_start_sb on filesystem freeze
Thanks for your help Jan,
A few months later, we've noticed the issue is actually still there. Using 3.11.0-17-generic on Ubuntu 12.04, we’re seeing this in the kernel logs:
[29243.606215] WARNING: CPU: 0 PID: 1785 at /build/buildd/linux-lts-saucy-3.11.0/fs/ext4/ext4_jbd2.c:48 ext4_journal_check_start+0x83/0x90()
Having a look at the Ubuntu source package for that version, it definitely does include commit 03d95eb2f2578083a3f6286262e1cb5d88a00c02, and the line generating the warning is still:
WARN_ON(sb->s_writers.frozen == SB_FREEZE_COMPLETE);
Are there any other obvious possibilities for what may be causing this? There seem to be some users of Oracle Linux experiencing similar problems at https://community.oracle.com/thread/2617418, which was apparently fixed in Oracle's kernel version '3.8.13-26.el6uek'. Any word on when this might be integrated into the official kernel?
Full call trace included below.
Thanks again!
Matthew
[29243.606212] ------------[ cut here ]------------
[29243.606215] WARNING: CPU: 0 PID: 1785 at /build/buildd/linux-lts-saucy-3.11.0/fs/ext4/ext4_jbd2.c:48 ext4_journal_check_start+0x83/0x90()
[29243.606216] Modules linked in: parport_pc ppdev nfsd nfs_acl auth_rpcgss nfs fscache lockd sunrpc ext2 cirrus ttm drm_kms_helper drm sysimgblt psmouse i2c_piix4 virtio_balloon sysfillrect mac_hid serio_raw syscopyarea virtio_console lp parport floppy
[29243.606227] CPU: 0 PID: 1785 Comm: nfsd Tainted: G W 3.11.0-17-generic #31~precise1-Ubuntu
[29243.606228] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2007
[29243.606228] 0000000000000030 ffff8801162f3b08 ffffffff8173c72d 0000000000000007
[29243.606230] 0000000000000000 ffff8801162f3b48 ffffffff8106540c 0000000000000000
[29243.606232] ffff880114892800 0000000000000007 0000000000000068 0000000000000000
[29243.606235] Call Trace:
[29243.606237] [<ffffffff8173c72d>] dump_stack+0x46/0x58
[29243.606239] [<ffffffff8106540c>] warn_slowpath_common+0x8c/0xc0
[29243.606241] [<ffffffff8106545a>] warn_slowpath_null+0x1a/0x20
[29243.606244] [<ffffffff8127ebb3>] ext4_journal_check_start+0x83/0x90
[29243.606246] [<ffffffff8127ec35>] __ext4_journal_start_sb+0x45/0x100
[29243.606249] [<ffffffff81258a03>] ? ext4_dirty_inode+0x33/0x70
[29243.606251] [<ffffffff81258a03>] ext4_dirty_inode+0x33/0x70
[29243.606254] [<ffffffff811de348>] __mark_inode_dirty+0x48/0x350
[29243.606256] [<ffffffff81256b53>] ext4_setattr+0x1b3/0x5b0
[29243.606259] [<ffffffff811d0903>] notify_change+0x1d3/0x390
[29243.606263] [<ffffffffa01c7fe2>] nfsd_setattr+0x232/0x2a0 [nfsd]
[29243.606267] [<ffffffffa01d00f6>] nfsd3_proc_setattr+0x76/0xc0 [nfsd]
[29243.606271] [<ffffffffa01c0d85>] nfsd_dispatch+0xe5/0x230 [nfsd]
[29243.606283] [<ffffffffa0128465>] svc_process_common+0x345/0x680 [sunrpc]
[29243.606289] [<ffffffffa0128af3>] svc_process+0x103/0x160 [sunrpc]
[29243.606293] [<ffffffffa01c08df>] nfsd+0xbf/0x130 [nfsd]
[29243.606297] [<ffffffffa01c0820>] ? nfsd_destroy+0x80/0x80 [nfsd]
[29243.606299] [<ffffffff81089170>] kthread+0xc0/0xd0
[29243.606302] [<ffffffff810890b0>] ? flush_kthread_worker+0xb0/0xb0
[29243.606304] [<ffffffff8175122c>] ret_from_fork+0x7c/0xb0
[29243.606307] [<ffffffff810890b0>] ? flush_kthread_worker+0xb0/0xb0
[29243.606308] ---[ end trace e9d4726f92c62d43 ]---
----- Original Message -----
From: "Jan Kara" <jack@...e.cz>
To: "Matthew Rahtz" <mrahtz@...itasystems.com>
Cc: linux-ext4@...r.kernel.org
Sent: Tuesday, 26 November, 2013 12:58:26 PM
Subject: Re: warning in ext4_journal_start_sb on filesystem freeze
Hello,
On Tue 26-11-13 08:20:51, Matthew Rahtz wrote:
> We're using qemu's guest agent daemon, qemu-ga, to freeze ext4
> filesystems in guest virtual machines before taking an LVM snapshot of
> the disk volume in the host. However, in the guests' dmesg, we're
> consistently seeing warnings like:
>
> [1246478.632936] WARNING: at /build/buildd/linux-lts-raring-3.8.0/fs/ext4/super.c:339 ext4_journal_start_sb+0x159/0x160()
>
> Looking at the source at
> https://github.com/torvalds/linux/blob/v3.8/fs/ext4/super.c#L339, this
> warning seems to be generated if the function is reached despite the
> filesystem being marked as frozen:
>
> WARN_ON(sb->s_writers.frozen == SB_FREEZE_COMPLETE);
>
> In 3.12, this has been moved to
> https://github.com/torvalds/linux/blob/v3.12/fs/ext4/ext4_jbd2.c#L48.
>
> Is this something we should be concerned about? The process that seems to
> be responsible for triggering it is mysqld, so we're concerned the
> databases in our snapshots have a higher possibility of being corrupt.
> (Taking online snapshots of databases like this is always risky, of
> course, but this just makes us a little more nervous :) ) Full kernel
> warning is attached below.
Yes, it's a bug in 3.8 kernel which got fixed by commit
03d95eb2f2578083a3f6286262e1cb5d88a00c02 (merged in 3.10). Looking into the
code there's really a chance the filesystem will be inconsistent because of
that bug so you might be better off updating to a kernel which has this bug
fixed if you rely on the snapshots heavily.
Honza
> [1246478.632930] ------------[ cut here ]------------
> [1246478.632936] WARNING: at /build/buildd/linux-lts-raring-3.8.0/fs/ext4/super.c:339 ext4_journal_start_sb+0x159/0x160()
> [1246478.632938] Hardware name: Bochs
> [1246478.632939] Modules linked in: cirrus(F) ttm(F) drm_kms_helper(F) drm(F) sysimgblt(F) psmouse(F) sysfillrect(F) serio_raw(F) syscopyarea(F) microcode(F) virtio_console(F) lp(F) virtio_balloon(F) mac_hid(F) i2c_piix4(F) ext2(F) parport(F) floppy(F) e1000(F)
> [1246478.632973] Pid: 2856, comm: mysqld Tainted: GF W 3.8.0-33-generic #48~precise1-Ubuntu
> [1246478.632975] Call Trace:
> [1246478.632981] [<ffffffff81059b6f>] warn_slowpath_common+0x7f/0xc0
> [1246478.632985] [<ffffffff81059bca>] warn_slowpath_null+0x1a/0x20
> [1246478.632989] [<ffffffff8125eb59>] ext4_journal_start_sb+0x159/0x160
> [1246478.632993] [<ffffffff8123f1c8>] ? _ext4_get_block+0x138/0x170
> [1246478.632997] [<ffffffff8123f1c8>] _ext4_get_block+0x138/0x170
> [1246478.633002] [<ffffffff8104e070>] ? get_user_pages_fast+0xe0/0x1a0
> [1246478.633006] [<ffffffff8123f263>] ext4_get_block_write+0x13/0x20
> [1246478.633009] [<ffffffff811d6d3a>] get_more_blocks+0x6a/0xa0
> [1246478.633013] [<ffffffff811d7a7e>] do_direct_IO+0x4be/0x1530
> [1246478.633018] [<ffffffff8107f9ab>] ? bit_waitqueue+0x1b/0xc0
> [1246478.633022] [<ffffffff81186221>] ? kmem_cache_alloc+0x31/0x140
> [1246478.633026] [<ffffffff811d8f22>] do_blockdev_direct_IO+0x432/0x13e0
> [1246478.633030] [<ffffffff8123f250>] ? noalloc_get_block_write+0x30/0x30
> [1246478.633035] [<ffffffff811d9f25>] __blockdev_direct_IO+0x55/0x60
> [1246478.633039] [<ffffffff8123f250>] ? noalloc_get_block_write+0x30/0x30
> [1246478.633042] [<ffffffff8123ab30>] ? ext4_journalled_invalidatepage+0x30/0x30
> [1246478.633046] [<ffffffff8123bcd0>] ext4_ext_direct_IO+0x130/0x250
> [1246478.633050] [<ffffffff8123f250>] ? noalloc_get_block_write+0x30/0x30
> [1246478.633053] [<ffffffff8123ab30>] ? ext4_journalled_invalidatepage+0x30/0x30
> [1246478.633057] [<ffffffff8123c1ad>] ext4_direct_IO+0x1ad/0x230
> [1246478.633061] [<ffffffff8108e3ca>] ? finish_task_switch+0x4a/0xf0
> [1246478.633065] [<ffffffff811368d6>] generic_file_direct_write+0xc6/0x180
> [1246478.633068] [<ffffffff81136c6d>] __generic_file_aio_write+0x2dd/0x3b0
> [1246478.633072] [<ffffffff816e5848>] ext4_file_dio_write+0x243/0x320
> [1246478.633076] [<ffffffff810b81b2>] ? unqueue_me+0x52/0x80
> [1246478.633079] [<ffffffff81236ed8>] ext4_file_write+0xc8/0xe0
> [1246478.633084] [<ffffffff8119b333>] do_sync_write+0xa3/0xe0
> [1246478.633089] [<ffffffff8119b9d3>] vfs_write+0xb3/0x180
> [1246478.633093] [<ffffffff8119be9a>] sys_pwrite64+0x9a/0xa0
> [1246478.633097] [<ffffffff816fd15d>] system_call_fastpath+0x1a/0x1f
> [1246478.633099] ---[ end trace f37019187d44de90 ]---
> Please Note: Rapita Systems has a new address and telephone number.
> Telephone: +44 1904 413945
> Address: Rapita Systems Ltd, Atlas House,
> Osbaldwick Link Road, YORK, YO10 3JB
> United Kingdom
> --
> To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
> the body of a message to majordomo@...r.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
--
Jan Kara <jack@...e.cz>
SUSE Labs, CR
Please Note: Rapita Systems has a new address and telephone number.
Telephone: +44 1904 413945
Address: Rapita Systems Ltd, Atlas House,
Osbaldwick Link Road, YORK, YO10 3JB
United Kingdom
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Powered by blists - more mailing lists