linux-ext4 - ext4: first write to large ext3 filesystem takes 96 seconds

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Date:	Mon, 7 Jul 2014 17:13:49 -0400
From:	Benjamin LaHaise <bcrl@...ck.org>
To:	linux-ext4@...r.kernel.org
Subject: ext4: first write to large ext3 filesystem takes 96 seconds

Hi folks,

I've just ran into a bug with the ext4 codebase in 3.4.91 that doesn't seem 
to exist in ext3, and was wondering if anyone has encountered this before.  
I have a 7.4TB ext3 filesystem that has been filled with 1.8TB of data.  
When this filesystem is freshly mounted, the first write to the filesystem 
takes a whopping 96 seconds to complete, during which time the system is 
reading about 1000 blocks per second.  Subsequent writes are much quicker.  
The problem seems to be that ext4 is loading all of the bitmaps on the 
filesystem before the first write proceeds.  The backtrace looks roughly as 
follows:

[ 4480.921288]  [<ffffffff81437472>] ? dm_request_fn+0x112/0x1c0
[ 4480.921292]  [<ffffffff81244bf5>] ? __blk_run_queue+0x15/0x20
[ 4480.921294]  [<ffffffff81246670>] ? queue_unplugged+0x20/0x50
[ 4480.921297]  [<ffffffff8154ed05>] schedule+0x45/0x60
[ 4480.921299]  [<ffffffff8154ef7c>] io_schedule+0x6c/0xb0
[ 4480.921301]  [<ffffffff810fb7a9>] sleep_on_buffer+0x9/0x10
[ 4480.921303]  [<ffffffff8154d435>] __wait_on_bit+0x55/0x80
[ 4480.921306]  [<ffffffff810fb7a0>] ? unmap_underlying_metadata+0x40/0x40
[ 4480.921308]  [<ffffffff810fb7a0>] ? unmap_underlying_metadata+0x40/0x40
[ 4480.921310]  [<ffffffff8154d4d8>] out_of_line_wait_on_bit+0x78/0x90
[ 4480.921312]  [<ffffffff8104d5b0>] ? autoremove_wake_function+0x40/0x40
[ 4480.921315]  [<ffffffff810fb756>] __wait_on_buffer+0x26/0x30
[ 4480.921318]  [<ffffffff81146258>] ext4_wait_block_bitmap+0x138/0x190
[ 4480.921321]  [<ffffffff8116c816>] ext4_mb_init_cache+0x1e6/0x5f0
[ 4480.921324]  [<ffffffff8109654a>] ? add_to_page_cache_locked+0x9a/0xd0
[ 4480.921327]  [<ffffffff810965b1>] ? add_to_page_cache_lru+0x31/0x50
[ 4480.921330]  [<ffffffff8116cd1f>] ext4_mb_init_group+0xff/0x1e0
[ 4480.921332]  [<ffffffff8116ce9f>] ext4_mb_good_group+0x9f/0x130
[ 4480.921334]  [<ffffffff8116e41f>] ext4_mb_regular_allocator+0x1bf/0x3d0
[ 4480.921337]  [<ffffffff8116c0ac>] ? ext4_mb_normalize_request+0x26c/0x4d0
[ 4480.921339]  [<ffffffff811716ce>] ext4_mb_new_blocks+0x2ee/0x490
[ 4480.921342]  [<ffffffff81174c11>] ? ext4_get_branch+0x101/0x130
[ 4480.921345]  [<ffffffff8117639c>] ext4_ind_map_blocks+0x9bc/0xc10
[ 4480.921347]  [<ffffffff810fae11>] ? __getblk+0x21/0x2b0
[ 4480.921350]  [<ffffffff8114c9a3>] ext4_map_blocks+0x293/0x390
[ 4480.921353]  [<ffffffff8117a462>] ? do_get_write_access+0x1d2/0x450
[ 4480.921355]  [<ffffffff810cb0b4>] ? kmem_cache_alloc+0xa4/0xc0
[ 4480.921358]  [<ffffffff8114d9e9>] _ext4_get_block+0xa9/0x140
[ 4480.921360]  [<ffffffff8114dab1>] ext4_get_block+0x11/0x20
[ 4480.921362]  [<ffffffff810fbda5>] __block_write_begin+0x2b5/0x470
[ 4480.921365]  [<ffffffff8114daa0>] ? noalloc_get_block_write+0x20/0x20
[ 4480.921368]  [<ffffffff81096679>] ? grab_cache_page_write_begin+0xa9/0x100
[ 4480.921370]  [<ffffffff8114c1e2>] ext4_write_begin+0x132/0x2f0
[ 4480.921373]  [<ffffffff81095869>] generic_file_buffered_write+0x119/0x260
[ 4480.921376]  [<ffffffff81096eef>] __generic_file_aio_write+0x27f/0x430
[ 4480.921379]  [<ffffffff810cfbba>] ? do_huge_pmd_anonymous_page+0x1ea/0x2d0
[ 4480.921382]  [<ffffffff81097101>] generic_file_aio_write+0x61/0xc0
[ 4480.921384]  [<ffffffff81147c18>] ext4_file_write+0x68/0x2a0
[ 4480.921387]  [<ffffffff8154e723>] ? __schedule+0x2c3/0x800
[ 4480.921389]  [<ffffffff810d2a41>] do_sync_write+0xe1/0x120
[ 4480.921392]  [<ffffffff8154eefa>] ? _cond_resched+0x2a/0x40
[ 4480.921395]  [<ffffffff810d31c9>] vfs_write+0xc9/0x170
[ 4480.921397]  [<ffffffff810d3910>] sys_write+0x50/0x90
[ 4480.921400]  [<ffffffff8155155f>] sysenter_dispatch+0x7/0x1a

Any thoughts?  Have there been any changes to this area of the ext4 code?

		-ben
-- 
"Thought is the essence of where you are now."
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html