linux-ext4 - Re: large file system & high object count testing

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-id: <20090831205608.GE4197@webber.adilger.int>
Date:	Mon, 31 Aug 2009 14:56:08 -0600
From:	Andreas Dilger <adilger@....com>
To:	Ric Wheeler <rwheeler@...hat.com>
Cc:	linux-ext4@...r.kernel.org, "Ted Ts'o" <tytso@...nk.org>
Subject: Re: large file system & high object count testing

On Aug 31, 2009  13:02 -0400, Ric Wheeler wrote:
> One more note - this file system was filled using fs_mark, but without  
> doing any fsync() calls.
>
> umount:
>
> Aug 31 10:19:27 megadeth kernel: EXT4-fs: mballoc: 2580708130 blocks  
> 516141626 reqs (511081408 success)
> Aug 31 10:19:27 megadeth kernel: EXT4-fs: mballoc: 5060218 extents  
> scanned, 0 goal hits, 5060218 2^N hits, 0 breaks, 0 lost
> Aug 31 10:19:27 megadeth kernel: EXT4-fs: mballoc: 85164 generated and  
> it took 471527376
> Aug 31 10:19:27 megadeth kernel: EXT4-fs: mballoc: 2590831616  
> preallocated, 10120312 discarded
>
> Mount after fsck:
> Aug 31 12:27:12 megadeth kernel: EXT4-fs (dm-75):  
> ext4_check_descriptors: Checksum for group 487 failed (59799!=46827)
> Aug 31 12:27:12 megadeth kernel: EXT4-fs (dm-75): group descriptors  
> corrupted!
>
> The MBALLOC messages are a bit worrying - what exactly gets discarded  
> during an unmount?

The in-memory preallocation areas are discarded.  This is reporting
that of the 2590M preallocation areas it reserved, only 10M of them
were discarded during the lifetime of the filesystem.

Of the other stats:
- 471 seconds were spent in total generating the 85k buddy bitmaps
  (this is done incrementally at runtime)
- 516M calls to mballoc to find a chunk of blocks, 511M calls were able
  to find the requested chunk (not surprising given it is a new filesystem,
  probably the 5M calls that failed were when the fs was nearly full)

Cheers, Andreas
--
Andreas Dilger
Sr. Staff Engineer, Lustre Group
Sun Microsystems of Canada, Inc.

--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html