linux-ext4 - Re: Large buffer cache in EXT4

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <51224751.3020002@redhat.com>
Date:	Mon, 18 Feb 2013 09:22:57 -0600
From:	Eric Sandeen <sandeen@...hat.com>
To:	Andreas Dilger <adilger@...ger.ca>
CC:	Subranshu Patel <spatel.ml@...il.com>,
	"linux-ext4@...r.kernel.org" <linux-ext4@...r.kernel.org>
Subject: Re: Large buffer cache in EXT4

On 2/17/13 12:28 AM, Andreas Dilger wrote:
> On 2013-02-16, at 21:04, Subranshu Patel <spatel.ml@...il.com> wrote:
> 
>> I created 2 filesystem on my system (RHEL 6.3, kernel version 2.6.32)
>> - XFS and EXT4 and mounted them.
>>
>> On both the filesystem I executed the mdtest tool(opensource tool) for
>> 64 concurrent process. Each process performed the following:
>> - Create large number of directories
>> - Remove all the directories
>>
>> During this time I monitored the memory usage of the system using sar
>> command. I checked the 3 components - kbmemused, kbbuffers and
>> kbcached
>>
>> kbmemused - Amount of used memory in kilobytes. This does not take
>> into account memory used by the kernel itself.
>> kbbuffers - buffer cache
>> kbcached - page cache
>>
>> While the kbmemused and kbcached component was almost similar in EXT4
>> and XFS (XFS being a little higher), the kbbuffer showed a totally
>> different trend.
>>
>> For EXT4, kbbuffers was:
>> 390999KB for dir creation
>> 364803KB for dir removal
>> For XFS, kbbuffers was:
>>
>> 1701KB for dir creation
>> 2738KB for dir removal
>>
>> In kernel 2.6, both buffer cache and page cache are merged. The page
>> cache caches pages of files. The buffer cache caches disk blocks which
>> consists of mainly metadata (not file data).
>>
>> Why is the buffer cache large in case of EXT4 and what is stored in
>> the buffer cache?
> 
> XFS does not use buffer cache, while ext[234] does use buffer cache. 
> 
> This is just a different code design. Ext4 uses the buffer cache to track metadata for journaling. 

Use slabtop or similar to see what xfs slab caches grow during the test; look at xfs_buf for example.

-Eric
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html