linux-ext4 - Re: ext3 leaking buffer

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:	Tue, 24 Mar 2009 17:43:10 -0500
From:	Don Porter <porterde@...utexas.edu>
To:	Eric Sandeen <sandeen@...hat.com>
CC:	linux-ext4@...r.kernel.org
Subject: Re: ext3 leaking buffer_heads

Thanks for the tip, Eric.

I spent some time tracing through this in a debugger.

I believe the lingering buffer_heads are allocated by 
journal_write_commit_record()->journal_get_descriptor_buffer()->__getblk().  
The reference count ends up at zero, but the buffer head is never freed.

As best I can tell, this is correct as long as the associated page is 
cached, so that the associated buffer_head can be looked up and reused 
later.

I plan to look into this more, but perhaps the issue is just that the 
kernel should be more aggressive about freeing cached journal pages?  
They are highly unlikely to be used again once a committed transaction 
is completely out on disk, as the journal is roughly an append-only log, 
right?

Thanks again,
Don

Eric Sandeen wrote:
> Don Porter wrote:
>   
>> Hi,
>>
>> It appears that the ext3 journal code has a slow leak of buffer_head 
>> structs.  Try this simple script:
>>
>> perl -e 'while(1){ `sync`; }'
>>
>> and monitor the count of allocated buffer_head structs in 
>> /proc/slabinfo, and it seems to increase without bound.  Even after this 
>> script is killed and the machine is left idle for several minutes, the 
>> count of buffer heads doesn't substantially decrease.
>>
>> Looking around at various machines I have access to, the count of 
>> allocated buffer_heads roughly correlates with uptime when using ext3.  
>> This is a slow leak - one would likely have to run this script for a day 
>> or more to drain enough lowmem to cause problems.
>>
>> Other info:  I have only tried this on x86 machines, but I have tried 
>> both 2.6.22.6 and 2.6.28.8, and both have the problem.  I am running 
>> Ubuntu 7.10 on top of these kernels, but the kernels were built directly 
>> from kernel.org tarballs.
>>
>> Any advice or help with this issue is greatly appreciated. 
>>     
>
> Without investigating too far yet, I did try this, and did indeed see
> the buffer_head usage go up while the script runs.
>
> However, if I did:
>
> # echo 3 > /proc/sys/vm/drop_caches
>
> before the script, and noted the total nr. of buffer heads in use, and
> then did it again after the script had been running a while, I got back
> to the same (low) count of buffer heads in use.   So I don't think this
> is a leak as in "the system has lost all accounting of these buffer
> heads" at least... but it'd be interesting to know what the reason for
> the increase is, I'm not sure offhand.
>
> -Eric
>
>
>   

--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html