linux-kernel - Re: The VFS cache is not freed when there is not enough free memory to allocate

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <6d6a94c50611220202t1d076b4cye70dcdcc19f56e55@mail.gmail.com>
Date:	Wed, 22 Nov 2006 18:02:30 +0800
From:	Aubrey <aubreylee@...il.com>
To:	"Peter Zijlstra" <a.p.zijlstra@...llo.nl>
Cc:	linux-kernel@...r.kernel.org, linux-mm@...ck.org
Subject: Re: The VFS cache is not freed when there is not enough free memory to allocate

On 11/22/06, Peter Zijlstra <a.p.zijlstra@...llo.nl> wrote:
> Please see the
> threads on Mel Gorman's Anti-Fragmentation and Linear/Lumpy reclaim in
> the linux-mm archives.
>

Thanks to point this. Is it already included in Linus' git tree?

> > The patch drop the page cache and slab and then give a new chance to
> > get more free pages. Applied this patch, my test application can
> > allocate memory sucessfully and drop the cache and slab as well. See
> > below:
> > ================================
> > root:/mnt> ./t
> > Alloc 8 MB !
> > alloc successful
>
> Pure luck, there are workloads where there just would not have been any
> order 9 contiguous block freeable (think where each 9th order block
> would contain at least one active inode).
>
> > I know performance is important for linux, and VFS cache obviously
> > improve the performance when implement file operation. But for
> > embedded system, we'll try our best to make the application executable
> > rather than hanging system to guarantee the system performance.
> >
> > Any suggestions and solutions are really appreciated!
>
> Try Mel's patches and wait for the next Lumpy reclaim posting.
>
> The lack of a MMU on your system makes it very hard not to rely on
> higher order allocations, because even user-space allocs need to be
> physically contiguous. But please take that into consideration when
> writing software.

Well, the test application just use an exaggerated way to replicate the issue.

Actually, In the real work, the application such as mplayer, asterisk,
etc will run into
the above problem when run them at the second time. I think I have no
reason to modify those kind of applications.

My patch let kernel drop VFS cache in the low memory situation when
the application requests more memory allocation, I don't think it's
luck. You know, the application just wants to allocate 8
1Mbyte-blocks(order =9) and releasing VFS cache we can get almost
50Mbyte free memory.

The patch indeedly enabled many failed test cases on our side. But
yes, I don't think it's the final solution. I'll try Mel's patch and
update the results.

Thanks,
-Aubrey
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/