linux-kernel - Re: VM/VFS bug with large amount of memory and file systems?

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-Id: <200709170828.01098.nickpiggin@yahoo.com.au>
Date:	Mon, 17 Sep 2007 08:28:00 +1000
From:	Nick Piggin <nickpiggin@...oo.com.au>
To:	Anton Altaparmakov <aia21@....ac.uk>,
	Andrew Morton <akpm@...ux-foundation.org>
Cc:	Andrew Morton <akpm@...ux-foundation.org>,
	Peter Zijlstra <peterz@...radead.org>,
	Linux Kernel Mailing List <linux-kernel@...r.kernel.org>,
	Linux Memory Management List <linux-mm@...ck.org>,
	marc.smith@...ail.mcc.edu
Subject: Re: VM/VFS bug with large amount of memory and file systems?

On Tuesday 18 September 2007 00:09, Anton Altaparmakov wrote:
> On 17 Sep 2007, at 15:04, Anton Altaparmakov wrote:
> > On 15 Sep 2007, at 11:52, Andrew Morton wrote:
> >> On Sat, 15 Sep 2007 12:08:17 +0200 Peter Zijlstra
> >>
> >> <peterz@...radead.org> wrote:
> >>> Anyway, looks like all of zone_normal is pinned in kernel
> >>>
> >>> allocations:
> >>>> Sep 13 15:31:25 escabot Normal free:3648kB min:3744kB low:4680kB
> >>>> high: 5616kB active:0kB inactive:3160kB present:894080kB
> >>>> pages_scanned:5336 all_unreclaimable? yes
> >>>
> >>> Out of the 870 odd mb only 3 is on the lru.
> >>>
> >>> Would be grand it you could have a look at slabinfo and the like.
> >>
> >> Definitely.
> >>
> >>>> Sep 13 15:31:25 escabot free:1090395 slab:198893 mapped:988
> >>>> pagetables:129 bounce:0
> >>
> >> 814,665,728 bytes of slab.
> >
> > Marc emailed me the contents of /proc/
> > {slabinfo,meminfo,vmstat,zoneinfo} taken just a few seconds before
> > the machine panic()ed due to running OOM completely...  They files
> > are attached this time rather than inlined so people don't complain
> > about line wrapping!  (No doubt people will not complain about them
> > being attached!  )-:)
> >
> > If I read it correctly it appears all of low memory is eaten up by
> > buffer_heads.
> >
> > <quote>
> > # name            <active_objs> <num_objs> <objsize> <objperslab>
> > <pagesperslab>
> >
> > : tunables <limit> <batchcount> <sharedfactor> : slabdata
> >
> > <active_slabs> <num_s
> > labs> <sharedavail>
> > buffer_head       12569528 12569535     56   67    1 : tunables
> > 120   60    8 :
> > slabdata 187605 187605      0
> > </quote>
> >
> > That is 671MiB of low memory in buffer_heads.
>
> I meant that is 732MiB of low memory in buffer_heads.  (12569535
> num_objs / 67 objperslab * 1 pagesperslab * 4096 PAGE_SIZE)

There is a sledgehammer in there which is supposed to alleviate
this problem. vmscan.c has buffer_heads_over_limit (could you check
if that is kicking in? (add a line in mm/page_alloc.c:show_free_areas()
to check it).

However, I'd guess the logic should be pretty robust. So it looks like
highmem is not getting scanned, so the buffer heads are not ever
getting a chance to be stripped off highmem pages!

(Rik has a patch sitting in -mm I believe which would make this problem
even worse, by doing even less highmem scanning in response to lowmem
allocations). However your user isn't using that kernel, so I wonder why it
isn't scanning highmem... it's been a while since I looked at reclaim, but
AFAIR it *should* be scanning it.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/