linux-kernel - Re: [PATCH 7/7] module: add debug stats to help identify memory pressure

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <ZCPU5T1PD+BwI9ri@bombadil.infradead.org>
Date:   Tue, 28 Mar 2023 23:04:21 -0700
From:   Luis Chamberlain <mcgrof@...nel.org>
To:     Greg KH <gregkh@...uxfoundation.org>
Cc:     david@...hat.com, patches@...ts.linux.dev,
        linux-modules@...r.kernel.org, linux-mm@...ck.org,
        linux-kernel@...r.kernel.org, pmladek@...e.com,
        petr.pavlu@...e.com, prarit@...hat.com,
        torvalds@...ux-foundation.org, rafael@...nel.org,
        christophe.leroy@...roup.eu, tglx@...utronix.de,
        peterz@...radead.org, song@...nel.org, rppt@...nel.org,
        willy@...radead.org, vbabka@...e.cz, mhocko@...e.com,
        dave.hansen@...ux.intel.com
Subject: Re: [PATCH 7/7] module: add debug stats to help identify memory
 pressure

On Wed, Mar 29, 2023 at 07:46:22AM +0200, Greg KH wrote:
> On Tue, Mar 28, 2023 at 10:31:49PM -0700, Luis Chamberlain wrote:
> > Loading modules with finit_module() can end up using vmalloc(), vmap()
> > and vmalloc() again, for a total of up to 3 separate allocations in the
> > worse case for a single module. We always kernel_read*() the module,
> > that's a vmalloc(). Then vmap() is used for the module decompression,
> > and if so the last read buffer is freed as we use the now decompressed
> > module buffer to stuff data into our copy module. The last one is
> > specific to architectures but pretty much that's generally a series
> > of vmalloc() for different ELF sections...
> > 
> > Evaluation with new stress-ng module support [1] with just 100 ops
> > us proving that you can end up using GiBs of data easily even if
> > we are trying to be very careful not to load modules which are already
> > loaded. 100 ops seems to resemble the sort of pressure a system with
> > about 400 CPUs can create on modules. Although those issues for so
> > many concurrent loads per CPU is silly and are being fixed, we lack
> > proper tooling to help diagnose easily what happened, when it happened
> > and what likely are the culprits -- userspace or kernel module
> > autoloading.
> > 
> > Provide an initial set of stats for debugfs which let us easily scrape
> > post-boot information about failed loads. This sort of information can
> > be used on production worklaods to try to optimize *avoiding* redundant
> > memory pressure using finit_module().
> > 
> > Screen shot:
> > 
> > root@...d ~ # cat /sys/kernel/debug/modules/stats
> >            Modules loaded       67
> 
> Is this "loaded now", or "ever successfully loaded"?  As in a
> modprobe/rmmod/modprobe would bump this by 2, right?

Ah, the later, so "how modules have I ever loaded". Maybe

Modules ever loaded

?

Will fix the nits, thanks!

> > diff --git a/kernel/module/debug.c b/kernel/module/debug.c
> 
> Why is this a whole separate file?

It's just a style preference, no real hard reason other than
module.c was huge before and now its split up. I find that
easier to review / manage. Certainly overkill for such as
simple thing but if its debug I think I rather see that
then some ifdef eyesore. But that's just preference.

> And as MODULE_DEBUG does not reference debugfs,

That should be fixed thanks.

> > diff --git a/kernel/module/internal.h b/kernel/module/internal.h
> > index 6ae29bb8836f..a645cb3fafc7 100644
> > --- a/kernel/module/internal.h
> > +++ b/kernel/module/internal.h
> > @@ -143,6 +143,41 @@ static inline bool set_livepatch_module(struct module *mod)
> >  #endif
> >  }
> >  
> > +#ifdef CONFIG_MODULE_STATS
> > +
> > +#define mod_stat_add64(count, var) atomic64_add(count, var)
> > +#define mod_stat_inc(name) atomic_inc(name)
> 
> Ok, but:
> 
> > +#define mod_stat_inc(name) atomic_inc(name)
> 
> Why do you still increment the variable here if the option is not
> enabled?

Whoops, will fix!

> Also, didn't we have some sort of "we want to use an atomic variable as
> statistics" type somewhere in the kernel? 

I didn't get the memo, nor do I recall, so it's not on my radar.

> Or did that never get accepted?

Not sure.

> And do all of these really need to be atomic variables?  Don't you have
> locks for some of this to not need the atomic-ness of them?  I guess it
> doesn't matter much as this isn't that fast of a code-path.

That was actually intentional, as this only *grows* I just care its not 0
so to help divide by the total number of modules to get average module
length and average module .text length. I used atomics and made it only
grow precisely to not have to lock anywhere.

  Luis