linux-kernel - Re: [PATCH v2 1/2] kstats: kernel metric collector

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <CAMOZA0LU_mGPre9gsJSZeG19fUjLWb+6xuG8-2yv5gJRHwWzqQ@mail.gmail.com>
Date:   Wed, 26 Feb 2020 11:31:01 -0800
From:   Luigi Rizzo <lrizzo@...gle.com>
To:     Peter Zijlstra <peterz@...radead.org>
Cc:     linux-kernel@...r.kernel.org,
        Masami Hiramatsu <mhiramat@...nel.org>,
        Andrew Morton <akpm@...ux-foundation.org>,
        Greg KH <gregkh@...uxfoundation.org>,
        naveen.n.rao@...ux.ibm.com, ardb@...nel.org,
        Luigi Rizzo <rizzo@....unipi.it>,
        Paolo Abeni <pabeni@...hat.com>, giuseppe.lettieri@...pi.it,
        Toke Høiland-Jørgensen <toke@...hat.com>,
        Jesper Dangaard Brouer <hawk@...nel.org>, mingo@...hat.com,
        acme@...nel.org, Steven Rostedt <rostedt@...dmis.org>
Subject: Re: [PATCH v2 1/2] kstats: kernel metric collector

On Wed, Feb 26, 2020 at 8:19 AM Peter Zijlstra <peterz@...radead.org> wrote:
>
> On Wed, Feb 26, 2020 at 05:46:36AM -0800, Luigi Rizzo wrote:
> > kstats is a helper to accumulate in-kernel metrics (timestamps, sizes,
> > etc.) and show distributions through debugfs.
> > Set CONFIG_KSTATS=m or y to enable it.
> >
> > Creating a metric takes one line of code (and one to destroy it):
> >
> >   struct kstats *key = kstats_new("foo", 3 /* frac_bits */);
> >   ...
> >   kstats_delete(key);
> >
> > The following line records a u64 sample:
> >
> >   kstats_record(key, value);
> >
> > kstats_record() is cheap (5ns hot cache, 250ns cold cache). Samples are
> > accumulated in a per-cpu array with 2^frac_bits slots for each power
> > of 2. Using frac_bits = 3 gives about 30 slots per decade.
>
> So I think everybody + dog has written code like this, although I never
> bothered with the log2 based buckets myself. Nor have I ever bothered
> with doing a debugfs interface.

the above is perhaps one excellent argument to why it may deserve to be in:
so that people don't need to write the measurement code time and again,
or, as I have done myself multiple times, use some inferior hack (racy
counter, coarse buckets) or give up measuring things and rely on guessing.

> I find it very hard to convince myself something like this deserves to
> live upstream, vs. remaining in the local debug/hack toolbox.
>
> Tracing has an aggregator (histogram), you can dump the raw deltas, or
> you can hack up a custom aggregator in a few lines, or you do BPF if
> you're so inclined.

And this is possibly another good argument: sometimes the systems where it
would be interesting to collect data are not accessible to the developers with
skills to write the monitoring code and run a modified kernel.

cheers
luigi