linux-kernel - perf tools regression testing was Re: [PATCH v3 1/9] perf hists: Move hists

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:	Tue, 25 Sep 2012 10:30:15 -0300
From:	Arnaldo Carvalho de Melo <acme@...hat.com>
To:	Namhyung Kim <namhyung@...nel.org>
Cc:	Feng Tang <feng.tang@...el.com>, mingo@...e.hu,
	a.p.zijlstra@...llo.nl, andi@...stfloor.org,
	David Ahern <dsahern@...il.com>, Jiri Olsa <jolsa@...hat.com>,
	linux-kernel@...r.kernel.org
Subject: perf tools regression testing was Re: [PATCH v3 1/9] perf hists:
 Move hists_init() from util/evsel.c to util/hist.c

Em Tue, Sep 25, 2012 at 09:59:02PM +0900, Namhyung Kim escreveu:
> 2012-09-25 (화), 08:05 -0300, Arnaldo Carvalho de Melo:
> > Em Tue, Sep 25, 2012 at 10:25:13AM +0900, Namhyung Kim escreveu:
> > > On Mon, 24 Sep 2012 13:02:39 -0300, Arnaldo Carvalho de Melo wrote:
> > > > Em Mon, Sep 24, 2012 at 11:24:03PM +0800, Feng Tang escreveu:
> > > >> Which looks more natural

> > > > It is there to avoid dragging the hist code into the python binding :-\

> > > Hmm... it's so hairy.  Can't we do better?

> > We always can do better :-)

> > I just stated why it was at that place.

> > When doing refactorings we're all the time trying to make it better in
> > many senses, one of them is trying to isolate code that is useful in a
> > general way and thus should be made available via a library/scripting
> > binding.
 
> Yeah, but the isolation sometimes got broken as code getting added like
> this.  So we need a automatic way of detecting breakage.

Agreed.
 
> I thought about adding a perf test entry running python/twatch.py, but
> it will not work for an installed perf binary since it cannot find the
> twatch.py script and perf.so extension files which are not installed.

I thought about doing it the way 'perf script' works with python, i.e.
calling the python interpreter, setting the PYTHONPATH thru it, then
asking for 'use python' to happen, catch the result.
 
> Now I'm thinking of making it build-time test so that it can be executed
> by make when specific argument is given - e.g. make C=1 ?

I think there is room for a 'make -C tools/perf check' that would use
the 'expect' tool to do not just this but also run record, report, etc
and check its output against what is expected, perf test is ok for
checking the APIs, but we need a test suite for the actual builtins as
called from the command line.

- Arnaldo
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/