linux-kernel - Re: optimized script [Was: ARM defconfig files]

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20100713180402.GA1422@lixom.net>
Date:	Tue, 13 Jul 2010 13:04:02 -0500
From:	Olof Johansson <olof@...om.net>
To:	Uwe Kleine-König 
	<u.kleine-koenig@...gutronix.de>
Cc:	Linus Torvalds <torvalds@...ux-foundation.org>,
	Grant Likely <grant.likely@...retlab.ca>,
	Nicolas Pitre <nico@...xnic.net>,
	Daniel Walker <dwalker@...eaurora.org>,
	Russell King - ARM Linux <linux@....linux.org.uk>,
	Kevin Hilman <khilman@...prootsystems.com>,
	linux-arm-msm@...r.kernel.org,
	Linux Kernel Mailing List <linux-kernel@...r.kernel.org>,
	linux-omap@...r.kernel.org, Eric Miao <eric.miao@...onical.com>,
	linux-arm-kernel@...ts.infradead.org,
	Stephen Rothwell <sfr@...b.auug.org.au>,
	Benjamin Herrenschmidt <benh@...nel.crashing.org>,
	linuxppc-dev@...ts.ozlabs.org
Subject: Re: optimized script [Was: ARM defconfig files]

On Tue, Jul 13, 2010 at 10:07:05AM +0200, Uwe Kleine-König wrote:
> Hello,
> 
> On Tue, Jul 13, 2010 at 09:07:41AM +0200, Uwe Kleine-König wrote:
> > Hi
> > 
> > On Mon, Jul 12, 2010 at 01:50:47PM -0600, Grant Likely wrote:
> > > On Mon, Jul 12, 2010 at 1:34 PM, Linus Torvalds
> > > <torvalds@...ux-foundation.org> wrote:
> > > > On Mon, Jul 12, 2010 at 12:17 PM, Nicolas Pitre <nico@...xnic.net> wrote:
> > > >> I think Uwe could provide his script and add it to the kernel tree.
> > > >> Then all architectures could benefit from it.  Having the defconfig
> > > >> files contain only those options which are different from the defaults
> > > >> is certainly more readable, even on x86.
> > > >
> > > > Quite possible. But maintainers would need to be on the lookout of
> > > > people actually using the script, and refusing to apply patches that
> > > > re-introduce the whole big thing.
> > > 
> > > I can (partially) speak for powerpc.  If ARM uses this approach, then
> > > I think we can do the same.  After the defconfigs are trimmed, I
> > > certainly won't pick up any more full defconfigs.
> > I just restarted my script on the powerpc defconfigs basing on rc5, I
> > assume they complete in a few days time.
> So Stephen was faster than me.  I don't know yet how he optimised my
> script, meanwhile I put some efforts into it, too by just checking lines
> that match "^(# )?CONFIG_".
> 
> Find it attached.
> 
> I will start to reduce the remaining configs (i.e. all but arm and
> powerpc).

I added just a simple heuristic: If I could remove a line, I attempted
to remove twice the amount next time around (and fall back to 1 if it failed).

I.e. main loop:

i = 0
lines = 1

while i < len(config):
    print 'test for %r + %d' % (config[i], lines)
    defconfig = open(defconfig_src, 'w')
    defconfig.writelines(config[:i])
    defconfig.writelines(config[i + lines:])
    defconfig.close()
    subprocess.check_call(['make', '-s', 'ARCH=%s' % arch, target])
    if os.stat('.config').st_size == config_size and list(open('.config')) == origconfig:
        del config[i:i+lines]
        lines *= 2
    else:
        if lines > 1:
            lines = 1
        else:
            i += 1


I didn't measure what the actual improvement was, but I saw a fair amount
of 2/4/8-attempts passing, so I let it run. Stephen beat me to posting
the resulting patch though. :P

While this script is great, it is somewhat painful to run given that it
attempts one config per line. Even on a fast machine that tends to take
a while.


-Olof
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/