linux-kernel - Re: [REGRESSION] Recent change to kernel spikes out ccache/distcc

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20090106181224.GC10903@mit.edu>
Date:	Tue, 6 Jan 2009 13:12:24 -0500
From:	Theodore Tso <tytso@....edu>
To:	David Miller <davem@...emloft.net>
Cc:	jbeulich@...ell.com, sam@...nborg.org,
	linux-kernel@...r.kernel.org, ccache@...ts.samba.org
Subject: Re: [REGRESSION] Recent change to kernel spikes out ccache/distcc

On Tue, Jan 06, 2009 at 08:26:19AM -0800, David Miller wrote:
> From: "Theodore Ts'o" <tytso@....edu>
> Date: Tue, 06 Jan 2009 10:15:26 -0500
> 
> I think the problem might be that the assembler source always
> "changes" because a different date or timestamp string is put in there
> by gcc every time the assembler source is built.

I'm not sure that's the problem, actually. Running "ccache -s" doesn't
show any additional cache misses *or* hits.  Instead the only
statistic which gets bumped is the "called for link" counter.  So I
suspect it's because ccache doesn't attempt to cache "gcc -S" or "gcc
-c foo.s" operations.  So it might be the case that a change to ccache
to cache compile-to-assemble and assemble-to-object code
transformation would help, but then we would still end up calling
ccache twice, and double the number of things that would be cached in
the ccache directories, not to mention doubling the number of calls to
remote hosts of distcc is involved.

So a full, long-term solution that gets back all of the efficiency
wins of ccache and distcc probably requires somehow passing a script
or program to ccache and distcc which does all of the steps which
kbuild needs to do to transform a kernel .c file to a kernel .o file,
which seems to involve interposing multiple steps between the foo.c to
foo.s and foo.s and foo.o steps, as well as post-processing on the
resulting .o file using objcopy.  If we can somehow get ccache to
cache all of that work, and distribute all of the steps of going from
foo.c to foo.o to other machines when distcc is involved, that would
be really cool.  I suspect this is not trivial to do, however....

   	  	   	   	       - Ted

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/