linux-kernel - [PATCH] kbuild: shrink Makefile cache when it exceeds 1000 lines

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Message-Id: <1507800569-30134-1-git-send-email-yamada.masahiro@socionext.com>
Date:   Thu, 12 Oct 2017 18:29:29 +0900
From:   Masahiro Yamada <yamada.masahiro@...ionext.com>
To:     linux-kbuild@...r.kernel.org
Cc:     Sam Ravnborg <sam@...nborg.org>,
        Douglas Anderson <dianders@...omium.org>,
        Masahiro Yamada <yamada.masahiro@...ionext.com>,
        Marcin Nowakowski <marcin.nowakowski@...tec.com>,
        Matthias Kaehlcke <mka@...omium.org>,
        Cao jin <caoj.fnst@...fujitsu.com>,
        Arnd Bergmann <arnd@...db.de>,
        Mark Charlebois <charlebm@...il.com>,
        linux-kernel@...r.kernel.org,
        Jan-Simon Möller <dl9pf@....de>,
        Josh Poimboeuf <jpoimboe@...hat.com>,
        Ingo Molnar <mingo@...nel.org>
Subject: [PATCH] kbuild: shrink Makefile cache when it exceeds 1000 lines

The cache files are only cleaned away by "make clean".  If you continue
incremental builds, the cache files will grow up little by little.
It is not a big deal in general use cases because $(call cc-option,...)
in not added/deleted quite often.

However, if you do build-test for various architectures, compilers, and
kernel configurations, you will end up with huge cache files soon.

The simple idea is to cut down the cache when it exceeds a certain
limit.  I wrote a simple method to check if nr_cache >= 1000.
If it is, shrink it by "tail -500".  This is not LRU strategy, but I
hope it will work well enough.

Signed-off-by: Masahiro Yamada <yamada.masahiro@...ionext.com>
---

If you have a better idea, please suggest it!


 scripts/Kbuild.include | 23 ++++++++++++++++++++++-
 1 file changed, 22 insertions(+), 1 deletion(-)

diff --git a/scripts/Kbuild.include b/scripts/Kbuild.include
index 4203fff..db81df3 100644
--- a/scripts/Kbuild.include
+++ b/scripts/Kbuild.include
@@ -99,9 +99,30 @@ cc-cross-prefix =  \
 
 # Include values from last time
 make-cache := $(if $(KBUILD_EXTMOD),$(KBUILD_EXTMOD)/,$(if $(obj),$(obj)/)).cache.mk
-$(make-cache): ;
 -include $(make-cache)
 
+nr_cache := $(words $(filter __cached_%, $(.VARIABLES)))
+
+# Strip the last digit.  Covert as follows:
+#    0   ->
+#    123 -> 12
+strip_last_digit = $(filter-out $1,$(firstword \
+		$(sort $(foreach i,0 1 2 3 4 5 6 7 8 9,$(patsubst %$i,%,$1)))))
+
+# If cache exceeds 1000 lines, shrink it down to 500.  The Least Recently Added
+# lines are deleted. (not Least Recently Used, unfortunately)
+#
+# Evalucate [ ${nr_cache} -ge 1000 ] without relying on external tools.
+# Check if nr_cache is not empty even after the last three digits are stripped.
+ifneq ($(call strip_last_digit,$(call strip_last_digit,$(call strip_last_digit,$(nr_cache)))),)
+.PHONY: $(make-cache)
+$(make-cache):
+	tail -500 $@ > $@...p
+	mv $@...p $@
+else
+$(make-cache): ;
+endif
+
 # Usage: $(call __sanitize-opt,Hello=Hola$(comma)Goodbye Adios)
 #
 # Convert all '$', ')', '(', '\', '=', ' ', ',', ':' to '_'
-- 
2.7.4