| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Date: Tue, 15 Sep 2009 19:49:37 -0700 (PDT) From: david@...g.hm To: Thomas Fjellstrom <tfjellstrom@...w.ca> cc: linux-kernel@...r.kernel.org Subject: Re: Tricks to speed up kernel builds On Tue, 15 Sep 2009, Thomas Fjellstrom wrote: > On Tue September 15 2009, Ozan ?a?layan wrote: >> Hi, >> >> I'd googled a lot about the $subject but couldn't find any detailed >> guide about it. So, the followings are what I understand from what I've >> found: >> >> 1. Build inside a tmpfs to avoid I/O bottlenecks, if you have the ram the system will use it to cache the data, avoiding the I/O bottlenecks. the first time you compile you have to wait for the source to be read into ram, but you would have to copy the source to the tmpfs anyway >> 2. Use ccache, >> 3. Switch governor to performance if supported, >> 4. Pass -j to gnu make to parallelize the build process, >> 5. Don't build debug symbols a.k.a set CONFIG_DEBUG_INFO=n >> 6. There are a bunch of sched patches in tip which apparently improves >> things during kbuild >> 7. Distribute the work into the local network using icecream, distcc. >> >> For the -j part, how many jobs should we pass to it? I've seen people >> passing >> >> - NR_of_cores+1 >> - NR_of_cores x 2 (I think that this is useful when there's 2 threads of >> execution per core like HyperThreading?) this is not due to hyperthreading. if you do not have the data cached in ram from a prior build, individual compile threads will stall waiting for the source to be read from disk. by having more threads than you have cores you can have some stalled waiting for data while others run and use the cpu cycles. exactly how much benfit you get, and where things taper off is very dependant on your system (how fast is your I/O, how effectivly does it handle parallel requests) it's probably higher than NR_of_cores +1, but if you have a 8-core or 16 core box it's probably less than NR_of_cores x 2 (unless you have a storage subsystem that works well in parallel, say something like a 8 or 16 disk raid array) >> - 64, 128, etc. regardless of the number of cores (Is it really useful >> regardless of the CPU) this will overload your system with lots of processes competing for the CPU. it's a good stress test of the system, but inefficiant in processor utilization due to the fact that each time a new thread takes the CPU the cache has been wiped out by other threads. >> The curious question is *what build times can you get* with your typical >> systems using your speedup tricks (quad core, dual core, etc.) ? What >> are other tricks that I can use to even more speed up the build? >> >> Do you really use icecream/distcc on your daily test builds? most people don't, some people do. some people have systems that can build the kernel in a handful of seconds (I think the record was 7.5 seconds several years ago) if it is in the memory disk cache and don't need to. others build on architecures that don't have systems that fast and use distcc all the time to get acceptable speed since many virtualization options will not let a guest OS use more than one real CPU at a time, people who build in virtual systems commonly use this sort of thing to use all the real cores on their system (same as the -j option above) Linus has been heard to complain that it takes longer to boot a system with a new kernel than to compile it (this was on an enterprise-class system that took a long time for the bios/raid/etc initialization) >> How about nicing/ionicing make on an idle system? Will it make any >> difference? if there is nothing to compete with the kernel build, these don't help. David Lang > I've only done a few compiles recently, and I only really use icecream and > make -jX to speed up compile times. > > I have a 4 node icecc cluster, three quad cores (one limited to 2 cores), and > one dual core (limited to a single core), which gives 11 total cores I can > distribute to, and I usually pass: make -j12 > > Any higher and I don't really see any speedups, any lower, same difference. > > My last kernel compile (2.6.31-git4) took about 9 minutes. > >> Thanks a lot >> Ozan Caglayan >> >> >> >> >> -- >> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in >> the body of a message to majordomo@...r.kernel.org >> More majordomo info at http://vger.kernel.org/majordomo-info.html >> Please read the FAQ at http://www.tux.org/lkml/ >> > > > -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@...r.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Powered by blists - more mailing lists