linux-ext4 - Re: [PATCH 0/4] e2fprogs: compiler fun

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Date:   Thu, 9 Aug 2018 12:18:32 -0700
From:   "Darrick J. Wong" <darrick.wong@...cle.com>
To:     "Theodore Y. Ts'o" <tytso@....edu>
Cc:     linux-ext4@...r.kernel.org
Subject: Re: [PATCH 0/4] e2fprogs: compiler fun

On Thu, Aug 09, 2018 at 02:55:23PM -0400, Theodore Y. Ts'o wrote:
> On Thu, Aug 09, 2018 at 11:14:33AM -0700, Darrick J. Wong wrote:
> >  566864   24272    1777  592913   90c11 build-x64/e2fsck/e2fsck
> >  561691   24336    1809  587836   8f83c build-x64-nonlto/e2fsck/e2fsck
> > 
> > Interesting that you got much smaller binaries than I did...
> 
> Were you building with configure --enable-elf-shlibs?  My stats were
> using binaries built with dpkg-buildpackage, with the only difference
> was changing --enable-lto to --disable-lto in debian/rules.

$ size build-x64/misc/e2image build-x64-nonlto/misc/e2image
   text    data     bss     dec     hex filename
 128293    3784    1601  133678   20a2e build-x64/misc/e2image
  27910    1416    1248   30574    776e build-x64-nonlto/misc/e2image

$ ldd build-x64/misc/e2image build-x64-nonlto/misc/e2image
build-x64/misc/e2image:
        linux-vdso.so.1 (0x00007ffc8699c000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f2bacd40000)
        libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f2bacb21000)
        libblkid.so.1 => /lib/x86_64-linux-gnu/libblkid.so.1 (0x00007f2bac8d4000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f2bad353000)
        libuuid.so.1 => /lib/x86_64-linux-gnu/libuuid.so.1 (0x00007f2bac6cd000)
build-x64-nonlto/misc/e2image:
        linux-vdso.so.1 (0x00007ffc4b968000)
        libext2fs.so.2 => /lib/x86_64-linux-gnu/libext2fs.so.2 (0x00007f60a718b000)
        libcom_err.so.2 => /lib/x86_64-linux-gnu/libcom_err.so.2 (0x00007f60a6f87000)
        libblkid.so.1 => /lib/x86_64-linux-gnu/libblkid.so.1 (0x00007f60a6d3a000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f60a6949000)
        libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f60a672a000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f60a75e3000)
        libuuid.so.1 => /lib/x86_64-linux-gnu/libuuid.so.1 (0x00007f60a6523000)

Now there's a side effect I didn't notice before -- LTO means we switch
to static libraries, presumably if the linker can find static libs with
LTO bytecode inside.

$ size build-x64-nonlto/lib/libcom_err.so build-x64-nonlto/lib/libext2fs.so
   text    data     bss     dec     hex filename
   9224     960     161   10345    2869 build-x64-nonlto/lib/libcom_err.so
 329135    8368     712  338215   52927 build-x64-nonlto/lib/libext2fs.so

Hmm.  So that's 329135+9224+27910 = 366269, which is a much bigger
footprint, except for the fact that most of it is a shared library and
could be amortized among whatever other processes are running.  Probably
not many in the case of libext2fs, but possibly more for libcomerr.

Ok, what about e2fsck?

$ size build-x64/*/e2fsck build-x64-nonlto/*/e2fsck
   text    data     bss     dec     hex filename
 578024   24272    1777  604073   937a9 build-x64/e2fsck/e2fsck
 291352   21712    1096  314160   4cb30 build-x64-nonlto/e2fsck/e2fsck

$ ldd build-x64/*/e2fsck build-x64-nonlto/*/e2fsck
build-x64/e2fsck/e2fsck:
        linux-vdso.so.1 (0x00007ffe5b38a000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f8c518fe000)
        libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f8c516df000)
        libblkid.so.1 => /lib/x86_64-linux-gnu/libblkid.so.1 (0x00007f8c51492000)
        libuuid.so.1 => /lib/x86_64-linux-gnu/libuuid.so.1 (0x00007f8c5128b000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f8c51f84000)
build-x64-nonlto/e2fsck/e2fsck:
        linux-vdso.so.1 (0x00007fffde3ee000)
        libext2fs.so.2 => /lib/x86_64-linux-gnu/libext2fs.so.2 (0x00007fc1cd4b1000)
        libcom_err.so.2 => /lib/x86_64-linux-gnu/libcom_err.so.2 (0x00007fc1cd2ad000)
        libblkid.so.1 => /lib/x86_64-linux-gnu/libblkid.so.1 (0x00007fc1cd060000)
        libuuid.so.1 => /lib/x86_64-linux-gnu/libuuid.so.1 (0x00007fc1cce59000)
        libe2p.so.2 => /lib/x86_64-linux-gnu/libe2p.so.2 (0x00007fc1ccc50000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fc1cc85f000)
        libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fc1cc640000)
        /lib64/ld-linux-x86-64.so.2 (0x00007fc1cd94e000)

$ size build-x64-nonlto/lib/libcom_err.so build-x64-nonlto/lib/libext2fs.so build-x64-nonlto/lib/libe2p.so
   text    data     bss     dec     hex filename
   9224     960     161   10345    2869 build-x64-nonlto/lib/libcom_err.so
 329135    8368     712  338215   52927 build-x64-nonlto/lib/libext2fs.so
  28112    2640     616   31368    7a88 build-x64-nonlto/lib/libe2p.so

Same general idea... 578024 vs 329135+9224+28112+291352 = 657823, though
60% of that could theoretically be shared.

That's an interesting implication -- libraries which are shared widely
among the "usual" set of running programs on a computer should not be
LTO'd because we're better off (in terms of memory consumption) to share
the library code; but libraries which are not widely shared should be
LTO if the reduction in code size outweighs the loss of amortization
possibilities.  I guess?

--D

> 
> 						- Ted