lists.openwall.net   lists  /  announce  owl-users  owl-dev  john-users  john-dev  passwdqc-users  yescrypt  popa3d-users  /  oss-security  kernel-hardening  musl  sabotage  tlsify  passwords  /  crypt-dev  xvendor  /  Bugtraq  Full-Disclosure  linux-kernel  linux-netdev  linux-ext4  PHC 
Open Source and information security mailing list archives
 
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [day] [month] [year] [list]
Date:   Thu, 9 Aug 2018 13:45:15 -0700
From:   "Darrick J. Wong" <darrick.wong@...cle.com>
To:     "Theodore Y. Ts'o" <tytso@....edu>
Cc:     linux-ext4@...r.kernel.org
Subject: Re: [PATCH 0/4] e2fprogs: compiler fun

On Thu, Aug 09, 2018 at 04:28:13PM -0400, Theodore Y. Ts'o wrote:
> On Thu, Aug 09, 2018 at 12:18:32PM -0700, Darrick J. Wong wrote:
> > 
> > Now there's a side effect I didn't notice before -- LTO means we switch
> > to static libraries, presumably if the linker can find static libs with
> > LTO bytecode inside.
> 
> Hmm, I'm not seeing this with my compiler toolchain (binaries built
> using dpkg-buildpackage -us -uc -b on a recently updated Debian
> unstable system):
> 
> <tytso@...c> {/tmp/debian/e2fsprogs/debian/e2fsprogs/sbin}   (next)
> 1391% ldd debugfs 
> 	linux-vdso.so.1 (0x00007ffc816f7000)
> 	libext2fs.so.2 => /lib/x86_64-linux-gnu/libext2fs.so.2 (0x00007f395fdcb000)
> 	libe2p.so.2 => /lib/x86_64-linux-gnu/libe2p.so.2 (0x00007f395fdc1000)
> 	libss.so.2 => /lib/x86_64-linux-gnu/libss.so.2 (0x00007f395fdb7000)
> 	libcom_err.so.2 => /lib/x86_64-linux-gnu/libcom_err.so.2 (0x00007f395fdb1000)
> 	libblkid.so.1 => /lib/x86_64-linux-gnu/libblkid.so.1 (0x00007f395fb62000)
> 	libuuid.so.1 => /lib/x86_64-linux-gnu/libuuid.so.1 (0x00007f395f95b000)
> 	libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f395f954000)
> 	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f395f797000)
> 	libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f395f776000)
> 	/lib64/ld-linux-x86-64.so.2 (0x00007f395fe7c000)
> <tytso@...c> {/tmp/debian/e2fsprogs/debian/e2fsprogs/sbin}   (next)
> 1392% ldd e2fsck
> 	linux-vdso.so.1 (0x00007fffae2e7000)
> 	libext2fs.so.2 => /lib/x86_64-linux-gnu/libext2fs.so.2 (0x00007f35f07cf000)
> 	libcom_err.so.2 => /lib/x86_64-linux-gnu/libcom_err.so.2 (0x00007f35f07c9000)
> 	libblkid.so.1 => /lib/x86_64-linux-gnu/libblkid.so.1 (0x00007f35f057a000)
> 	libuuid.so.1 => /lib/x86_64-linux-gnu/libuuid.so.1 (0x00007f35f0373000)
> 	libe2p.so.2 => /lib/x86_64-linux-gnu/libe2p.so.2 (0x00007f35f0369000)
> 	libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f35f0364000)
> 	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f35f01a5000)
> 	libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f35f0184000)
> 	/lib64/ld-linux-x86-64.so.2 (0x00007f35f089c000)
> <tytso@...c> {/tmp/debian/e2fsprogs/debian/e2fsprogs/sbin}   (next)
> 1393% grep -- -lto ../../rules
> COMMON_CONF_FLAGS = --enable-lto --disable-ubsan --disable-addrsan \
> 
> And given the bloated size of libext2fs.a, I know that we actually
> were building with LTO:
> 
> <tytso@...c> {/tmp/debian/e2fsprogs/debian/e2fsprogs/sbin}   (next)
> 1394% ls ../../libext2fs-dev/usr/lib/x86_64-linux-gnu/libext2fs.a 
> 3696 ../../libext2fs-dev/usr/lib/x86_64-linux-gnu/libext2fs.a
> 
> And just in case the problem was shared object files with -fpic
> weren't built with LTO, I can confirm they were:
> 
> <tytso@...c> {/tmp/debian/e2fsprogs/debian/e2fsprogs/sbin}   (next)
> 1395% cd ../../BUILD-STD/lib/ext2fs/
> <tytso@...c> {/tmp/debian/e2fsprogs/debian/BUILD-STD/lib/ext2fs}   (next)
> 1396% objdump -x elfshared/alloc.o | grep lto | head -10
>   3 .gnu.lto_.profile.dc6c1beb1dcdd1cc 00000014  0000000000000000  0000000000000000  00000e1a  2**0
>   4 .gnu.lto_.icf.dc6c1beb1dcdd1cc 00000082  0000000000000000  0000000000000000  00000e2e  2**0
>   5 .gnu.lto_.jmpfuncs.dc6c1beb1dcdd1cc 0000033c  0000000000000000  0000000000000000  00000eb0  2**0
>   6 .gnu.lto_.inline.dc6c1beb1dcdd1cc 000004c1  0000000000000000  0000000000000000  000011ec  2**0
>   7 .gnu.lto_.pureconst.dc6c1beb1dcdd1cc 00000035  0000000000000000  0000000000000000  000016ad  2**0
>   8 .gnu.lto_check_inode_uninit.dc6c1beb1dcdd1cc 000006e8  0000000000000000  0000000000000000  000016e2  2**0
>   9 .gnu.lto_ext2fs_clear_block_uninit.part.7.dc6c1beb1dcdd1cc 000002bf  0000000000000000  0000000000000000  00001dca  2**0
>  10 .gnu.lto_ext2fs_get_free_blocks2.part.8.dc6c1beb1dcdd1cc 00000703  0000000000000000  0000000000000000  00002089  2**0
>  11 .gnu.lto_ext2fs_alloc_range.part.9.dc6c1beb1dcdd1cc 000003ca  0000000000000000  0000000000000000  0000278c  2**0
>  12 .gnu.lto_ext2fs_clear_block_uninit.dc6c1beb1dcdd1cc 000003c6  0000000000000000  0000000000000000  00002b56  2**0
> 
> And just double checking to be sure:
> 
> <tytso@...c> {/tmp/debian/e2fsprogs/debian/BUILD-STD/e2fsck}   (next)
> 1399% objdump -x e2fsck | grep lto | head -10
> 000000000000f6d3 l     F .text	00000000000002c9              new_table_block.lto_priv.31
> 000000000000f610 l     F .text	00000000000000c3              finish_processing_inode.part.27.lto_priv.28
> 0000000000016730 l     F .text	00000000000000c7              mark_inode_bad.lto_priv.30
> 00000000000503a0 l     O .bss	0000000000000038              ext2_max_sizes.lto_priv.15
> 00000000000503d8 l     O .bss	0000000000000008              inodes_to_process.lto_priv.16
> 0000000000034900 l     F .text	0000000000000197              e2fsck_handle_read_error.lto_priv.22
> 0000000000050740 l     O .bss	0000000000000008              operation.lto_priv.8
> 0000000000012e60 l     F .text	000000000000006a              pass1_read_inode.lto_priv.19
> 000000000002bcc0 l     F .text	00000000000001fb              check_large_ea_inode.lto_priv.13
> 000000000000f99c l     F .text	000000000000014a              adjust_extattr_refcount.lto_priv.27
> <tytso@...c> {/tmp/debian/e2fsprogs/debian/BUILD-STD/e2fsck}   (next)
> 1400% ldd e2fsck
> 	linux-vdso.so.1 (0x00007fff889a6000)
> 	libext2fs.so.2 => /lib/x86_64-linux-gnu/libext2fs.so.2 (0x00007f2512abe000)
> 	libcom_err.so.2 => /lib/x86_64-linux-gnu/libcom_err.so.2 (0x00007f2512ab8000)
> 	libblkid.so.1 => /lib/x86_64-linux-gnu/libblkid.so.1 (0x00007f2512869000)
> 	libuuid.so.1 => /lib/x86_64-linux-gnu/libuuid.so.1 (0x00007f2512662000)
> 	libe2p.so.2 => /lib/x86_64-linux-gnu/libe2p.so.2 (0x00007f2512658000)
> 	libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f2512653000)
> 	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f2512494000)
> 	libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f2512473000)
> 	/lib64/ld-linux-x86-64.so.2 (0x00007f2512b8b000)
> 
> > That's an interesting implication -- libraries which are shared widely
> > among the "usual" set of running programs on a computer should not be
> > LTO'd because we're better off (in terms of memory consumption) to share
> > the library code; but libraries which are not widely shared should be
> > LTO if the reduction in code size outweighs the loss of amortization
> > possibilities.  I guess?
> 
> I think the bigger take home is that LTO is very much an emerging
> technology, and may be true for one version of the toolchain may not
> be true for another.
> 
> So I'm wondering if we should default LTO to be off, rather than
> --enable-lto=probe for now.  Mainly because I'm a bit more nervous how
> mature the LTO support is for all distributions.

Probably a reasonable default, at least for a year or two. :)

--D

> 
>        	       	       	      	  - Ted
> 
> 
> 
> > 
> > --D
> > 
> > > 
> > > 						- Ted

Powered by blists - more mailing lists