linux-ext4 - Re: [PATCH RFC 2/5] libext2fs: add threading support to the I/O manager abstraction

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20201208011705.GB52960@mit.edu>
Date:   Mon, 7 Dec 2020 20:17:05 -0500
From:   "Theodore Y. Ts'o" <tytso@....edu>
To:     Andreas Dilger <adilger@...ger.ca>
Cc:     Ext4 Developers List <linux-ext4@...r.kernel.org>,
        Saranya Muruganandam <saranyamohan@...gle.com>,
        Wang Shilong <wshilong@....com>
Subject: Re: [PATCH RFC 2/5] libext2fs: add threading support to the I/O
 manager abstraction

On Mon, Dec 07, 2020 at 11:15:30AM -0700, Andreas Dilger wrote:
> 
> Do you know how often we get into the "bounce_read" IO path?  It seems like
> locking around the read would kill parallelism, but this code path also
> looks like a fallback, but maybe 100% used for blocksize != PAGE_SIZE?

It should be extremely rare.  It only happens when Direct I/O is
requested (which is rare to begin with, although it looks like there
are people who are playing with some out of tree patchets to force
e2image to use Direct I/O?), and the offset or the size of I/O isn't
aligned with the system's direct I/O alignment requirements.  (See
ext2fs_get_dio_alignment() in lib/ext2fs/getsectsize.c)

> At one point you talked about using dlopen() or similar to link in the
> pthread library only if it is actually needed?  Or is the linkage of
> the pthread library avoided by these functions not being called unless
> IO_FLAG_THREADS is set? 

So what I'm doing is just not trying to call those functions unless
threading is required (e.g., IO_FLAG_THREADS is set, which would imply
that EXT2_FLAG_THREADS was passed to ext2fs_open()).  This won't help
if the application is using static linking, but if the application is
compiled statically, dlopen(3) is not guaranteed to work in any case.

So it's not perfect, and there may be some ancient AIX machine for
which this might be problematic.  But for all modern OS's that are
linking to want to compile e2fsprogs, it should work.  And I don't
have access to an AIX machine these days, and I don't work for IBM
anymore, so....  ¯\_(ツ)_/¯

:-)

					- Ted