linux-kernel - Re: [PATCH 0/3] readfile(2): a new syscall to make open/read/close faster

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:   Sun, 5 Jul 2020 13:50:21 +0200
From:   Greg KH <gregkh@...uxfoundation.org>
To:     Jan Ziak <0xe2.0x9a.0x9b@...il.com>
Cc:     linux-api@...r.kernel.org, linux-fsdevel@...r.kernel.org,
        linux-kernel@...r.kernel.org, linux-kselftest@...r.kernel.org,
        linux-man@...r.kernel.org, mtk.manpages@...il.com,
        shuah@...nel.org, viro@...iv.linux.org.uk
Subject: Re: [PATCH 0/3] readfile(2): a new syscall to make open/read/close
 faster

On Sun, Jul 05, 2020 at 04:06:22AM +0200, Jan Ziak wrote:
> Hello
> 
> At first, I thought that the proposed system call is capable of
> reading *multiple* small files using a single system call - which
> would help increase HDD/SSD queue utilization and increase IOPS (I/O
> operations per second) - but that isn't the case and the proposed
> system call can read just a single file.

If you want to do this for multple files, use io_ring, that's what it
was designed for.  I think Jens was going to be adding support for the
open/read/close pattern to it as well, after some other more pressing
features/fixes were finished.

> Without the ability to read multiple small files using a single system
> call, it is impossible to increase IOPS (unless an application is
> using multiple reader threads or somehow instructs the kernel to
> prefetch multiple files into memory).

There's not much (but it is mesurable) need to prefetch virtual files
into memory first, which is primarily what this syscall is for (procfs,
sysfs, securityfs, etc.)  If you are dealing with real-disks, then yes,
the overhead of the syscall might be in the noise compared to the i/o
path of the data.

> While you are at it, why not also add a readfiles system call to read
> multiple, presumably small, files? The initial unoptimized
> implementation of readfiles syscall can simply call readfile
> sequentially.

Again, that's what io_uring is for.

thanks,

greg k-h