linux-kernel - Re: [PATCH 0/3] readfile(2): a new syscall to make open/read/close faster

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20200705120003.GC1227929@kroah.com>
Date:   Sun, 5 Jul 2020 14:00:03 +0200
From:   Greg KH <gregkh@...uxfoundation.org>
To:     Jan Ziak <0xe2.0x9a.0x9b@...il.com>
Cc:     Andreas Dilger <adilger@...ger.ca>,
        Matthew Wilcox <willy@...radead.org>,
        linux-api@...r.kernel.org, linux-fsdevel@...r.kernel.org,
        linux-kernel@...r.kernel.org, linux-kselftest@...r.kernel.org,
        linux-man@...r.kernel.org, mtk.manpages@...il.com,
        shuah@...nel.org, viro@...iv.linux.org.uk
Subject: Re: [PATCH 0/3] readfile(2): a new syscall to make open/read/close
 faster

On Sun, Jul 05, 2020 at 09:25:39AM +0200, Jan Ziak wrote:
> On Sun, Jul 5, 2020 at 8:32 AM Andreas Dilger <adilger@...ger.ca> wrote:
> >
> > On Jul 4, 2020, at 8:46 PM, Jan Ziak <0xe2.0x9a.0x9b@...il.com> wrote:
> > >
> > > On Sun, Jul 5, 2020 at 4:16 AM Matthew Wilcox <willy@...radead.org> wrote:
> > >>
> > >> On Sun, Jul 05, 2020 at 04:06:22AM +0200, Jan Ziak wrote:
> > >>> Hello
> > >>>
> > >>> At first, I thought that the proposed system call is capable of
> > >>> reading *multiple* small files using a single system call - which
> > >>> would help increase HDD/SSD queue utilization and increase IOPS (I/O
> > >>> operations per second) - but that isn't the case and the proposed
> > >>> system call can read just a single file.
> > >>>
> > >>> Without the ability to read multiple small files using a single system
> > >>> call, it is impossible to increase IOPS (unless an application is
> > >>> using multiple reader threads or somehow instructs the kernel to
> > >>> prefetch multiple files into memory).
> > >>
> > >> What API would you use for this?
> > >>
> > >> ssize_t readfiles(int dfd, char **files, void **bufs, size_t *lens);
> > >>
> > >> I pretty much hate this interface, so I hope you have something better
> > >> in mind.
> > >
> > > I am proposing the following:
> > >
> > > struct readfile_t {
> > >  int dirfd;
> > >  const char *pathname;
> > >  void *buf;
> > >  size_t count;
> > >  int flags;
> > >  ssize_t retval; // set by kernel
> > >  int reserved; // not used by kernel
> > > };
> >
> > If you are going to pass a struct from userspace to the kernel, it
> > should not mix int and pointer types (which may be 64-bit values,
> > so that there are not structure packing issues, like:
> >
> > struct readfile {
> >         int     dirfd;
> >         int     flags;
> >         const char *pathname;
> >         void    *buf;
> >         size_t  count;
> >         ssize_t retval;
> > };
> >
> > It would be better if "retval" was returned in "count", so that
> > the structure fits nicely into 32 bytes on a 64-bit system, instead
> > of being 40 bytes per entry, which adds up over many entries, like.
> 
> I know what you mean and it is a valid point, but in my opinion it
> shouldn't (in most cases) be left to the programmer to decide what the
> binary layout of a data structure is - instead it should be left to an
> optimizing compiler to decide it.

We don't get that luxury when creating user/kernel apis in C, sorry.

I suggest using the pahole tool if you are interested in seeing the
"best" way a structure can be layed out, it can perform that
optimization for you so that you know how to fix your code.

thanks,

greg k-h