linux-ext4 - Re: [PATCH 0/8 v2] Non-blocking AIO

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <c8c3cede-d69a-a47a-0b08-85072b386d77@scylladb.com>
Date:   Mon, 6 Mar 2017 17:59:26 +0200
From:   Avi Kivity <avi@...lladb.com>
To:     Jens Axboe <axboe@...nel.dk>, Jan Kara <jack@...e.cz>
Cc:     Goldwyn Rodrigues <rgoldwyn@...e.de>, jack@...e.com,
        hch@...radead.org, linux-fsdevel@...r.kernel.org,
        linux-block@...r.kernel.org, linux-btrfs@...r.kernel.org,
        linux-ext4@...r.kernel.org, linux-xfs@...r.kernel.org
Subject: Re: [PATCH 0/8 v2] Non-blocking AIO

On 03/06/2017 05:38 PM, Jens Axboe wrote:
> On 03/06/2017 08:29 AM, Avi Kivity wrote:
>>
>> On 03/06/2017 05:19 PM, Jens Axboe wrote:
>>> On 03/06/2017 01:25 AM, Jan Kara wrote:
>>>> On Sun 05-03-17 16:56:21, Avi Kivity wrote:
>>>>>> The goal of the patch series is to return -EAGAIN/-EWOULDBLOCK if
>>>>>> any of these conditions are met. This way userspace can push most
>>>>>> of the write()s to the kernel to the best of its ability to complete
>>>>>> and if it returns -EAGAIN, can defer it to another thread.
>>>>>>
>>>>> Is it not possible to push the iocb to a workqueue?  This will allow
>>>>> existing userspace to work with the new functionality, unchanged. Any
>>>>> userspace implementation would have to do the same thing, so it's not like
>>>>> we're saving anything by pushing it there.
>>>> That is not easy because until IO is fully submitted, you need some parts
>>>> of the context of the process which submits the IO (e.g. memory mappings,
>>>> but possibly also other credentials). So you would need to somehow transfer
>>>> this information to the workqueue.
>>> Outside of technical challenges, the API also needs to return EAGAIN or
>>> start blocking at some point. We can't expose a direct connection to
>>> queue work like that, and let any user potentially create millions of
>>> pending work items (and IOs).
>> You wouldn't expect more concurrent events than the maxevents parameter
>> that was supplied to io_setup syscall; it should have reserved any
>> resources needed.
> Doesn't matter what limit you apply, my point still stands - at some
> point you have to return EAGAIN, or block. Returning EAGAIN without
> the caller having flagged support for that change of behavior would
> be problematic.

Doesn't it already return EAGAIN (or some other error) if you exceed 
maxevents?

> And for this to really work, aio would need some serious help in
> how it applies limits. It looks like a hot mess.

For sure.  I think it would be a shame to create more user-facing 
complexity.