Message-ID: <976fcec0-d132-3a27-bbd2-01b21571bca2@redhat.com>
Date:   Wed, 3 May 2023 14:53:27 +0200
From:   David Hildenbrand <david@...hat.com>
To:     Matthew Rosato <mjrosato@...ux.ibm.com>,
        Lorenzo Stoakes <lstoakes@...il.com>, linux-mm@...ck.org,
        linux-kernel@...r.kernel.org,
        Andrew Morton <akpm@...ux-foundation.org>
Cc:     Jason Gunthorpe <jgg@...pe.ca>, Jens Axboe <axboe@...nel.dk>,
        Matthew Wilcox <willy@...radead.org>,
        Dennis Dalessandro <dennis.dalessandro@...nelisnetworks.com>,
        Leon Romanovsky <leon@...nel.org>,
        Christian Benvenuti <benve@...co.com>,
        Nelson Escobar <neescoba@...co.com>,
        Bernard Metzler <bmt@...ich.ibm.com>,
        Peter Zijlstra <peterz@...radead.org>,
        Ingo Molnar <mingo@...hat.com>,
        Arnaldo Carvalho de Melo <acme@...nel.org>,
        Mark Rutland <mark.rutland@....com>,
        Alexander Shishkin <alexander.shishkin@...ux.intel.com>,
        Jiri Olsa <jolsa@...nel.org>,
        Namhyung Kim <namhyung@...nel.org>,
        Ian Rogers <irogers@...gle.com>,
        Adrian Hunter <adrian.hunter@...el.com>,
        Bjorn Topel <bjorn@...nel.org>,
        Magnus Karlsson <magnus.karlsson@...el.com>,
        Maciej Fijalkowski <maciej.fijalkowski@...el.com>,
        Jonathan Lemon <jonathan.lemon@...il.com>,
        "David S . Miller" <davem@...emloft.net>,
        Eric Dumazet <edumazet@...gle.com>,
        Jakub Kicinski <kuba@...nel.org>,
        Paolo Abeni <pabeni@...hat.com>,
        Christian Brauner <brauner@...nel.org>,
        Richard Cochran <richardcochran@...il.com>,
        Alexei Starovoitov <ast@...nel.org>,
        Daniel Borkmann <daniel@...earbox.net>,
        Jesper Dangaard Brouer <hawk@...nel.org>,
        John Fastabend <john.fastabend@...il.com>,
        linux-fsdevel@...r.kernel.org, linux-perf-users@...r.kernel.org,
        netdev@...r.kernel.org, bpf@...r.kernel.org,
        Oleg Nesterov <oleg@...hat.com>,
        Jason Gunthorpe <jgg@...dia.com>,
        John Hubbard <jhubbard@...dia.com>, Jan Kara <jack@...e.cz>,
        "Kirill A . Shutemov" <kirill@...temov.name>,
        Pavel Begunkov <asml.silence@...il.com>,
        Mika Penttila <mpenttil@...hat.com>,
        Dave Chinner <david@...morbit.com>,
        Theodore Ts'o <tytso@....edu>, Peter Xu <peterx@...hat.com>,
        "Paul E . McKenney" <paulmck@...nel.org>,
        Christian Borntraeger <borntraeger@...ux.ibm.com>
Subject: Re: [PATCH v8 0/3] mm/gup: disallow GUP writing to file-backed
 mappings by default

On 03.05.23 13:25, Matthew Rosato wrote:
> On 5/3/23 3:08 AM, David Hildenbrand wrote:
>> On 03.05.23 02:31, Matthew Rosato wrote:
>>> On 5/2/23 6:51 PM, Lorenzo Stoakes wrote:
>>>> Writing to file-backed mappings which require folio dirty tracking using
>>>> GUP is a fundamentally broken operation, as kernel write access to GUP
>>>> mappings do not adhere to the semantics expected by a file system.
>>>>
>>>> A GUP caller uses the direct mapping to access the folio, which does not
>>>> cause write notify to trigger, nor does it enforce that the caller marks
>>>> the folio dirty.
>>>>
>>>> The problem arises when, after an initial write to the folio, writeback
>>>> results in the folio being cleaned and then the caller, via the GUP
>>>> interface, writes to the folio again.
>>>>
>>>> As a result of the use of this secondary, direct, mapping to the folio no
>>>> write notify will occur, and if the caller does mark the folio dirty, this
>>>> will be done so unexpectedly.
>>>>
>>>> For example, consider the following scenario:-
>>>>
>>>> 1. A folio is written to via GUP which write-faults the memory, notifying
>>>>      the file system and dirtying the folio.
>>>> 2. Later, writeback is triggered, resulting in the folio being cleaned and
>>>>      the PTE being marked read-only.
>>>> 3. The GUP caller writes to the folio, as it is mapped read/write via the
>>>>      direct mapping.
>>>> 4. The GUP caller, now done with the page, unpins it and sets it dirty
>>>>      (though it does not have to).
>>>>
>>>> This change updates both the PUP FOLL_LONGTERM slow and fast APIs. As
>>>> pin_user_pages_fast_only() does not exist, we can rely on a slightly
>>>> imperfect whitelisting in the PUP-fast case and fall back to the slow case
>>>> should this fail.
>>>>
>>>> v8:
>>>> - Fixed typo writeable -> writable.
>>>> - Fixed bug in writable_file_mapping_allowed() - must check combination of
>>>>     FOLL_PIN AND FOLL_LONGTERM not either/or.
>>>> - Updated vma_needs_dirty_tracking() to include write/shared to account for
>>>>     MAP_PRIVATE mappings.
>>>> - Move to open-coding the checks in folio_pin_allowed() so we can
>>>>     READ_ONCE() the mapping and avoid unexpected compiler loads. Rename to
>>>>     account for fact we now check flags here.
>>>> - Disallow mapping == NULL or mapping & PAGE_MAPPING_FLAGS other than
>>>>     anon. Defer to slow path.
>>>> - Perform GUP-fast check _after_ the lowest page table level is confirmed to
>>>>     be stable.
>>>> - Updated comments and commit message for final patch as per Jason's
>>>>     suggestions.
>>>
>>> Tested again on s390 using QEMU with a memory backend file (on ext4) and vfio-pci -- This time both vfio_pin_pages_remote (which will call pin_user_pages_remote(flags | FOLL_LONGTERM)) and the pin_user_pages_fast(FOLL_WRITE | FOLL_LONGTERM) in kvm_s390_pci_aif_enable are being allowed (e.g. returning positive pin count)
>>
>> At least it's consistent now ;) And it might be working as expected ...
>>
>> In v7:
>> * pin_user_pages_fast() succeeded
>> * vfio_pin_pages_remote() failed
>>
>> But also in v7:
>> * GUP-fast allows pinning (anonymous) pages in MAP_PRIVATE file
>>    mappings
>> * Ordinary GUP allows pinning pages in MAP_PRIVATE file mappings
>>
>> In v8:
>> * pin_user_pages_fast() succeeds
>> * vfio_pin_pages_remote() succeeds
>>
>> But also in v8:
>> * GUP-fast allows pinning (anonymous) pages in MAP_PRIVATE file
>>    mappings
>> * Ordinary GUP allows pinning pages in MAP_PRIVATE file mappings
>>
>>
>> I have to speculate, but ... could it be that you are using a private mapping?
>>
>> In QEMU, unfortunately, the default for memory-backend-file is "share=off" (private) ... for memory-backend-memfd it is "share=on" (shared). The default is stupid ...
>>
>> If you invoke QEMU manually, can you specify "share=on" for the memory-backend-file? I thought libvirt would always default to "share=on" for file mappings (everything else doesn't make much sense) ... but you might have to specify
>>      <access mode="shared"/>
>> in addition to
>>      <source type="file"/>
>>
> 
> Ah, there we go.  Yes, I was using the default of share=off.  When I instead specify share=on, now the pins will fail in both cases.
> 

Out of curiosity, how does that manifest?

I assume the VM is successfully created and as Linux tries initializing 
and using the device, we get a bunch of errors inside the VM, correct?

-- 
Thanks,

David / dhildenb

Powered by blists - more mailing lists