[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-Id: <20190130081336.GC17937@rapoport-lnx>
Date: Wed, 30 Jan 2019 10:13:36 +0200
From: Mike Rapoport <rppt@...ux.ibm.com>
To: Andrea Arcangeli <aarcange@...hat.com>
Cc: lsf-pc@...ts.linux-foundation.org, linux-mm@...ck.org,
linux-kernel@...r.kernel.org, Peter Xu <peterx@...hat.com>,
Blake Caldwell <blake.caldwell@...orado.edu>,
Mike Rapoport <rppt@...ux.vnet.ibm.com>,
Mike Kravetz <mike.kravetz@...cle.com>,
Michal Hocko <mhocko@...nel.org>, Mel Gorman <mgorman@...e.de>,
Vlastimil Babka <vbabka@...e.cz>,
David Rientjes <rientjes@...gle.com>,
Andrei Vagin <avagin@...il.com>,
Pavel Emelyanov <xemul@...tuozzo.com>
Subject: [LSF/MM TOPIC]: userfaultfd (was: [LSF/MM TOPIC] NUMA remote THP vs
NUMA local non-THP under MADV_HUGEPAGE)
Hi,
(changed the subject and added CRIU folks)
On Tue, Jan 29, 2019 at 06:40:58PM -0500, Andrea Arcangeli wrote:
> Hello,
>
> --
>
> In addition to the above "NUMA remote THP vs NUMA local non-THP
> tradeoff" topic, there are other developments in "userfaultfd" land that
> are approaching merge readiness and that would be possible to provide a
> short overview about:
>
> - Peter Xu made significant progress in finalizing the userfaultfd-WP
> support over the last few months. That feature was planned from the
> start and it will allow userland to do some new things that weren't
> possible to achieve before. In addition to synchronously blocking
> write faults to be resolved by an userland manager, it has also the
> ability to obsolete the softdirty feature, because it can provide
> the same information, but with O(1) complexity (as opposed of the
> current softdirty O(N) complexity) similarly to what the Page
> Modification Logging (PML) does in hardware for EPT write accesses.
We (CRIU) have some concerns about obsoleting soft-dirty in favor of
uffd-wp. If there are other soft-dirty users these concerns would be
relevant to them as well.
With soft-dirty we collect the information about the changed memory every
pre-dump iteration in the following manner:
* freeze the tasks
* find entries in /proc/pid/pagemap with SOFT_DIRTY set
* unfreeze the tasks
* dump the modified pages to disk/remote host
While we do need to traverse the /proc/pid/pagemap to identify dirty pages,
in between the pre-dump iterations and during the actual memory dump the
tasks are running freely.
If we are to switch to uffd-wp, every write by the snapshotted/migrated
task will incur latency of uffd-wp processing by the monitor.
We'd need to see how this affects overall slowdown of the workload under
migration before moving forward with obsoleting soft-dirty.
> - Blake Caldwell maintained the UFFDIO_REMAP support to atomically
> remove memory from a mapping with userfaultfd (which can't be done
> with a copy as in UFFDIO_COPY and it requires a slow TLB flush to be
> safe) as an alternative to host swapping (which of course also
> requires a TLB flush for similar reasons). Notably UFFDIO_REMAP was
> rightfully naked early on and quickly replaced by UFFDIO_COPY which
> is more optimal to add memory to a mapping is small chunks, but we
> can't remove memory with UFFDIO_COPY and UFFDIO_REMAP should be as
> efficient as it gets when it comes to removing memory from a
> mapping.
If we are to discuss userfaultfd, I'd like also to bring the subject of COW
mappings.
The pages populated with UFFDIO_COPY cannot be COW-shared between related
processes which unnecessarily increases memory footprint of a migrated
process tree.
I've posted a patch [1] a (real) while ago, but nobody reacted and I've put
this aside.
Maybe it's time to discuss it again :)
> Thank you,
> Andrea
>
[1] https://lwn.net/ml/linux-api/20180328101729.GB1743%40rapoport-lnx/
--
Sincerely yours,
Mike.
Powered by blists - more mailing lists