Date:	Fri, 31 Oct 2014 16:11:24 +0800
From:	zhanghailiang <zhang.zhanghailiang@...wei.com>
To:	Andres Lagar-Cavilla <andreslc@...gle.com>
CC:	Peter Feiner <pfeiner@...gle.com>,
	Andrea Arcangeli <aarcange@...hat.com>,
	<qemu-devel@...gnu.org>, <kvm@...r.kernel.org>,
	<linux-kernel@...r.kernel.org>, Dave Hansen <dave@...1.net>,
	Paolo Bonzini <pbonzini@...hat.com>,
	Rik van Riel <riel@...hat.com>, Mel Gorman <mgorman@...e.de>,
	Andy Lutomirski <luto@...capital.net>,
	Andrew Morton <akpm@...ux-foundation.org>,
	Sasha Levin <sasha.levin@...cle.com>,
	"Hugh Dickins" <hughd@...gle.com>,
	"Dr. David Alan Gilbert" <dgilbert@...hat.com>,
	Christopher Covington <cov@...eaurora.org>,
	Johannes Weiner <hannes@...xchg.org>,
	Android Kernel Team <kernel-team@...roid.com>,
	"Robert Love" <rlove@...gle.com>,
	Dmitry Adamushko <dmitry.adamushko@...il.com>,
	"Neil Brown" <neilb@...e.de>, Mike Hommey <mh@...ndium.org>,
	Taras Glek <tglek@...illa.com>, Jan Kara <jack@...e.cz>,
	KOSAKI Motohiro <kosaki.motohiro@...il.com>,
	Michel Lespinasse <walken@...gle.com>,
	"Minchan Kim" <minchan@...nel.org>,
	Keith Packard <keithp@...thp.com>,
	"Huangpeng (Peter)" <peter.huangpeng@...wei.com>,
	Isaku Yamahata <yamahata@...inux.co.jp>,
	Anthony Liguori <anthony@...emonkey.ws>,
	"Stefan Hajnoczi" <stefanha@...il.com>,
	Wenchao Xia <wenchaoqemu@...il.com>,
	"Andrew Jones" <drjones@...hat.com>,
	Juan Quintela <quintela@...hat.com>
Subject: Re: [PATCH 00/17] RFC: userfault v2

On 2014/10/31 13:17, Andres Lagar-Cavilla wrote:
> On Thu, Oct 30, 2014 at 9:38 PM, zhanghailiang
> <zhang.zhanghailiang@...wei.com> wrote:
>> On 2014/10/31 11:29, zhanghailiang wrote:
>>>
>>> On 2014/10/31 10:23, Peter Feiner wrote:
>>>>
>>>> On Thu, Oct 30, 2014 at 07:31:48PM +0800, zhanghailiang wrote:
>>>>>
>>>>> On 2014/10/30 1:46, Andrea Arcangeli wrote:
>>>>>>
>>>>>> On Mon, Oct 27, 2014 at 05:32:51PM +0800, zhanghailiang wrote:
>>>>>>>
>>>>>>> I want to confirm a question:
>>>>>>> Can we support distinguishing between writing and reading memory for
>>>>>>> userfault?
>>>>>>> That is, we can decide whether writing a page, reading a page or both
>>>>>>> trigger userfault.
>>>>>>
>>>>>> Mail is going to be long enough already so I'll just assume tracking
>>>>>> dirty memory in userland (instead of doing it in kernel) is worthy
>>>>>> feature to have here.
>>>>
>>>>
>>>> I'll open that can of worms :-)
>>>>
>>>>> [...]
>>>>> Er, maybe i didn't describe clearly. What i really need for live memory
>>>>> snapshot
>>>>> is only wrprotect fault, like kvm's dirty tracing mechanism, *only
>>>>> tracing write action*.
>>>>>
>>>>> So, what i need for userfault is supporting only wrprotect fault. i
>>>>> don't
>>>>> want to get notification for non present reading faults, it will
>>>>> influence
>>>>> VM's performance and the efficiency of doing snapshot.
>>>>
>>>>
>>>> Given that you do care about performance Zhanghailiang, I don't think
>>>> that a
>>>> userfault handler is a good place to track dirty memory. Every dirtying
>>>> write
>>>> will block on the userfault handler, which is an expensively slow
>>>> proposition
>>>> compared to an in-kernel approach.
>>>>
>>>
>>> Agreed, but for doing live memory snapshot (VM is running when do
>>> snapsphot),
>>> we have to do this (block the write action), because we have to save the
>>> page before it
>>> is dirtied by writing action. This is the difference, compared to pre-copy
>>> migration.
>>>
>>
>> Again;) For snapshot, i don't use its dirty tracing ability, i just use it
>> to block write action,
>> and save page, and then i will remove its write protect.
>
> You could do a CoW in the kernel, post a notification, keep going, and
> expose an interface for user-space to mmap the preserved copy. Getting
> the life-cycle of the preserved page(s) right is tricky, but doable.
> Anyway, it's easy to hand-wave without knowing your specific
> requirements.
>

Yes, what i need is very much like user-space COW feature, but i don't want to modify
any code of kvm to relize COW, usefault is a more generic way and more grace.
Besides, I'm not an expert in kernel:(

> Opening the discussion a bit, this does look similar to the xen-access
> interface, in which a xen domain vcpu could be stopped in its tracks

Right;)

> while user-space was notified (and acknowledged) a variety of
> scenarios: page was written to, page was read from, vcpu is attempting
> to execute from page, etc. Very applicable to anti-viruses right away,
> for example you can enforce W^X properties on pages.
>
> I don't know that Andrea wants to open the game so broadly for
> userfault, and the code right now is very specific to triggering on
> pte_none(), but that's a nice reward down this road.
>

I hope he will consider it. IMHO, it is a good extension for userfault
(write fault);)

Best Regards,
zhanghailiang

>>
>>>>> Also, i think this feature will benefit for migration of ivshmem and
>>>>> vhost-scsi
>>>>> which have no dirty-page-tracing now.
>>>>
>>>>
>>>> I do agree wholeheartedly with you here. Manually tracking non-guest
>>>> writes
>>>> adds to the complexity of device emulation code. A central fault-driven
>>>> means
>>>> for dirty tracking writes from the guest and host would be a welcome
>>>> simplification to implementing pre-copy migration. Indeed, that's exactly
>>>> what
>>>> I'm working on! I'm using the softdirty bit, which was introduced
>>>> recently for
>>>> CRIU migration, to replace the use of KVM's dirty logging and manual
>>>> dirty
>>>> tracking by the VMM during pre-copy migration. See
>>>
>>>
>>> Great! Do you plan to issue your patches to community? I mean is your work
>>> based on
>>> qemu? or an independent tool (CRIU migration?) for live-migration?
>>> Maybe i could fix the migration problem for ivshmem in qemu now,
>>> based on softdirty mechanism.
>>>
>>>> Documentation/vm/soft-dirty.txt and pagemap.txt in case you aren't
>>>> familiar. To
>>>
>>>
>>> I have read them cursorily, it is useful for pre-copy indeed. But it seems
>>> that
>>> it can not meet my need for snapshot.
>>>
>>>> make softdirty usable for live migration, I've added an API to atomically
>>>> test-and-clear the bit and write protect the page.
>>>
>>>
>>> How can i find the API? Is it been merged in kernel's master branch
>>> already?
>>>
>>>
>>> Thanks,
>>> zhanghailiang
>>>
>>> --
>>> To unsubscribe from this list: send the line "unsubscribe kvm" in
>>> the body of a message to majordomo@...r.kernel.org
>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>> .
>>>
>>
>
>
>

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Powered by blists - more mailing lists