linux-kernel - Re: [PATCH 00/17] RFC: userfault v2

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives
Hash Suite for Android: free password hash cracker in your pocket
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <54531258.1060908@huawei.com>
Date:	Fri, 31 Oct 2014 12:38:48 +0800
From:	zhanghailiang <zhang.zhanghailiang@...wei.com>
To:	Peter Feiner <pfeiner@...gle.com>
CC:	Andrea Arcangeli <aarcange@...hat.com>, <qemu-devel@...gnu.org>,
	<kvm@...r.kernel.org>, <linux-kernel@...r.kernel.org>,
	Andres Lagar-Cavilla <andreslc@...gle.com>,
	Dave Hansen <dave@...1.net>,
	Paolo Bonzini <pbonzini@...hat.com>,
	Rik van Riel <riel@...hat.com>, Mel Gorman <mgorman@...e.de>,
	Andy Lutomirski <luto@...capital.net>,
	Andrew Morton <akpm@...ux-foundation.org>,
	Sasha Levin <sasha.levin@...cle.com>,
	"Hugh Dickins" <hughd@...gle.com>,
	"Dr. David Alan Gilbert" <dgilbert@...hat.com>,
	Christopher Covington <cov@...eaurora.org>,
	Johannes Weiner <hannes@...xchg.org>,
	Android Kernel Team <kernel-team@...roid.com>,
	"Robert Love" <rlove@...gle.com>,
	Dmitry Adamushko <dmitry.adamushko@...il.com>,
	"Neil Brown" <neilb@...e.de>, Mike Hommey <mh@...ndium.org>,
	Taras Glek <tglek@...illa.com>, Jan Kara <jack@...e.cz>,
	KOSAKI Motohiro <kosaki.motohiro@...il.com>,
	Michel Lespinasse <walken@...gle.com>,
	"Minchan Kim" <minchan@...nel.org>,
	Keith Packard <keithp@...thp.com>,
	"Huangpeng (Peter)" <peter.huangpeng@...wei.com>,
	Isaku Yamahata <yamahata@...inux.co.jp>,
	Anthony Liguori <anthony@...emonkey.ws>,
	"Stefan Hajnoczi" <stefanha@...il.com>,
	Wenchao Xia <wenchaoqemu@...il.com>,
	"Andrew Jones" <drjones@...hat.com>,
	Juan Quintela <quintela@...hat.com>
Subject: Re: [PATCH 00/17] RFC: userfault v2

On 2014/10/31 11:29, zhanghailiang wrote:
> On 2014/10/31 10:23, Peter Feiner wrote:
>> On Thu, Oct 30, 2014 at 07:31:48PM +0800, zhanghailiang wrote:
>>> On 2014/10/30 1:46, Andrea Arcangeli wrote:
>>>> On Mon, Oct 27, 2014 at 05:32:51PM +0800, zhanghailiang wrote:
>>>>> I want to confirm a question:
>>>>> Can we support distinguishing between writing and reading memory for userfault?
>>>>> That is, we can decide whether writing a page, reading a page or both trigger userfault.
>>>> Mail is going to be long enough already so I'll just assume tracking
>>>> dirty memory in userland (instead of doing it in kernel) is worthy
>>>> feature to have here.
>>
>> I'll open that can of worms :-)
>>
>>> [...]
>>> Er, maybe i didn't describe clearly. What i really need for live memory snapshot
>>> is only wrprotect fault, like kvm's dirty tracing mechanism, *only tracing write action*.
>>>
>>> So, what i need for userfault is supporting only wrprotect fault. i don't
>>> want to get notification for non present reading faults, it will influence
>>> VM's performance and the efficiency of doing snapshot.
>>
>> Given that you do care about performance Zhanghailiang, I don't think that a
>> userfault handler is a good place to track dirty memory. Every dirtying write
>> will block on the userfault handler, which is an expensively slow proposition
>> compared to an in-kernel approach.
>>
>
> Agreed, but for doing live memory snapshot (VM is running when do snapsphot),
> we have to do this (block the write action), because we have to save the page before it
> is dirtied by writing action. This is the difference, compared to pre-copy migration.
>

Again;) For snapshot, i don't use its dirty tracing ability, i just use it to block write action,
and save page, and then i will remove its write protect.

>>> Also, i think this feature will benefit for migration of ivshmem and vhost-scsi
>>> which have no dirty-page-tracing now.
>>
>> I do agree wholeheartedly with you here. Manually tracking non-guest writes
>> adds to the complexity of device emulation code. A central fault-driven means
>> for dirty tracking writes from the guest and host would be a welcome
>> simplification to implementing pre-copy migration. Indeed, that's exactly what
>> I'm working on! I'm using the softdirty bit, which was introduced recently for
>> CRIU migration, to replace the use of KVM's dirty logging and manual dirty
>> tracking by the VMM during pre-copy migration. See
>
> Great! Do you plan to issue your patches to community? I mean is your work based on
> qemu? or an independent tool (CRIU migration?) for live-migration?
> Maybe i could fix the migration problem for ivshmem in qemu now,
> based on softdirty mechanism.
>
>> Documentation/vm/soft-dirty.txt and pagemap.txt in case you aren't familiar. To
>
> I have read them cursorily, it is useful for pre-copy indeed. But it seems that
> it can not meet my need for snapshot.
>
>> make softdirty usable for live migration, I've added an API to atomically
>> test-and-clear the bit and write protect the page.
>
> How can i find the API? Is it been merged in kernel's master branch already?
>
>
> Thanks,
> zhanghailiang
>
> --
> To unsubscribe from this list: send the line "unsubscribe kvm" in
> the body of a message to majordomo@...r.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> .
>

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/