linux-kernel - MMU notifier callbacks and forking

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [day] [month] [year] [list]

Date:	Wed, 18 Apr 2012 10:51:01 -0400
From:	Kyle Hubert <khubert@...il.com>
To:	linux-kernel@...r.kernel.org
Subject: MMU notifier callbacks and forking

Hi,

I'm writing a driver that pins user memory and stores it in a page
table so the device's own IOMMU can translate into host physical
addresses. I built up the MMU notifier callbacks (invalidate_page and
invalidate_range_start) to unpin the pages when the VM state changes.
This works as I expect for mmap/munmap. However, when fork() occurs,
copy_page_range will call invalidate_range_[start|end] if the range is
a COW mapping. I think it makes sense because the page range of both
the parent and child will be changed to remove VM_MAYWRITE. If anyone
tries to write, we take a page fault, then copy and mark the page as
VM_MAYWRITE in both MMs. Do I understand this correctly?

My problem is, in my callbacks I can't distinguish between
invalidate_range_start indicating the page should be freed or the
pages permissions are changing. Is it sufficient to set the VM flags
to VM_SHARED? What is the API to do this (or is it just find_vma and
modify the vm_flags while holding the mmap_sem)? If I understand the
code, this will prevent invalidate_range_[start|end] from being called
when forking. Are there other side effects when setting this?

Thanks for your help,
-Kyle Hubert
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/