linux-kernel - Re: mmap/munmap latency on multithreaded apps, because pagefaults hold mmap

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <48CACCCD.3020400@gmail.com>
Date:	Fri, 12 Sep 2008 23:10:53 +0300
From:	Török Edwin <edwintorok@...il.com>
To:	Mike Waychison <mikew@...gle.com>
CC:	Andi Kleen <andi@...stfloor.org>, Theodore Tso <tytso@....edu>,
	Peter Zijlstra <peterz@...radead.org>,
	Ingo Molnar <mingo@...e.hu>,
	Linux Kernel <linux-kernel@...r.kernel.org>,
	"Thomas Gleixner mingo@...hat.com" <tglx@...utronix.de>,
	"H. Peter Anvin" <hpa@...or.com>,
	Benjamin Herrenschmidt <benh@...nel.crashing.org>,
	linux-mm@...ck.org
Subject: Re: mmap/munmap latency on multithreaded apps, because pagefaults
 hold mmap_sem during disk read

On 2008-09-12 22:35, Mike Waychison wrote:
> Török Edwin wrote:
>> On 2008-09-08 22:10, Török Edwin wrote:
>>> [snip]
>>> There is however a problem with mmap [mmap with N threads is as slow as
>>> mmap with 1 thread, i.e. it is sequential :(], pagefaults and disk I/O,
>>> I think I am hitting the problem described in this thread (2 years
>>> ago!)
>>> http://lwn.net/Articles/200215/
>>> http://lkml.org/lkml/2006/9/19/260
>>>
>>> It looks like such a patch is still not part of 2.6.27, what
>>> happened to it?
>>> I will see if that patch applies to 2.6.27, and will rerun my test with
>>> that patch applied too.
>>>   
>>
>> The patch doesn't apply to 2.6.27-rc6, I tried manually applying the
>> patch.
>> There have been many changes since 2.6.18 (like replacing find_get_page
>> with find_lock_page, filemap returning VM_FAULT codes, etc.).
>> I have probably done something wrong, because the resulting kernel won't
>> boot: I  get abnormal exits and random sigbus during boot.
>>
>> Can you please help porting the patch to 2.6.27-rc6? I have attached my
>> 2 attempts at the end of this mail.
>
> I actually have to forward port this and a bunch of other mm speed-ups
> in the coming two weeks, though they'll be ports from 2.6.18 to
> 2.6.26.  I'll be sending them out to linux-mm once I've done so and
> completed some testing.
>


That would be great, thanks!

>>
>> Also it looks like the original patch just releases the mmap_sem if
>> there is lock contention on the page, but keeps mmap_sem during read?
>> I would like mmap_sem be released during disk I/O too.
>
> The 'lock'ing of the page is the part that waits for the read to
> finish, and is the part that is contentious.

Didn't know that, thanks for explaining.

>
>>
>> I also tried changing i_mmap_lock into a semaphore, however I that won't
>> work since some users of i_mmap_lock can't sleep.
>> Taking the i_mmap_lock spinlock in filemap fault is also not possible,
>> since we would sleep while holding a spinlock.
>
> You shouldn't be seeing much contention on the i_mmap_lock, at least
> not in the fault path (it's mostly just painful when you have a lot of
> threads in direct reclaim and you have a large file mmaped).

I was thinking of using i_mmap_lock as an alternative to holding
mmap_sem, but it didn't seem like a good idea after all.

Best regards,
--Edwin
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/