linux-kernel - Re: increased vmap_area_lock contentions on "n_tty: Move buffers into n_tty

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <5238F8AF.4050309@hurleysoftware.com>
Date:	Tue, 17 Sep 2013 20:49:51 -0400
From:	Peter Hurley <peter@...leysoftware.com>
To:	Fengguang Wu <fengguang.wu@...el.com>
CC:	Greg KH <gregkh@...uxfoundation.org>,
	LKML <linux-kernel@...r.kernel.org>, lkp@...org,
	Tejun Heo <tj@...nel.org>
Subject: Re: increased vmap_area_lock contentions on "n_tty: Move buffers
 into n_tty_data"

On 09/12/2013 09:09 PM, Fengguang Wu wrote:
> On Fri, Sep 13, 2013 at 08:51:33AM +0800, Fengguang Wu wrote:
>> Hi Peter,
>>
>> FYI, we noticed much increased vmap_area_lock contentions since this
>> commit:
>>
>> commit 20bafb3d23d108bc0a896eb8b7c1501f4f649b77
>> Author: Peter Hurley <peter@...leysoftware.com>
>> Date:   Sat Jun 15 10:21:19 2013 -0400
>>
>>      n_tty: Move buffers into n_tty_data
>>
>>      Reduce pointer reloading and improve locality-of-reference;
>>      allocate read_buf and echo_buf within struct n_tty_data.
>
> Here are some comparison between this commit [o] with its parent commit [*].

[...]

>      8cb06c983822103da1cf      20bafb3d23d108bc0a89
> ------------------------  ------------------------
>                    976.67      +108.3%      2034.67  lkp-a04/micro/netperf/120s-200%-TCP_STREAM
>                   8971.36       +11.4%      9997.05  nhm-white/micro/aim7/exec_test
>                   9948.03       +20.9%     12031.72  TOTAL slabinfo.kmalloc-128.active_objs
>
>      8cb06c983822103da1cf      20bafb3d23d108bc0a89
> ------------------------  ------------------------
>                    976.67      +108.3%      2034.67  lkp-a04/micro/netperf/120s-200%-TCP_STREAM
>                   9127.64       +11.4%     10164.15  nhm-white/micro/aim7/exec_test
>                  10104.31       +20.7%     12198.82  TOTAL slabinfo.kmalloc-128.num_objs

The dramatic increase in 128-byte kmalloc blocks is from vmalloc overhead
with associated with each allocation. On a x86_64, struct vmap_area is 104 bytes,
rounded to 128, allocated with every vmalloc allocation. This is approx 1% overhead
(which seems high to me).

The reason this is still visible after the test completes is the vmap area is
lazily reclaimed (see mm/vmalloc.c:__purge_vmap_area_lazy()).

1% memory overhead coupled with the unwanted vmap_area_lock contention (even though it is
test-induced) -- I might revert this anyway.

Regards,
Peter Hurley
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/