netdev - Re: [PATCH net-next-2.6] inetpeer: lower false sharing effect

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <1307680287.3210.2.camel@edumazet-laptop>
Date:	Fri, 10 Jun 2011 06:31:27 +0200
From:	Eric Dumazet <eric.dumazet@...il.com>
To:	Tim Chen <tim.c.chen@...ux.intel.com>
Cc:	David Miller <davem@...emloft.net>,
	netdev <netdev@...r.kernel.org>, Andi Kleen <andi@...stfloor.org>
Subject: Re: [PATCH net-next-2.6] inetpeer: lower false sharing effect

Le jeudi 09 juin 2011 à 17:03 -0700, Tim Chen a écrit :
> On Thu, 2011-06-09 at 08:26 +0200, Eric Dumazet wrote:
> > Profiles show false sharing in addr_compare() because refcnt/dtime
> > changes dirty the first inet_peer cache line, where are lying the keys
> > used at lookup time. If many cpus are calling inet_getpeer() and
> > inet_putpeer(), or need frag ids, addr_compare() is in 2nd position in
> > "perf top".
> > 
> 
> I've applied both inetpeer patches.  I also no longer have inet_getpeer
> and inet_putpeer and addr_compare in my profile.  Instead, neighbor
> lookup is now dominant.  See profile below.
> 
> When I retest with original 3.0-rc2 kernel, inet_putpeer no longer shows
> up, wonder if dst->peer was not set for some reason. 
> 
> Tim
> 
> -     27.06%     memcached  [kernel.kallsyms]             [k] atomic_add_unless.clone.34
>    - atomic_add_unless.clone.34
>       - 99.97% neigh_lookup
>            __neigh_lookup_errno.clone.17
>            arp_bind_neighbour
>            rt_intern_hash
>            __ip_route_output_key
>            ip_route_output_flow
>            udp_sendmsg
>            inet_sendmsg
>            __sock_sendmsg
>            sock_sendmsg
>            __sys_sendmsg
>            sys_sendmsg
>            system_call_fastpath
>            __sendmsg
> -     13.33%     memcached  [kernel.kallsyms]             [k] atomic_dec_and_test
>    - atomic_dec_and_test
>       - 99.89% dst_destroy
>          - dst_release
>             - 98.12% skb_dst_drop.clone.55
>                  dev_hard_start_xmit
>                + sch_direct_xmit
>             + 1.88% skb_release_head_state
> -      3.26%     memcached  [kernel.kallsyms]             [k] do_raw_spin_lock
>    - do_raw_spin_lock
>       - 92.24% _raw_spin_lock
>          + 41.39% sch_direct_xmit
> 
> 

Thanks Tim

I have some questions for further optimizations.

1) How many different destinations are used in your stress load ?
2) Could you provide a distribution of the size of packet lengthes ?
   Or maybe the average length would be OK




--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html