lists.openwall.net   lists  /  announce  owl-users  owl-dev  john-users  john-dev  passwdqc-users  yescrypt  popa3d-users  /  oss-security  kernel-hardening  musl  sabotage  tlsify  passwords  /  crypt-dev  xvendor  /  Bugtraq  Full-Disclosure  linux-kernel  linux-netdev  linux-ext4  linux-hardening  linux-cve-announce  PHC 
Open Source and information security mailing list archives
 
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <20110301135053.GA6684@canuck.infradead.org>
Date:	Tue, 1 Mar 2011 08:50:53 -0500
From:	Thomas Graf <tgraf@...radead.org>
To:	Herbert Xu <herbert@...dor.apana.org.au>
Cc:	Eric Dumazet <eric.dumazet@...il.com>,
	David Miller <davem@...emloft.net>, rick.jones2@...com,
	therbert@...gle.com, wsommerfeld@...gle.com,
	daniel.baluta@...il.com, netdev@...r.kernel.org
Subject: Re: SO_REUSEPORT - can it be done in kernel?

On Tue, Mar 01, 2011 at 08:19:51PM +0800, Herbert Xu wrote:
> On Tue, Mar 01, 2011 at 07:18:29AM -0500, Thomas Graf wrote:
> >
> > 
> > ... makes it use CPU 5 for rxq2 and the qps goes up from 250kqps to 270kqps
> 
> I think the increase here comes from the larger number of packets
> in flight more than anything.
> 
> The bottleneck is still the TX queue (both software and hardware).

Disabled netfilter and reran test

Now does ~316kqps (rx was split over 2 queues)

----------------------------------------------------------------------------------------------------------------------
   PerfTop:   30608 irqs/sec  kernel:66.1%  exact:  0.0% [1000Hz cpu-clock-msecs],  (all, CPU: 1)
----------------------------------------------------------------------------------------------------------------------

             samples  pcnt function                      DSO
             _______ _____ _____________________________ ___________________________________________________________

            19237.00  5.6% _raw_spin_unlock_irqrestore   /lib/modules/2.6.38-rc5+/build/vmlinux                     
            17170.00  5.0% get_rx_page_info              /lib/modules/2.6.38-rc5+/kernel/drivers/net/benet/be2net.ko
            11411.00  3.3% be_poll_rx                    /lib/modules/2.6.38-rc5+/kernel/drivers/net/benet/be2net.ko
            11320.00  3.3% isc_rwlock_lock               /usr/lib64/libisc.so.62.0.1                                
            10669.00  3.1% __do_softirq                  /lib/modules/2.6.38-rc5+/build/vmlinux                     
            10655.00  3.1% get_page_from_freelist        /lib/modules/2.6.38-rc5+/build/vmlinux                     
             9523.00  2.8% intel_idle                    /lib/modules/2.6.38-rc5+/build/vmlinux                     
             8677.00  2.5% __udp4_lib_lookup             /lib/modules/2.6.38-rc5+/build/vmlinux                     
             8379.00  2.4% sock_queue_rcv_skb            /lib/modules/2.6.38-rc5+/build/vmlinux                     
             8226.00  2.4% sk_run_filter                 /lib/modules/2.6.38-rc5+/build/vmlinux                     
             6724.00  1.9% __netif_receive_skb           /lib/modules/2.6.38-rc5+/build/vmlinux                     
             6553.00  1.9% __alloc_skb                   /lib/modules/2.6.38-rc5+/build/vmlinux                     
             6205.00  1.8% udp_queue_rcv_skb             /lib/modules/2.6.38-rc5+/build/vmlinux                     
             6038.00  1.7% _raw_spin_lock                /lib/modules/2.6.38-rc5+/build/vmlinux                     
             5868.00  1.7% isc_rwlock_unlock             /usr/lib64/libisc.so.62.0.1                                
             5696.00  1.6% dns_rbt_findnode              /usr/lib64/libdns.so.69.0.1                                
             5647.00  1.6% read_tsc                      /lib/modules/2.6.38-rc5+/build/vmlinux                     
             5633.00  1.6% getnstimeofday                /lib/modules/2.6.38-rc5+/build/vmlinux                     
             5448.00  1.6% kmem_cache_alloc_node_trace   /lib/modules/2.6.38-rc5+/build/vmlinux                     
             5272.00  1.5% finish_task_switch            /lib/modules/2.6.38-rc5+/build/vmlinux                     
             4719.00  1.4% sock_def_readable             /lib/modules/2.6.38-rc5+/build/vmlinux                     
             4002.00  1.2% is_swiotlb_buffer             /lib/modules/2.6.38-rc5+/build/vmlinux                     
             3914.00  1.1% memcpy                        /lib/modules/2.6.38-rc5+/build/vmlinux                     
             3717.00  1.1% isc_stats_increment           /usr/lib64/libisc.so.62.0.1                                
             3706.00  1.1% __udp4_lib_rcv                /lib/modules/2.6.38-rc5+/build/vmlinux                     
             3653.00  1.1% ip_rcv                        /lib/modules/2.6.38-rc5+/build/vmlinux                     
             3598.00  1.0% kmem_cache_alloc_node         /lib/modules/2.6.38-rc5+/build/vmlinux                     
             3407.00  1.0% ip_route_input_common         /lib/modules/2.6.38-rc5+/build/vmlinux                     
             2683.00  0.8% be_post_rx_frags              /lib/modules/2.6.38-rc5+/kernel/drivers/net/benet/be2net.ko
             2666.00  0.8% __pthread_mutex_lock_internal /lib64/libpthread-2.12.so                                  
             2331.00  0.7% __phys_addr                   /lib/modules/2.6.38-rc5+/build/vmlinux                     
             2230.00  0.6% __alloc_pages_nodemask        /lib/modules/2.6.38-rc5+/build/vmlinux                     
             2023.00  0.6% dns_name_fullcompare          /usr/lib64/libdns.so.69.0.1                                
             1972.00  0.6% packet_rcv                    /lib/modules/2.6.38-rc5+/build/vmlinux                     
             1902.00  0.6% eth_type_trans                /lib/modules/2.6.38-rc5+/build/vmlinux                     
             1860.00  0.5% __pthread_mutex_unlock        /lib64/libpthread-2.12.so                                  
             1804.00  0.5% fget_light                    /lib/modules/2.6.38-rc5+/build/vmlinux                     
             1739.00  0.5% alloc_pages_current           /lib/modules/2.6.38-rc5+/build/vmlinux                     
             1736.00  0.5% dns_rbtnodechain_init         /usr/lib64/libdns.so.69.0.1                            

----------------------------------------------------------------------------------------------------------------------
   PerfTop:   29038 irqs/sec  kernel:48.0%  exact:  0.0% [1000Hz cpu-clock-msecs],  (all, CPU: 11)
----------------------------------------------------------------------------------------------------------------------

             samples  pcnt function                      DSO
             _______ _____ _____________________________ ___________________________________________________________

            12833.00  7.5% intel_idle                    /lib/modules/2.6.38-rc5+/build/vmlinux                     
            10771.00  6.3% isc_rwlock_lock               /usr/lib64/libisc.so.62.0.1                                
             8713.00  5.1% be_tx_compl_process           /lib/modules/2.6.38-rc5+/kernel/drivers/net/benet/be2net.ko
             6452.00  3.8% kfree                         /lib/modules/2.6.38-rc5+/build/vmlinux                     
             5935.00  3.5% skb_release_data              /lib/modules/2.6.38-rc5+/build/vmlinux                     
             5552.00  3.2% kmem_cache_free               /lib/modules/2.6.38-rc5+/build/vmlinux                     
             5292.00  3.1% isc_rwlock_unlock             /usr/lib64/libisc.so.62.0.1                                
             4893.00  2.9% dns_rbt_findnode              /usr/lib64/libdns.so.69.0.1                                
             4413.00  2.6% kfree_skb                     /lib/modules/2.6.38-rc5+/build/vmlinux                     
             3802.00  2.2% be_poll_tx_mcc                /lib/modules/2.6.38-rc5+/kernel/drivers/net/benet/be2net.ko
             3515.00  2.1% isc_stats_increment           /usr/lib64/libisc.so.62.0.1                                
             3016.00  1.8% _raw_spin_unlock_irqrestore   /lib/modules/2.6.38-rc5+/build/vmlinux                     
             2202.00  1.3% __do_softirq                  /lib/modules/2.6.38-rc5+/build/vmlinux                     
             2027.00  1.2% _raw_spin_lock                /lib/modules/2.6.38-rc5+/build/vmlinux                     
             1935.00  1.1% finish_task_switch            /lib/modules/2.6.38-rc5+/build/vmlinux                     
             1906.00  1.1% __pthread_mutex_lock_internal /lib64/libpthread-2.12.so                                  
             1837.00  1.1% dns_name_fullcompare          /usr/lib64/libdns.so.69.0.1                                
             1702.00  1.0% dns_rbtnodechain_init         /usr/lib64/libdns.so.69.0.1                                
             1561.00  0.9% fget_light                    /lib/modules/2.6.38-rc5+/build/vmlinux                     
             1559.00  0.9% dns_name_getlabelsequence     /usr/lib64/libdns.so.69.0.1                                
             1491.00  0.9% dns_name_equal                /usr/lib64/libdns.so.69.0.1                                
             1464.00  0.9% __pthread_mutex_unlock        /lib64/libpthread-2.12.so                                  
             1454.00  0.9% dns_acl_match                 /usr/lib64/libdns.so.69.0.1                                
             1293.00  0.8% dns_zone_attach               /usr/lib64/libdns.so.69.0.1                                
             1245.00  0.7% be_xmit                       /lib/modules/2.6.38-rc5+/kernel/drivers/net/benet/be2net.ko
             1159.00  0.7% dns_message_rendersection     /usr/lib64/libdns.so.69.0.1                                
             1115.00  0.7% isc___mempool_get             /usr/lib64/libisc.so.62.0.1                                
             1100.00  0.6% copy_user_generic_string      /lib/modules/2.6.38-rc5+/build/vmlinux                     
             1030.00  0.6% dns_name_fromwire             /usr/lib64/libdns.so.69.0.1                                
             1015.00  0.6% dns_name_hash                 /usr/lib64/libdns.so.69.0.1                                
             1013.00  0.6% isc_radix_search              /usr/lib64/libisc.so.62.0.1                                
              970.00  0.6% __ip_route_output_key         /lib/modules/2.6.38-rc5+/build/vmlinux                     
              917.00  0.5% fput                          /lib/modules/2.6.38-rc5+/build/vmlinux                     
              817.00  0.5% dev_queue_xmit                /lib/modules/2.6.38-rc5+/build/vmlinux                     
              812.00  0.5% sk_run_filter                 /lib/modules/2.6.38-rc5+/build/vmlinux                     
              806.00  0.5% avc_has_perm_noaudit          /lib/modules/2.6.38-rc5+/build/vmlinux                     
              802.00  0.5% sock_wfree                    /lib/modules/2.6.38-rc5+/build/vmlinux                     
              793.00  0.5% dns_name_towire               /usr/lib64/libdns.so.69.0.1                                
              754.00  0.4% sock_alloc_send_pskb          /lib/modules/2.6.38-rc5+/build/vmlinux                     
              752.00  0.4% dns_message_parse             /usr/lib64/libdns.so.69.0.1                                
              749.00  0.4% dns_rdata_towire              /usr/lib64/libdns.so.69.0.1                                
              728.00  0.4% dns_rdataset_init             /usr/lib64/libdns.so.69.0.1                                
              709.00  0.4% isc___mempool_put             /usr/lib64/libisc.so.62.0.1                                
              699.00  0.4% skb_release_head_state        /lib/modules/2.6.38-rc5+/build/vmlinux                     
              685.00  0.4% _raw_spin_lock_bh             /lib/modules/2.6.38-rc5+/build/vmlinux                     
              683.00  0.4% dns_name_concatenate          /usr/lib64/libdns.so.69.0.1                                
              678.00  0.4% __ip_append_data              /lib/modules/2.6.38-rc5+/build/vmlinux                     
              673.00  0.4% tick_nohz_stop_sched_tick     /lib/modules/2.6.38-rc5+/build/vmlinux                     
              662.00  0.4% sys_sendmsg                   /lib/modules/2.6.38-rc5+/build/vmlinux                     
              654.00  0.4% dns_compress_findglobal       /usr/lib64/libdns.so.69.0.1                                
              654.00  0.4% memcpy                        /lib64/libc-2.12.so                                        
              637.00  0.4% dns_compress_invalidate       /usr/lib64/libdns.so.69.0.1                                
              597.00  0.3% isc__buffer_init              /usr/lib64/libisc.so.62.0.1                                
              595.00  0.3% dns_zone_detach               /usr/lib64/libdns.so.69.0.1                                
WARNING: failed to keep up with mmap data.
WARNING: failed to keep up with mmap data.


----------------------------------------------------------------------------------------------------------------------
   PerfTop:   29539 irqs/sec  kernel:47.0%  exact:  0.0% [1000Hz cpu-clock-msecs],  (all, CPU: 11)
----------------------------------------------------------------------------------------------------------------------

             samples  pcnt function                      DSO
             _______ _____ _____________________________ ___________________________________________________________

            14478.00  7.5% intel_idle                    /lib/modules/2.6.38-rc5+/build/vmlinux                     
            12279.00  6.3% isc_rwlock_lock               /usr/lib64/libisc.so.62.0.1                                
             9844.00  5.1% be_tx_compl_process           /lib/modules/2.6.38-rc5+/kernel/drivers/net/benet/be2net.ko
             7368.00  3.8% kfree                         /lib/modules/2.6.38-rc5+/build/vmlinux                     
             6696.00  3.5% skb_release_data              /lib/modules/2.6.38-rc5+/build/vmlinux                     
             6240.00  3.2% kmem_cache_free               /lib/modules/2.6.38-rc5+/build/vmlinux                     
             6034.00  3.1% isc_rwlock_unlock             /usr/lib64/libisc.so.62.0.1                                
             5547.00  2.9% dns_rbt_findnode              /usr/lib64/libdns.so.69.0.1                                
             5012.00  2.6% kfree_skb                     /lib/modules/2.6.38-rc5+/build/vmlinux                     
             4290.00  2.2% be_poll_tx_mcc                /lib/modules/2.6.38-rc5+/kernel/drivers/net/benet/be2net.ko
             4024.00  2.1% isc_stats_increment           /usr/lib64/libisc.so.62.0.1                                
             3417.00  1.8% _raw_spin_unlock_irqrestore   /lib/modules/2.6.38-rc5+/build/vmlinux                     
             2470.00  1.3% __do_softirq                  /lib/modules/2.6.38-rc5+/build/vmlinux                     
             2312.00  1.2% _raw_spin_lock                /lib/modules/2.6.38-rc5+/build/vmlinux                     
             2138.00  1.1% finish_task_switch            /lib/modules/2.6.38-rc5+/build/vmlinux                     
             2136.00  1.1% __pthread_mutex_lock_internal /lib64/libpthread-2.12.so                                  
             2061.00  1.1% dns_name_fullcompare          /usr/lib64/libdns.so.69.0.1                                
             1961.00  1.0% dns_rbtnodechain_init         /usr/lib64/libdns.so.69.0.1                                
             1797.00  0.9% dns_name_getlabelsequence     /usr/lib64/libdns.so.69.0.1                                
             1743.00  0.9% fget_light                    /lib/modules/2.6.38-rc5+/build/vmlinux                     
             1723.00  0.9% dns_name_equal                /usr/lib64/libdns.so.69.0.1                                
             1673.00  0.9% __pthread_mutex_unlock        /lib64/libpthread-2.12.so                                  
             1671.00  0.9% dns_acl_match                 /usr/lib64/libdns.so.69.0.1                                
             1488.00  0.8% dns_zone_attach               /usr/lib64/libdns.so.69.0.1                                
             1428.00  0.7% be_xmit                       /lib/modules/2.6.38-rc5+/kernel/drivers/net/benet/be2net.ko
             1369.00  0.7% dns_message_rendersection     /usr/lib64/libdns.so.69.0.1                                
             1278.00  0.7% isc___mempool_get             /usr/lib64/libisc.so.62.0.1                                
             1251.00  0.6% copy_user_generic_string      /lib/modules/2.6.38-rc5+/build/vmlinux                     
             1193.00  0.6% dns_name_fromwire             /usr/lib64/libdns.so.69.0.1                                
             1182.00  0.6% isc_radix_search              /usr/lib64/libisc.so.62.0.1                        
--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Powered by blists - more mailing lists

Powered by Openwall GNU/*/Linux Powered by OpenVZ