[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Date: Fri, 09 Oct 2009 03:46:09 +0200
From: Eric Dumazet <eric.dumazet@...il.com>
To: "David S. Miller" <davem@...emloft.net>
CC: Vegard Nossum <vegard.nossum@...il.com>,
Linux Netdev List <netdev@...r.kernel.org>,
Ingo Molnar <mingo@...e.hu>,
Christoph Lameter <cl@...ux-foundation.org>
Subject: Re: [PATCH] net: Fix struct sock bitfield annotation
Eric Dumazet a écrit :
>
> I am currently writing a tool to re-organize 'struct sock' fields,
> for net-next-2.6 using offsetof() macro, this is how I spot the problem.
>
For networking guys, here is the actual mess with "struct sock" on x86_64,
related to UDP handling (critical latencies for some people). We basically touch
all cache lines, in every paths, bad effects on SMP...
- rx softirq reception
- rx recvmsg() time
- tx sendto() time
- tx completion time
sizeof(struct sk_buff) =232 (0x e8)
sizeof(struct sock) =528 (0x210)
sizeof(struct inet_sock)=680 (0x2a8)
Following offsets in hexadecimal
offsetof(struct sock, sk_refcnt) = 10 (rw by reception of udp frame, __udp4_lib_lookup())
(unavoidable hot spot unfortunatly, but not anymore touched by sock_wfree())
offsetof(struct sock, sk_hash) = 14 (read rx softirq )
offsetof(struct sock, sk_family) = 18 (read " "
offsetof(struct inet_sock, daddr) =210 (read " "
offsetof(struct inet_sock, rcv_saddr) =214 (read " "
offsetof(struct inet_sock, dport) =218 (read " "
offsetof(struct inet_sock, mc_list) =240 (read " multicast reception
offsetof(struct sock, sk_bound_dev_if) = 1c (read rx softirq)
(First patch I'll submit is move daddr/rcv_saddr/dport to struct sock_common,
so that lookup() use one cache line instead of two per socket in hash chain)
offsetof(struct sock, sk_prot) = 30 (read by sk_has_account())
offsetof(struct sock, sk_rcvbuf) = 3c (read)
offsetof(struct sock, sk_protocol) = 39
offsetof(struct sock, sk_allocation) = e0 (read at send() time)
offsetof(struct sock, sk_flags) = f8 (read)
offsetof(struct sock, sk_lock) = 40 (rw by udp_sendmsg())
offsetof(struct sock, sk_dst_lock) = 90 (rw by udp_sendmsg() on connected socks)
offsetof(struct sock, sk_dst_cache) = 78 (read by udp_sendmsg() on connected socks)
offsetof(struct sock, sk_mark) =1d8 (read at sendto() time)
offsetof(struct sock, sk_write_queue) = c0 (rw by sendto())
offsetof(struct inet_sock, id) =232 (rw by sendto() on connected socks)
offsetof(struct sock, sk_wmem_alloc) = 98 (RW, both at sendto() and tx completion handler time)
offsetof(struct sock, sk_sndbuf) = a0 (read at tx completion time and sendto())
offsetof(struct sock, sk_sndmsg_page) =1b8 (rw by send())
offsetof(struct sock, sk_send_head) =1c0 (rw by send(), tcp)
offsetof(struct sock, sk_rmem_alloc) = 94 (RW, both when frame is received by softirq and dequeued at recvmsg() time)
offsetof(struct sock, sk_receive_queue)= a8 (RW, both when frame is received by softirq and dequeued at recvmsg() time)
offsetof(struct sock, sk_forward_alloc)= dc (rw rx softirq and recvmsg() time)
offsetof(struct sock, sk_drops) =134 (read when udp frame is received in softirq handler)
offsetof(struct sock, sk_stamp) =1a0 (write at recvmsg() time)
offsetof(struct sock, sk_sleep) = 70 (read by softirq handlers (rx/tx))
offsetof(struct sock, sk_filter) =160 (read when udp frame is received in softirq handler)
offsetof(struct sock, sk_socket) =1a8 (read)
offsetof(struct sock, sk_callback_lock)=128 (rw at softirq time
offsetof(struct sock, sk_data_ready) =1e8 (read)
offsetof(struct sock, sk_write_space) =1f0 (read at TX completion time)
used by TCP
offsetof(struct sock, sk_timer) =170
offsetof(struct sock, sk_ack_backlog) =138 (listen socks)
Almost never used
offsetof(struct sock, sk_lingertime) =100
offsetof(struct sock, sk_write_pending)=1cc
offsetof(struct sock, sk_prot_creator )=120
--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Powered by blists - more mailing lists