netdev - [RFC] fclone layout suboptimal

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Date:	Fri, 26 Sep 2014 06:07:39 -0700
From:	Eric Dumazet <eric.dumazet@...il.com>
To:	David Miller <davem@...emloft.net>
Cc:	netdev <netdev@...r.kernel.org>
Subject: [RFC] fclone layout suboptimal

Fast clones have following layout :

[sk_buff 1]
[sk_buff 2]
[atomic_t fclone_ref]


Main consumer is TCP stack for its write queue.

When tcp_ack()/tcp_clean_rtx_queue() frees skb,
kfree_skbmem() needs to fetch a cold cache line :

  0.72 │      je     b0
       │6f:   add    $0x8,%rsp
       │      pop    %rbx
  0.07 │      pop    %rbp
       │      retq
       │      nop
  0.52 │80:   lock   decl   0x1b0(%rbx)
 90.23 │   ┌──je     90
       │   │  jmp    6f
       │   │  nop
       │90:└─ mov    0x5c4e29(%rip),%rdi
       │      mov    %rbx,%rsi
       │      callq  kmem_cache_free
  1.37 │      add    $0x8,%rsp
       │      pop    %rbx
       │      pop    %rbp
  0.91 │      retq
       │      nop
       │b0:   mov    0x5c4e09(%rip),%rdi
       │      lea    -0xd8(%rbx),%rsi
       │      callq  kmem_cache_free

It might be better to have :

[sk_buff skb1]
[atomic_t fclone_ref]
[sk_buff skb2]

__alloc(skb) would not have to dirty a cache line to perform the
atomic_set(fclone_ref, 1);

kfree_skbmem() would do the atomic_dec_and_test() on a hot cache line
(Because we had access to skb_shinfo() a bit earlier while doing
skb_release_all()

When TX completions has to free the cloned sk_buff, fetching fclone_ref
would use an already hot cache line as well (skb2->next / skb2->sk are
already in cpu cache)



--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html