netdev - Re: [PATCH] net: reduce number of reference taken on sk

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:	Sun, 10 May 2009 09:09:18 +0200
From:	Eric Dumazet <dada1@...mosbay.com>
To:	David Miller <davem@...emloft.net>
CC:	khc@...waw.pl, netdev@...r.kernel.org
Subject: Re: [PATCH] net: reduce number of reference taken on sk_refcnt

David Miller a écrit :
> From: David Miller <davem@...emloft.net>
> Date: Sat, 09 May 2009 13:34:54 -0700 (PDT)
> 
>> Consider the case where we always send some message on CPU A and
>> then process the ACK on CPU B.  We'll always be cancelling the
>> timer on a foreign cpu.
> 
> I should also mention that TCP has a peculiar optimization of timers
> that is likely being thwarted by your workload.  It never deletes
> timers under normal operation, it simply lets them still expire
> and the handler notices that there is "nothing to do" and returns.

Yes, you refer to INET_CSK_CLEAR_TIMERS condition, never set.

> 
> But when the connection does shut down, we have to purge all of
> these timers.
> 
> That could be another part of why you see timers in your profile.
> 
> 

Well, in my workload they should never expire, since application exchange
enough data on both direction, and they are no losses (Gigabit LAN context)

On machine acting as a server (the one I am focusing to, of course),
each incoming frame :

- Contains ACK for the previous sent frame
- Contains data provided by the client.
- Starts a timer for delayed ACK

Then server applications reacts and sends a new payload, and TCP stack
- Sends a frame including ACK for previous received frame
- Contains data provided by server application
- Starts a timer for retransmiting this frame if no ACK is received later.

So yes, each incoming and each outgoing frame is going to call mod_timer()

Problem is that incoming process is done by CPU 0 (the one that is dedicated
to NAPI processing because of stress situation, cpu 100% in softirq land),
and outgoing processing done by other cpus in the machine.

offsetof(struct inet_connection_sock, icsk_retransmit_timer)=0x208
offsetof(struct inet_connection_sock, icsk_delack_timer)=0x238

So there are cache line ping-pongs, but oprofile seems to point
to a spinlock contention in lock_timer_base(), I dont know why...
shouldnt (in my workload) delack_timer all belongs to cpu 0, and 
retransmit_timers to other cpus ? 

Or is mod_timer never migrates an already established timer ?

That would explain the lock contention on timer_base, we should
take care of it if possible.

Thanks David


--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html