linux-kernel - Re: [PATCH RFC 3/3] virtio

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <87pqmwj3am.fsf@rustcorp.com.au>
Date:	Thu, 02 Jun 2011 13:24:57 +0930
From:	Rusty Russell <rusty@...tcorp.com.au>
To:	"Michael S. Tsirkin" <mst@...hat.com>, linux-kernel@...r.kernel.org
Cc:	Carsten Otte <cotte@...ibm.com>,
	Christian Borntraeger <borntraeger@...ibm.com>,
	linux390@...ibm.com, Martin Schwidefsky <schwidefsky@...ibm.com>,
	Heiko Carstens <heiko.carstens@...ibm.com>,
	Shirley Ma <xma@...ibm.com>, lguest@...ts.ozlabs.org,
	linux-kernel@...r.kernel.org,
	virtualization@...ts.linux-foundation.org, netdev@...r.kernel.org,
	linux-s390@...r.kernel.org, kvm@...r.kernel.org,
	Krishna Kumar <krkumar2@...ibm.com>,
	Tom Lendacky <tahm@...ux.vnet.ibm.com>, steved@...ibm.com,
	habanero@...ux.vnet.ibm.com
Subject: Re: [PATCH RFC 3/3] virtio_net: limit xmit polling

On Wed, 1 Jun 2011 12:50:03 +0300, "Michael S. Tsirkin" <mst@...hat.com> wrote:
> Current code might introduce a lot of latency variation
> if there are many pending bufs at the time we
> attempt to transmit a new one. This is bad for
> real-time applications and can't be good for TCP either.
> 
> Free up just enough to both clean up all buffers
> eventually and to be able to xmit the next packet.

OK, I found this quite confusing to read.

> -	while ((skb = virtqueue_get_buf(vi->svq, &len)) != NULL) {
> +	while ((r = virtqueue_min_capacity(vi->svq) < MAX_SKB_FRAGS + 2) ||
> +	       min_skbs-- > 0) {
> +		skb = virtqueue_get_buf(vi->svq, &len);
> +		if (unlikely(!skb))
> +			break;
>  		pr_debug("Sent skb %p\n", skb);
>  		vi->dev->stats.tx_bytes += skb->len;
>  		vi->dev->stats.tx_packets++;
>  		dev_kfree_skb_any(skb);
>  	}
> +	return r;
>  }

Gah... what a horrible loop.

Basically, this patch makes hard-to-read code worse, and we should try
to make it better.

Currently, xmit *can* fail when an xmit interrupt wakes the queue, but
the packet(s) xmitted didn't free up enough space for the new packet.
With indirect buffers this only happens if we hit OOM (and thus go to
direct buffers).

We could solve this by only waking the queue in skb_xmit_done if the
capacity is >= 2 + MAX_SKB_FRAGS.  But can we do it without a race?

If not, then I'd really prefer to see this, because I think it's clearer:

        // Try to free 2 buffers for every 1 xmit, to stay ahead.
        free_old_buffers(2)

        if (!add_buf()) {
                // Screw latency, free them all.
                free_old_buffers(UINT_MAX)
                // OK, this can happen if we are using direct buffers,
                // and the xmit interrupt woke us but the packets
                // xmitted were smaller than this one.  Rare though.
                if (!add_buf())
                        Whinge and stop queue, maybe loop.
        }

        if (capacity < 2 + MAX_SKB_FRAGS) {
                // We don't have enough for the next packet?  Try
                // freeing more.
                free_old_buffers(UINT_MAX);
                if (capacity < 2 + MAX_SKB_FRAGS) {
                        Stop queue, maybe loop.
        }

The current code makes my head hurt :(

Thoughts?
Rusty.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/