netdev - Re: [PATCH 0/3] x86_64, sfc: 128-bit memory-mapped I/O

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <1345601423.2659.100.camel@bwh-desktop.uk.solarflarecom.com>
Date:	Wed, 22 Aug 2012 03:10:23 +0100
From:	Ben Hutchings <bhutchings@...arflare.com>
To:	"H. Peter Anvin" <hpa@...or.com>
CC:	Thomas Gleixner <tglx@...utronix.de>,
	Ingo Molnar <mingo@...hat.com>, <netdev@...r.kernel.org>,
	<linux-net-drivers@...arflare.com>, <x86@...nel.org>
Subject: Re: [PATCH 0/3] x86_64, sfc: 128-bit memory-mapped I/O

On Tue, 2012-08-21 at 18:59 -0700, H. Peter Anvin wrote:
> On 08/21/2012 06:43 PM, Ben Hutchings wrote:
> > On Tue, 2012-08-21 at 18:38 -0700, H. Peter Anvin wrote:
> >> On 08/21/2012 06:17 PM, Ben Hutchings wrote:
> >>> Current Solarflare network controllers have 128-bit memory-mapped
> >>> registers which are normally accessed through a series of I/O
> >>> operations.  However, it is also possible to access them with a single
> >>> MOVAPS instruction on x86_64, and this is measurably faster as it
> >>> requires only one PCIe transaction.
> >>
> >> Also, have you considered doing this with write combining instead?
> > 
> > We tried it, and it goes horribly wrong.  On some systems, the writes
> > are not combined, but they are reordered in a way the hardware doesn't
> > support.  See the comment at the top of drivers/net/ethernet/sfc/io.h.
> > 
> 
> Yes, you have to make sure you properly enforce the necessary ordering
> requirements manually (I think you can do that with sfence).

We did put an sfence after the writes to each register.  But some
systems only want to combine writes that cover an entire cache line, and
the writes covering a 128-bit register get broken back up into multiple
writes at the PCIe level.  And on some systems these are sent in
decreasing address order, which breaks the rules for writing to
TX_DESC_UPD.

To avoid this we'd have to put an sfence in between the writes to a
register, leaving us back where we started.

Ben.

-- 
Ben Hutchings, Staff Engineer, Solarflare
Not speaking for my employer; that's the marketing department's job.
They asked us to note that Solarflare product names are trademarked.

--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html