linux-kernel - Re: Ordering between PCI config space writes and MMIO reads?

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Date:	Wed, 1 Nov 2006 19:05:11 -0800
From:	Jeremy Higdon <jeremy@....com>
To:	Jack Steiner <steiner@....com>
Cc:	Matthew Wilcox <matthew@....cx>, Roland Dreier <rdreier@...co.com>,
	Jeff Garzik <jeff@...zik.org>,
	linux-pci@...ey.karlin.mff.cuni.cz, linux-ia64@...r.kernel.org,
	linux-kernel@...r.kernel.org, openib-general@...nib.org,
	John Partridge <johnip@....com>
Subject: Re: Ordering between PCI config space writes and MMIO reads?

On Tue, Oct 24, 2006 at 06:27:55PM -0500, Jack Steiner wrote:
> On Tue, Oct 24, 2006 at 04:36:32PM -0600, Matthew Wilcox wrote:
> > On Tue, Oct 24, 2006 at 02:51:30PM -0700, Roland Dreier wrote:
> > >  > I think the right way to fix this is to ensure mmio write ordering in
> > >  > the pci_write_config_*() implementations.  Like this.
> > > 
> > > I'm happy to fix this in the PCI core and not force drivers to worry
> > > about this.
> > > 
> > > John, can you confirm that this patch fixes the issue for you?
> > 
> > Hang on.  I wasn't thinking clearly.  mmiowb() only ensures the write
> > has got as far as the shub.  
> 
> I think mmiowb() should work on SN hardware. mmiowb() delays until shub
> reports that all previously issued PIO writes have completed. 
> 
> The processor "mf.a" guarantees "platform acceptance" which on SN means
> that shub has accepted the write - not that it has actually completed (or
> even forwarded anywhere by shub). That makes "mf.a" more-or-less useless
> on SN. However, shub has an additional MMR register (PIO_WRITE_COUNT) that
> counts actual outstanding PIOs.  mmiob() delays until that count goes to
> zero.
> 
> I'll check if there is any additional reordering that can occur AFTER the
> PIO_WRITE_COUNT goes to zero.  If so, it would be at bus level - not in
> shub or routers.

As I understand it, the mmiowb on the shub waits only for the PIO write
to be accepted by the destination node (shub or tio) that the I/O device
is attached to, thus guaranteeing that no reordering will happen within
the NL.

If the PPB can reorder the write, then mmiowb is not sufficient.  You'd
have to do a readback from a chip register (assuming you can trust the
PPB not to reorder reads and writes), or some other work around I haven't
thought of.

jeremy
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/