linux-kernel - Re: ioremap_uc() followed by set_memory_wc()

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <20150422170731.GH5622@wotan.suse.de>
Date:	Wed, 22 Apr 2015 19:07:31 +0200
From:	"Luis R. Rodriguez" <mcgrof@...e.com>
To:	Andy Lutomirski <luto@...capital.net>
Cc:	Jason Gunthorpe <jgunthorpe@...idianresearch.com>,
	Mike Marciniszyn <mike.marciniszyn@...el.com>,
	Mike Marciniszyn <infinipath@...el.com>,
	linux-rdma@...r.kernel.org, Andy Walls <awalls@...metrocast.net>,
	Toshi Kani <toshi.kani@...com>,
	"H. Peter Anvin" <hpa@...or.com>, Ingo Molnar <mingo@...nel.org>,
	"linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
	Hal Rosenstock <hal.rosenstock@...il.com>,
	Sean Hefty <sean.hefty@...el.com>,
	Suresh Siddha <sbsiddha@...il.com>,
	Rickard Strandqvist <rickard_strandqvist@...ctrumdigital.se>,
	Roland Dreier <roland@...estorage.com>,
	Juergen Gross <jgross@...e.com>,
	Mauro Carvalho Chehab <mchehab@....samsung.com>,
	Borislav Petkov <bp@...e.de>, Mel Gorman <mgorman@...e.de>,
	Vlastimil Babka <vbabka@...e.cz>,
	Davidlohr Bueso <dbueso@...e.de>,
	Dave Hansen <dave.hansen@...ux.intel.com>,
	Jean-Christophe Plagniol-Villard <plagnioj@...osoft.com>,
	Thomas Gleixner <tglx@...utronix.de>,
	Ville Syrj?l? <syrjala@....fi>,
	Linux Fbdev development list <linux-fbdev@...r.kernel.org>,
	linux-media@...r.kernel.org, X86 ML <x86@...nel.org>,
	"Luis R. Rodriguez" <mcgrof@...not-panic.com>
Subject: Re: ioremap_uc() followed by set_memory_wc() - burrying MTRR

On Wed, Apr 22, 2015 at 09:53:03AM -0700, Andy Lutomirski wrote:
> On Wed, Apr 22, 2015 at 8:23 AM, Luis R. Rodriguez <mcgrof@...e.com> wrote:
> > On Tue, Apr 21, 2015 at 11:39:39PM -0600, Jason Gunthorpe wrote:
> >> On Wed, Apr 22, 2015 at 01:39:07AM +0200, Luis R. Rodriguez wrote:
> >> > > Mike, do you think the time is right to just remove the iPath driver?
> >> >
> >> > With PAT now being default the driver effectively won't work
> >> > with write-combining on modern kernels. Even if systems are old
> >> > they likely had PAT support, when upgrading kernels PAT will work
> >> > but write-combing won't on ipath.
> >>
> >> Sorry, do you mean the driver already doesn't get WC? Or do you mean
> >> after some more pending patches are applied?
> >
> > No, you have to consider the system used and the effects of calls used
> > on the driver in light of this table:
> >
> > ----------------------------------------------------------------------
> > MTRR Non-PAT   PAT    Linux ioremap value        Effective memory type
> > ----------------------------------------------------------------------
> >                                                   Non-PAT |  PAT
> >      PAT
> >      |PCD
> >      ||PWT
> >      |||
> > WC   000      WB      _PAGE_CACHE_MODE_WB            WC   |   WC
> > WC   001      WC      _PAGE_CACHE_MODE_WC            WC*  |   WC
> > WC   010      UC-     _PAGE_CACHE_MODE_UC_MINUS      WC*  |   UC
> > WC   011      UC      _PAGE_CACHE_MODE_UC            UC   |   UC
> > ----------------------------------------------------------------------
> >
> > (*) denotes implementation defined and is discouraged
> >
> > ioremap_nocache() will use _PAGE_CACHE_MODE_UC_MINUS by default today,
> > in the future we want to flip the switch and make _PAGE_CACHE_MODE_UC
> > the default. When that flip occurs it will mean ipath cannot get
> > write-combining on both non-PAT and PAT systems. Now that is for
> > the future, lets review the current situation for ipath.
> >
> > For PAT capable systems if mtrr_add() is used today on a Linux system on a
> > region mapped with ioremap_nocache() that will mean you effectively nullify the
> > mtrr_add() effect as the combinatorial effect above yields an effective memory
> > type of UC.
> 
> Are you sure?

Well lets double check.

>  I thought that ioremap_nocache currently is UC-,

It is.

> so mtrr_add + ioremap_nocache gets WC even on PAT systems.

https://www-ssl.intel.com/content/dam/www/public/us/en/documents/manuals/64-ia-32-architectures-software-developer-manual-325462.pdf

    As per Intel SDM "11.5.2.2 Selecting Memory Types for Pentium
    III and More Recent Processor Families" the ffect of a WC MTRR
    for a region with a PAT entry value of UC will be UC. The effect
    of a WC MTRR on a region with a PAT entry UC- will be WC. The
    effect of a WC MTRR on a regoin with PAT entry WC is WC.

And indeed as per table 11-7 mtrr WC on PAT UC- yields WC. So ineed the above
table needs adjustment for this. So for PAT systems write-combing would be
effective with mtrr_add(), but once strong UC (_PAGE_CACHE_MODE_UC) is used by
default for ioremap_nocache() what I mentioned will be true. Furhtermore if we
switch the drivers to use arch_phys_wc_add() then for sure write-combining will
also not be effective.

Jason, Andy, is the change still a reasonable compromise? We'd just be asking
users to boot with noat for users for ipath, ivtv until the drivers gets proper
PAT support with a split.

There are two motivations for this:

  * help move to strong UC as default
  * bury MTRR

> Going forward, when mtrr_add is gone, this will change, of course.

Indeed.

  Luis
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/