Date:	Fri, 23 Aug 2013 19:53:35 +0200
From:	wernerandy@....de
To:	"Andy Lutomirski" <luto@...capital.net>
Cc:	"Andreas Werner" <wernerandy@....de>,
	"linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>
Subject: Re: Re: question about ioremap_cache and PAT

> On Fri, Aug 23, 2013 at 9:59 AM, Andreas Werner <wernerandy@....de> wrote:
>>
>>
>> Hi,
>> thank you for your answer.
>>
>> So we are two persons for now who need WT :-)
>>
>> Im currently working on an ethernet driver for our own ETH core.
>> The problem is that one requirement is to not use DMA to transmit or
>> receive the data.
>> This means the that the ethernet buffer are not located in the main
>> memory. They are located in
>> the FPGA.
>>
>> To transmit or receive a frame, i have to read or write to mmio to get
>> the data.
>>
>> Intel has introduced the command "clflush" which can flush a cache line.
>> I wanted to activate the caches for those mmio (eth buffer) to speed up
>> the transmit or receive.
>> After that the transfer over PCIe uses burst read/write.
>>
>> The problem was if i set the buffer to Write-Back and call clflush on
>> those mmio-addresses, the system crashed without any output.
>> I found this articel
>> http://software.intel.com/en-us/forums/topic/393070.[http://software.intel.com/en-us/forums/topic/393070]
>>
>> After that i configured the transmit buffer to be Write-Combining (only
>> write to that adresses) using ioremap_wc, and
>> the receive buffer to be Write-Through (ioremap_cache + mtrr Write-Back
>> + my Kernel Hack :-)) everything worked fine.
>> The other configuration Register on the FPGA are just mapped with
>> ioremap.
>
> I'm curious: have you tried movntdqa on UC memory for this?
> (Certainly WP or WT is easier.)
>
> In any case, I hope to have patches to support WP and WT without using
> PAT reasonably soon.
>
>>
>> On PCIe Tracer i can see the burst read/write on my device.
>>
>> Is it possible to get hits into the Kernel?
>>
>> My modification in arch/x86/mm/pat.c:
>>
>> --- pat.c.orig 2013-02-03 01:18:49.491879407 +0100
>> +++ pat.c 2013-02-03 01:19:19.053509836 +0100
>> @@ -149,10 +149,16 @@ static unsigned long pat_x_mtrr_type(u64
>>    u8 mtrr_type;
>>
>>    mtrr_type = mtrr_type_lookup(start, end);
>> -  if (mtrr_type != MTRR_TYPE_WRBACK)
>> +
>> +  if (mtrr_type == MTRR_TYPE_WRTHROUGH) {
>> +   return _PAGE_CACHE_WB;
>> +  }
>> +  else if( mtrr_type == MTRR_TYPE_WRBACK )
>> +   return _PAGE_CACHE_WB;
>> +  else
>>     return _PAGE_CACHE_UC_MINUS;
>> -
>> -  return _PAGE_CACHE_WB;
>> +
>>   }
>>
>>   return req_type;
>>
>
> That seems more or less reasonable to me.  If you want it included,
> send it to x86@...nel.org (cc lkml) and see what they say.
>
> It would be prettier if you combined the conditions into a single
> if/else, though.
>
>>
>> Best regards.
>>
>>
>> Gesendet: Montag, 12. August 2013 um 19:53 Uhr
>> Von: "Andy Lutomirski" <luto@...capital.net>
>> An: "Andreas Werner" <wernerandy@....de>
>> Cc: linux-kernel@...r.kernel.org
>> Betreff: Re: question about ioremap_cache and PAT
>> On 08/11/2013 09:50 AM, Andreas Werner wrote:
>>> Hi i have a question about ioremap_cache and the resulting PAT
>>> attribute on X86 system. If I configure the mtrr to Write-Through for
>>> an adress range, and call ioremap_cache to map the mmio, the resulting
>>> PAT attribute is set to UC.
>>> If I check the Intel document IA-32 SDM vol 3a, the resulting PAT
>>> attribute should be WB.
>>>
>>> I found the function pat_x_mtrr_type in arch/x86/mm/pat.c where the
>>> resulting attribute is returned. There will be always UC return expect
>>> if the MTRR is set to WB.
>>>
>>> Why is there only WB or UC returned? In the Intel document there are a
>>> lot of combinations "allowed".
>>>
>>> I need a Attribute of WT, so what i did is to modify the
>>> pat_x_mtrr_type function to return also WB if the MTRR is set to WT.
>>>
>>> Is this a solution to solve that or whats the reasion why the kernel
>>> doesn´t support this combination?
>>
>> The kernel doesn't support it because I'm apparently the only person who
>> ever wanted it and I haven't implemented it yet.
>>
>> This stuff is handled in hardware, so modifying the kernel's idea of
>> what hardware does won't do much. Also, the kernel using MTRRs is on
>> its (very slow) way out. You could probably hack something up, but I
>> can almost guarantee that hpa, etc won't accept the patches.
>>
>> That being said, I'm planning to support WT directly using PAT in the
>> near future. This will work on most recent cpus (there are errata that
>> will prevent use of the high PAT entries on some cpus).
>>
>> What do you need WT for? I want it for NVDIMMs, and all I need to get
>> started now is a heatsink*, so I'll hopefully start implementing this
>> stuff in the next week or so.
>>
>> --Andy
>>
>> * Damnit, Intel, it's not 2003 any more. You already figured out that
>> heatsinks want screw holes. But why couldn't you make sure that all
>> so-called "LGA 2011" sockets have the screw holes in the same place?
>>
>>
>>>
>>> Best regards
>>>
>>> B
>>> B
>>> B
>>> B
>>> B
>>> B
>>> B
>>> B
>>> B
>>> B
>>> B
>>> B
>>> B
>>> B
>>> B
>>> B
>>> B
>>> B
>>> A
>>> A
>>> A
>>> A
>>> A
>>> A
>>> A
>>> A
>>> A
>>> A
>>> A
>>> A
>>> A
>>> A
>>> A
>>> B
>>> B
>>> B
>>> Best regards
>>>
>>
>
>
>
> --
> Andy Lutomirski
> AMA Capital Management, LLC
>


--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Powered by blists - more mailing lists