linux-kernel - Re: problems doing direct dma from a pci device to pci-e device

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <4AAAAD30.4040704@compro.net>
Date:	Fri, 11 Sep 2009 16:04:00 -0400
From:	Mark Hounschell <markh@...pro.net>
To:	markh@...pro.net
CC:	Alan Cox <alan@...rguk.ukuu.org.uk>, linux-pci@...r.kernel.org,
	Mark Hounschell <dmarkh@....rr.com>,
	linux-kernel@...r.kernel.org
Subject: Re: problems doing direct dma from a pci device to pci-e device

Mark Hounschell wrote:
> Alan Cox wrote:
>>> I now have an AM3 based DFI DK 790FXB-M3H5 motherboard. This board has 3 regular
>>> PCI slots and 3 PCI-E (16x) slots. I also have a PCI-E (x4) version of the VMIC-5565
>>> reflective memory  card in one of the PCI-E slots and our gpiohsd card in one of the regular
>>> PCI slots. All on the motherboard. No expansion slots being used. However I cannot get
>>> data from our gpiohsd into the PCI-E VMIC-5565 cards memory. I can certainly get the data there
>>> from a userland buffer, no problem. Just not from one card to the other directly. Oh and when
>>> I put the regular PCI version of the VMIC into one of the regular PCI slots everything works
>>> as expected. They are then both on the same PCI bus and no bridges are involved though.
>> Have you verified with the vendor that such DMA works properly ? There is
>> a long history of there being boards where some device to device DMA
>> exploded or vanished. The arrival of PCI capture cards doing direct to
>> video DMA cleaned the world up (eg the BT848) but I wouldn't be suprised
>> if this recurred somewhere since they were popular and nobody really
>> noticed as they didn't run such an unusual config.
>>
> 
> I have not made an inquiry with the vendor yet. I'm pretty sure I have done 
> this with this very card in another mother board. I guess I need to dig that one 
> up and verify it before I go much further.  
> 
> But it's much like you describe above. The data seems to just vanish. I can't
> see any way to use my v-metro to find out where it's going either.
> 
>> Also does the board have a true IOMMU in the PCI-E side of the system ?
>> It's not a chipset I know.
>>
> 
> That I don't know for sure but I thought they all did??
> 

A quick readup on the IOMMU makes me think that if it does use one, that could certainly
cause my problem. I'm using bus addresses plugged into page tables that live in the gpiohsd card
obtained from virt2bus. They do match what lcpsi says they are. But it would appear from what
I read, that an IOMMU, could still be causing me grief if there is one being used. 
   
> I'll verify that the VMIC card is in fact capable in that other MB before
> I do anything else.

I still haven't done this yet. Maybe the MB this all worked on didn't have an IOMMU though?

I'll investigate futher.

Regards
Mark
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/