linux-kernel - Re: [PATCH 4/9] dmaengine: dw-edma: HDMA: Add memory barrier before starting the DMA transfer in remote setup

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20230621151948.36125997@kmaincent-XPS-13-7390>
Date:   Wed, 21 Jun 2023 15:19:48 +0200
From:   Köry Maincent <kory.maincent@...tlin.com>
To:     Serge Semin <fancer.lancer@...il.com>
Cc:     Cai Huoqing <cai.huoqing@...ux.dev>,
        Manivannan Sadhasivam <mani@...nel.org>,
        Vinod Koul <vkoul@...nel.org>,
        Gustavo Pimentel <Gustavo.Pimentel@...opsys.com>,
        dmaengine@...r.kernel.org, linux-kernel@...r.kernel.org,
        Thomas Petazzoni <thomas.petazzoni@...tlin.com>,
        Herve Codina <herve.codina@...tlin.com>
Subject: Re: [PATCH 4/9] dmaengine: dw-edma: HDMA: Add memory barrier before
 starting the DMA transfer in remote setup

On Wed, 21 Jun 2023 12:45:35 +0300
Serge Semin <fancer.lancer@...il.com> wrote:

> > I thought that using a read will solve the issue like the gpio_nand driver
> > (gpio_nand_dosync)   
> 
> AFAICS The io_sync dummy-read there is a workaround to fix the
> bus-reordering within the SoC bus. In this case we have a PCIe bus
> which is supposed to guarantee the strong order with the exception I
> described above or unless there is a bug someplace in the PCIe fabric.
> 
> > but I didn't thought of a cache that could return the value
> > of the read even if the write doesn't fully happen. In the case of a cache
> > how could we know that the write is done without using a delay?   
> 
> MMIO mapping is platform dependent and low-level driver dependent.
> That's why I asked many times about the platform you are using and the
> low-level driver that probes the eDMA engine. It would be also useful
> to know what PCIe host controller is utilized too.
> 
> Mainly MMIO spaces are mapped in a way to bypass the caching. But in
> some cases it might be useful to map an MMIO space with additional
> optimizations like Write-combining. For instance it could be
> effectively done for the eDMA linked-list BAR mapping. Indeed why
> would you need to send each linked-list byte/word/dword right away to
> the device while you can combine them and send all together, then
> flush the cache and only after that start the DMA transfer? Another
> possible reason of the writes reordering could be in a way the PCIe
> host outbound memory window (a memory region accesses to which are
> translated to the PCIe bus transfers) is configured. For instance DW
> PCIe Host controller outbound MW config CSR has a special flag which
> enables setting a custom PCIe bus TLPs (packets) attribute. As I
> mentioned above that attribute can affect the TLPs order: make it
> relaxed or ID-based.
> 
> Of course we can't reject a possibility of having some delays hidden
> inside your device which may cause writes to the internal memory
> landing after the writes to the CSRs. But that seems too exotic to be
> considered as the real one for sure until the alternatives are
> thoroughly checked.
> 
> What I was trying to say that your problem can be caused by some much
> more frequently met reason. If I were you I would have checked them
> first and only then considered a workaround like you suggest.

Thanks for you detailed answer, this was instructive.
I will come back with more information if TLP flags are set.
FYI the PCIe board I am currently working with is the one from Brainchip:
Here is the driver:
https://github.com/Brainchip-Inc/akida_dw_edma

Köry