linux-kernel - [QUERY] Flushing cache from userspace using VFIO

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [thread-next>] [day] [month] [year] [list]
Message-ID: <d338414f-ed88-20d4-7da0-6742dedb8579@ti.com>
Date:   Mon, 6 Sep 2021 21:22:15 +0530
From:   Kishon Vijay Abraham I <kishon@...com>
To:     Cornelia Huck <cohuck@...hat.com>, <alex.williamson@...hat.com>
CC:     <kvm@...r.kernel.org>, Ohad Ben-Cohen <ohad@...ery.com>,
        Bjorn Andersson <bjorn.andersson@...aro.org>,
        Mathieu Poirier <mathieu.poirier@...aro.org>,
        linux-remoteproc <linux-remoteproc@...r.kernel.org>,
        "linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
        "Vutla, Lokesh" <lokeshvutla@...com>,
        Vignesh Raghavendra <vigneshr@...com>,
        "Strashko, Grygorii" <grygorii.strashko@...com>
Subject: [QUERY] Flushing cache from userspace using VFIO

Hi Alex, Cornelia,

I'm trying to see if I can use VFIO (Versatile Framework for userspace I/O
[1]) for communication between two cores within the same SoC. I've tried to put
down a picture like below which tries to communicate between ARM64 (running
Linux) and CORTEX R5 (running firmware). It uses rpmsg/remoteproc for the
control messages and the actual data buffers are directly accessed from the
userspace. The location of the data buffers can be informed to the userspace via
rpmsg_vfio (which has to be built as a rpmsg endpoint).

My question is after the userspace application in ARM64 writes to a buffer in
the SYSTEM MEMORY, can it flush it (through a VFIO IOCTL) before handing the
buffer to the CORTEX R5.

If it's implemented within kernel either we use dma_alloc_coherent() for
allocating coherent memory or streaming DMA APIs like
dma_map_single()/dma_unmap_single() for flushing/invalidate the cache.

Trying to see if that is already supported in VFIO or if not, would it be
acceptable to implement it.

Please let me know your thoughts.

┌───────────────────────────────────────────────────────────────────────────┐
│                                                                           │
│ ┌────────────────────┐                                                    │
│ │                    │                                                    │
│ │ ┌──────────────┐   │                                                    │
│ │ │ userspace    │   │  Data Buffers                                      │
│ │ │ Application  ├───┼──────────────┐                                     │
│ │ │              │   │              │                                     │
│ │ └──────▲──┬────┘   │              │                                     │
│ │        │  │        │              │                                     │
│ │        │  │ user   │              │                 ┌─────────────────┐ │
│ │  ──────┼──┼─────── │              │                 │                 │ │
│ │        │  │ kernel │              │                 │                 │ │
│ │  ┌─────┴──▼────┐   │    ┌─────────┼────────────┐    │                 │ │
│ │  │             │   │    │         │            │    │  Data           │ │
│ │  │  rpmsg_vfio │   │    │  ┌──────▼─────────┐  │    │  Buffers        │ │
│ │  │             │   │    │  │ Reserved Region◄──┼────┼────────┐        │ │
│ │  └─────▲──┬────┘   │    │  │                │  │    │        │        │ │
│ │        │  │        │    │  └────────────────┘  │    │        │        │ │
│ │  ┌─────┴──▼────┐   │    │                      │    │        │        │ │
│ │  │             │   │    │                      │    │ ┌──────┴──────┐ │ │
│ │  │  rpmsg      │   │    │     SYSTEM MEMORY    │    │ │ Application │ │ │
│ │  │             │   │    │       (DDR)          │    │ │   Logic     │ │ │
│ │  └─────▲──┬────┘   │    └──────────────────────┘    │ └───▲────┬────┘ │ │
│ │        │  │        │                                │     │    │      │ │
│ │  ┌─────┴──▼────┐   │Notify Firmware/Control Message │ ┌───┴────▼────┐ │ │
│ │  │             ├───┼────────────────────────────────┼─►             │ │ │
│ │  │  remoteproc │   │Interrupt ARM/Control Message   │ │   Firmware  │ │ │
│ │  │             ◄───┼────────────────────────────────┼─┤             │ │ │
│ │  └─────────────┘   │                                │ └─────────────┘ │ │
│ │      ARM64(Linux)  │                                │   ARM CORTEX R5 │ │
│ └────────────────────┘                                └─────────────────┘ │
│                                                                           │
│                                                                       SoC │
└───────────────────────────────────────────────────────────────────────────┘

Thank You,
Kishon

[1] -> https://youtu.be/WFkdTFTOTpA