| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Message-ID: <00a76912-497f-482d-b2c2-675e34ea92c1@nvidia.com>
Date: Mon, 20 Oct 2025 15:08:16 -0700
From: John Hubbard <jhubbard@...dia.com>
To: Joel Fernandes <joelagnelf@...dia.com>, linux-kernel@...r.kernel.org,
rust-for-linux@...r.kernel.org, dri-devel@...ts.freedesktop.org,
dakr@...nel.org, acourbot@...dia.com
Cc: Alistair Popple <apopple@...dia.com>, Miguel Ojeda <ojeda@...nel.org>,
Alex Gaynor <alex.gaynor@...il.com>, Boqun Feng <boqun.feng@...il.com>,
Gary Guo <gary@...yguo.net>, bjorn3_gh@...tonmail.com,
Benno Lossin <lossin@...nel.org>, Andreas Hindborg <a.hindborg@...nel.org>,
Alice Ryhl <aliceryhl@...gle.com>, Trevor Gross <tmgross@...ch.edu>,
David Airlie <airlied@...il.com>, Simona Vetter <simona@...ll.ch>,
Maarten Lankhorst <maarten.lankhorst@...ux.intel.com>,
Maxime Ripard <mripard@...nel.org>, Thomas Zimmermann <tzimmermann@...e.de>,
Timur Tabi <ttabi@...dia.com>, joel@...lfernandes.org,
Elle Rhumsaa <elle@...thered-steel.dev>,
Daniel Almeida <daniel.almeida@...labora.com>, nouveau@...ts.freedesktop.org
Subject: Re: [PATCH 4/7] docs: gpu: nova-core: Document the PRAMIN aperture
mechanism
On 10/20/25 11:55 AM, Joel Fernandes wrote:
> While not terribly complicated, a little bit of documentation will help
> augment the code for this very important mechanism.
>
> Signed-off-by: Joel Fernandes <joelagnelf@...dia.com>
> ---
> Documentation/gpu/nova/core/pramin.rst | 113 +++++++++++++++++++++++++
> Documentation/gpu/nova/index.rst | 1 +
> 2 files changed, 114 insertions(+)
> create mode 100644 Documentation/gpu/nova/core/pramin.rst
Hi Joel,
Here's a more thorough review.
>
> diff --git a/Documentation/gpu/nova/core/pramin.rst b/Documentation/gpu/nova/core/pramin.rst
> new file mode 100644
> index 000000000000..19615e504db9
> --- /dev/null
> +++ b/Documentation/gpu/nova/core/pramin.rst
> @@ -0,0 +1,113 @@
> +.. SPDX-License-Identifier: GPL-2.0
> +
> +=========================
> +PRAMIN aperture mechanism
> +=========================
> +
> +.. note::
> + The following description is approximate and current as of the Ampere family.
> + It may change for future generations and is intended to assist in understanding
> + the driver code.
> +
> +Introduction
> +============
> +
> +PRAMIN is a hardware aperture mechanism that provides CPU access to GPU Video RAM (VRAM) before
> +the GPU's Memory Management Unit (MMU) and page tables are initialized. This 1MB sliding window,
> +located at a fixed offset within BAR0, is essential for setting up page tables and other critical
> +GPU data structures without relying on the GPU's MMU.
> +
> +Architecture Overview
> +=====================
> +
> +Logically, the PRAMIN aperture mechanism is implemented by the GPU's PBUS (PCIe Bus Controller Unit)
How about:
The PRAMIN aperture mechanism is logically implemented by the GPU's PBUS (PCIe Bus Controller Unit)
> +and provides a CPU-accessible window into VRAM through the PCIe interface::
> +
> + +-----------------+ PCIe +------------------------------+
> + | CPU |<----------->| GPU |
> + +-----------------+ | |
> + | +----------------------+ |
> + | | PBUS | |
> + | | (Bus Controller) | |
> + | | | |
> + | | +--------------.<------------ (window always starts at
> + | | | PRAMIN | | | BAR0 + 0x700000)
> + | | | Window | | |
> + | | | (1MB) | | |
> + | | +--------------+ | |
> + | | | | |
> + | +---------|------------+ |
> + | | |
> + | v |
> + | .----------------------.<------------ (Program PRAMIN to any
> + | | VRAM | | 64KB VRAM physical boundary)
> + | | (Several GBs) | |
> + | | | |
> + | | FB[0x000000000000] | |
> + | | ... | |
> + | | FB[0x7FFFFFFFFFF] | |
> + | +----------------------+ |
> + +------------------------------+
> +
> +PBUS (PCIe Bus Controller) among other things is responsible in the GPU for handling MMIO
How about:
PBUS (PCIe Bus Controller) is responsible for, among other things, handling MMIO
> +accesses to the BAR registers.
> +
> +PRAMIN Window Operation
> +=======================
> +
> +The PRAMIN window provides a 1MB sliding aperture that can be repositioned over
> +the entire VRAM address space using the NV_PBUS_BAR0_WINDOW register.
> +
> +Window Control Mechanism
> +-------------------------
> +
> +The window position is controlled via the PBUS BAR0_WINDOW register::
> +
> + NV_PBUS_BAR0_WINDOW Register
> + +-----+-----+--------------------------------------+
> + |31-26|25-24| 23-0 |
Should we use ":" notation here?
31:26 | 25:24 | 23:0
> + | |TARG | BASE_ADDR |
> + | | ET | (bits 39:16 of VRAM address) |
> + +-----+-----+--------------------------------------+
> +
> + TARGET field values:
> + - 0x0: VID_MEM (Video Memory / VRAM)
> + - 0x1: SYS_MEM_COHERENT (Coherent system memory)
> + - 0x2: SYS_MEM_NONCOHERENT (Non-coherent system memory)
These SYS_MEM_* items imply to the reader that PRAMIN can target
sysmem. However, pramin.rs in this series hard-codes this field
to VID_MEM:
let window_reg = regs::NV_PBUS_BAR0_WINDOW::default().set_window_addr(fb_offset);
It's not clear why that is done (I seem to recall a potential deadlock
in older chips, in similar situations, but that's probably stale info).
If, however, there is some limitation that prevents using SYS_MEM*, then
we should document it, probably here.
> +
> +64KB Alignment Requirement
> +---------------------------
> +
> +The PRAMIN window must be aligned to 64KB boundaries in VRAM. This is enforced
> +by the BASE_ADDR field representing bits [39:16] of the target address::
> +
> + VRAM Address Calculation:
> + actual_vram_addr = (BASE_ADDR << 16) + pramin_offset
> + Where:
> + - BASE_ADDR: 24-bit value from NV_PBUS_BAR0_WINDOW[23:0]
> + - pramin_offset: 20-bit offset within PRAMIN window [0x00000-0xFFFFF]
How about:
- pramin_offset: 20-bit offset within the PRAMIN window [0x00000-0xFFFFF]
> + Example Window Positioning:
> + +---------------------------------------------------------+
> + | VRAM Space |
> + | |
> + | 0x000000000 +-----------------+ <-- 64KB aligned |
> + | | PRAMIN Window | |
> + | | (1MB) | |
> + | 0x0000FFFFF +-----------------+ |
> + | |
> + | | ^ |
> + | | | Window can slide |
> + | v | to any 64KB boundary |
Maybe:
s/any 64KB boundary/any 64KB-aligned boundary/
> + | |
> + | 0x123400000 +-----------------+ <-- 64KB aligned |
> + | | PRAMIN Window | |
> + | | (1MB) | |
> + | 0x1234FFFFF +-----------------+ |
> + | |
> + | ... |
> + | |
> + | 0x7FFFF0000 +-----------------+ <-- 64KB aligned |
> + | | PRAMIN Window | |
> + | | (1MB) | |
> + | 0x7FFFFFFFF +-----------------+ |
> + +---------------------------------------------------------+
> diff --git a/Documentation/gpu/nova/index.rst b/Documentation/gpu/nova/index.rst
> index 46302daace34..e77d3ee336a4 100644
> --- a/Documentation/gpu/nova/index.rst
> +++ b/Documentation/gpu/nova/index.rst
> @@ -33,3 +33,4 @@ vGPU manager VFIO driver and the nova-drm driver.
> core/fwsec
> core/falcon
> core/msgq
> + core/pramin
thanks,
--
John Hubbard
Powered by blists - more mailing lists