linux-kernel - [PATCH 1/6] nova-core: doc: Add code comments related to devinit

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <20250423225405.139613-2-joelagnelf@nvidia.com>
Date: Wed, 23 Apr 2025 18:53:57 -0400
From: Joel Fernandes <joelagnelf@...dia.com>
To: linux-kernel@...r.kernel.org,
	Danilo Krummrich <dakr@...nel.org>,
	David Airlie <airlied@...il.com>,
	Simona Vetter <simona@...ll.ch>
Cc: nouveau@...ts.freedesktop.org,
	dri-devel@...ts.freedesktop.org,
	Alexandre Courbot <acourbot@...dia.com>,
	John Hubbard <jhubbard@...dia.com>,
	Shirish Baskaran <sbaskaran@...dia.com>,
	Alistair Popple <apopple@...dia.com>,
	Timur Tabi <ttabi@...dia.com>,
	Ben Skeggs <bskeggs@...dia.com>,
	Joel Fernandes <joelagnelf@...dia.com>
Subject: [PATCH 1/6] nova-core: doc: Add code comments related to devinit

Add several code comments to reduce acronym soup and explain how devinit magic
and bootflow works before driver loads. These are essential for debug and
development of the nova driver.

Signed-off-by: Joel Fernandes <joelagnelf@...dia.com>
---
 drivers/gpu/nova-core/devinit.rs | 36 ++++++++++++++++++++++++++++----
 drivers/gpu/nova-core/regs.rs    | 17 +++++++++++++--
 2 files changed, 47 insertions(+), 6 deletions(-)

diff --git a/drivers/gpu/nova-core/devinit.rs b/drivers/gpu/nova-core/devinit.rs
index ee5685aff845..890bdc4d9522 100644
--- a/drivers/gpu/nova-core/devinit.rs
+++ b/drivers/gpu/nova-core/devinit.rs
@@ -1,6 +1,21 @@
 // SPDX-License-Identifier: GPL-2.0
 
 //! Methods for device initialization.
+//!
+//! A clarification about devinit terminology:
+//! devinit is a sequence of register read/writes after reset that performs tasks
+//! such as:
+//! 1. Programming VRAM memory controller timings.
+//! 2. Power sequencing.
+//! 3. Clock and PLL configuration.
+//! 4. Thermal management.
+//! 5. Performs VRAM memory scrubbing (ECC initialization) - on some GPUs it scrubs
+//!    only part of memory and then kicks of 'async scrubbing'.
+//!
+//! devinit itself is a 'script' which is interpreted by the PMU microcontroller of
+//! the GPU by an interpreter program.
+//!
+//! Note that the devinit sequence also needs to run during suspend/resume at runtime.
 
 use kernel::bindings;
 use kernel::devres::Devres;
@@ -9,15 +24,28 @@
 use crate::driver::Bar0;
 use crate::regs;
 
-/// Wait for devinit FW completion.
+/// Wait for gfw (GPU firmware) boot completion signal (GFW_BOOT).
 ///
-/// Upon reset, the GPU runs some firmware code to setup its core parameters. Most of the GPU is
-/// considered unusable until this step is completed, so it must be waited on very early during
-/// driver initialization.
+/// Upon reset, several microcontrollers (such as PMU, SEC2, GSP etc) on the GPU run some GPU
+/// firmware (gfw) code to setup its core parameters. Most of the GPU is considered unusable until
+/// this step is completed, so it must be waited on very early during driver initialization.
+///
+/// The GPU firmware (gfw) code includes several components that execute before the driver loads.
+/// These components are located in the VBIOS ROM and are executed in a sequence on these different
+/// microcontrollers. The devinit sequence itself runs on the PMU, and the FWSEC runs on the GSP.
+///
+/// This function specifically waits for a signal indicating core initialization is complete before
+/// which not much can be done. This signal is setup by the FWSEC running on the GSP in high secure
+/// mode.
 pub(crate) fn wait_gfw_boot_completion(bar: &Devres<Bar0>) -> Result<()> {
     let mut timeout = 2000;
 
     loop {
+        // Before accessing the completion status in PGC6_AON_SECURE_SCRATCH_GROUP_05, we must first
+        // check PGC6_AON_SECURE_SCRATCH_GROUP_05_PRIV_LEVEL_MASK. This is because the register is
+        // accessible only after secure firmware (FWSEC) lowers the privilege level to allow CPU
+        // (LS/low-secure) access. We can only safely read the status register from CPU (LS/low-secure)
+        // once the mask indicates the privilege level has been appropriately lowered.
         let gfw_booted = with_bar!(
             bar,
             |b| regs::Pgc6AonSecureScratchGroup05PrivLevelMask::read(b)
diff --git a/drivers/gpu/nova-core/regs.rs b/drivers/gpu/nova-core/regs.rs
index eae5b7c13155..f4a6a382e83f 100644
--- a/drivers/gpu/nova-core/regs.rs
+++ b/drivers/gpu/nova-core/regs.rs
@@ -59,13 +59,26 @@
     31:4    hi_val => as u32
 );
 
-/* GC6 */
-
+/*
+ * GC6:
+ *
+ * GC6 is a GPU low-power state where VRAM is in self-refresh and the GPU
+ * is powered down (except for power rails needed to keep self-refresh working).
+ *
+ * These scratch registers are "always-on" even in a low-power state and have a
+ * designated group number.
+ */
+
+/// Privilege level mask register (PGC6_AON_SECURE_SCRATCH_GROUP_05_PRIV_LEVEL_MASK)
+/// which dictates whether the host CPU has privilege to access to the
+/// PGC6_AON_SECURE_SCRATCH_GROUP_05 register.
 register!(Pgc6AonSecureScratchGroup05PrivLevelMask@...0118128;
     0:0     read_protection_level0_enabled => as_bit bool
 );
 
 /* TODO: This is an array of registers. */
+/// PGC6_AON_SECURE_SCRATCH_GROUP_05 scratch register used to check for
+/// GFW boot completion status.
 register!(Pgc6AonSecureScratchGroup05@...0118234;
     31:0    value => as u32
 );
-- 
2.43.0