Message-ID: <ZxMFUpBnNTRC_cD3@Boquns-Mac-mini.local>
Date: Fri, 18 Oct 2024 18:03:14 -0700
From: Boqun Feng <boqun.feng@...il.com>
To: Lyude Paul <lyude@...hat.com>
Cc: rust-for-linux@...r.kernel.org, Danilo Krummrich <dakr@...hat.com>,
	airlied@...hat.com, Ingo Molnar <mingo@...hat.com>,
	Will Deacon <will@...nel.org>, Waiman Long <longman@...hat.com>,
	Peter Zijlstra <peterz@...radead.org>,
	Thomas Gleixner <tglx@...utronix.de>, linux-kernel@...r.kernel.org,
	Benno Lossin <benno.lossin@...ton.me>,
	Miguel Ojeda <ojeda@...nel.org>,
	Alex Gaynor <alex.gaynor@...il.com>, Gary Guo <gary@...yguo.net>,
	Björn Roy Baron <bjorn3_gh@...tonmail.com>,
	Andreas Hindborg <a.hindborg@...nel.org>,
	Alice Ryhl <aliceryhl@...gle.com>, Trevor Gross <tmgross@...ch.edu>,
	Martin Rodriguez Reboredo <yakoyoku@...il.com>,
	Valentin Obst <kernel@...entinobst.de>,
	Filipe Xavier <felipe_life@...e.com>, Fiona Behrens <me@...enk.dev>,
	Wedson Almeida Filho <walmeida@...rosoft.com>,
	Danilo Krummrich <dakr@...nel.org>
Subject: Re: [PATCH v8 2/3] rust: sync: Introduce lock::Backend::Context and
 lock::BackendWithContext

On Fri, Oct 18, 2024 at 07:22:26PM -0400, Lyude Paul wrote:
> Now that we've introduced an `IrqDisabled` token for marking contexts in
> which IRQs are disabled, we need a way to be able to pass it to locks that
> require that IRQs are disabled. In order to continue using the
> `lock::Backend` type instead of inventing our own thing, we accomplish this
> by adding the associated Context type, along with a `lock_with()` function
> that can accept a Context when acquiring a lock. To allow current users of
> context-less locks to keep using the normal `lock()` method, we take an
> example from Wedson Almeida Filho's work and add a `where T<'a>: Default`
> bound to `lock()` so that it can only be called on lock types where the
> context is simply a placeholder value, then re-implement it through the new
> `lock_with()` function.
> 
> We additionally add a BackendWithContext trait, to handle calling the
> various locking primatives necessary for these types - along with providing
> a `lock_with_new()` function for using those primitives and creating the
> relevant context tokens.
> 
> Signed-off-by: Lyude Paul <lyude@...hat.com>
> Suggested-by: Benno Lossin <benno.lossin@...ton.me>
> 
> ---
> 
> V3:
> * Use explicit lifetimes in lock_with() to ensure self and _context have
>   the same lifetime (Benno)
> * Use () for locks that don't need a Context instead of PhantomData (Benno)
> V4:
> * Fix typo (Dirk)
> V7:
> * Introduce BackendWithContext and lock_with_new following tglx's feedback
> * Name functions lock_with_context_saved and unlock_with_context_restored
> 
> Signed-off-by: Lyude Paul <lyude@...hat.com>
> ---
>  rust/kernel/sync/lock.rs          | 118 ++++++++++++++++++++++++++++--
>  rust/kernel/sync/lock/mutex.rs    |   1 +
>  rust/kernel/sync/lock/spinlock.rs |   1 +
>  3 files changed, 115 insertions(+), 5 deletions(-)
> 
> diff --git a/rust/kernel/sync/lock.rs b/rust/kernel/sync/lock.rs
> index 7b4859b05d3a7..8949a69dd53c5 100644
> --- a/rust/kernel/sync/lock.rs
> +++ b/rust/kernel/sync/lock.rs
> @@ -12,7 +12,7 @@
>      str::CStr,
>      types::{NotThreadSafe, Opaque, ScopeGuard},
>  };
> -use core::{cell::UnsafeCell, marker::PhantomPinned};
> +use core::{cell::UnsafeCell, marker::PhantomPinned, mem::ManuallyDrop};
>  use macros::pin_data;
>  
>  pub mod mutex;
> @@ -43,6 +43,11 @@ pub unsafe trait Backend {
>      /// [`unlock`]: Backend::unlock
>      type GuardState;
>  
> +    /// The context which must be provided to acquire the lock.
> +    ///
> +    /// Can be `()`, or another type if [`BackendWithContext`] is implemented.
> +    type Context<'a>;
> +
>      /// Initialises the lock.
>      ///
>      /// # Safety
> @@ -89,6 +94,54 @@ unsafe fn relock(ptr: *mut Self::State, guard_state: &mut Self::GuardState) {
>      }
>  }
>  
> +/// An additional trait for [`Backend`] implementations with a non-`()` [`Context`].
> +///
> +/// Restricts the context in which a [`Lock`] may be locked. It can initially be created using
> +/// [`Lock::lock_with_new`], and can be reused to acquire additional [`Lock`] objects using
> +/// [`Lock::lock_with`].
> +///
> +/// An example of a locking context would be a context in which local CPU interrupts are disabled,
> +/// where we must restrict the ability to acquire the [`Lock`] so that it can only be acquired
> +/// within that context.
> +///
> +/// [`Context`]: `Backend::Context`
> +pub trait BackendWithContext: Backend {
> +    /// The type that represents the state of this [`Context`].
> +    ///
> +    /// [`Context`]: Backend::Context
> +    type ContextState;
> +
> +    /// Fulfills the invariants of [`State`] and acquires the lock, making the caller its owner.
> +    ///
> +    /// This returns any state data ([`Context::State`]) needed upon unlock.
> +    ///
> +    /// # Safety
> +    ///
> +    /// * Same as [`Backend::lock`].
> +    ///
> +    /// [`State`]: Context::State
> +    unsafe fn lock_with_context_saved<'a>(
> +        ptr: *mut Self::State,
> +    ) -> (Self::Context<'a>, Self::ContextState, Self::GuardState);
> +
> +    /// Performs the final unlock within [`Context`].
> +    ///
> +    /// Passes the [`Context::State`] returned from [`first_lock`].
> +    ///
> +    /// # Safety
> +    ///
> +    /// * This function may only be called after [`first_lock`].
> +    /// * `state` must be the value returned from [`first_lock`].
> +    ///
> +    /// [`first_lock`]: Backend::first_lock
> +    /// [`Context`]: Backend::Context
> +    unsafe fn unlock_with_context_restored(
> +        ptr: *mut Self::State,
> +        guard_state: &Self::GuardState,
> +        context_state: Self::ContextState,
> +    );
> +}
> +
>  /// A mutual exclusion primitive.
>  ///
>  /// Exposes one of the kernel locking primitives. Which one is exposed depends on the lock
> @@ -132,8 +185,9 @@ pub fn new(t: T, name: &'static CStr, key: &'static LockClassKey) -> impl PinIni
>  }
>  
>  impl<T: ?Sized, B: Backend> Lock<T, B> {
> -    /// Acquires the lock and gives the caller access to the data protected by it.
> -    pub fn lock(&self) -> Guard<'_, T, B> {
> +    /// Acquires the lock with the given context and gives the caller access to the data protected
> +    /// by it.
> +    pub fn lock_with<'a>(&'a self, _context: B::Context<'a>) -> Guard<'a, T, B> {
>          // SAFETY: The constructor of the type calls `init`, so the existence of the object proves
>          // that `init` was called.
>          let state = unsafe { B::lock(self.state.get()) };
> @@ -141,14 +195,68 @@ pub fn lock(&self) -> Guard<'_, T, B> {
>          unsafe { Guard::new(self, state) }
>      }
>  
> -    /// Tries to acquire the lock.
> +    /// Acquires the lock and gives the caller access to the data protected by it.
> +    #[inline]
> +    pub fn lock<'a>(&'a self) -> Guard<'a, T, B>
> +    where
> +        B::Context<'a>: Default,
> +    {
> +        self.lock_with(Default::default())
> +    }
> +
> +    /// Tries to acquire the lock with the given context.
>      ///
>      /// Returns a guard that can be used to access the data protected by the lock if successful.
> -    pub fn try_lock(&self) -> Option<Guard<'_, T, B>> {
> +    pub fn try_lock_with<'a>(&'a self, _context: B::Context<'a>) -> Option<Guard<'a, T, B>> {
>          // SAFETY: The constructor of the type calls `init`, so the existence of the object proves
>          // that `init` was called.
>          unsafe { B::try_lock(self.state.get()).map(|state| Guard::new(self, state)) }
>      }
> +
> +    /// Tries to acquire the lock.
> +    ///
> +    /// Returns a guard that can be used to access the data protected by the lock if successful.
> +    #[inline]
> +    pub fn try_lock<'a>(&'a self) -> Option<Guard<'a, T, B>>
> +    where
> +        B::Context<'a>: Default,
> +    {
> +        self.try_lock_with(Default::default())
> +    }
> +}
> +
> +impl<T: ?Sized, B: BackendWithContext> Lock<T, B> {
> +    /// Acquire the lock with a new [`Context`].
> +    ///
> +    /// Creates a new instance of [`Context`], and then calls `cb` with said [`Context`] and a
> +    /// [`Guard`] for `self`. The [`Context`] will be dropped once `cb` finishes, and it may be used
> +    /// within `cb` to acquire additional locks.
> +    ///
> +    /// [`Context`]: Backend::Context
> +    pub fn lock_with_new<'a, R>(
> +        &self,
> +        cb: impl FnOnce(&mut Guard<'_, T, B>, B::Context<'a>) -> R,

I think this needs to be:

        cb: impl FnOnce(&mut Guard<'_, T, B>, B::Context<'_>) -> R,

i.e. using wildcard life for B::Context, which is equal to:

        cb: impl for<'b> FnOnce(&mut Guard<'b, T, B>, B::Context<'b>) -> R,

, which makes the lifetime of B::Context bound to the closure instead of
`lock_with_new()`. Otherwise, users can leak the `Context` outside:

    let irq_disabled_leak = lock1.lock_with_new(|_, irq_disabled| {
        irq_disabled
    });

playground:

	https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=520dc80444e6f3cc8c5782c7f0095cfe

> +    ) -> R {
> +        let ptr = self.state.get();
> +
> +        // SAFETY: The constructor of the type calls `init`, so the existence of the object proves
> +        // that `init` was called.
> +        let (context, context_state, guard_state) = unsafe { B::lock_with_context_saved(ptr) };
> +
> +        // We don't want Guard's destructor to get called, since we'll drop the lock manually with
> +        // B::unlock_with_context_restored later. So we store it in a ManuallyDrop and pass it to cb
> +        // via reference.
> +        // SAFETY: We acquired the lock when we called [`B::lock_with_context_saved`] above.
> +        let mut guard = ManuallyDrop::new(unsafe { Guard::new(self, guard_state) });
> +
> +        let result = cb(&mut guard, context);
> +
> +        // SAFETY: We called `B::lock_with_context_saved` above, `context_state` was returned from
> +        // there.
> +        unsafe { B::unlock_with_context_restored(ptr, &guard.state, context_state) };
> +

I think we have a soundness issue here, users can do:

(let's say we support static locks, which is a solid thing we want to
have)

static l1: SpinLockIrq<32> = ...;

    <in a function>

    let l2: &SpinLockIrq<i32> = ..;

    l2.lock_with_new(|guard2, context| {
        let mut guard1 = l1.lock_with(context);
	core::mem::swap(&mut guard1, guard2);
	drop(guard1); // actually unlock l2.
    }) // but when the cb returns, we dropped `l2` as well.

I have played this for a while, looks to me, only a static lock `l1` can
make the code pass the borrow checker, I'm not sure whether it is a
borrow checker implementation limitation, or this is by design. Because
lifetime of `guard1` should be `static` and lifetime of `guard2` should 
be `for<'a>, 'a`, seems they are interchangeable right now? A simplified
example:

	https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=8bcc1132734a7bd2420f766524af56a7

Regards,
Boqun

> +        result
> +    }
>  }
>  
>  /// A lock guard.
> diff --git a/rust/kernel/sync/lock/mutex.rs b/rust/kernel/sync/lock/mutex.rs
> index 9ce43ccb45158..9a873cb5b438b 100644
> --- a/rust/kernel/sync/lock/mutex.rs
> +++ b/rust/kernel/sync/lock/mutex.rs
> @@ -93,6 +93,7 @@ macro_rules! new_mutex {
>  unsafe impl super::Backend for MutexBackend {
>      type State = bindings::mutex;
>      type GuardState = ();
> +    type Context<'a> = ();
>  
>      unsafe fn init(
>          ptr: *mut Self::State,
> diff --git a/rust/kernel/sync/lock/spinlock.rs b/rust/kernel/sync/lock/spinlock.rs
> index 040dc16975a68..9fbfd96ffba3e 100644
> --- a/rust/kernel/sync/lock/spinlock.rs
> +++ b/rust/kernel/sync/lock/spinlock.rs
> @@ -92,6 +92,7 @@ macro_rules! new_spinlock {
>  unsafe impl super::Backend for SpinLockBackend {
>      type State = bindings::spinlock_t;
>      type GuardState = ();
> +    type Context<'a> = ();
>  
>      unsafe fn init(
>          ptr: *mut Self::State,
> -- 
> 2.47.0
> 

Powered by blists - more mailing lists