[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <Zv6TU_p4WMELMj_G@boqun-archlinux>
Date: Thu, 3 Oct 2024 05:51:31 -0700
From: Boqun Feng <boqun.feng@...il.com>
To: Thomas Gleixner <tglx@...utronix.de>
Cc: Lyude Paul <lyude@...hat.com>, rust-for-linux@...r.kernel.org,
Danilo Krummrich <dakr@...hat.com>, airlied@...hat.com,
Ingo Molnar <mingo@...hat.com>, Will Deacon <will@...nel.org>,
Waiman Long <longman@...hat.com>,
Peter Zijlstra <peterz@...radead.org>, linux-kernel@...r.kernel.org,
Benno Lossin <benno.lossin@...ton.me>,
Daniel Almeida <daniel.almeida@...labora.com>,
Gary Guo <gary@...yguo.net>, Miguel Ojeda <ojeda@...nel.org>,
Alex Gaynor <alex.gaynor@...il.com>,
Wedson Almeida Filho <wedsonaf@...il.com>,
Björn Roy Baron <bjorn3_gh@...tonmail.com>,
Andreas Hindborg <a.hindborg@...sung.com>,
Alice Ryhl <aliceryhl@...gle.com>, Trevor Gross <tmgross@...ch.edu>,
Martin Rodriguez Reboredo <yakoyoku@...il.com>,
Valentin Obst <kernel@...entinobst.de>
Subject: Re: [PATCH v6 3/3] rust: sync: Add SpinLockIrq
On Wed, Oct 02, 2024 at 10:53:32PM +0200, Thomas Gleixner wrote:
> On Mon, Sep 16 2024 at 17:28, Lyude Paul wrote:
> > A variant of SpinLock that is expected to be used in noirq contexts, and
> > thus requires that the user provide an kernel::irq::IrqDisabled to prove
> > they are in such a context upon lock acquisition. This is the rust
> > equivalent of spin_lock_irqsave()/spin_lock_irqrestore().
>
> This fundamentally does not work with CONFIG_PREEMPT_RT. See:
>
> https://www.kernel.org/doc/html/latest/locking/locktypes.html
>
> for further information. TLDR:
>
> On RT enabled kernels spin/rw_lock are substituted by sleeping locks. So
> you _cannot_ disable interrupts before taking the lock on RT enabled
> kernels. That will result in a 'might_sleep()' splat.
>
One thing I was missing when I suggested Lyude with the current API is
that local_irq_save() disables interrupts even on RT. I was under the
impression that local_irq_save() will only disable preemption per:
https://lwn.net/Articles/146861/
but seems it's not the case right now: we move the RT vs non-RT games
and hardware interrupt disabling vs preemption/migration disabling to
local_lock_*() I guess?
> There is a reason why the kernel has two distinct spinlock types:
>
> raw_spinlock_t and spinlock_t
>
> raw_spinlock_t is a real spinning lock independent of CONFIG_PREEMPT_RT,
> spinlock_t is mapped to raw_spinlock_t on CONFIG_PREEMPT_RT=n and to a
> rtmutex based implementation for CONFIG_PREEMPT_RT=y.
>
> As a consequence spin_lock_irq() and spin_lock_irqsave() will _NOT_
> disable interrupts on a CONFIG_PREEMPT_RT=y kernel.
>
> The proposed rust abstraction is not suitable for that.
>
> At this phase of rust integration there is no need to wrap
> raw_spinlock_t, so you have two options to solve that:
>
> 1) Map Rust's SpinLockIrq() to spin_lock_irqsave() and
> spin_unlock_irqrestore() which does the right thing
>
> 2) Play all the PREEMPT_RT games in the local irq disable abstraction
>
> #1 is the right thing to do because no driver should rely on actually
> disabling interrupts on the CPU. If there is a driver which does that,
> then it's not compatible with RT and should use a local lock instead.
>
> local locks aside of being RT compatible have the benefit that they give
> scope to the protected region/data, while a plain local_irq_disable()
> does not.
>
> Don't even think about exposing this 'with_irq_disabled' interface
> unless you are trying to move actual core code like the scheduler or low
> level interrupt handling to rust.
>
> Create explicit interrupt safe interfaces which map to the underlying
> locking primitives instead.
>
Then we should have a SpinLockIrq<T> type, and a function:
fn with_locked<U>(&self, cb: impl FnOnce(&mut T) -> U) -> U {
<spin_lock_irqsave()>
let ret = cb(...);
<spin_lock_irqrestore()>
ret
}
FYI, the reason that we cannot have a SpinLockIrq::lock() return a guard
that holds the irq state is because:
https://github.com/Rust-for-Linux/linux/issues/998
namely:
// interrupts are enabled here.
let a = spin_lock_irq1.lock();
let b = spin_lock_irq2.lock();
drop(a); // releasing spin_lock_irq1 and restore the irq state.
// `b` exists with interrupts enabled, which breaks the
// invariants of b.
(technically we can, but that requires a rework of how nested
irq_save()s are handled, that's another can of worms)
Regards,
Boqun
> Thanks,
>
> tglx
>
>
Powered by blists - more mailing lists