| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Message-ID: <20130724232642.GB30349@somewhere>
Date: Thu, 25 Jul 2013 01:26:44 +0200
From: Frederic Weisbecker <fweisbec@...il.com>
To: "Paul E. McKenney" <paulmck@...ux.vnet.ibm.com>
Cc: linux-kernel@...r.kernel.org, mingo@...e.hu, laijs@...fujitsu.com,
dipankar@...ibm.com, akpm@...ux-foundation.org,
mathieu.desnoyers@...ymtl.ca, josh@...htriplett.org,
niv@...ibm.com, tglx@...utronix.de, peterz@...radead.org,
rostedt@...dmis.org, dhowells@...hat.com, edumazet@...gle.com,
darren@...art.com, sbw@....edu
Subject: Re: [PATCH RFC nohz_full 6/7] nohz_full: Add full-system-idle state
machine
On Wed, Jul 24, 2013 at 03:09:02PM -0700, Paul E. McKenney wrote:
> On Wed, Jul 24, 2013 at 08:09:04PM +0200, Frederic Weisbecker wrote:
> > On Thu, Jul 18, 2013 at 10:06:25PM -0700, Paul E. McKenney wrote:
> > > > Lets summarize the last sequence, the following happens ordered by time:
> > > >
> > > > CPU 0 CPU 1
> > > >
> > > > cmpxchg(&full_sysidle_state,
> > > > RCU_SYSIDLE_SHORT,
> > > > RCU_SYSIDLE_LONG);
> > > >
> > > > smp_mb() //cmpxchg
> > > >
> > > > atomic_read(rdtp(1)->dynticks_idle)
> > > >
> > > > //CPU 0 goes to sleep
> > > > //CPU 1 wakes up
> > > > atomic_inc(rdtp(1)->dynticks_idle)
> > > >
> > > > smp_mb()
> > > >
> > > > ACCESS_ONCE(full_sysidle_state)
> > > >
> > > >
> > > > Are you suggesting that because the CPU 1 executes its atomic_inc() _after_ (in terms
> > > > of absolute time) the atomic_read of CPU 0, the ordering settled in both sides guarantees
> > > > that the value read from CPU 1 is the one from the cmpxchg that precedes the atomic_read,
> > > > or FULL or FULL_NOTED that happen later.
> > > >
> > > > If so that's a big lesson for me.
> > >
> > > It is not absolute time that matters. Instead, it is the fact that
> > > CPU 0, when reading from ->dynticks_idle, read the old value before the
> > > atomic_inc(). Therefore, anything CPU 0 did before that memory barrier
> > > preceding CPU 0's read must come before anything CPU 1 did after that
> > > memory barrier following the atomic_inc(). For this to work, there
> > > must be some access to the same variable on each CPU.
> >
> > Aren't we in the following situation?
> >
> > CPU 0 CPU 1
> >
> > STORE A STORE B
> > LOAD B LOAD A
> >
> >
> > If so and referring to your perfbook, this is an "ears to mouth" situation.
> > And it seems to describe there is no strong guarantee in that situation.
>
> "Yes" to the first, but on modern hardware, "no" to the second. The key
> paragraph is Section 12.2.4.5:
>
> The following pairings from Table 12.1 can be used on modern
> hardware, but might fail on some systems that were produced in
> the 1990s. However, these can safely be used on all mainstream
> hardware introduced since the year 2000.
Right I missed that!
>
> That said, you are not the first to be confused by this, so I might need
> to rework this section to make it clear that each can in fact be used on
> modern hardware.
>
> If you happen to have an old Sequent NUMA-Q or Symmetry box lying around,
> things are a bit different. On the other hand, I don't believe that any
> of these old boxes are still running Linux. (Hey, I am as sentimental as
> the next guy, but there are limits!)
>
> I updated this section and pushed it, please let me know if this helps!
I don't know because I encountered some troubles to build it, I'm seeing thousand
lines like this:
Name "main::opt_help" used only once: possible typo at /usr/bin/a2ping line 534.
/usr/bin/a2ping: not a GS output from gs -dSAFER
./cartoons/whippersnapper300.eps -> ./cartoons/whippersnapper300.pdf
Name "main::opt_extra" used only once: possible typo at /usr/bin/a2ping line 546.
Name "main::opt_help" used only once: possible typo at /usr/bin/a2ping line 534.
/usr/bin/a2ping: not a GS output from gs -dSAFER
make: *** [embedfonts] Error 1
Anyway I looked at the diff and it looks indeed clearer, thanks!
So back to the issue, I think we made nice progresses with my rusty brain ;-)
But just to be clear, I'm pasting that again for just a few precisions:
CPU 0 CPU 1
cmpxchg(&full_sysidle_state, //CPU 1 wakes up
RCU_SYSIDLE_SHORT, atomic_inc(rdtp(1)->dynticks_idle)
RCU_SYSIDLE_LONG);
smp_mb() //cmpxchg smp_mb()
atomic_read(rdtp(1)->dynticks_idle) ACCESS_ONCE(full_sysidle_state
//CPU 0 goes to sleep
1) If CPU 0 sets RCU_SYSIDLE_LONG and sees dynticks_idle as even, do we have the _guarantee_
that later CPU 1 sees full_sysidle_state == RCU_SYSIDLE_LONG (or any later full_sysidle_state value)
due to the connection between atomic_read / atomic_inc and the barriers that come along?
2) You taught me once that barrier != memory committed, and it has been one of the hardest trauma
in my life. How can we be sure that CPU 1 sees memory as committed from CPU 0? The only fact that
we read an even value from CPU 0 is enough for the connection between the atomic_read() and atomic_inc()
and all the barriers that come along?
3) In your book it says: "recent hardware would guarantee that at least one of the loads saw the value
stored by the corresponding store".
At least one? So in our example, CPU 0 could see dynticks_idle as even (success to see some prior store done in
CPU 1) but following the above statement reasoning, CPU 1 might not see the corresponding store and see, for example
RCU_SYSIDLE_SHORT?
I'm really sorry to bother you with that... :-(
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Powered by blists - more mailing lists