linux-kernel - Re: [PATCH] sched_clock: prevent scd->clock from moving backwards

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20081009151703.GA8010@elte.hu>
Date:	Thu, 9 Oct 2008 17:17:03 +0200
From:	Ingo Molnar <mingo@...e.hu>
To:	Peter Zijlstra <a.p.zijlstra@...llo.nl>
Cc:	Dave Kleikamp <shaggy@...ux.vnet.ibm.com>,
	Jeremy Fitzhardinge <jeremy@...p.org>,
	Steven Rostedt <srostedt@...hat.com>,
	Linux Kernel Mailing List <linux-kernel@...r.kernel.org>
Subject: Re: [PATCH] sched_clock: prevent scd->clock from moving backwards


* Ingo Molnar <mingo@...e.hu> wrote:

> 
> * Peter Zijlstra <a.p.zijlstra@...llo.nl> wrote:
> 
> > On Wed, 2008-10-08 at 08:00 -0500, Dave Kleikamp wrote:
> > > sched_clock: prevent scd->clock from moving backwards
> > > 
> > > When sched_clock_cpu() couples the clocks between two cpus, it may
> > > increment scd->clock beyond the GTOD tick window that __update_sched_clock()
> > > uses to clamp the clock.  A later call to __update_sched_clock() may move
> > > the clock back to scd->tick_gtod + TICK_NSEC, violating the clock's
> > > monotonic property.
> > > 
> > > This patch ensures that scd->clock will not be set backward.
> > 
> > Ah, yes indeed, this comes from the tick not happening at the same time
> > on different cpus, so if we use a local timestamp to move a remote clock
> > forward, this scenario could indeed happen.
> > 
> > The fix looks good to me, good catch, thanks shaggy!
> > 
> > A related 'fix' which I'm still not quite sure about is making the
> > window 'tick_gtod + 2*TICK_NSEC'. That increases the max observed
> > difference to 4 jiffies, but allows ticks to be 'late' a bit without
> > first holding back time and then jumping ahead again.
> > 
> > > Signed-off-by: Dave Kleikamp <shaggy@...ux.vnet.ibm.com>
> > > Cc: Ingo Molnar <mingo@...e.hu>
> > > Cc: Peter Zijlstra <a.p.zijlstra@...llo.nl>
> > 
> > Acked-by: Peter Zijlstra <a.p.zijlstra@...llo.nl>
> 
> applied to tip/sched/core, thanks!

hm, -tip testing found a sporadic hard lockup during bootup, and i've 
bisected it back to this patch. They happened on 64-bit test-systems. 
I've attached the .config that produced the problem.

i reverted the patch and the lockups went away. But i cannot see what's 
wrong with it ...

The lockups were at random places during bootup, such as:

[    0.000000] initcall init_mbcache+0x0/0x20 returned 0 after 0 msecs
[    0.000000] calling  dquot_init+0x0/0x100 @ 1
[    0.000000] VFS: Disk quotas dquot_6.5.1
[    0.000000] Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
[    0.000000] initcall dquot_init+0x0/0x100 returned 0 after 10 msecs
[    0.000000] calling  init_v2_quota_format+0x0/0x20 @ 1
[    0.000000] initcall init_v2_quota_format+0x0/0x20 returned 0 after 0 msecs
[    0.000000] calling  dnotify_init+0x0/0x30 @ 1
[... hard lockup ...]

weird ...

	Ingo

View attachment "config" of type "text/plain" (57046 bytes)