[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <b647ffbd0803081631y282cfe1blb7343eb1c979b871@mail.gmail.com>
Date: Sun, 9 Mar 2008 01:31:24 +0100
From: "Dmitry Adamushko" <dmitry.adamushko@...il.com>
To: "Gregory Haskins" <ghaskins@...ell.com>
Cc: "Ingo Molnar" <mingo@...e.hu>, ego@...ibm.com,
suresh.b.siddha@...el.com, yi.y.yang@...el.com, tglx@...utronix.de,
akpm@...ux-foundation.org, rjw@...k.pl, oleg@...sign.ru,
linux-kernel@...r.kernel.org
Subject: Re: [PATCH] adjust root-domain->online span in response to hotplugevent
On 08/03/2008, Gregory Haskins <ghaskins@...ell.com> wrote:
> >>> On Sat, Mar 8, 2008 at 3:41 AM, in message <20080308084118.GA24552@...e.hu>,
> Ingo Molnar <mingo@...e.hu> wrote:
>
> > * Gregory Haskins <ghaskins@...ell.com> wrote:
> >
> >> Unfortunately, this patch will introduce its own set of bugs.
> >> However, your analysis was spot-on. I think I see the problem now.
> >> It was introduced when I put a hack in to "fix" s2ram problems in -mm
> >> as a result of the new root-domain logic. I think the following patch
> >> will fix both issues:
> >>
> >> (I verified that I could take a cpu offline/online, but I don't have
> >> an s2ram compatible machine handy. Andrew, I believe you could
> >> reproduce the s2ram problem a few months ago when that issue popped
> >> up. So if you could, please verify that s2ram also works with this
> >> patch applied, in addition to the hotplug problem.
> >
> > thanks Gregory, i've queued up your fix. If it passes all tests over the
> > day i'll send it to Linus in about 12 hours.
> >
>
>
>
> After thinking about it some more, I am not sure if I got this fix quite right. The first two hunks are technically fine and should ultimately go in. The last hunk is questionable.
>
> Ultimately, I believe the root cause of these reported issues is that cpu_online_map and rd->online can get out of sync. I fear that while the current patch may fix the hotplug/s2ram case, it potentially breaks the disjoint cpuset topology change case. I will run a few cpuset tests ASAP to confirm, though I may not have a chance until Monday.
>
> What I probably need is to tie the code that sets/clears cpu_online_map to the root-domain rd->online map somehow. However, a cursory cscope inspection has failed to reveal the location that this cpu_online_map manipulation occurs. If anyone knows where cpu_online_map is actually updated, please let me know.
I guess, it's in arch-specific code. e.g. for x86-64,
down:
kernel/cpu.c :: _cpu_down -> take_cpu_down() -> __cpu_disable() [
arch/x86/kernel/smpboot_64.c ] -> cpu_clear(cpu, cpu_online_map)
up:
kernel/cpu.c :: _cpu_up() -> __cpu_up [ arch/x86/kernel/smpboot_64.c ]
-> do_boot_cpu() -> start_secondary() [ runs on a to-be-online cpu ]
-> cpu_set(smp_processor_id(), cpu_online_map)
there are cpu-notification events available before a cpu gets removed
from cpu_online_map or after it gets added, so I guess (a first guess.
I'll also look at the code) they should be a sync. point.
I have a small patch to record some events during the cpu_down/up. I
have a trace but will already analyze it tomorrow.
[ the patch is enclosed (err.. it also includes Gregory's fix... I
just forgot to commit it previously :-/) ]
e.g. it should print a message if a task is placed on the 'offline'
cpu. Note, the 'cpu' is removed from the cpu_online_map before
migrate_live_tasks() runs. So if any task remains on the offline cpu,
it has been likely placed there already after migrate_live_tasks().
> Regards,
> -Greg
>
--
Best regards,
Dmitry Adamushko
View attachment "debug-stop_machine-2.patch" of type "text/x-patch" (7008 bytes)
Powered by blists - more mailing lists