Message-ID: <20070729010039.GA11661@gnuppy.monkey.org>
Date:	Sat, 28 Jul 2007 18:00:39 -0700
From:	Bill Huey (hui) <billh@...ppy.monkey.org>
To:	Linus Torvalds <torvalds@...ux-foundation.org>
Cc:	Diego Calleja <diegocg@...il.com>,
	jos poortvliet <jos@...nkamer.nl>, ck@....kolivas.org,
	Michael Chang <thenewme91@...il.com>,
	Kasper Sandberg <lkml@...anurb.dk>,
	Linux Kernel Mailing List <linux-kernel@...r.kernel.org>,
	"Bill Huey (hui)" <billh@...ppy.monkey.org>
Subject: Re: [ck] Re: Linus 2.6.23-rc1

On Sat, Jul 28, 2007 at 03:18:24PM -0700, Linus Torvalds wrote:
> I don't think anything was suppressed here.

I disagree. See below.

> You seem to say that more modular code would have helped make for a nicer 
> way to do schedulers, but if so, where were those patches to do that? 
> Con's patches didn't do that either. They just replaced the code.

They replaced code because he would have liked to have taken scheduler code
in possibly a completely different direction. This is a large conceptual
change from what is currently there. That might also mean how the notion of
bandwidth with regards to core frequency might be expressed in the system
with regards to power saving and other things. Things get dropped often
not because of pure technical reasons but because of person preference
and the lack of willingness to ask where this might take us.

The way that Con works and conceptualizes things is quite a bit different
and more comprehensive in a lot of ways compared to how the regular kernel
community operates. He's strong in this area and weak in general kernel
hackery as a function of time and experience. That doesn't mean that he,
his ideas and his code should be subject to an either/or situation with the
scheduler and other ideas that have been rejected by various folks. He
maintained -ck branch successfully for a long time and is a very capable
developer.

I do acknowledge that having a maintainer that you can trust is more
important, but it should not be exclusionary in this way. I totally
understand his reaction.

> In fact, Ingo's patches _do_ add some modularity, and might make it easier 
> to replace the scheduler. So it would seem that you would argue for CFS, 
> not against it?

It's not the same as sched plugin. Some folks might not like to use the
rbtree that's in place and express things in a completely different
manner. Take for instance, Tong Li's stuff with CFS a bit of a conceptual
mismatch with his attempt at expression rebalancing in terms expiry rounds
yet would be more seamlessly integrated with something like either the old
O(1) scheduler or Con's stuff. It's also the only method posted to lkml
that can deal with fairness across SMP situtations with low error. Yet
what's happening here is that his implementation is being rejected because
of size and complexity because of a data structure conceptual mismatch.

Because of this, his notion of trio as a general method of getting
aggressive group fairness (by far the most complete conceptually on lkml,
over design is a different topic altogether) may never see the light of
day in Linux because of people's collective lack of foresight.

To answer the question that you posed, no. I'm not arguing against it. I'm
in favor of it going into the kernel like any dead line mechanism since
it can be generalized, but the current developement processes in Linux
kernel should not create an either/or situation with the scheduler code.
There has been multipule rejection of ideas with regards to the scheduler
code over the years that could have take things in a very different and
possibly complete kick ass way that was suppress because of the development
attitude of various Linux kernel developers.

It's all of a sudden because of Con's work there's a flurry of development
in this area when this idea is shown to be superior and even then, it's
conceptually incomplete and subject to a lot of arbitrary hacking. This
is very different than Con's development style and mine as well.

This is an area that could have been addressed sooner if the general
community admitted that there was a problem earlier and permitted more
conscious and open change. I've seen changes in this area from Con be
reject time and time again which effect the technical direction he
originally wanted to take this.

Now, Con might have a communication problem here, but nobody asked to
clarify what he might have wanted and why, yet folks were very quick at
dismissing him, nitpick him to death,  even when he explained why he might
have wanted a particular change in the first place. This is the
"facilitation" part that's missing in the current kernel culture.

This is a very important idea as the community grows, because I see folks
that are capable of doing work get discouraged and locked out because of
code maintainability issues and an inability to get folks to move that
direction because of a missing concensus mechanism in the community
other that sucking up to developers.

Con and folks like him should be permitted the opportunity to fail on
their own account. If Linux was truely open, it would have dealt with
issue by now and there wouldn't be so much flammage from the general
community.

> > I think that's kind of a bogus assumption from the very get go. Scheduling
> > in Linux is one of the most unevolved systems in the kernel that still
> > could go through a large transformation and get big gains like what
> > we've had over the last few months. This evident with both schedulers,
> > both do well and it's a good thing overall the CFS is going in.
> > 
> > Now, the way it happened is completely screwed up in so many ways that I
> > don't see how folks can miss it. This is not just Ingo versus Con, this
> > is the current Linux community and how it makes decision from the top down
> > and the current cultural attitude towards developers doing things that
> > are:
> 
> I don't think so.
> 
> I think you're barking up the totally wrong tree here.

Did in any place in your reply did you ask what I meant by this ? Where
somebody like me, Con or Tong (sorry to drag you into this) and others
might like to take this and the -rt ? and other things in the Linux
kernel ? Well, this is a failure of the concensus process in Linux.

> I think that what happened was very simple: somebody showed that we did 
> badly and had benchmarks to show for it, and that in turn resulted in a 
> huge spurt of coding from the people involved.

Or ass kissing from large companies that are afraid to challenge the
Linux kernel norm. And a lot of that stuff is a complete ugly hack. That's
a self fullfulling attribute of the Linux culture that indicative of how
many ass kisser we have in this community that can snow ball any arbitary
phenomenon like CFS.

> The fact that you think this is "broken" is interesting. I can point to a 
> very real example of where this also happened, and where I bet you don't 
> think the process was "broken".

This lack of communication is an indicator of it being broken. Folks not
asking where we could take the Linux kernel, versus hacking it without
background on particular area and listening, is indicative of it. General
lack of direction with development is an indicator of it. Lack of design
discussion and inclusiveness of various designs is an indicator of failure.

> Do you remember the mindcraft study?
> 
> Exact same thing. Somebody came in, and showed that Linux did really badly 
> on some benchmark, and that an alternate approach was much better.
> 
> What happened? A huge spurt of development in a pretty short timeframe, 
> that totally _obliterated_ the mindcraft results. 
> 
> It could have happened independently, but the fact is, it didn't. These 
> kinds of events where somebody shows (with real numbers and code) that 
> things can be done better really *are* a good way to do development, and 
> it's how development generally ends up happening. It's hugely 
> motivational, both because competition is motivational in itself, but also 
> because somebody shows that things can be done so much better opens 
> peoples eyes to it.

Everybody here wants Linux to be better. Everybody, me included. Make no
mistake. But collectively we should not be in a state of "reaction" to
external forces as the only known method of development.

The scheduler could have and still can undertake good solid transformation,
but getting folks to listen is another story which is why Con quit. CFS
basically locks him and his ideas out, not just from a technical stand
point, and sends a personal message that we don't care about him because
we've made zero effort to reach out to him. This is exactly how you laid
it out in previous messages. That's broken. He has his own problems, but
he also produces good work and recognizes the development problems with
the community in his /. article that many folks agree with approach
lkml with bug reports.

It's hack versus design. There needs to be a balance between the two and
a culture to receive both and not create and either/or situation. Con's
method of development should be welcomed.

> And if you think the scheduler situation is different, why? Was it just 
> because the mindcraft study compared against Windows NT, not another 
> version of Linux patches?
> 
> The thing is, development is slow and gradual, but at the same time, it 
> happens in spurts (btw, if you have ever studied evolution, you'll find 
> the exact same thing: evolution is slow and gradual, but it also happens 
> in sudden "spurts" where you have relatively much bigger changes happening 
> because you get into a feedback loop).

This time it was Con being the Mindcraft catalyst. But he's on *our* side
and he got beat down by the Linux kernel community. That's the tragedy here.
He was beaten down by the very people he was trying to help out and
support. It should have been handled better.
 
> Another comparison to evolution: most of the competitive pressure actually 
> comes from the _same_ species, not from outside. It's not so much rabbits 
> competing against foxes (although that happens too), quite a lot of it is 
> rabbits competing against other rabbits!

This is different topic and a distraction from inclusiveness issues. All
groups have a mono-culture. Groups that recognized it and push out of it
make it a win for the entire group and not just for a small set of individuals.

Even as your "arm chair kernel" hacker, I've done things that are out of
the box in Linux and I believe are interesting but never cared for the
politics of mainline.

bill

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Powered by blists - more mailing lists