Message-ID: <20140821140211.GD19293@ulmo.nvidia.com>
Date:	Thu, 21 Aug 2014 16:02:13 +0200
From:	Thierry Reding <thierry.reding@...il.com>
To:	Grant Likely <grant.likely@...aro.org>
Cc:	Alexander Holler <holler@...oftware.de>,
	linux-kernel@...r.kernel.org, devicetree@...r.kernel.org,
	Jon Loeliger <jdl@....com>,
	Russell King <linux@....linux.org.uk>,
	Greg Kroah-Hartman <gregkh@...uxfoundation.org>,
	Rob Herring <robh+dt@...nel.org>,
	Arnd Bergmann <arnd@...db.de>,
	linux-arm-kernel@...ts.infradead.org
Subject: Re: [RFC PATCH 0/9] dt: dependencies (for deterministic driver
 initialization order based on the DT)

On Wed, May 14, 2014 at 03:19:14PM +0100, Grant Likely wrote:
> On Mon, 12 May 2014 18:47:51 +0200, Alexander Holler <holler@...oftware.de> wrote:
> > 
> > Hello,
> > 
> > if I would have to describe the Linux kernels init system (before userspace
> > starts), it would be like:
> > Unknown functions with almost unknown functionality are called in an almost
> > random order.
> > 
> > That reminded me that a kernel-maintainer once said to me:
> > "We should aim to make things synchronous, deterministic and
> > stuff-happens-in-the-correct-order."
> > 
> > Looks like either the target moved or no Wilhelm Tell was around. ;)
> > 
> > This is an attempt to reach the target for the case of (platform-)drivers.
> > It is a mere starting point to reach the final target but it works on two
> > DT enabled ARM devices I have and it hasn't any implications on other
> > architectures, platforms or whatever. If the new configuration option,
> > which is only available if DT is enabled, isn't turned on, there is no
> > increase of code size or similiar.
> > 
> > So what are these patches I'm posting here?
> > They offer an imho solid base to fix the 3. problem. they build a deterministic
> > order in which (platform-)drivers should be initialized, based on datas
> > (dependencies) found in the device tree. They also offer a starting point to fix
> > the other 2 problems (unknown functions and unknown functionality) by showing a
> > way how the long range target of known functions with known functionality could
> > be reached.
> > 
> > Unfortunately work still isn't done. As written above, this is just a starting
> > point, neiter complete nor perfect. It is what I could do behind closed doors,
> > by spending a limited amount of time and resources (I've started to look at
> > that stuff 3-4 weeks ago, sometimes after 3.14 appeared), so it can be blamed
> > quick & dirty. But it should be already enough to explain and test the concepts.
> > 
> > Enough forewords.
> > 
> > This is a small patch series to use a deterministic dependency based device
> > and driver initialization order on machines which are using device tree.
> > The dependency graph will not only be build based on the device tree itself,
> > but will use dependencies based on phandle references in the .dts,
> > automatically added by dtc through a new property.
> > Manualy adding dependencies to the .dts is possible too.
> > 
> > Advantages:
> > 
> > - Correct order of initialization without any "dirty tricks" in drivers or the
> >   machine init code. The order in which devices/drivers will be initialized
> >   depends only on the DT and the topological sort algorithm used by the
> >   kernel, not some code elsewhere. That means less code and more homogeneity
> >   across different SOCs.
> > - Might be(come) a little faster because the number of deferred probes should
> >   be minimized (they might not even be necessary anymore at all).
> > - Using a modified algorithm, it's possible to build a list of drivers which
> >   can be initialized in parallel, e.g. using n threads, where n equals the
> >   number of cores. I have not tested nor implemented it, because I don't have
> >   any multicore DT based board with which I easily can use a (patched) mainline
> >   kernel, just locked down multicore things I don't want to go through the
> >   pain of unlocking them.
> > - Funny dependency graphs when using Graphviz.
> > 
> > Disadvantages:
> > 
> > - To use this feature correctly, binary blobs must be regenerated by
> >   recompiling them to include properties with dependencies. Without
> >   recompiling them there will be no advantage.
> 
> Rather than a dtb schema change, for the most common properties (irqs,
> clocks, gpios), we could extract dependencies at boot time. I don't like
> the idea of adding a separate depends-on property because it is very
> easy to get it out of sync with the actual binding data (dtc is not the
> only tool that manipulates .dtbs. Firmware will fiddle with it too).

Sorry for reviving this old thread, but this issue keeps popping up time
and time again and people are starting to think about ways to workaround
deferred probe being slow.

I'd very much prefer to avoid that and just keep using deferred probing
as the one means to resolve dependencies at boot time, but at the same
time I can't ignore when several people start complaining about the same
thing.

This very recently popped up again in the context of DRM and panels. One
user complaints that in some cases, deferred probe increases boot time
by 1.5 seconds (best case) and 5-6 seconds (worst case). That doesn't
sound all that much, objectively, but if the boot time of the device is
on the order of 6 seconds or so, then it's quite a lot.

The reason why this happens is that the DRM driver looks up a panel
connected to it and return -EPROBE_DEFER if the panel hasn't registered
yet. Typically the panel would be probed after the DRM driver, therefore
causing the DRM driver to defer probing. But it can also happen that the
panel driver defers probing because it needs a regulator that hasn't
been probed yet.

Anyway, those are all fairly standard reasons for where deferred probe
triggers, and since I do like deferred probe for it's simplicity and
reliability I'd rather not try to work around it if boot time is all
that people are concerned about.

In order to solve the above problem I started investigating a bit what
it would take to parse dependency information from a device tree at boot
time. As it happens this can easily be done for things like clocks and
interrupts properties, because they have very well-defined generic
bindings. Unfortunately they are also the two types of resources where
deferred probing matters the least because they are so special that they
often get probed earlier than normal drivers (and in some cases even way
before initcalls).

Unfortunately for other bindings it's not that easy. Take GPIOs for
example. GPIOs can be specified in fairly arbitrary properties. There's
a concensus on using a -gpio or -gpios suffix, but I don't think there
are any guarantees. It could also technically be that a -gpio or -gpios
property doesn't in fact contain a phandle at all.

The same is true for regulators (*-supply) and likely a couple other
bindings. So one of the hurdles to take trying to obtain the dependency
information from existing DTBs is that there may be false positives (and
the code could therefore try to resolve non-phandles as phandles and
even succeed) and the code resolving the dependencies would need to know
a whole lot about bindings.

I suppose the latter problem could be solved by adding a way for
subsystems to register a "dependency resolver" that would implement the
knowledge about existing generic bindings of the subsystems. Another way
to push that even further down into drivers would be to make drivers
list the types of resources that they need in some structure. Arnd
(Cc'ed) proposed something akin to that a while ago with the goal of
removing a lot of boilerplate from device drivers. Such a list would
automatically "implement" the driver binding and could be used to
resolve dependencies when the driver is registered.

On the other hand, some of the solutions that workaround the problems
caused by deferred probing may actually be worthwhile to have in any
case. One such solution for the above-mentioned example of panels is to
make the DRM subsystem deal better with hotplugging panels. This is in a
way counterintuitive because those panels are not really hotpluggable
(you'd have to disassemble the device at runtime and remove or unsolder
some connector and even then there's no GPIO detecting their presence or
absence). However I can imagine that it would round things off nicely if
a panel driver could simply be unloaded and the result being that the OS
simply registers this as a display going away, much like with external
monitors.

Thierry

Content of type "application/pgp-signature" skipped

Powered by blists - more mailing lists