Message-ID: <20161004232804.GX3296@wotan.suse.de>
Date:   Wed, 5 Oct 2016 01:28:04 +0200
From:   "Luis R. Rodriguez" <mcgrof@...nel.org>
To:     "Herbert, Marc" <marc.herbert@...el.com>
Cc:     "Luis R. Rodriguez" <mcgrof@...nel.org>,
        Linus Torvalds <torvalds@...ux-foundation.org>,
        Bjorn Andersson <bjorn.andersson@...aro.org>,
        Daniel Vetter <daniel.vetter@...ll.ch>,
        Mimi Zohar <zohar@...ux.vnet.ibm.com>,
        Felix Fietkau <nbd@....name>,
        David Woodhouse <dwmw2@...radead.org>,
        Roman Pen <r.peniaev@...il.com>,
        Ming Lei <ming.lei@...onical.com>,
        Andrew Morton <akpm@...ux-foundation.org>,
        Michal Marek <mmarek@...e.com>,
        Greg KH <gregkh@...uxfoundation.org>,
        Linux Kernel Mailing List <linux-kernel@...r.kernel.org>,
        Vikram Mulukutla <markivx@...eaurora.org>,
        Stephen Boyd <stephen.boyd@...aro.org>,
        Mark Brown <broonie@...nel.org>, Takashi Iwai <tiwai@...e.de>,
        Johannes Berg <johannes@...solutions.net>,
        Christian Lamparter <chunkeey@...glemail.com>,
        Hauke Mehrtens <hauke@...ke-m.de>,
        Jeff Mahoney <jeffm@...e.com>
Subject: Re: [RFC] fs: add userspace critical mounts event support

On Fri, Sep 23, 2016 at 07:51:41PM -0700, Herbert, Marc wrote:
> On 06/09/2016 16:04, Luis R. Rodriguez wrote:
> > They claim that without it there is the race between /lib/firmware
> > being ready and driver asking for the firmware.
> 
> Hope it's understood by now.
> 
> > I was told there were quite a bit of out-of-tree hacks to address
> > this without using the usermode helper,
> 
> There are:
> https://chromium-review.googlesource.com/#/c/354089/
>  wait until SYSTEM_RUNNING before loading DMC firmware.

Jeesh. Good thing its not merged yet upstream, but indeed
I can understand why out of tree kernels are picking these
sorts of solutions up in the meantime.

> > the goal of this patch was to create the discussion needed to a
> > proper resolution to this.
> 
> Sincere thanks.
> 
> >>> On Tue 06 Sep 11:32 PDT 2016, Linus Torvalds wrote:
> >>>
> >>>> On Tue, Sep 6, 2016 at 10:46 AM, Bjorn Andersson
> >>>> Nobody has actually answered the "why don't we just tie the
> >>>> firmware and module together" question.
> >>>
> >>> The answer to this depends on the details of the suggestion; but
> >>> generally there's a much stronger bond between the kernel and the
> >>> driver than between the driver and the firmware in my cases.
> 
> Indeed.
> 
> The i915 DMC firmware is an interesting example. First of all it’s
> _optional_!  It’s critical for battery-powered systems but the i915
> driver works without it.

You obviously may want to upgrade firmware too, and a driver may
want to provide support for a series of old and new firmware.

An alternative idea hinted to me recently was a new system call for
drivers that need firmware early, so we'd have system call deal with
loading *both* the module and firmware -- but indeed optional firmware is one
possible issue that throws a wrench into this. We can surely add a flags
option but not yet sure if that alone would suffice for most of our needs.
You may need code to generate the firmware name dynamically as well, so a
system call would only be useful for a few cases where firmware requirement
information can be inferred by userspace by just looking at the module object.

> Dan wrote:
> > Plus all gpu drivers which need firmware. And yes we must load them
> > at probe because people are generally pissed when they boot their
> > machine and the screen goes black.

Thanks for the clarification BTW.

> > On top of that a lot of people
> > want their gpu drivers to be built-in, but can't ship the firmware
> > blobs in the kernel image because gpl. Yep, there's a bit a
> > contradiction there ...
> 
> Eppur si muove:
> 1) As Dan just wrote, users expect the screen to light up as soon as they
> press the power button so the i915 driver is built-in
> 2) ... yet they’ll never notice the nanojoules of battery loss caused
> by the DMC firmware being on a filesystem and loaded a tiny bit later.
> 
> SoCs and platforms have become some new kind of distributed systems
> where other processors run their own, specific software/OS/firmware.
> From this perspective the kernel plays a role similar to a boot server;
> and choke point. Granted: booting various and heterogeneous
> distributed systems doesn’t look like a simple problem to solve
> generically. Yet at the moment the kernel  doesn’t help by not
> even supporting something as basic as being told when the files it’s
> (unfortunately) in charge to deploy to other nodes become available and
> ready to deploy.

When you consider the problem more from a directed acyclic graph point of view
you soon realize the issue is really about the *need* for certain files upon
driver load and the lack of of semantics for a deterministic assurance that
when we look for files its a valid hunt. What you describe in terms of
SoCs is just that the complexity of the DAG increases considerably.

> It can’t be assumed that the driver and the firmware are two parts of
> the same software piece whereas they actually run on two different
> processors, are most likely developed and validated by completely
> different teams and released on different lifecycles. Especially in
> the Linux case.
> 
> I hope this distributed systems analogy captures the essence of the
> examples and rationales detailed elsewhere in this thread.

You also need to upgrade firmware, and users should be able to opt-in
for firmware, and pick any firmware, or roll back to older versions
as they see fit.

  Luis

Powered by blists - more mailing lists