Message-ID: <CAD=FV=X5=AK4XMANv+fatn3d9Y5vo9AXsHWb_47HmNaMWUtMwA@mail.gmail.com>
Date: Mon, 1 Dec 2025 16:44:08 -0800
From: Doug Anderson <dianders@...omium.org>
To: Linus Walleij <linusw@...nel.org>
Cc: devicetree-spec@...r.kernel.org, boot-architecture@...ts.linaro.org, 
	Chen-Yu Tsai <wenst@...omium.org>, Rob Herring <robh@...nel.org>, 
	Krzysztof Kozlowski <krzk@...nel.org>, 
	"open list:OPEN FIRMWARE AND FLATTENED DEVICE TREE BINDINGS" <devicetree@...r.kernel.org>, LKML <linux-kernel@...r.kernel.org>, 
	Linux ARM <linux-arm-kernel@...ts.infradead.org>, 
	William McVicker <willmcvicker@...gle.com>, Julius Werner <jwerner@...omium.org>, 
	Conor Dooley <conor+dt@...nel.org>, Peter Griffin <peter.griffin@...aro.org>, 
	Tudor Ambarus <tudor.ambarus@...aro.org>, André Draszik <andre.draszik@...aro.org>
Subject: Re: Proposal: Officially allow "incomplete" trees as a base

Hi,

On Mon, Dec 1, 2025 at 3:52 PM Linus Walleij <linusw@...nel.org> wrote:
>
> Hi Dough,
>
> thanks for your mail!
>
> Let me just see if I understand correctly what you're trying to
> do  (and apparently actively doing in production):

Thanks for your thoughts. Just to be clear, even though I've dealt
with device trees on a lot of ChromeOS boards, this is something that
_Android_ phones are doing in production and is what I'm focusing on
here. The whole scheme is at least roughly documented at:

https://source.android.com/docs/core/architecture/dto/partitions

This is all stuff that predates me looking at Android. I'm just coming
in and trying to make sense of what's there and trying to upstream
what I can.

Separately, it can be noted that we always _wanted_ a similar solution
for ChromeOS, but we never quite got there... ;-)


> On Tue, Nov 18, 2025 at 11:43 PM Doug Anderson <dianders@...omium.org> wrote:
>
> > We would like an officially accepted scheme that lets us more
> > efficiently ship compiled device trees for a handful of related
> > products by breaking the device trees up into a common "base" device
> > tree and then applying "overlay" device trees atop the base to make a
> > full and complete device tree.
> (...)
> > In Android, we do a pretty good job of solving the stated problem
> > using device tree overlays. We describe the SoCs in "dts" files and
> > compile them into "dtb"s. We describe boards in "dtso" files and
> > compile them into "dtbo" files. A bootloader can handle identifying
> > the correct base and overlay (the scheme for doing this is a separate
> > but related topic) and applying the overlay atop the base "dtb". This
> > solution is fully implemented downstream for Android phones and is
> > well documented [2].
>
> So in the kernel we have several arch/*/boot/dts/vendor/ folders
> where they have:
>
> soc.dtsi included by board.dtsi included by system.dts
> and the top system.dts is all that get compiled.
>
> So what you say is that you do the same thing but at runtime?

Essentially. In this case, we don't ship the device tree _sources_ but
ship _compiled_ device tree. We compile the SoC into a "dtb" and then
combine anything above the SoC into a single "dtbo". We ship several
"dtb" files and several "dtbo" files and combine them at runtime after
we identify which hardware we're running on.


> Can you just describe why this is done like this with
> overlays?
>
> I can only guess that you have one bootloader that needs
> to compose device trees to many systems and the bootloader
> only discovers the system it is running on and its constraints
> at runtime, so shipping the static system.dtb as many people
> do is not an option for you?
>
> And the reason the bootloader doesn't already know what
> it is running on and doesn't just pass that one device tree
> or is just prepared with that one device tree has something
> to do with manufacturing or so?
>
> Sorry it just evades me.

Our builder creates a single "image" that can be flashed onto any
number of devices. Users (or manufacturers) can download and install
this single image and it will work on a wide variety of devices. So
you can download a single "Pixel 10" image that can be installed on
any of the Pixel 10 devices (Pixel 10, Pixel 10 Pro, and Pixel 10 Pro
XL). It will also work on our development board and early
pre-production variants of those boards.

In order for this to work, we need a lot of device trees and
_something_ needs to pick the correct one. Right now, the bootloader
is in charge of this task.

If we had less variety of products that we needed to ship then, yes,
we could just ship the one device tree and have a separate build for
each product. ...but that doesn't scale terribly well.


> Probably becayse even U-Boot these days use the provided
> device tree dtb, for the system targeted, to initialize itself. I
> suppose your bootloader is generic enough to avoid that
> chicken-and-egg problem, right?

FWIW: I believe even U-boot supports shipping a pile of device trees
and detecting things at runtime. See the "FIT" image:

https://docs.u-boot.org/en/latest/usage/fit/index.html


> I guess if I had this problem:
>
> soc.dtsi
> board-a.dtsi
> board-b.dtsi
> system-board-a-v1.dts
> system-board-a-v2.dts
> system-board-b-v1.dts
>
> etc having to be combined at runtime, in a bootloader, I would
> consider shipping exactly these files in a memory-based
> filesystem in my bootloader, and bake a DTS compiler into
> my bootloader so it can just build a device tree for what
> it detects.
>
> But I guess you didn't want to hear that :D

You're saying to just ship device tree source instead of binary? Hmmm,
it's an interesting idea. It wouldn't be _terrible_ since "dtb" stores
a lot of strings to begin with, but I think it would still add enough
of a bloat to make it a no-go for us... I would also imagine it would
be a pain to deal with #include of header files with #defines. Do we
somehow partially pre-process the device tree files but don't apply
the #include files. It might be possible to solve some specific cases,
but having this work in a generic way seems miserable.


> Anyway, please describe how you ended up in this situation,
> I'm trying to understand this.

The main goal is just building/shipping one image to support a variety
of similar products and trying to save space by not repeating the SoC
bits over and over again.


> It is quite clear that device tree overlays were intended for
> say plug-n-play additions and minor fragments to be added
> to a basically complete device tree, what you are doing was
> probably not how it was intended. (Or let's see if someone
> proves me wrong on that.)

Not sure what makes that so clear, since it's not clear to me. In any
case, even if device tree overlays weren't intended for such a usage,
they actually seem to work fairly well for it, even if upstream
currently rejects device tree usage like this.

FWIW, the fact that bootloaders (like U-Boot) have the ability to
apply overlays makes me think that _someone_ intended them to be used
similar to how Android is using them. ;-) If it was just expansion
cards then (presumably) someone would boot to Linux and just apply
overlays from there. See:

https://docs.u-boot.org/en/latest/usage/fit/overlay-fdt-boot.html


-Doug

Powered by blists - more mailing lists