Message-ID: <DM6PR11MB4657003794552DC98ACF31669B179@DM6PR11MB4657.namprd11.prod.outlook.com>
Date:   Fri, 2 Dec 2022 11:27:24 +0000
From:   "Kubalewski, Arkadiusz" <arkadiusz.kubalewski@...el.com>
To:     Jiri Pirko <jiri@...nulli.us>,
        Vadim Fedorenko <vfedorenko@...ek.ru>
CC:     Jakub Kicinski <kuba@...nel.org>,
        Jonathan Lemon <jonathan.lemon@...il.com>,
        Paolo Abeni <pabeni@...hat.com>,
        "netdev@...r.kernel.org" <netdev@...r.kernel.org>,
        "linux-arm-kernel@...ts.infradead.org" 
        <linux-arm-kernel@...ts.infradead.org>,
        "linux-clk@...r.kernel.org" <linux-clk@...r.kernel.org>
Subject: RE: [RFC PATCH v4 0/4] Create common DPLL/clock configuration API

>From: Jiri Pirko <jiri@...nulli.us>
>Sent: Wednesday, November 30, 2022 1:32 PM
>
>Tue, Nov 29, 2022 at 10:37:20PM CET, vfedorenko@...ek.ru wrote:
>>Implement common API for clock/DPLL configuration and status reporting.
>>The API utilises netlink interface as transport for commands and event
>>notifications. This API aim to extend current pin configuration and
>>make it flexible and easy to cover special configurations.
>
>Overall, I see a lot of issues on multiple levels. I will go over them in
>follow-up emails. So far, after couple of hours looking trought this, I
>have following general feelings/notes:

Hi Jiri,

As we have been participating in last version, feel obligated to answer to
the concerns.
 
>
>1) Netlink interface looks much saner than in previous versions. I will
>   send couple of notes, mainly around events and object mixtures and
>   couple of bugs/redundancies. But overall, looks fineish.
>
>2) I don't like that concept of a shared pin, at all. It makes things
>   unnecessary complicated. Just have a pin created for dpll instance
>   and that's it. If another instance has the same pin, it should create
>   it as well. Keeps things separate and easy to model. Let the
>   hw/fw/driver figure out the implementation oddities.
>   Why exactly you keep pushing the shared pin idea? Perhaps I'm missing
>   something crucial.


If the user request change on pin#0 of dpll#0, the dpll#0 knows about the
change, it reacts accordingly, and notifies the user the something has changed.
Which is rather simple.

Now, if the dpll#1 is using the same pin (pin#0 of dpll#0), the complicated
part starts. First we have to assume:
- it was initialized with the same description (as it should, to prevent
confusing the user)
- it was initialized with the same order (this is at least nice to have from
user POV, as pin indices are auto generated), and also in case of multiple pins
being shared it would be best for the user to have exactly the same number of
pins initialized, so they have same indices and initialization order doesn't
introduce additional confusion.

Thus, one reason of shared pins was to prevent having this assumptions ever.
If the pin is shared, all dplls sharing a pin would have the same description
and pin index for a shared pin out of the box.

Pin attribute changes
The change on dpll#0 pin#0 impacts also dpll#1 pin#0. Notification about the
change shall be also requested from the driver that handles dpll#1. In such
case the driver has to have some dpll monitoring/notifying mechanics, which at
first doesn't look very hard to do, but most likely only if both dplls are
initialized and managed by a single instance of a driver/firmware.

If board has 2 dplls but each one is managed by its own firmware/driver
instance. User changes frequency of pin#0 signal, the driver of dpll#0 must
also notify driver of dpll#1 that pin#0 frequency has changed, dpll#1 reacts on
the change, notifies the user.
But this is only doable with assumption, that the board is internally capable
of such internal board level communication, which in case of separated
firmwares handling multiple dplls might not be the case, or it would require
to have some other sw component feel that gap.

For complex boards with multiple dplls/sync channels, multiple ports,
multiple firmware instances, it seems to be complicated to share a pin if
each driver would have own copy and should notify all the other about changes.

To summarize, that is certainly true, shared pins idea complicates stuff
inside of dpll subsystem.
But at the same time it removes complexity from all the drivers which would use
it and is easier for the userspace due to common identification of pins.
This solution scales up without any additional complexity in the driver,
and without any need for internal per-board communication channels.

Not sure if this is good or bad, but with current version, both approaches are
possible, so it pretty much depending on the driver to initialize dplls with
separated pin objects as you have suggested (and take its complexity into
driver) or just share them.

>
>3) I don't like the concept of muxed pins and hierarchies of pins. Why
>   does user care? If pin is muxed, the rest of the pins related to this
>   one should be in state disabled/disconnected. The user only cares
>   about to see which pins are related to each other. It can be easily
>   exposed by "muxid" like this:
>   pin 1
>   pin 2
>   pin 3 muxid 100
>   pin 4 muxid 100
>   pin 5 muxid 101
>   pin 6 muxid 101
>   In this example pins 3,4 and 5,6 are muxed, therefore the user knows
>   if he connects one, the other one gets disconnected (or will have to
>   disconnect the first one explicitly first).
>

Currently DPLLA_PIN_PARENT_IDX is doing the same thing as you described, it
groups MUXed pins, the parent pin index here was most straightforward to me,
as in case of DPLL_MODE_AUTOMATIC, where dpll auto-selects highest priority
available signal. The priority can be only assigned to the pins directly
connected to the dpll. The rest of pins (which would have present
attribute DPLLA_PIN_PARENT_IDX) are the ones that require manual selection
even if DPLL_MODE_AUTOMATIC is enabled.

Enabling a particular pin and sub-pin in DPLL_MODE_AUTOMATIC requires from user
to select proper priority on on a dpll-level MUX-pin and manually select one of
the sub-pins.  
On the other hand for DPLL_MODE_FORCED, this might be also beneficial, as the
user could select a directly connected pin and muxed pin with two separated
commands, which could be handled in separated driver instances (if HW design
requires such approach) or either it can be handled just by one select call
for the pin connected directly and handled entirely in the one driver instance.

>4) I don't like the "attr" indirection. It makes things very tangled. It
>   comes from the concepts of classes and objects and takes it to
>   extreme. Not really something we are commonly used to in kernel.
>   Also, it brings no value from what I can see, only makes things very
>   hard to read and follow.
>

Yet again, true, I haven't find anything similar in the kernel, it was more
like a try to find out a way to have a single structure with all the stuff that
is passed between netlink/core/driver parts. Came up with this, and to be
honest it suits pretty well, those are well defined containers. They store
attributes that either user or driver have set, with ability to obtain a valid
value only if it was set. Thus whoever reads a struct, knows which of those
attributes were actually set.
As you said, seems a bit revolutionary, but IMHO it simplifies stuff, and
basically it is value and validity bit, which I believe is rather common in the
kernel, this differs only with the fact it is encapsulated. No direct access to
the fields of structure is available for the users.
Most probably there are some things that could be improved with it, but in
general it is very easy to use and understand how it works.
What could be improved:
- naming scheme as function names are a bit long right now, although mostly
still fits the line-char limits, thus not that problematic
- bit mask values are capable of storing 32 bits and bit(0) is always used as
unspec, which ends up with 31 values available for the enums so if by any
chance one of the attribute enums would go over 32 it could be an issue.
 
It is especially useful for multiple values passed with the same netlink
attribute id. I.e. please take a look at dpll_msg_add_pin_types_supported(..)
function.

>   Please keep things direct and simple:
>   * If some option could be changed for a pin or dpll, just have an
>     op that is directly called from netlink handler to change it.
>     There should be clear set of ops for configuration of pin and
>     dpll object. This "attr" indirection make this totally invisible.

In last review you have asked to have rather only set and get ops defined
with a single attribute struct. This is exactly that, altough encapsulated.

>   * If some attribute is const during dpll or pin lifetime, have it
>     passed during dpll or pin creation.
>
>

Only driver knows which attributes are const and which are not, this shall
be also part of driver implementation.
As I understand all the fields present in (dpll/dpll_pin)_attr, used in get/set
ops, could be altered in run-time depending on HW design.

Thanks,
Arkadiusz

>
>>
>>v3 -> v4:
>> * redesign framework to make pins dynamically allocated (Arkadiusz)
>> * implement shared pins (Arkadiusz)
>>v2 -> v3:
>> * implement source select mode (Arkadiusz)
>> * add documentation
>> * implementation improvements (Jakub)
>>v1 -> v2:
>> * implement returning supported input/output types
>> * ptp_ocp: follow suggestions from Jonathan
>> * add linux-clk mailing list
>>v0 -> v1:
>> * fix code style and errors
>> * add linux-arm mailing list
>>
>>
>>Arkadiusz Kubalewski (1):
>>  dpll: add dpll_attr/dpll_pin_attr helper classes
>>
>>Vadim Fedorenko (3):
>>  dpll: Add DPLL framework base functions
>>  dpll: documentation on DPLL subsystem interface
>>  ptp_ocp: implement DPLL ops
>>
>> Documentation/networking/dpll.rst  | 271 ++++++++
>> Documentation/networking/index.rst |   1 +
>> MAINTAINERS                        |   8 +
>> drivers/Kconfig                    |   2 +
>> drivers/Makefile                   |   1 +
>> drivers/dpll/Kconfig               |   7 +
>> drivers/dpll/Makefile              |  11 +
>> drivers/dpll/dpll_attr.c           | 278 +++++++++
>> drivers/dpll/dpll_core.c           | 760 +++++++++++++++++++++++
>> drivers/dpll/dpll_core.h           | 176 ++++++
>> drivers/dpll/dpll_netlink.c        | 963 +++++++++++++++++++++++++++++
>> drivers/dpll/dpll_netlink.h        |  24 +
>> drivers/dpll/dpll_pin_attr.c       | 456 ++++++++++++++
>> drivers/ptp/Kconfig                |   1 +
>> drivers/ptp/ptp_ocp.c              | 123 ++--
>> include/linux/dpll.h               | 261 ++++++++
>> include/linux/dpll_attr.h          | 433 +++++++++++++
>> include/uapi/linux/dpll.h          | 263 ++++++++
>> 18 files changed, 4002 insertions(+), 37 deletions(-) create mode
>> 100644 Documentation/networking/dpll.rst create mode 100644
>> drivers/dpll/Kconfig create mode 100644 drivers/dpll/Makefile create
>> mode 100644 drivers/dpll/dpll_attr.c create mode 100644
>> drivers/dpll/dpll_core.c create mode 100644 drivers/dpll/dpll_core.h
>> create mode 100644 drivers/dpll/dpll_netlink.c create mode 100644
>> drivers/dpll/dpll_netlink.h create mode 100644
>> drivers/dpll/dpll_pin_attr.c create mode 100644 include/linux/dpll.h
>> create mode 100644 include/linux/dpll_attr.h create mode 100644
>> include/uapi/linux/dpll.h
>>
>>--
>>2.27.0
>>

Powered by blists - more mailing lists