Message-ID: <DM6PR11MB465767D550C3DF2CE24E70EA9BAEA@DM6PR11MB4657.namprd11.prod.outlook.com>
Date: Fri, 10 Nov 2023 08:50:34 +0000
From: "Kubalewski, Arkadiusz" <arkadiusz.kubalewski@...el.com>
To: Jiri Pirko <jiri@...nulli.us>
CC: Vadim Fedorenko <vadim.fedorenko@...ux.dev>, "netdev@...r.kernel.org"
	<netdev@...r.kernel.org>, "Michalik, Michal" <michal.michalik@...el.com>,
	"Olech, Milena" <milena.olech@...el.com>, "pabeni@...hat.com"
	<pabeni@...hat.com>, "kuba@...nel.org" <kuba@...nel.org>
Subject: RE: [PATCH net 3/3] dpll: fix register pin with unregistered parent
 pin

>From: Jiri Pirko <jiri@...nulli.us>
>Sent: Friday, November 10, 2023 7:48 AM
>
>Fri, Nov 10, 2023 at 12:21:11AM CET, arkadiusz.kubalewski@...el.com wrote:
>>>From: Jiri Pirko <jiri@...nulli.us>
>>>Sent: Thursday, November 9, 2023 7:04 PM
>>>
>>>Thu, Nov 09, 2023 at 05:02:48PM CET, arkadiusz.kubalewski@...el.com
>>>wrote:
>>>>>From: Vadim Fedorenko <vadim.fedorenko@...ux.dev>
>>>>>Sent: Thursday, November 9, 2023 11:56 AM
>>>>>To: Kubalewski, Arkadiusz <arkadiusz.kubalewski@...el.com>; Jiri Pirko
>>>>>
>>>>>On 09/11/2023 09:59, Kubalewski, Arkadiusz wrote:
>>>>>>> From: Jiri Pirko <jiri@...nulli.us>
>>>>>>> Sent: Wednesday, November 8, 2023 4:08 PM
>>>>>>>
>>>>>>> Wed, Nov 08, 2023 at 11:32:26AM CET, arkadiusz.kubalewski@...el.com
>>>>>>> wrote:
>>>>>>>> In case of multiple kernel module instances using the same dpll
>>>>>>>>device:
>>>>>>>> if only one registers dpll device, then only that one can register
>>>>>>>
>>>>>>> They why you don't register in multiple instances? See mlx5 for a
>>>>>>> reference.
>>>>>>>
>>>>>>
>>>>>> Every registration requires ops, but for our case only PF0 is able to
>>>>>> control dpll pins and device, thus only this can provide ops.
>>>>>> Basically without PF0, dpll is not able to be controlled, as well
>>>>>> as directly connected pins.
>>>>>>
>>>>>But why do you need other pins then, if FP0 doesn't exist?
>>>>>
>>>>
>>>>In general we don't need them at that point, but this is a corner case,
>>>>where users for some reason decided to unbind PF 0, and I treat this
>>>>state
>>>>as temporary, where dpll/pins controllability is temporarily broken.
>>>
>>>So resolve this broken situation internally in the driver, registering
>>>things only in case PF0 is present. Some simple notification infra would
>>>do. Don't drag this into the subsystem internals.
>>>
>>
>>Thanks for your feedback, but this is already wrong advice.
>>
>>Our HW/FW is designed in different way than yours, it doesn't mean it is
>>wrong.
>>As you might recall from our sync meetings, the dpll subsystem is to unify
>>approaches and reduce the code in the drivers, where your advice is
>>exactly
>
>No. Your driver knows when what objects are valid or not. Of a pin of
>PF1 is not valid because the "master" PF0 is gone, it is responsibility
>of your driver to resolve that. Don't bring this internal dependencies
>to the dpll core please, does not make any sense to do so. Thanks!
>

No, a driver doesn't know it, those are separated instances, and you already
suggested to implement special notification bus in the driver.
This is not needed and prone for another errors. The dpll subsystem is here to
make driver life easier.

Thank you!
Arkadiusz

>
>>opposite, suggested fix would require to implement extra synchronization
>>of the
>>dpll and pin registration state between driver instances, most probably
>>with
>>use of additional modules like aux-bus or something similar, which was
>>from the
>>very beginning something we tried to avoid.
>>Only ice uses the infrastructure of muxed pins, and this is broken as it
>>doesn't allow unbind the driver which have registered dpll and pins
>>without
>>crashing the kernel, so a fix is required in dpll subsystem, not in the
>>driver.
>>
>>Thank you!
>>Arkadiusz
>>
>>>
>>>>
>>>>The dpll at that point is not registered, all the direct pins are also
>>>>not registered, thus not available to the users.
>>>>
>>>>When I do dump at that point there are still 3 pins present, one for
>>>>each
>>>>PF, although they are all zombies - no parents as their parent pins are
>>>>not
>>>>registered (as the other patch [1/3] prevents dump of pin parent if the
>>>>parent is not registered). Maybe we can remove the REGISTERED mark for
>>>>all
>>>>the muxed pins, if all their parents have been unregistered, so they
>>>>won't
>>>>be visible to the user at all. Will try to POC that.
>>>>
>>>>>>>
>>>>>>>> directly connected pins with a dpll device. If unregistered parent
>>>>>>>> determines if the muxed pin can be register with it or not, it
>>>>>>>>forces
>>>>>>>> serialized driver load order - first the driver instance which
>>>>>>>> registers the direct pins needs to be loaded, then the other
>>>>>>>> instances
>>>>>>>> could register muxed type pins.
>>>>>>>>
>>>>>>>> Allow registration of a pin with a parent even if the parent was
>>>>>>>>not
>>>>>>>> yet registered, thus allow ability for unserialized driver instance
>>>>>>>
>>>>>>> Weird.
>>>>>>>
>>>>>>
>>>>>> Yeah, this is issue only for MUX/parent pin part, couldn't find
>>>>>>better
>>>>>> way, but it doesn't seem to break things around..
>>>>>>
>>>>>
>>>>>I just wonder how do you see the registration procedure? How can parent
>>>>>pin exist if it's not registered? I believe you cannot get it through
>>>>>DPLL API, then the only possible way is to create it within the same
>>>>>driver code, which can be simply re-arranged. Am I wrong here?
>>>>>
>>>>
>>>>By "parent exist" I mean the parent pin exist in the dpll subsystem
>>>>(allocated on pins xa), but it doesn't mean it is available to the
>>>>users,
>>>>as it might not be registered with a dpll device.
>>>>
>>>>We have this 2 step init approach:
>>>>1. dpll_pin_get(..) -> allocate new pin or increase reference if exist
>>>>2.1. dpll_pin_register(..) -> register with a dpll device
>>>>2.2. dpll_pin_on_pin_register -> register with a parent pin
>>>>
>>>>Basically:
>>>>- PF 0 does 1 & 2.1 for all the direct inputs, and steps: 1 & 2.2 for
>>>>its
>>>>  recovery clock pin,
>>>>- other PF's only do step 1 for the direct input pins (as they must get
>>>>  reference to those in order to register recovery clock pin with them),
>>>>  and steps: 1 & 2.2 for their recovery clock pin.
>>>>
>>>>
>>>>Thank you!
>>>>Arkadiusz
>>>>
>>>>>> Thank you!
>>>>>> Arkadiusz
>>>>>>
>>>>>>>
>>>>>>>> load order.
>>>>>>>> Do not WARN_ON notification for unregistered pin, which can be
>>>>>>>> invoked
>>>>>>>> for described case, instead just return error.
>>>>>>>>
>>>>>>>> Fixes: 9431063ad323 ("dpll: core: Add DPLL framework base
>>>>>>>>functions")
>>>>>>>> Fixes: 9d71b54b65b1 ("dpll: netlink: Add DPLL framework base
>>>>>>>> functions")
>>>>>>>> Signed-off-by: Arkadiusz Kubalewski
>>>>>>>><arkadiusz.kubalewski@...el.com>
>>>>>>>> ---
>>>>>>>> drivers/dpll/dpll_core.c    | 4 ----
>>>>>>>> drivers/dpll/dpll_netlink.c | 2 +-
>>>>>>>> 2 files changed, 1 insertion(+), 5 deletions(-)
>>>>>>>>
>>>>>>>> diff --git a/drivers/dpll/dpll_core.c b/drivers/dpll/dpll_core.c
>>>>>>>> index
>>>>>>>> 4077b562ba3b..ae884b92d68c 100644
>>>>>>>> --- a/drivers/dpll/dpll_core.c
>>>>>>>> +++ b/drivers/dpll/dpll_core.c
>>>>>>>> @@ -28,8 +28,6 @@ static u32 dpll_xa_id;
>>>>>>>> 	WARN_ON_ONCE(!xa_get_mark(&dpll_device_xa, (d)->id,
>>>>>>>> DPLL_REGISTERED))
>>>>>>>> #define ASSERT_DPLL_NOT_REGISTERED(d)	\
>>>>>>>> 	WARN_ON_ONCE(xa_get_mark(&dpll_device_xa, (d)->id,
>>>>>>>> DPLL_REGISTERED))
>>>>>>>> -#define ASSERT_PIN_REGISTERED(p)	\
>>>>>>>> -	WARN_ON_ONCE(!xa_get_mark(&dpll_pin_xa, (p)->id,
>>>>>>>> DPLL_REGISTERED))
>>>>>>>>
>>>>>>>> struct dpll_device_registration {
>>>>>>>> 	struct list_head list;
>>>>>>>> @@ -641,8 +639,6 @@ int dpll_pin_on_pin_register(struct dpll_pin
>>>>>>>> *parent,
>>>>>>>> struct dpll_pin *pin,
>>>>>>>> 	    WARN_ON(!ops->state_on_pin_get) ||
>>>>>>>> 	    WARN_ON(!ops->direction_get))
>>>>>>>> 		return -EINVAL;
>>>>>>>> -	if (ASSERT_PIN_REGISTERED(parent))
>>>>>>>> -		return -EINVAL;
>>>>>>>>
>>>>>>>> 	mutex_lock(&dpll_lock);
>>>>>>>> 	ret = dpll_xa_ref_pin_add(&pin->parent_refs, parent, ops,
>>>>>>>> priv); diff
>>>>>>>> --git a/drivers/dpll/dpll_netlink.c b/drivers/dpll/dpll_netlink.c
>>>>>>>> index
>>>>>>>> 963bbbbe6660..ff430f43304f 100644
>>>>>>>> --- a/drivers/dpll/dpll_netlink.c
>>>>>>>> +++ b/drivers/dpll/dpll_netlink.c
>>>>>>>> @@ -558,7 +558,7 @@ dpll_pin_event_send(enum dpll_cmd event, struct
>>>>>>>> dpll_pin *pin)
>>>>>>>> 	int ret = -ENOMEM;
>>>>>>>> 	void *hdr;
>>>>>>>>
>>>>>>>> -	if (WARN_ON(!xa_get_mark(&dpll_pin_xa, pin->id,
>>>>>>>> DPLL_REGISTERED)))
>>>>>>>> +	if (!xa_get_mark(&dpll_pin_xa, pin->id, DPLL_REGISTERED))
>>>>>>>> 		return -ENODEV;
>>>>>>>>
>>>>>>>> 	msg = genlmsg_new(NLMSG_GOODSIZE, GFP_KERNEL);
>>>>>>>> --
>>>>>>>> 2.38.1
>>>>>>>>
>>>>>
>>>>

Powered by blists - more mailing lists