Date:   Tue, 23 Jan 2018 19:31:27 +0300
From:   Kirill Tkhai <ktkhai@...tuozzo.com>
To:     Eric Dumazet <edumazet@...gle.com>
Cc:     Eric Dumazet <eric.dumazet@...il.com>,
        netdev <netdev@...r.kernel.org>,
        David Miller <davem@...emloft.net>,
        David Ahern <dsahern@...il.com>,
        Florian Westphal <fw@...len.de>,
        Xin Long <lucien.xin@...il.com>,
        Daniel Borkmann <daniel@...earbox.net>,
        mschiffer@...verse-factory.net, jakub.kicinski@...ronome.com,
        Vladislav Yasevich <vyasevich@...il.com>,
        Jiri Benc <jbenc@...hat.com>
Subject: Re: [PATCH] net: Make synchronize_net() be expedited only when it's
 really need

On 23.01.2018 19:05, Eric Dumazet wrote:
> On Tue, Jan 23, 2018 at 7:57 AM, Kirill Tkhai <ktkhai@...tuozzo.com> wrote:
>> On 23.01.2018 18:45, Eric Dumazet wrote:
>>> On Tue, Jan 23, 2018 at 7:29 AM, Kirill Tkhai <ktkhai@...tuozzo.com> wrote:
>>>> On 23.01.2018 18:12, Eric Dumazet wrote:
>>>>> On Tue, Jan 23, 2018 at 6:41 AM, Kirill Tkhai <ktkhai@...tuozzo.com> wrote:
>>>>>> Hi, Eric,
>>>>>>
>>>>>> thanks for your review.
>>>>>>
>>>>>> On 22.01.2018 20:15, Eric Dumazet wrote:
>>>>>>> On Mon, 2018-01-22 at 12:41 +0300, Kirill Tkhai wrote:
>>>>>>>> Commit be3fc413da9e "net: use synchronize_rcu_expedited()" introducing
>>>>>>>> synchronize_net() says:
>>>>>>>>
>>>>>>>>     >When we hold RTNL mutex, we would like to spend some cpu cycles but not
>>>>>>>>     >block too long other processes waiting for this mutex.
>>>>>>>>     >We also want to setup/dismantle network features as fast as possible at
>>>>>>>>     >boot/shutdown time.
>>>>>>>>     >This patch makes synchronize_net() call the expedited version if RTNL is
>>>>>>>>     >locked.
>>>>>>>>
>>>>>>>> At the time of the commit (May 23 2011) there was no possible to differ,
>>>>>>>> who is the actual owner of the mutex. Only the fact that it's locked
>>>>>>>> by someone at the moment. So (I guess) this is the only reason the generic
>>>>>>>> primitive mutex_is_locked() was used.
>>>>>>>>
>>>>>>>> But now mutex owner is available outside the locking subsystem and
>>>>>>>> __mutex_owner() may be used instead (there is an example in audit_log_start()).
>>>>>>>> So, let's make expensive synchronize_rcu_expedited() be used only
>>>>>>>> when a caller really owns rtnl_mutex().
>>>>>>>>
>>>>>>>> There are several possibilities to fix that. The first one is
>>>>>>>> to fix synchronize_net(), the second is to change rtnl_is_locked().
>>>>>>>>
>>>>>>>> I prefer the second, as it seems it's more intuitive for people
>>>>>>>> to think that rtnl_is_locked() is about current process, not
>>>>>>>> about the fact mutex is locked in general. Grep over kernel
>>>>>>>> sources just proves this fact:
>>>>>>>>
>>>>>>>> drivers/staging/rtl8723bs/os_dep/osdep_service.c:297
>>>>>>>> drivers/staging/rtl8723bs/os_dep/osdep_service.c:316
>>>>>>>>
>>>>>>>>         if (!rtnl_is_locked())
>>>>>>>>                 ret = register_netdev(pnetdev);
>>>>>>>>         else
>>>>>>>>                 ret = register_netdevice(pnetdev);
>>>>>>>>
>>>>>>>> drivers/staging/wilc1000/linux_mon.c:310
>>>>>>>>
>>>>>>>>      if (rtnl_is_locked()) {
>>>>>>>>              rtnl_unlock();
>>>>>>>>              rollback_lock = true;
>>>>>>>>      }
>>>>>>>>
>>>>>>>> Side effect of this patch is three BUGs in above examples
>>>>>>>> become fixed.
>>>>>>>>
>>>>>>>> Signed-off-by: Kirill Tkhai <ktkhai@...tuozzo.com>
>>>>>>>> ---
>>>>>>>>  net/core/rtnetlink.c |    2 +-
>>>>>>>>  1 file changed, 1 insertion(+), 1 deletion(-)
>>>>>>>>
>>>>>>>> diff --git a/net/core/rtnetlink.c b/net/core/rtnetlink.c
>>>>>>>> index 16d644a4f974..a5ddf373ffa9 100644
>>>>>>>> --- a/net/core/rtnetlink.c
>>>>>>>> +++ b/net/core/rtnetlink.c
>>>>>>>> @@ -117,7 +117,7 @@ EXPORT_SYMBOL(rtnl_trylock);
>>>>>>>>
>>>>>>>>  int rtnl_is_locked(void)
>>>>>>>>  {
>>>>>>>> -    return mutex_is_locked(&rtnl_mutex);
>>>>>>>> +    return __mutex_owner(&rtnl_mutex) == current;
>>>>>>>>  }
>>>>>>>>  EXPORT_SYMBOL(rtnl_is_locked);
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>> Seems good to me, but this looks a net-next candidate to me.
>>>>>>
>>>>>> No objections. What for this may be need for net tree?! Only to fix
>>>>>> the staging drivers above. But AFAIR, staging drivers guarantees, which
>>>>>> the kernel gives, are that they may be compiled. If so, we do not need
>>>>>> this in net tree.
>>>>>>
>>>>>>> Note that this does not catch illegal uses from BH, where current is
>>>>>>> not related to our context of execution.
>>>>>>
>>>>>> It's true, but the patch is about reducing of synchronize_rcu_expedited()
>>>>>> calls.
>>>>>
>>>>> You have not touched only this path, but all paths using ASSERT_RTNL()
>>>>>
>>>>> This is why I think your patch would target net-next, not net tree.
>>>>>
>>>>>> There was no an objective to limit area of the places, where
>>>>>> rtnl_is_locked() can be used. For me it looks like another logical change.
>>>>>> If we really need that, one more patch on top of this may be submitted.
>>>>>> But honestly, I can't imagine someone really needs that check.
>>>>>
>>>>> I believe you missed ASSERT_RTNL(), used all over the place.
>>>>
>>>> Not missed. I grepped all over the kernel source, and this is how BUGs
>>>> in staging drivers were found. I just can't believe we really need
>>>> this check. Ok, then how about something like this:
>>>>
>>>> int rtnl_is_locked(void)
>>>> {
>>>> #ifdef CONFIG_DEBUG_KERNEL
>>>>         BUG_ON(!in_task());
>>>> #endif
>>>>         return __mutex_owner(&rtnl_mutex) == current;
>>>> }
>>>>
>>>> CONFIG_DEBUG_KERNEL is because of rtnl_is_locked() is used widely,
>>>> and the check has only the debug purpose.
>>>>
>>>
>>> So it looks you want to fix 3 bugs in staging, by changing
>>> rtnl_is_locked() semantic.
>>> This semantic had no recent changes (for last 10 years at least)
>>
>> No, I don't care about the staging. I care about excess actions
>> (interrupts), that synchronize_rcu_expedited() sends. I wrote about
>> that in patch description :)
> 
> This behavior is years old. What is suddenly the concern here?

This place is/has ambiguous/double sense and this is the reason
I sent the patch to make the behaviour unambiguous.
 
> If we are concerned about expedited stuff, why do we allow it in the
> first place ?

Since rtnl lock is the biggest network lock and it's used by many drivers
and net subsystems, we expedite RCU just to release it earlier. This is
the main reason it's expedited.

> Please provide numbers, experiments, bugs caused by this expedited thing,
> because right now I have no idea what problem you really have.
> 
> You are mixing several things in your patch attempt, and this is very confusing.

Your original patch did not provide any test results. Only the fact synchronize_rcu_expedited()
completes faster than plain synchronize_rcu(). But this is an obvious fact
just because of the design, and this is described even in the documentation.
Beleive me, I don't want to offend you this words, but it's strange you hadn't
gone the way you suggest me.

>>
>>> I am fine with such a change but for net-next tree.
>>> We are too late in linux-4.15 for such a change.
>>
>> Thanks for your review again. Could you, please, clarify, which change is
>> OK for you relatively to net-next: 1)w/o BUG_ON() or 2)with BUG_ON().
>> Sorry for that I ask, but I hadn't understand, which change you mean :(
> 
> I mean that I see nothing urgent needing a change in rtnl_is_locked()
> in net tree.
> 
> Now, if you need a temporary new rtnl_is_locked_by_me() I would not be
> against that.
> (to address staging bugs)
> 
> (See sock_owned_by_me() for one example...)

Oh. As I said I don't interested in stable net tree and have no objections
if this change is aimed for net-next tree only. Again, I don't interested
in staging bug and they are not the reason for the patch I send. But I'll
report to appropriate maintainers.

>>
>>> For net tree, please independently fix the staging bugs, that is less
>>> controversial
>>
>> Since I had no the staging devices, I'll report to their maintainers
>> after we found the final decision.

Powered by blists - more mailing lists