netdev - Re: [PATCH net] team: fix deadlock during setting of MTU

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:   Thu, 14 Jan 2021 16:34:24 -0800
From:   Saeed Mahameed <saeed@...nel.org>
To:     Ivan Vecera <ivecera@...hat.com>, netdev@...r.kernel.org
Cc:     Jiri Pirko <jiri@...nulli.us>
Subject: Re: [PATCH net] team: fix deadlock during setting of MTU

On Thu, 2021-01-14 at 12:55 +0100, Ivan Vecera wrote:
> Team driver protects port list traversal in team_change_mtu()
> by its team->lock mutex. This causes a deadlock with certain
> devices that calls netdev_update_features() during their
> .ndo_change_mtu() callback. In this case netdev_update_features()
> calls team's netdevice notifier team_device_event() that in case
> of NETDEV_FEAT_CHANGE tries lock team->lock mutex again.
> 
> Example (r8169 case):
> ...
> [ 6391.348202]  __mutex_lock.isra.6+0x2d0/0x4a0
> [ 6391.358602]  team_device_event+0x9d/0x160 [team]
> [ 6391.363756]  notifier_call_chain+0x47/0x70
> [ 6391.368329]  netdev_update_features+0x56/0x60
> [ 6391.373207]  rtl8169_change_mtu+0x14/0x50 [r8169]
> [ 6391.378457]  dev_set_mtu_ext+0xe1/0x1d0
> [ 6391.387022]  dev_set_mtu+0x52/0x90
> [ 6391.390820]  team_change_mtu+0x64/0xf0 [team]
> [ 6391.395683]  dev_set_mtu_ext+0xe1/0x1d0
> [ 6391.399963]  do_setlink+0x231/0xf50
> ...
> 
> To fix the problem the port list traversal in team_change_mtu() can
> be protected by RCU read lock. In case of failure the failing port
> is marked and unwind code-path is done also under RCU read lock
> protection (but not in reverse order).
> 
> Fixes: 3d249d4ca7d0 ("net: introduce ethernet teaming device")
> Cc: Jiri Pirko <jiri@...nulli.us>
> Signed-off-by: Ivan Vecera <ivecera@...hat.com>
> ---
>  drivers/net/team/team.c | 20 ++++++++++----------
>  1 file changed, 10 insertions(+), 10 deletions(-)
> 
> diff --git a/drivers/net/team/team.c b/drivers/net/team/team.c
> index c19dac21c468..69d4b28beb17 100644
> --- a/drivers/net/team/team.c
> +++ b/drivers/net/team/team.c
> @@ -1802,35 +1802,35 @@ static int team_set_mac_address(struct
> net_device *dev, void *p)
>  static int team_change_mtu(struct net_device *dev, int new_mtu)
>  {
>  	struct team *team = netdev_priv(dev);
> -	struct team_port *port;
> +	struct team_port *port, *fail_port;
>  	int err;
>  
> -	/*
> -	 * Alhough this is reader, it's guarded by team lock. It's not
> possible
> -	 * to traverse list in reverse under rcu_read_lock
> -	 */
> -	mutex_lock(&team->lock);
> +	rcu_read_lock();
>  	team->port_mtu_change_allowed = true;
> -	list_for_each_entry(port, &team->port_list, list) {
> +	list_for_each_entry_rcu(port, &team->port_list, list) {
>  		err = dev_set_mtu(port->dev, new_mtu);

some netdevs will need to sleep in their set_mtu, and you can't sleep
under rcu!

according to your explanation in the commit message the team->lock
mutex will be also taken under this rcu lock, so this is bad even
if dev_set_mtu does not sleep.