linux-kernel - Re: [PATCH] staging: rtl8712: reduce stack usage

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <56C52A5F.2030302@lwfinger.net>
Date:	Wed, 17 Feb 2016 20:20:15 -0600
From:	Larry Finger <Larry.Finger@...inger.net>
To:	Arnd Bergmann <arnd@...db.de>,
	Florian Schilhabel <florian.c.schilhabel@...glemail.com>,
	Greg Kroah-Hartman <gregkh@...uxfoundation.org>
Cc:	linux-arm-kernel@...ts.infradead.org,
	Luis de Bethencourt <luisbg@....samsung.com>,
	Joshua Clayton <stillcompiling@...il.com>,
	devel@...verdev.osuosl.org, linux-kernel@...r.kernel.org
Subject: Re: [PATCH] staging: rtl8712: reduce stack usage

On 02/17/2016 02:32 AM, Arnd Bergmann wrote:
> The "translate_scan" function in rtl8712 uses a lot of stack, and
> gets inlined into its single caller, r8711_wx_get_scan, which
> in some configurations now blows the 1024 byte stack warning
> limit:
>
> drivers/staging/rtl8712/rtl871x_ioctl_linux.c: In function 'r8711_wx_get_scan':
> drivers/staging/rtl8712/rtl871x_ioctl_linux.c:1227:1: error: the frame size of 1032 bytes is larger than 1024 bytes [-Werror=frame-larger-than=]
>
> This somewhat reduces the stack usage by moving the translate_scan
> function out of line with the noinline_for_stack annotation.
> It might be possible to modify translate_scan() a little further
> to reduce the stack usage, but with this patch, we can build without
> the warning, the the call chain to get here is rather predictable
> (sys_ioctl->vfs_ioctl->sock_ioctl->dev_ioctl->wext_ioctl->
> r8711_wx_get_scan).
>
> Signed-off-by: Arnd Bergmann <arnd@...db.de>
> ---
>   drivers/staging/rtl8712/rtl871x_ioctl_linux.c | 2 +-
>   1 file changed, 1 insertion(+), 1 deletion(-)

As there is no reason for translate_scan() to be inlined, this patch seems 
reasonable The largest user of stack likely comes from the line "struct iw_event 
iwe". Changing that to a pointer, and using kalloc to acquire the space would 
likely clear the compile message, but that would require many more changes.

Acked-by: Larry Finger <Larry.Finger@...inger.net>

Larry

>
> diff --git a/drivers/staging/rtl8712/rtl871x_ioctl_linux.c b/drivers/staging/rtl8712/rtl871x_ioctl_linux.c
> index db2e31bcdd77..a15f3ce70223 100644
> --- a/drivers/staging/rtl8712/rtl871x_ioctl_linux.c
> +++ b/drivers/staging/rtl8712/rtl871x_ioctl_linux.c
> @@ -137,7 +137,7 @@ static inline void handle_group_key(struct ieee_param *param,
>   	}
>   }
>
> -static inline char *translate_scan(struct _adapter *padapter,
> +static noinline_for_stack char *translate_scan(struct _adapter *padapter,
>   				   struct iw_request_info *info,
>   				   struct wlan_network *pnetwork,
>   				   char *start, char *stop)
>