[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <CACSApvZU=G6D7rM+dfem0TAyk8Qbu4EkM=SehNSqRDwPTf0nmQ@mail.gmail.com>
Date: Thu, 15 Apr 2021 14:10:07 -0400
From: Soheil Hassas Yeganeh <soheil@...gle.com>
To: Eric Dumazet <eric.dumazet@...il.com>
Cc: "David S . Miller" <davem@...emloft.net>,
Jakub Kicinski <kuba@...nel.org>,
netdev <netdev@...r.kernel.org>,
Eric Dumazet <edumazet@...gle.com>
Subject: Re: [PATCH net-next] scm: optimize put_cmsg()
On Thu, Apr 15, 2021 at 1:38 PM Eric Dumazet <eric.dumazet@...il.com> wrote:
>
> From: Eric Dumazet <edumazet@...gle.com>
>
> Calling two copy_to_user() for very small regions has very high overhead.
>
> Switch to inlined unsafe_put_user() to save one stac/clac sequence,
> and avoid copy_to_user().
>
> Signed-off-by: Eric Dumazet <edumazet@...gle.com>
> Cc: Soheil Hassas Yeganeh <soheil@...gle.com>
Acked-by: Soheil Hassas Yeganeh <soheil@...gle.com>
Very nice! Thank you, Eric!
> ---
> net/core/scm.c | 21 ++++++++++++++-------
> 1 file changed, 14 insertions(+), 7 deletions(-)
>
> diff --git a/net/core/scm.c b/net/core/scm.c
> index 8156d4fb8a3966122fdfcfd0ebc9e5520aa7b67c..bd96c922041d22a2f3b7ee73e4b3183316f9b616 100644
> --- a/net/core/scm.c
> +++ b/net/core/scm.c
> @@ -228,14 +228,16 @@ int put_cmsg(struct msghdr * msg, int level, int type, int len, void *data)
>
> if (msg->msg_control_is_user) {
> struct cmsghdr __user *cm = msg->msg_control_user;
> - struct cmsghdr cmhdr;
>
> - cmhdr.cmsg_level = level;
> - cmhdr.cmsg_type = type;
> - cmhdr.cmsg_len = cmlen;
> - if (copy_to_user(cm, &cmhdr, sizeof cmhdr) ||
> - copy_to_user(CMSG_USER_DATA(cm), data, cmlen - sizeof(*cm)))
> - return -EFAULT;
> + if (!user_write_access_begin(cm, cmlen))
> + goto efault;
> +
> + unsafe_put_user(len, &cm->cmsg_len, efault_end);
> + unsafe_put_user(level, &cm->cmsg_level, efault_end);
> + unsafe_put_user(type, &cm->cmsg_type, efault_end);
> + unsafe_copy_to_user(CMSG_USER_DATA(cm), data,
> + cmlen - sizeof(*cm), efault_end);
> + user_write_access_end();
> } else {
> struct cmsghdr *cm = msg->msg_control;
>
> @@ -249,6 +251,11 @@ int put_cmsg(struct msghdr * msg, int level, int type, int len, void *data)
> msg->msg_control += cmlen;
> msg->msg_controllen -= cmlen;
> return 0;
> +
> +efault_end:
> + user_write_access_end();
> +efault:
> + return -EFAULT;
> }
> EXPORT_SYMBOL(put_cmsg);
>
> --
> 2.31.1.368.gbe11c130af-goog
>
Powered by blists - more mailing lists