Message-Id: <20251015151718.3927-1-dawei.li@linux.dev>
Date: Wed, 15 Oct 2025 23:17:15 +0800
From: Dawei Li <dawei.li@...ux.dev>
To: andersson@...nel.org,
	mathieu.poirier@...aro.org
Cc: linux-remoteproc@...r.kernel.org,
	linux-kernel@...r.kernel.org,
	dawei.li@...ux.dev,
	set_pte_at@...look.com
Subject: [PATCH v5 0/3] rpmsg: Introduce RPMSG_CREATE_EPT_FD_IOCTL uAPI

Hi,

This is V5 of series which introduce new uAPI(RPMSG_CREATE_EPT_FD_IOCTL)
for rpmsg subsystem.

Current uAPI implementation for rpmsg ctrl & char device manipulation is
abstracted in procedures below:
- fd = open("/dev/rpmsg_ctrlX")
- ioctl(fd, RPMSG_CREATE_EPT_IOCTL, &info); /dev/rpmsgY devnode is
  generated.
- fd_ep = open("/dev/rpmsgY", O_RDWR)
- operations on fd_ep(write, read, poll ioctl)
- ioctl(fd_ep, RPMSG_DESTROY_EPT_IOCTL)
- close(fd_ep)
- close(fd)

This /dev/rpmsgY abstraction is less favorable for:
- Performance issue: It's time consuming for some operations are
involved:
  - Device node creation.
    Depends on specific config, especially CONFIG_DEVTMPFS, the overall
    overhead is based on coordination between DEVTMPFS and userspace
    tools such as udev and mdev.
  - Extra kernel-userspace switch cost.
  - Other major costs brought by heavy-weight logic like device_add().

- /dev/rpmsgY node can be opened only once. It doesn't make much sense
    that a dynamically created device node can be opened only once.

- For some container application such as docker, a client can't access
  host's dev unless specified explicitly. But in case of /dev/rpmsgY, which
  is generated dynamically and whose existence is unknown for clients in
  advance, this uAPI based on device node doesn't fit well.

An anonymous inode based approach is introduced to address the issues
above. Rather than generating device node and opening it, rpmsg code just
creates an anonymous inode representing eptdev and return the fd to
userspace.

Performance demo

A simple C application is tested to verify performance of new uAPI.
Please be noted that all '#' in code are preceded with space to suppress
checkpatch complaints.

$ cat test.c

 #include <linux/rpmsg.h>

 #include <sys/types.h>
 #include <sys/stat.h>
 #include <sys/ioctl.h>
 #include <fcntl.h>
 #include <string.h>
 #include <stdio.h>
 #include <unistd.h>
 #include <stdlib.h>
 #include <errno.h>
 #include <sys/time.h>

 #define N (1 << 20)

int main(int argc, char *argv[])
{
	int ret, fd, ep_fd, loop;
	struct rpmsg_endpoint_info info;
	struct rpmsg_endpoint_fd_info fd_info;
	struct timeval start, end;
	int i = 0;
	double t1, t2;

	fd = -1;
	ep_fd = -1;
	loop = N;

	if (argc == 1) {
		loop = N;
	} else if (argc > 1) {
		loop = atoi(argv[1]);
	}

	printf("loop[%d]\n", loop);

	strcpy(info.name, "epx");
	info.src = -1;
	info.dst = -1;

	strcpy(fd_info.name, "epx");
	fd_info.src = -1;
	fd_info.dst = -1;
	fd_info.fd = -1;

	while (fd < 0) {
		fd = open("/dev/rpmsg_ctrl0", O_RDWR);
		if (fd < 0) {
			printf("open rpmsg_ctrl0 failed, fd[%d]\n", fd);
		}
	}

	gettimeofday(&start, NULL);

	while (loop--) {
		ret = ioctl(fd, RPMSG_CREATE_EPT_IOCTL, &info);
		if (ret < 0) {
			printf("ioctl[RPMSG_CREATE_EPT_IOCTL] failed,
			ret[%d]\n", ret);
		}

		ep_fd = -1;
		i = 0;

		while (ep_fd < 0) {
			ep_fd = open("/dev/rpmsg0", O_RDWR);
			if (ep_fd < 0) {
				i++;
				printf("open rpmsg0 failed, epfd[%d]\n", ep_fd);
			}
		}

		ret = ioctl(ep_fd, RPMSG_DESTROY_EPT_IOCTL, &info);
		if (ret < 0) {
			printf("old RPMSG_DESTROY_EPT_IOCTL failed, ret[%d], errno[%d]\n",
			ret, errno);
		}

		close(ep_fd);
	}
	
	gettimeofday(&end, NULL);

	printf("time for old way: [%ld] us\n",
		1000000 * (end.tv_sec - start.tv_sec) + end.tv_usec - start.tv_usec);
	t1 = 1000000 * (end.tv_sec - start.tv_sec) + end.tv_usec - start.tv_usec;

	if (argc == 1) {
		loop = N;
	} else if (argc > 1) {
		loop = atoi(argv[1]);
	}

	printf("loop[%d]\n", loop);

	gettimeofday(&start, NULL);

	while (loop--) {
		fd_info.fd = -1;
		fd_info.flags = O_RDWR | O_CLOEXEC | O_NONBLOCK;
		ret = ioctl(fd, RPMSG_CREATE_EPT_FD_IOCTL, &fd_info);
		if (ret < 0 || fd_info.fd < 0) {
			printf("ioctl[RPMSG_CREATE_EPT_FD_IOCTL] failed, ret[%d]\n", ret);
		}

		ret = ioctl(fd_info.fd, RPMSG_DESTROY_EPT_IOCTL, &info);
		if (ret < 0) {
			printf("new ioctl[RPMSG_DESTROY_EPT_IOCTL] failed, ret[%d]\n", ret);
		}

		close(fd_info.fd);
	}
	
	gettimeofday(&end, NULL);

	printf("time for new way: [%ld] us\n",
	1000000 * (end.tv_sec - start.tv_sec) + end.tv_usec - start.tv_usec);
	t2 = 1000000 * (end.tv_sec - start.tv_sec) + end.tv_usec - start.tv_usec;

	printf("t1(old) / t2(new) = %f\n", t1 / t2);

	close(fd);
}

Performance benchmark

- Legacy means benchmark based on old uAPI
- New means benchmark based on new uAPI(the one this series introduce)
- Time are in units of us(10^-6 s)

Test	loops	Total time(legacy)	Total time(new)	legacy/new	
1	1000	148362			1153		128.674761	
2	1000	145640			1121		129.919715	
3	1000	145303			1174		123.767462	
4	1000	150294			1142		131.605954	
5	1000	160877			1175		136.916596	
6	1000	154400			1134		136.155203	
7	1000	143252			1163		123.174549	
8	1000	148758			1161		128.129199
9	1000	149044			1112		134.032374
10	1000	146895			1192		123.234060
11	10000	1428967			11627		122.900748
12	10000	1367015			10557		129.488965
13	10000	1371919			11663		117.630027
14	10000	1358447			11080		122.603520
15	10000	1375463			11245		122.317741
16	10000	1364901			11153		122.379718
17	10000	1352665			10735		126.005123
18	10000	1400873			11341		123.522882
19	10000	1391276			10892		127.733750
20	10000	1394367			11110		125.505581
21	100000	14069671		115569		121.742604
22	100000	13663364		117074		116.707074
23	100000	13735740		115638		118.782234
24	100000	13714441		119362		114.897882
25	100000	13904366		118282		117.552679
26	100000	13870560		117717		117.829710
27	100000	13713605		118312		115.910516
28	100000	13872852		114347		121.322396
29	100000	13777964		119072		115.711200
30	100000	13725654		116296		118.023440

Changelog:

Changes in v5:
- Rebased on v6.18.rc1.
- Fix checkpatch warning on commit msg on patch[1/3].
- Other minor commit msg tweaks.
- Update performance testing results.
- Link to v4:
  https://lore.kernel.org/all/20250609151531.22621-1-dawei.li@linux.dev/

Changes in v4:
- Build warning of copy_to_user (Dan).
- ioctl() branches reorder (Beleswar).
- Remove local variable fd and pass &ept_fd_info.fd to
  rpmsg_anonymous_eptdev_create().
- Link to v3:
  https://lore.kernel.org/all/20250519150823.62350-1-dawei.li@linux.dev/

Changes in v3:
- s/anon/anonymous (Mathieu)
- API naming adjustment (Mathieu)
  - __rpmsg_chrdev_eptdev_alloc ->  rpmsg_eptdev_alloc
  - __rpmsg_chrdev_eptdev_add ->  rpmsg_eptdev_add
- Add parameter 'flags' to uAPI so user can specify file flags
  explicitly on creating anonymous inode.
- Link to v2:
  https://lore.kernel.org/all/20250509155927.109258-1-dawei.li@linux.dev/

Changes in v2:
- Fix compilation error for !CONFIG_RPMSG_CHAR config(Test robot).
- Link to v1:
  https://lore.kernel.org/all/20250507141712.4276-1-dawei.li@linux.dev/

Dawei Li (3):
  rpmsg: char: Reuse eptdev logic for anonymous device
  rpmsg: char: Implement eptdev based on anonymous inode
  rpmsg: ctrl: Introduce RPMSG_CREATE_EPT_FD_IOCTL uAPI

 drivers/rpmsg/rpmsg_char.c | 129 ++++++++++++++++++++++++++++++-------
 drivers/rpmsg/rpmsg_char.h |  23 +++++++
 drivers/rpmsg/rpmsg_ctrl.c |  35 ++++++++--
 include/uapi/linux/rpmsg.h |  27 +++++++-
 4 files changed, 182 insertions(+), 32 deletions(-)

---

Thanks,

	Dawei

-- 
2.25.1

Powered by blists - more mailing lists