[<prev] [next>] [day] [month] [year] [list]
Date: Fri, 5 May 2017 18:00:17 +0200
From: DESBRUS Maxime <maxime.desbrus@...lesgroup.com>
To: "netdev@...r.kernel.org" <netdev@...r.kernel.org>
Subject: Question about packet_mmap & huge pages
Hello,
I am currently evaluating various 0-copy packet capture frameworks on Linux.
One of them is packet mmap, provided by the mainline Linux kernel.
I want to capture on one network interface with several raw sockets, each being used in a dedicated thread (to spread the processing load on several cores).
My first test program works well, however I am now trying to gain even more performance by mapping RX rings inside 1GB huge pages (other 0-copy network frameworks like DPDK do this).
My first question is: is it possible to map the rx ring of packet_mmap using huge pages, with the current Linux kernel?
If it is possible, what specific flags must be passed to mmap?
Here is what I tried so far:
* Calling mmap on the socket file descriptor (to set the rx ring address) with MAP_SHARED | MAP_HUGETLB | MAP_HUGE_1GB
=> mmap returns error EINVAL
* Calling mmap on the socket file descriptor (to set the rx ring address) with MAP_SHARED | MAP_ANONYMOUS | MAP_HUGETLB | MAP_HUGE_1GB
=> mmap succeeds and a huge page is allocated, but the memory area is all zero and does not contain the expected ring structures. I suspect the MAP_ANONYMOUS flag makes it ignore the socket file descriptor, and the mapping is thus unrelated to the ring.
* Calling mmap first to map a huge page area by opening a file on a hugetlbfs mount, and then calling mmap on the socket file descriptor (to set the ring address) with an address inside the previously mapped area
=> the address hint is ignored, and thus the ring is not mapped inside a huge page
I am using "tpacket_v3" packet_mmap version and my kernel version is based on 4.8 (on Ubuntu 16.04).
I reserve huge pages with the kernel boot command line, as recommended in Documentation/vm/hugetlbpage.txt.
Thank you in advance for your guidance
Powered by blists - more mailing lists