Message-Id: <1473692159-4017-27-git-send-email-kan.liang@intel.com>
Date:   Mon, 12 Sep 2016 07:55:59 -0700
From:   kan.liang@...el.com
To:     davem@...emloft.net, linux-kernel@...r.kernel.org,
        netdev@...r.kernel.org
Cc:     jeffrey.t.kirsher@...el.com, mingo@...hat.com,
        peterz@...radead.org, kuznet@....inr.ac.ru, jmorris@...ei.org,
        yoshfuji@...ux-ipv6.org, kaber@...sh.net,
        akpm@...ux-foundation.org, keescook@...omium.org,
        viro@...iv.linux.org.uk, gorcunov@...nvz.org,
        john.stultz@...aro.org, aduyck@...antis.com, ben@...adent.org.uk,
        decot@...glers.com, fw@...len.de, alexander.duyck@...il.com,
        daniel@...earbox.net, tom@...bertland.com, rdunlap@...radead.org,
        xiyou.wangcong@...il.com, hannes@...essinduktion.org,
        stephen@...workplumber.org, alexei.starovoitov@...il.com,
        jesse.brandeburg@...el.com, andi@...stfloor.org,
        Kan Liang <kan.liang@...el.com>
Subject: [RFC V3 PATCH 26/26] Documentation/networking: Document NET policy

From: Kan Liang <kan.liang@...el.com>

Signed-off-by: Kan Liang <kan.liang@...el.com>
---
 Documentation/networking/netpolicy.txt | 157 +++++++++++++++++++++++++++++++++
 1 file changed, 157 insertions(+)
 create mode 100644 Documentation/networking/netpolicy.txt

diff --git a/Documentation/networking/netpolicy.txt b/Documentation/networking/netpolicy.txt
new file mode 100644
index 0000000..b8e3d4c
--- /dev/null
+++ b/Documentation/networking/netpolicy.txt
@@ -0,0 +1,157 @@
+What is Linux Net Policy?
+
+It is a big challenge to get good network performance. First, the network
+performance is not good with default system settings. Second, it is too
+difficult to do automatic tuning for all possible workloads, since workloads
+have different requirements. Some workloads may want high throughput. Some may
+need low latency. Last but not least, there are lots of manual configurations.
+Fine grained configuration is too difficult for users.
+
+"NET policy" intends to simplify the network configuration and get a
+good network performance according to the hints(policy) which is applied by
+user. It provides some typical "policies" for user which can be set
+per-socket, per-task or per-device. The kernel automatically figures out
+how to merge different requests to get good network performance.
+
+"Net policy" is designed for multiqueue network devices. This document
+describes the concepts and APIs of "net policy" support.
+
+NET POLICY CONCEPTS
+
+Scope of Net Policies
+
+    Device net policy: this policy applies to the whole device. Once the
+    device net policy is set, it automatically configures the system
+    according to the applied policy. The configuration usually includes IRQ
+    affinity, IRQ balance disable, interrupt moderation, and so on. But the
+    device net policy does not change the packet direction.
+
+    Task net policy: this is a per-task policy. When it is applied to specific
+    task, all packet transmissions of the task will be redirected to the
+    assigned queues accordingly. If a task does not define a task policy,
+    it "falls back" to the system default way to direct the packets. The
+    per-task policy must be compatible with device net policy.
+
+    Socket net policy: this is a per-socket policy. When it is applied to
+    specific socket, all packet transmissions of the socket will be redirected
+    to the assigned queues accordingly. If a socket does not define a socket
+    policy, it "falls back" to the system default way to direct the packets.
+    The per-socket policy must be compatible with both device net policy and
+    per-task policy.
+
+Components of Net Policies
+
+    Net policy object: it is a combination of CPU and queue. The queue IRQ has
+    to set affinity with the CPU. It can be shared between sockets and tasks.
+    A reference counter is used to track the sharing number.
+
+    Net policy object list: each device policy has an object list. Once the
+    device policy is determined, the net policy object will be inserted into
+    the net policy object list. The net policy object list does not change
+    unless the CPU/queue number is changed, the netpolicy is disabled or
+    the device policy is changed.
+    The network performance for objects could be different because of the
+    CPU/queue topology and dev location. The objects which can bring high
+    performance are in the front of the list.
+
+    RCU hash table: an RCU hash table to maintain the relationship between
+    the task/socket and the assigned object. The task/socket can get the
+    assigned object by searching the table.
+    If it is the first time, there is no assigned object in the table. It will
+    go through the object list to find the available object based on position
+    and reference number.
+    If the net policy object list changes, all the assigned objects will become
+    invalid.
+
+NET POLICY APIs
+
+Interfaces between net policy and device driver
+
+    int (*ndo_netpolicy_init)(struct net_device *dev,
+                              struct netpolicy_info *info);
+
+    The device driver who has NET policy support must implement this interface.
+    In this interface, the device driver does necessory initialization, and fill
+    the info for net policy module. The information could include supported
+    policy, MIX policy support, queue pair support and so on.
+
+    int (*ndo_get_irq_info)(struct net_device *dev,
+                            struct netpolicy_dev_info *info);
+
+    This interface is used to get more accurate device IRQ information.
+
+    int (*ndo_set_net_policy)(struct net_device *dev,
+                              enum netpolicy_name name);
+
+    This interface is used to set device net policy by name. It is device driver's
+    responsibility to set driver specific configuration for the given policy.
+
+Interfaces between net policy and kernel
+
+    int netpolicy_register(struct netpolicy_instance *instance);
+    void netpolicy_unregister(struct netpolicy_instance *instance);
+
+    This interface is used to register per task/socket net policy.
+    The socket/task can only be benefited when it register itself with specific
+    policy. After registeration, a record will be created and inserted into RCU
+    hash table, which include all the NET policy related information for the
+    socket/task, such as pointor, policy, object and so on.
+
+    int netpolicy_pick_queue(struct netpolicy_instance *instance, bool is_rx);
+
+    This interface is used to find the proper queue(object) for packet
+    receiving and transmitting. The proper queue is picked from object list
+    according to policy, reference, location and so on.
+
+
+    int netpolicy_set_rules(struct netpolicy_instance *instance);
+
+    This interface is used to add device specific rules. Once the rule is
+    applied, the packet from specific IP and port will be redirected to the
+    given queue. This interface is usually used on receive side.
+
+NET POLICY INTERFACE
+
+Device net policy setting
+
+    /proc/net/netpolicy/$DEV/policy
+
+    Concatenating(cat) the "policy" file can show the available device
+    policies, if there is no device policy applied. Otherwise, the device
+    policy name will be printed out. If it is MIX policy, the policy for each
+    queue will also be printed out.
+    User can set device net policy by writing policy name.
+
+Task policy setting
+
+    /proc/$PID/net_policy
+
+    Concatenating(cat) the "net_policy" file can show the applied per task
+    policy.
+    User can set per task net policy by writing policy name.
+
+    OR
+
+    prctl(PR_SET_NETPOLICY, POLICY_NAME, NULL, NULL, NULL)
+
+    "prctl" is an alternative way to set/get per task policy.
+
+Socket policy setting
+
+    setsockopt(sockfd,SOL_SOCKET,SO_NETPOLICY,&policy,sizeof(int))
+
+    The socket net policy can be set by option SO_NETPOLICY of setsockopt.
+
+AVAILABLE NET POLICIES
+
+    The available net policies are defined as below:
+    - CPU: intends to get higher throughput and lower CPU% (power saving).
+           This policy can be applied as either device net policy or
+           task/socket net policy.
+    - BULK: intends to get highest throughput. This policy can be applied as
+            either device net policy or task/socket net policy.
+    - LATENCY: intends to get lowest latency. This policy can be applied as
+               either device net policy or task/socket net policy.
+    - MIX: combination of other policies, which allows each queue to have a
+           different policy. This policy can only be set as device net policy.
+
-- 
2.5.5

Powered by blists - more mailing lists