| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Message-Id: <1423100070-31848-1-git-send-email-dsahern@gmail.com>
Date: Wed, 4 Feb 2015 18:34:01 -0700
From: David Ahern <dsahern@...il.com>
To: netdev@...r.kernel.org
Cc: ebiederm@...ssion.com, David Ahern <dsahern@...il.com>
Subject: [RFC PATCH 00/29] net: VRF support
Kernel patches are also available here:
https://github.com/dsahern/linux.git vrf-3.19
iproute2 patches are also available here:
https://github.com/dsahern/iproute2 vrf-3.19
Background
----------
The concept of VRFs (Virtual Routing and Forwarding) has been around for over
15 years. Support for VRFs in the Linux kernel has been an often requested
feature for almost as long. For a while support was available via an out of
tree patch [1]. Since network namespaces came along, the response to queries
about VRF support for Linux was 'use namespaces'. But as mentioned previously
[2] network namespaces are not a good match for VRFs. Of the list of problems
noted the big one is that namespaces do not scale efficiently to the number
of VRFs supported by networking gear (> 1000 VRFs). Networking vendors that
want to use Linux as the OS have to carry custom solutions to this problem --
be it userspace networking stacks, extensive kernel patches (to add VRF
support or bend the implementation of namespaces), and/or patches to many
open source components. The recent addition of switchdev support in the
kernel suggests that people expect the use of Linux as a switch networking
OS to increase. Hopefully the time is right to re-open the discussion on a
salable VRF implementation for the Linux kernel.
The intent of this RFC is to get feedback on the overall idea - namely VRFs
as integer id and the nesting of VRFs within a namespace. This set includes
changes only to core IPv4 code which shows the concept; changes to the rest
of the network stack are fairly repetitive.
This patch set has a number of similarities to the original VRF patch - most
notably VRF ids as an integer index and plumbing through iproute2 and
netlink. But this set is really a complete re-implementation of the feature,
integrating VRF within a namespace and leveraging existing support for
network namespaces.
Design
------
Namespaces provide excellent separation of the networking stack from the
netdevices and up. The intent of VRFs is to provide an additional,
logical separation at the L3 layer within a namespace.
+----------------------------------------------------------+
| Namespace foo |
| +---------------+ |
| +------+ | L3/L4 service | |
| | lldp | | (VRF any) | |
| +------+ +---------------+ |
| |
| +-------------------------+ |
| | VRF M | |
| +---------------------+ +-------------------------+ | |
| | VRF 1 (default) | | VRF N | | |
| | +---------------+ | | +---------------+ | | |
| | | L3/L4 service | | | | L3/L4 service | | | |
| | | (VRF unaware) | | | | (VRF unaware) | | | |
| | +---------------+ | | +---------------+ | | |
| | | | | | |
| |+-----+ +----------+ | | +-----+ +----------+ | | |
| || FIB | | neighbor | | | | FIB | | neighbor | | | |
| |+-----+ +----------+ | | +-----+ +----------+ | | |
| | | | |-+ |
| | {dev 1} {dev 2} | | {dev 3} {dev 4} {dev 5} | |
| +---------------------+ +-------------------------+ |
+----------------------------------------------------------+
This is accomplished by enhancing the current namespace checks to a
broader network context that is both a namepsace and a VRF id. The VRF
id is a tag applied to relevant structures, an integer between 1 and 4095
which allows for 4095 VRFs (could have 0 be the default VRF and then the
range is 0-4095 = 4096s VRFs). (The limitation is arguably artificial. It
is based on the genid scheme for versioning networking data which is a
32-bit integer. The VRF id is the lower 12 bits of the genid's.)
Netdevices, sk_buffs, sockets, and tasks are all tagged with a VRF id.
Network lookups (devices, sockets, addresses, routes, neighbors) require a
match of both network namespace and VRF id (or the special 'vrf any' tag;
more on that later).
Beyond the 4-byte tag in various data structures, there are no resources
allocated to a VRF so there is no need to create or destroy a VRF which is
in-line with the concept of keeping it lightweight for scalability. The
trade-off is that VRFs use the the same sysctl settings as the namespace
they are part of and, for example, MIB counters.
The VRF id of tasks defaults to 1 and is inherited parent to child. It can
be read via the file '/proc/<pid>/vrf' and can be changed anytime by writing
to this file (if preferred this can be made a prctl to change the VRF id).
This allows services to be launched in a VRF context using ip, similar to
what is done for network namespaces.
e.g., ip vrf exec 99 /usr/sbin/sshd
(or a simpler chvrf alias/command can be used to just write the VRF id
to the proc file.)
The task's VRF id also affects viewing and modifying network configuration.
For example, 'ip addr show', 'ip route ls', 'ifconfig', 'arp -n', etc, only
show network data for the VRF associated with the task's VRF id; devices
are at the L2 layer so a command listing devices is not impacted by VRF id.
When a socket is created the VRF id is taken from the task. Socket-vrf
association for non-connected sockets can be changed using a setsockopt
(e.g., create a socket then change VRF id prior to calling bind or connect).
Network devices belong to a single VRF context which defaults to VRF 1.
They can be assigned to another VRF using IFLA_VRF attribute in link
messages. Similarly the VRF assignment is returned in the IFLA_VRF
attribute. The ip command has been modified to display the VRF id of a
device. L2 applications like lldp are not VRF aware and still work through
through all network devices within the namespace.
On RX skbs get their VRF context from the netdevice the packet is received
on. For TX the VRF context for an skb is taken from the socket. The
intention is for L3/raw sockets to be able to set the VRF context for a
packet TX using cmsg (not coded in this patch set).
VRF aware apps (e.g., L3 VPNs) can have sockets in multiple VRFs for
forwarding packets.
The special 'ANY VRF' context allows a single instance of a daemon to
provide a service across all VRFs.
e.g., ip vrf exec any /usr/sbin/sshd
The 'any' context applies to listen sockets only; connected sockets are in
a VRF context. Child sockets accepted by the daemon acquire the VRF context
of the network device the connection originated on.
The 'ANY VRF' context can also be used to display all addresses, routes
or neighbors in the kernel cache. That is, 'ip addr show', 'ip route ls',
'ifconfig', 'arp -n', etc, show all network data for the namespace.
About this Patch Set
--------------------
This is not a complete conversion of the networking stack, only a small
sampling to test the waters. Only changes are to core IPv4 code [2] which
is sufficient to illustrate the fundamental concept. Changes from
struct net to net_ctx are very repetitive.
I'm sure there are a lot of oversights and bugs, but the intent here is
to solicit feedback on the overall idea.
Examples
--------
To illustrate the VRF patches consider a system with 18 NICs:
- eth0, eth17 are in default namespace (e.g., management namespace)
- eth1 - eth8 are in group1 namespace
- eth1 - eth4 are in VRF 11
- eth5 - eth8 are in VRF 13
- eth9 - eth16 are in group2 namespace
- eth9 - eth12 are in VRF 21
- eth13 - eth16 are in VRF 23
- Addresses assigned to each interface:
- eth1: 1.1.1.1/24
- eth2: 2.2.2.1/24
- eth3: 3.3.3.1/24
- eth4: 4.4.4.1/24
- eth5: 1.1.1.1/24 (not a typo, duplicate address in different vrfs)
- eth6: 6.6.6.1/24
- eth7: 7.7.7.1/24
- eth8: 8.8.8.1/24
- openlldpd is started in each namespace
1. device list is VRF agnostic
- ifconfig -a, ip link show, /proc/net/dev
--> default namespace shows only eth0 and eth17
--> group1 namespace shows only eth1 - eth8
--> group2 namespace shows only eth9 - eth16
- ip shows vrf assignment of each link
3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 vrf 11 qdisc pfifo_fast state UP mode DEFAULT group default qlen 1000
link/ether 02:ab:cd:02:00:01 brd ff:ff:ff:ff:ff:ff
2. address, route, neighbor list is VRF aware
- ifconfig, ip addr show, ip route ls, /proc/net/route
--> shows only addresses for VRF id of task unless id is 'any'
in VRF 1:
ifconfig eth1
eth1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
ether 02:ab:cd:02:00:01 txqueuelen 1000 (Ethernet)
...
No addresses are shown. But if the command is run in VRF 11 or VRF 'any'
ip vrf exec 11 ip addr show dev eth1
3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 vrf 11 qdisc pfifo_fast state UP group default qlen 1000
link/ether 02:ab:cd:02:00:01 brd ff:ff:ff:ff:ff:ff
inet 1.1.1.1/24 brd 1.1.1.255 scope global eth1
valid_lft forever preferred_lft forever
3. start ssh in group1 namespace
ip netns exec group1 ip vrf exec 11 /usr/sbin/sshd -d
ssh to 1.1.1.1 via eth1
ip netns exec group1 ip vrf exec 13 /usr/sbin/sshd -d
ssh to 1.1.1.1 via eth5
--> same namespace but different VRFs
4. One ssh instance handles VRFs in group1 namespace
ip netns exec group1 ip vrf exec any /usr/sbin/sshd
--> ssh to any address in the namespace works
References
----------
[1] http://sourceforge.net/projects/linux-vrf
[2] http://www.spinics.net/lists/netdev/msg298368.html
[3] To build only enable core ipv4 code. Disable IPv6, netfilter, ipsec, etc.
David Ahern (29):
net: Introduce net_ctx and macro for context comparison
net: Flip net_device to use net_ctx
net: Flip sock_common to net_ctx
net: Add net_ctx macros for skbuffs
net: Flip seq_net_private to net_ctx
net: Flip fib_rules and fib_rules_ops to use net_ctx
net: Flip inet_bind_bucket to net_ctx
net: Flip fib_info to net_ctx
net: Flip ip6_flowlabel to net_ctx
net: Flip neigh structs to net_ctx
net: Flip nl_info to net_ctx
net: Add device lookups by net_ctx
net: Convert function arg from struct net to struct net_ctx
net: vrf: Introduce vrf header file
net: vrf: Add vrf to net_ctx struct
net: vrf: Set default vrf
net: vrf: Add vrf context to task struct
net: vrf: Plumbing for vrf context on a socket
net: vrf: Add vrf context to skb
net: vrf: Add vrf context to flow struct
net: vrf: Add vrf context to genid's
net: vrf: Set VRF id in various network structs
net: vrf: Enable vrf checks
net: vrf: Add support to get/set vrf context on a device
net: vrf: Handle VRF any context
net: vrf: Change single_open_net to pass net_ctx
net: vrf: Add vrf checks and context to ipv4 proc files
iproute2: vrf: Add vrf subcommand
iproute2: Add vrf option to ip link command
fs/proc/base.c | 94 +++++++++++++++++++++++++
fs/proc/proc_net.c | 22 +++++-
include/linux/inetdevice.h | 12 ++--
include/linux/init_task.h | 1 +
include/linux/netdevice.h | 44 +++++++++++-
include/linux/sched.h | 2 +
include/linux/seq_file_net.h | 10 +--
include/linux/skbuff.h | 5 ++
include/net/addrconf.h | 22 +++---
include/net/arp.h | 2 +-
include/net/dst.h | 16 ++---
include/net/fib_rules.h | 10 ++-
include/net/flow.h | 10 ++-
include/net/inet6_hashtables.h | 19 +++---
include/net/inet_hashtables.h | 60 ++++++++++------
include/net/inet_sock.h | 1 +
include/net/inet_timewait_sock.h | 1 +
include/net/ip.h | 10 +--
include/net/ip6_fib.h | 4 +-
include/net/ip6_route.h | 24 +++----
include/net/ip_fib.h | 38 +++++++----
include/net/ipv6.h | 14 +++-
include/net/neighbour.h | 93 +++++++++++++++++++++----
include/net/net_namespace.h | 39 +++++++++--
include/net/netlink.h | 5 +-
include/net/route.h | 46 +++++++------
include/net/sock.h | 21 ++++--
include/net/tcp.h | 1 +
include/net/transp_v6.h | 2 +-
include/net/udp.h | 8 +--
include/net/vrf.h | 36 ++++++++++
include/net/xfrm.h | 28 ++++----
include/uapi/linux/if_link.h | 1 +
include/uapi/linux/in.h | 1 +
kernel/fork.c | 2 +
net/core/dev.c | 95 +++++++++++++++++++++++---
net/core/fib_rules.c | 36 ++++++----
net/core/flow.c | 5 +-
net/core/neighbour.c | 106 +++++++++++++++--------------
net/core/rtnetlink.c | 12 ++++
net/core/skbuff.c | 12 ++++
net/core/sock.c | 2 +
net/ipv4/af_inet.c | 20 ++++--
net/ipv4/arp.c | 76 ++++++++++++---------
net/ipv4/datagram.c | 6 +-
net/ipv4/devinet.c | 64 ++++++++++++------
net/ipv4/fib_frontend.c | 83 ++++++++++++++---------
net/ipv4/fib_rules.c | 12 ++--
net/ipv4/fib_semantics.c | 38 +++++++----
net/ipv4/fib_trie.c | 24 +++++--
net/ipv4/icmp.c | 40 ++++++-----
net/ipv4/igmp.c | 53 +++++++++------
net/ipv4/inet_connection_sock.c | 23 ++++---
net/ipv4/inet_diag.c | 13 ++--
net/ipv4/inet_hashtables.c | 42 +++++++-----
net/ipv4/inet_timewait_sock.c | 1 +
net/ipv4/ip_input.c | 6 +-
net/ipv4/ip_options.c | 20 +++---
net/ipv4/ip_output.c | 16 +++--
net/ipv4/ip_sockglue.c | 32 +++++++--
net/ipv4/ipconfig.c | 6 +-
net/ipv4/ipmr.c | 53 +++++++++------
net/ipv4/netfilter.c | 13 ++--
net/ipv4/ping.c | 41 +++++------
net/ipv4/proc.c | 10 +--
net/ipv4/raw.c | 48 ++++++++-----
net/ipv4/route.c | 143 +++++++++++++++++++++++----------------
net/ipv4/syncookies.c | 6 +-
net/ipv4/tcp_ipv4.c | 57 +++++++++-------
net/ipv4/tcp_minisocks.c | 1 +
net/ipv4/udp.c | 122 ++++++++++++++++++---------------
net/ipv4/udp_diag.c | 11 +--
net/ipv4/xfrm4_policy.c | 14 ++--
net/netlink/af_netlink.c | 12 ++++
net/sctp/protocol.c | 10 +--
net/xfrm/xfrm_policy.c | 9 +--
76 files changed, 1415 insertions(+), 682 deletions(-)
create mode 100644 include/net/vrf.h
--
1.9.3 (Apple Git-50)
--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Powered by blists - more mailing lists