| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Message-Id: <BC6E48CC-B149-4467-AC9F-FAB71C58EFCE@sztaki.hu>
Date: Wed, 12 Oct 2011 17:10:47 +0200
From: Andras Takacs <andras.takacs@...aki.hu>
To: netdev@...r.kernel.org
Cc: József Kovacs <jk@...aki.hu>,
Emmanuel Thierry <emmanuel.thierry@...ecom-bretagne.eu>,
Virág László <lvirag@...aki.hu>
Subject: ip6/ip6 tunnel add/change failure
Dear All,
We're using Mobile IPv6 Daemon (mip6d) on Linux 2.6.35.14. Previously we have used 2.6.22 without any kernel-related issues. Unfortunately this kernel version produces a very annoying problem with the IP6/IP6 tunnels. We're having two tunnels, which are linked to two different eth interfaces.
The issue: All of the packets, independently from the tunnel interface leaves the box via eth1 with the correct source address. Instead of the expected way, when the packets sent via ip6tnl1 go out on eth1, and the others (sent via ip6tnl2) go out on eth2.
We tried to reproduce this issue without mip6d, only with the ip command. I attached the setup and the test scripts. In this case, the routing works fine.
After a lot of debugging we found the following difference: mip6d creates the tunnel with remote address A. Later, it changes it to B. We found that somewhere here is the key.
1) The following data shows the internals of ip6tnl1 device's net_device structure. we dumped it from the rtnl_fill_ifinfo, which was triggered by ip -6 link ls:
Breakpoint 3, 0xc0399069 in rtnl_fill_ifinfo (skb=0xcee55300, dev=0xcede7000, type=16, pid=537, seq=1318434857, change=0, flags=2)
at /mnt/hgfs/svnOnMyMac/SZTAKI/ITSSv6/trunk/kernel/linux-2.6.35.14/net/core/rtnetlink.c:1014
1014 if (tb[IFLA_ADDRESS] &&
(gdb) p *dev
$7 = {name = "ip6tnl1\000\000\000\000\000\000\000\000", pm_qos_req = 0x0, name_hlist = {next = 0x0, pprev = 0xcf95fac8}, ifalias = 0x0, mem_end = 0, mem_start = 0, base_addr = 0,
irq = 0, if_port = 0 '\0', dma = 0 '\0', state = 3, dev_list = {next = 0xcfbd1838, prev = 0xcec1d038}, napi_list = {next = 0xcede7040, prev = 0xcede7040}, unreg_list = {
next = 0xcede7048, prev = 0xcede7048}, features = 8192, ifindex = 7, iflink = 4, stats = {rx_packets = 0, tx_packets = 2, rx_bytes = 0, tx_bytes = 176, rx_errors = 0,
tx_errors = 0, rx_dropped = 0, tx_dropped = 0, multicast = 0, collisions = 0, rx_length_errors = 0, rx_over_errors = 0, rx_crc_errors = 0, rx_frame_errors = 0, rx_fifo_errors = 0,
rx_missed_errors = 0, tx_aborted_errors = 0, tx_carrier_errors = 0, tx_fifo_errors = 0, tx_heartbeat_errors = 0, tx_window_errors = 0, rx_compressed = 0, tx_compressed = 0},
netdev_ops = 0xc0461c78, ethtool_ops = 0x0, header_ops = 0x0, flags = 145, gflags = 0, priv_flags = 1024, padded = 0, operstate = 0 '\0', link_mode = 0 '\0', mtu = 1460, type = 769,
hard_header_len = 54, needed_headroom = 0, needed_tailroom = 0, master = 0x0, perm_addr = '\0' <repeats 31 times>, addr_len = 16 '\020', dev_id = 0, addr_list_lock = {{rlock = {
raw_lock = {slock = 1}, magic = 3735899821, owner_cpu = 4294967295, owner = 0xffffffff, dep_map = {key = 0xc0b979dc, class_cache = 0xc072dd30,
name = 0xc057b8d5 "_xmit_TUNNEL6"}}, {__padding = "\001\000\000\000?N??????????", dep_map = {key = 0xc0b979dc, class_cache = 0xc072dd30,
name = 0xc057b8d5 "_xmit_TUNNEL6"}}}}, uc = {list = {next = 0xcede7120, prev = 0xcede7120}, count = 0}, mc = {list = {next = 0xcede712c, prev = 0xcede712c}, count = 0},
uc_promisc = 0, promiscuity = 0, allmulti = 0, atalk_ptr = 0x0, ip_ptr = 0xceeb5800, dn_ptr = 0x0, ip6_ptr = 0xceeb5e00, ec_ptr = 0x0, ax25_ptr = 0x0, ieee80211_ptr = 0x0,
last_rx = 0, dev_addr = 0xced57588 " \001\a8\0200\020\005PT", dev_addrs = {list = {next = 0xced57580, prev = 0xced57580}, count = 1},
broadcast = " \001\a8\0200\000\001\000\000\000\000\000\000\020", '\0' <repeats 16 times>, rx_queue = {dev = 0xcede7000, qdisc = 0xc05e16cc, state = 0, qdisc_sleeping = 0xc05e16cc,
_xmit_lock = {{rlock = {raw_lock = {slock = 1}, magic = 3735899821, owner_cpu = 4294967295, owner = 0xffffffff, dep_map = {key = 0xc0b97bb4, class_cache = 0x0,
name = 0xc057b8d5 "_xmit_TUNNEL6"}}, {__padding = "\001\000\000\000?N??????????", dep_map = {key = 0xc0b97bb4, class_cache = 0x0, name = 0xc057b8d5 "_xmit_TUNNEL6"}}}},
xmit_lock_owner = -1, trans_start = 0, tx_bytes = 0, tx_packets = 0, tx_dropped = 0}, _tx = 0xced57dc0, num_tx_queues = 1, real_num_tx_queues = 1, qdisc = 0xc05e176c,
tx_queue_len = 0, tx_global_lock = {{rlock = {raw_lock = {slock = 1}, magic = 3735899821, owner_cpu = 4294967295, owner = 0xffffffff, dep_map = {key = 0xc0b978dc, class_cache = 0x0,
name = 0xc057b3df "&(&dev->tx_global_lock)->rlock"}}, {__padding = "\001\000\000\000?N??????????", dep_map = {key = 0xc0b978dc, class_cache = 0x0,
name = 0xc057b3df "&(&dev->tx_global_lock)->rlock"}}}}, trans_start = 0, watchdog_timeo = 0, watchdog_timer = {entry = {next = 0x0, prev = 0x0}, expires = 0,
base = 0xc06609c0, function = 0xc039eb7f <dev_watchdog>, data = 3470684160, slack = -1, start_site = 0x0, start_comm = '\0' <repeats 15 times>, start_pid = -1, lockdep_map = {
key = 0xc0b98144, class_cache = 0x0, name = 0xc057ca77 "&dev->watchdog_timer"}}, refcnt = {counter = 13}, todo_list = {next = 0x0, prev = 0x0}, index_hlist = {next = 0x0,
pprev = 0xcf95fc1c}, link_watch_list = {next = 0xcede7260, prev = 0xcede7260}, reg_state = NETREG_REGISTERED, rtnl_link_state = RTNL_LINK_INITIALIZED,
destructor = 0xc038f1b1 <free_netdev>, ml_priv = 0x0, br_port = 0x0, macvlan_port = 0x0, garp_port = 0x0, dev = {parent = 0x0, p = 0xcedfe600, kobj = {name = 0xcfb24ae0 "ip6tnl1",
entry = {next = 0xcfbd1a8c, prev = 0xcf80e7fc}, parent = 0xcf99edc0, kset = 0xcf8142a0, ktype = 0xc05d414c, sd = 0xced575c0, kref = {refcount = {counter = 2}},
state_initialized = 1, state_in_sysfs = 1, state_add_uevent_sent = 1, state_remove_uevent_sent = 0, uevent_suppress = 0}, init_name = 0x0, type = 0x0, mutex = {count = {
counter = 1}, wait_lock = {{rlock = {raw_lock = {slock = 1}, magic = 3735899821, owner_cpu = 4294967295, owner = 0xffffffff, dep_map = {key = 0xc0661d9c, class_cache = 0x0,
name = 0xc051d978 "&(&lock->wait_lock)->rlock"}}, {__padding = "\001\000\000\000?N??????????", dep_map = {key = 0xc0661d9c, class_cache = 0x0,
name = 0xc051d978 "&(&lock->wait_lock)->rlock"}}}}, wait_list = {next = 0xcede72d4, prev = 0xcede72d4}, owner = 0x0, name = 0x0, magic = 0xcede72b4, dep_map = {
key = 0xc066e010, class_cache = 0x0, name = 0xc05493c7 "&__lockdep_no_validate__"}}, bus = 0x0, driver = 0x0, platform_data = 0xcede7000, power = {power_state = {event = 0},
can_wakeup = 0, should_wakeup = 0, async_suspend = 0, status = DPM_INVALID}, dma_mask = 0x0, coherent_dma_mask = 0, dma_parms = 0x0, dma_pools = {next = 0xcede731c,
prev = 0xcede731c}, dma_mem = 0x0, archdata = {acpi_handle = 0x0}, devt = 0, devres_lock = {{rlock = {raw_lock = {slock = 1}, magic = 3735899821, owner_cpu = 4294967295,
owner = 0xffffffff, dep_map = {key = 0xc0b92e54, class_cache = 0x0, name = 0xc05493e0 "&(&dev->devres_lock)->rlock"}}, {__padding = "\001\000\000\000?N??????????",
dep_map = {key = 0xc0b92e54, class_cache = 0x0, name = 0xc05493e0 "&(&dev->devres_lock)->rlock"}}}}, devres_head = {next = 0xcede734c, prev = 0xcede734c}, knode_class = {
n_klist = 0xcf9aca4c, n_node = {next = 0xcfbd1b58, prev = 0xcec1d358}, n_ref = {refcount = {counter = 1}}}, class = 0xc05e1008, groups = 0xcede7370, release = 0},
sysfs_groups = {0xc05e1044, 0x0, 0x0, 0x0}, rtnl_link_ops = 0x0, vlan_features = 0, gso_max_size = 65536, ethtool_ntuple_list = {list = {next = 0xcede738c, prev = 0xcede738c},
count = 0}}
I would highlight the ifindex and the iflink values! (ifindex = 7, iflink = 4)
2) Next, we duped out the same for the tunnels, which was created by ip -6 tunnel add. We found the following:
Breakpoint 4, 0xc0399069 in rtnl_fill_ifinfo (skb=0xced2d300, dev=0xceeb4800, type=16, pid=568, seq=1318435116, change=0, flags=2)
at /mnt/hgfs/svnOnMyMac/SZTAKI/ITSSv6/trunk/kernel/linux-2.6.35.14/net/core/rtnetlink.c:1014
1014 if (tb[IFLA_ADDRESS] &&
(gdb) p *dev
$15 = {name = "ip6test1\000\000\000\000\000\000\000", pm_qos_req = 0x0, name_hlist = {next = 0x0, pprev = 0xcf95f974}, ifalias = 0x0, mem_end = 0, mem_start = 0, base_addr = 0,
irq = 0, if_port = 0 '\0', dma = 0 '\0', state = 3, dev_list = {next = 0xceeb6838, prev = 0xcec1d038}, napi_list = {next = 0xceeb4840, prev = 0xceeb4840}, unreg_list = {
next = 0xceeb4848, prev = 0xceeb4848}, features = 8192, ifindex = 7, iflink = 7, stats = {rx_packets = 0, tx_packets = 0, rx_bytes = 0, tx_bytes = 0, rx_errors = 0, tx_errors = 0,
rx_dropped = 0, tx_dropped = 0, multicast = 0, collisions = 0, rx_length_errors = 0, rx_over_errors = 0, rx_crc_errors = 0, rx_frame_errors = 0, rx_fifo_errors = 0,
rx_missed_errors = 0, tx_aborted_errors = 0, tx_carrier_errors = 0, tx_fifo_errors = 0, tx_heartbeat_errors = 0, tx_window_errors = 0, rx_compressed = 0, tx_compressed = 0},
netdev_ops = 0xc0461c78, ethtool_ops = 0x0, header_ops = 0x0, flags = 145, gflags = 0, priv_flags = 1024, padded = 0, operstate = 0 '\0', link_mode = 0 '\0', mtu = 1460, type = 769,
hard_header_len = 54, needed_headroom = 0, needed_tailroom = 0, master = 0x0, perm_addr = '\0' <repeats 31 times>, addr_len = 16 '\020', dev_id = 0, addr_list_lock = {{rlock = {
raw_lock = {slock = 1}, magic = 3735899821, owner_cpu = 4294967295, owner = 0xffffffff, dep_map = {key = 0xc0b979dc, class_cache = 0xc072c2a0,
name = 0xc057b8d5 "_xmit_TUNNEL6"}}, {__padding = "\001\000\000\000?N??????????", dep_map = {key = 0xc0b979dc, class_cache = 0xc072c2a0,
name = 0xc057b8d5 "_xmit_TUNNEL6"}}}}, uc = {list = {next = 0xceeb4920, prev = 0xceeb4920}, count = 0}, mc = {list = {next = 0xceeb492c, prev = 0xceeb492c}, count = 0},
uc_promisc = 0, promiscuity = 0, allmulti = 0, atalk_ptr = 0x0, ip_ptr = 0xced77400, dn_ptr = 0x0, ip6_ptr = 0xced77e00, ec_ptr = 0x0, ax25_ptr = 0x0, ieee80211_ptr = 0x0,
last_rx = 0, dev_addr = 0xced58c88 " \001\a8\0200\020\004PT", dev_addrs = {list = {next = 0xced58c80, prev = 0xced58c80}, count = 1},
broadcast = " \001\a8\0200\000\001\000\000\000\000\000\000\020", '\0' <repeats 16 times>, rx_queue = {dev = 0xceeb4800, qdisc = 0xc05e16cc, state = 0, qdisc_sleeping = 0xc05e16cc,
_xmit_lock = {{rlock = {raw_lock = {slock = 1}, magic = 3735899821, owner_cpu = 4294967295, owner = 0xffffffff, dep_map = {key = 0xc0b97bb4, class_cache = 0x0,
name = 0xc057b8d5 "_xmit_TUNNEL6"}}, {__padding = "\001\000\000\000?N??????????", dep_map = {key = 0xc0b97bb4, class_cache = 0x0, name = 0xc057b8d5 "_xmit_TUNNEL6"}}}},
xmit_lock_owner = -1, trans_start = 0, tx_bytes = 0, tx_packets = 0, tx_dropped = 0}, _tx = 0xced58c40, num_tx_queues = 1, real_num_tx_queues = 1, qdisc = 0xc05e176c,
tx_queue_len = 0, tx_global_lock = {{rlock = {raw_lock = {slock = 1}, magic = 3735899821, owner_cpu = 4294967295, owner = 0xffffffff, dep_map = {key = 0xc0b978dc, class_cache = 0x0,
name = 0xc057b3df "&(&dev->tx_global_lock)->rlock"}}, {__padding = "\001\000\000\000?N??????????", dep_map = {key = 0xc0b978dc, class_cache = 0x0,
name = 0xc057b3df "&(&dev->tx_global_lock)->rlock"}}}}, trans_start = 0, watchdog_timeo = 0, watchdog_timer = {entry = {next = 0x0, prev = 0x0}, expires = 0,
base = 0xc06609c0, function = 0xc039eb7f <dev_watchdog>, data = 3471525888, slack = -1, start_site = 0x0, start_comm = '\0' <repeats 15 times>, start_pid = -1, lockdep_map = {
key = 0xc0b98144, class_cache = 0x0, name = 0xc057ca77 "&dev->watchdog_timer"}}, refcnt = {counter = 11}, todo_list = {next = 0x0, prev = 0x0}, index_hlist = {next = 0x0,
pprev = 0xcf95fc1c}, link_watch_list = {next = 0xceeb4a60, prev = 0xceeb4a60}, reg_state = NETREG_REGISTERED, rtnl_link_state = RTNL_LINK_INITIALIZED,
destructor = 0xc038f1b1 <free_netdev>, ml_priv = 0x0, br_port = 0x0, macvlan_port = 0x0, garp_port = 0x0, dev = {parent = 0x0, p = 0xceea7380, kobj = {name = 0xcfb54780 "ip6test1",
entry = {next = 0xceeb6a8c, prev = 0xcf80d7fc}, parent = 0xcf99fdc0, kset = 0xcf8142a0, ktype = 0xc05d414c, sd = 0xced58780, kref = {refcount = {counter = 2}},
state_initialized = 1, state_in_sysfs = 1, state_add_uevent_sent = 1, state_remove_uevent_sent = 0, uevent_suppress = 0}, init_name = 0x0, type = 0x0, mutex = {count = {
counter = 1}, wait_lock = {{rlock = {raw_lock = {slock = 1}, magic = 3735899821, owner_cpu = 4294967295, owner = 0xffffffff, dep_map = {key = 0xc0661d9c, class_cache = 0x0,
name = 0xc051d978 "&(&lock->wait_lock)->rlock"}}, {__padding = "\001\000\000\000?N??????????", dep_map = {key = 0xc0661d9c, class_cache = 0x0,
name = 0xc051d978 "&(&lock->wait_lock)->rlock"}}}}, wait_list = {next = 0xceeb4ad4, prev = 0xceeb4ad4}, owner = 0x0, name = 0x0, magic = 0xceeb4ab4, dep_map = {
key = 0xc066e010, class_cache = 0x0, name = 0xc05493c7 "&__lockdep_no_validate__"}}, bus = 0x0, driver = 0x0, platform_data = 0xceeb4800, power = {power_state = {event = 0},
can_wakeup = 0, should_wakeup = 0, async_suspend = 0, status = DPM_INVALID}, dma_mask = 0x0, coherent_dma_mask = 0, dma_parms = 0x0, dma_pools = {next = 0xceeb4b1c,
prev = 0xceeb4b1c}, dma_mem = 0x0, archdata = {acpi_handle = 0x0}, devt = 0, devres_lock = {{rlock = {raw_lock = {slock = 1}, magic = 3735899821, owner_cpu = 4294967295,
owner = 0xffffffff, dep_map = {key = 0xc0b92e54, class_cache = 0x0, name = 0xc05493e0 "&(&dev->devres_lock)->rlock"}}, {__padding = "\001\000\000\000?N??????????",
dep_map = {key = 0xc0b92e54, class_cache = 0x0, name = 0xc05493e0 "&(&dev->devres_lock)->rlock"}}}}, devres_head = {next = 0xceeb4b4c, prev = 0xceeb4b4c}, knode_class = {
n_klist = 0xcf9ada4c, n_node = {next = 0xceeb6b58, prev = 0xcec1d358}, n_ref = {refcount = {counter = 1}}}, class = 0xc05e1008, groups = 0xceeb4b70, release = 0},
sysfs_groups = {0xc05e1044, 0x0, 0x0, 0x0}, rtnl_link_ops = 0x0, vlan_features = 0, gso_max_size = 65536, ethtool_ntuple_list = {list = {next = 0xceeb4b8c, prev = 0xceeb4b8c},
count = 0}}
Highlight again: ifindex = 7 iflink = 7. I would like to repeat, that it is the working version, and the mip6d version is the wrong from the usage's point-of-view.
3) What we see from userspace? The ip -6 addr ls is a good help.
If we just added the interface we see:
7: ip6test1: <POINTOPOINT,NOARP,UP,LOWER_UP> mtu 1460
inet6 2001:738:1030:1::3/128 scope global
valid_lft forever preferred_lft forever
inet6 fe80::5054:ff:fe12:3459/64 scope link
valid_lft forever preferred_lft forever
If it has been changed (by mip6d or with the ip -6 tunnel change command):
7: ip6tnl1@...1: <POINTOPOINT,NOARP,UP,LOWER_UP> mtu 1460
inet6 2001:738:1030:1::3/128 scope global home nodad
valid_lft forever preferred_lft forever
inet6 fe80::5054:ff:fe12:3459/64 scope link
valid_lft forever preferred_lft forever
The '@' signs that the ifindex and the iflink values are different.
4) One more addition (maybe it helps): The link-local address all of the tunnels are the same, and equals the LL address of eth0. It is quite interesting, because any of the tunnels linked to eth0. (The tunnels are linked to eth1 and eth2).
--- --- --- --- --
We would like to fix this issue very much, but we don't know what was the concept of the existing code. We're sure that the exisiting mechanism is wrong, when the change method procedures a totally different result that the add. We would appreciate any help or comment!
Thank you very much!
Regards,
András
Download attachment "test-ip6-tnls.sh" of type "application/octet-stream" (3799 bytes)
András Takács
MTA SZTAKI
Computer and Automation Research Institute
Hungarian Academy of Sciences
Email: andras.takacs@...aki.hu
Office: +36-1-279-6288
Mobile: +36-70-282-5316
Powered by blists - more mailing lists