netdev - Bug in MACSec - stops passing traffic after approx 5TB

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Message-ID: <CACcUnf8uKOFjJQq6AAroKjArUTwOuLx4HhteZg8y0wDT9NdZDA@mail.gmail.com>
Date:   Sun, 14 Oct 2018 10:59:31 -0400
From:   Josh Coombs <jcoombs@...ff.gwi.net>
To:     netdev@...r.kernel.org
Subject: Bug in MACSec - stops passing traffic after approx 5TB

I initially mistook this for a traffic control issue, but after
stripping the test beds down to just the MACSec component, I can still
replicate the issue.  After approximately 5TB of transfer / 4 billion
packets over a MACSec link it stops passing traffic.  I have
replicated this now on both Ubuntu Server 18.04 with their patched
4.15 kernel and Gentoo with a vanilla 4.18.13 kernel.

As noted before, my test setup consists of two machines with a direct
ethernet connection to each other.  MACSec is setup on the link, along
with a /30 network.  I then hammer the link with iperf3 in both
directions.  On a gigabit link it takes me one to two days to trip the
bug.  Nothing is logged to dmesg.  If I remove and re-add the MACSec
link, or reboot the machines traffic flow resumes.  I can replicate on
physical hardware or in VMs.

How should I proceed to help diag and correct this?

To replicate, setup two machines with a direct ethernet connection.
If simulating in ESXi setup a dedicated vSwitch, allow promiscuous,
forged and MAC changes should be enabled, MTU increased to 9000, then
setup a dedicated port group and VLAN to simulate the direct
connection.

I use the following script to setup the MACSec connection, adjusting
keys, rxmac, interface and IPs as appropriate:

-----------
The script I used on each host (keys, rxmacs and IPs updated as appropriate):
#!/bin/bash

# Interfaces:
# dif = Egress physical interface (Dest)
# eif = Encrypted interface
dif=ens224
eif=macsec0

# MACSec Keys:
# txkey = Transmit (Local) key
# rxkey = Receive (Remote) key
# rxmac = Receive (Remote) MAC addy
txkey=60995924232808431491190820961556
rxkey=87345530111733181210202106249824
rxmac=00:0c:29:c5:95:df

# Clear any existing IP config
ifconfig $dif 0.0.0.0

# Bring up macsec:
echo "* Enable MACSec"
modprobe macsec
ip link add link "$dif" "$eif" type macsec
ip macsec add "$eif" tx sa 0 pn 1 on key 02 "$txkey"
ip macsec add "$eif" rx address "$rxmac" port 1
ip macsec add "$eif" rx address "$rxmac" port 1 sa 0 pn 1 on key 01 "$rxkey"
ip link set "$eif" type macsec encrypt on

# Bring up the interfaces:
echo "* Light tunnel NICS"
ip link set "$dif" up
ip link set "$eif" up

# Set IP
ifconfig $eif 192.168.211.1/30

Once you can ping across the link, use iperf3 or a similar network
stress tool to flood the link with traffic in both directions and wait
for the bug to trigger.

Josh Coombs