As you noticed from the previous articles, lately I have been playing with some various tunnelling techniques and today I am presenting MACsec.
Most of the documentation resources about MACsec implementation on the web at this moment, are the ones showing various vendors implementation, especially Cisco's approach.
Although it's not a new topic, support for MACsec in the Linux kernel was added only recently, in version 4.6.

Quick Overview

MAC Security (MACsec), defined in IEEE 802.1AE standard, is intended to provide secure access to the network, ensuring data integrity, data origin authentication and, optionally, encryption for the traffic between the host and the access switch - everything at Layer 2 !
Its primary use case is to secure communication to & from endpoints at the access edge, and in this role, it is usually used together with 802.1X that provides port-based authentication and transmits the necessary keying material to both host and switch. 802.1X had several revisions that cover MACsec Key Agreement protocol (MKA), a protocol that discovers MACsec enabled peers and dynamically distributes keying material to them.

Additional use cases could be to protect switch to switch links or, why not, host to host, and usually in these cases you have to use static association keys (with Cisco you can also do switch-to-switch MACsec link security with Radius and AAA, as well as manual). In this case, MACsec represents an alternative to IPsec for WAN links if they use Ethernet.

MACsec does a hop-by-hop, wire speed, link protection - as opposed to IPsec that provides end-to-end (over multiple hops) layer 3 security.
For this reason, MACsec is sometimes referred to as Linksec.

Features

Here are some notes about MACsec without covering the details:

  • protection (integrity and/or encryption) is performed at Layer 2, so it is transparent for the network
  • as opposed to IPsec that raises performance challenges, MACsec is intended to run at line-rate, in hardware's ASIC (for this reason, not all hardware supports MACsec)
  • the Ethertype for the protected MACsec frame is 0x88e5
  • the Ethernet Header and SecTag are sent in clear text but they are always integrity-protected (by ICV)
  • default crypto is AES-GCM-128
  • supports optional replay protection with a configurable replay window

Implementation on Linux

Before I start, I would like to mention the name Sabrina Dubroca - she is the person behind the work on bringing MACsec support in the Linux kernel.

In order to test MACsec I am going to use two Virtual Boxes (managed via vagrant), similar to the lab described here with Ubuntu. At the date when this article was written, July 2016, the following are necessary to make it work:

  • install the latest version of the kernel, kernel-4-7-rc7 (yes! a release candidate version)
  • compile the macsec module support
  • install the latest version of iproute2 tools

Let's start!

Install latest kernel and compile MACsec support

I am only briefly describing this process, since internet contains a lot of articles on this topic. Here is one official document.

# preparation
cd $HOME
mkdir kernel-4-7-rc7
cd kernel-4-7-rc7

# clone the git or download a release candidate version
wget https://cdn.kernel.org/pub/linux/kernel/v4.x/testing/linux-4.7-rc7.tar.xz
tar xf linux-4.7-rc7.tar.xz

# copy existing kernel config to keep previous settings
cp /boot/config-`uname -r` .config

# make the below changes
vi .config   
# unset CONFIG_DEBUG_INFO to speed up time and decrease size on disk
# enable MACsec support 
    CONFIG_DEBUG_INFO is not set
    CONFIG_MACSEC=m

# bring file up to date and cleanup
make oldconfig
make clean

# build the dev files
make -j `getconf _NPROCESSORS_ONLN` deb-pkg LOCALVERSION=-custom

# change directory then install linux-image and linux-header files 
sudo dpkg -i linux-image-*-custom-*.deb
sudo dpkg -i linux-headers-*-custom-*.deb

That's it! When that finishes, just do sudo reboot.

Install the latest version of iproute2

# some prerequisites needed
sudo apt-get install pkg-config bison flex xtables-addons-common xtables-addons-source

git clone git://git.kernel.org/pub/scm/linux/kernel/git/shemminger/iproute2.git
cd iproute2/
./configure
make
sudo make install

# verify that the below command works  
ip macsec show

Configure MACsec between Two Linux Machines

Repeat the above steps on both virtual machines and then you are ready to configure and test MACsec. If you use the same setup as me, it means that you already have two Host-Only adaptors between the VMs - I am going to use one of them, enp0s8, to create the MACsec interface/device on top of it.
Here is how you configure MACsec on Linux - instructions are inline. The highlighted MAC address corresponds to the other end (it's like on IPsec you provide the IP address of the peer endpoint):

MACsec Implementation on Linux

On the first Virtual Machine

# on vagrant box-1
# ----------------

# Clear IP configuration on the Host-Only adaptor between the VMs
sudo ifconfig enp0s8 0.0.0.0

# Load the MACsec kernel
sudo modprobe macsec

# Create the MACsec device on top of the physical one
sudo ip link add link enp0s8 macsec0 type macsec

# Configure the Transmit SA and keys
sudo ip macsec add macsec0 tx sa 0 pn 100 on key 01 11111111111111111111111111111111

# Configure the Receive Channel and SA:
# MAC address of the peer
# port number, packet number and key
sudo ip macsec add macsec0 rx address 08:00:27:f2:1d:8c port 1
sudo ip macsec add macsec0 rx address 08:00:27:f2:1d:8c port 1 sa 0 pn 100 on key 02 22222222222222222222222222222222


# Bring up the interface
sudo ip link set dev macsec0 up

# Configure an IP address on it for connectivity between the hosts
sudo ifconfig macsec0 1.1.1.1/24

On the second Virtual Machine

Follow the same steps - again, make sure that you have the correct MAC addresses for the Layer 2 endpoints:

# on vagrant box-1
# ----------------
sudo modprobe macsec

sudo ip link add link enp0s8 macsec0 type macsec
sudo ip macsec add macsec0 tx sa 0 pn 100 on key 02 22222222222222222222222222222222

sudo ip macsec add macsec0 rx address 08:00:27:ae:4d:62 port 1
sudo ip macsec add macsec0 rx address 08:00:27:ae:4d:62 port 1 sa 0 pn 100 on key 01 11111111111111111111111111111111

sudo ip link set dev macsec0 up

sudo ifconfig macsec0 1.1.1.2/24




Verification and Troubleshooting

Common Problems

If you encounter any problems, here are some things that you can check:

  1. command sudo modprobe macsec fails with error modprobe: FATAL: Module macsec not found in directory /lib/modules/.
    As indicated in the error message, your kernel does not contain a module for macsec support. You need to re-compile the kernel as per above-mentioned procedure and make sure that the .config file contains this line CONFIG_MACSEC=m (which enables MACsec support as a module)

  2. command ip macsec fails with error Object "macsec" is unknown, try "ip help"..
    This means that you do not have the latest version of iproute2 utilities. Follow the procedure above to fix this.

  3. command sudo ip macsec add macsec0 fails with error RTNETLINK answers: Cannot allocate memory.
    This means that you do not use the correct version of the kernel - something higher than version 4.7.0 release candidate 7 (rc7).

  4. command ip macsec show fails with error RTNETLINK answers: No such file or directory - Error talking to the kernel.
    In this case make sure that you loaded the MACsec module into the kernel with command sudo modprobe macsec.

Verification

To verify that everything works, we can check the following:

  • connectivity over the MACsec interface 

    ubuntu@box-1 ~$ ping 1.1.1.2
PING 1.1.1.2 (1.1.1.2) 56(84) bytes of data.
64 bytes from 1.1.1.2: icmp_seq=1 ttl=64 time=0.663 ms
64 bytes from 1.1.1.2: icmp_seq=2 ttl=64 time=0.489 ms
64 bytes from 1.1.1.2: icmp_seq=3 ttl=64 time=0.527 ms
^C
--- 1.1.1.2 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2004ms
rtt min/avg/max/mdev = 0.489/0.559/0.663/0.079 ms

  • output of ip command showing increased Packet Number (PN) counter - please note that I provided an initial pn 100 in the above configuration commands, so the starting point is 100 

    ubuntu@box-1 ~$ sudo ip macsec show
5: macsec0: protect on validate strict sc off sa off encrypt off send_sci on end_station off scb off replay off
    cipher suite: GCM-AES-128, using ICV length 16
    TXSC: 0100624dae270008 on SA 0
        0: PN 113, state on, key 01000000000000000000000000000000
    RXSC: 01008c1df2270008, state on
        0: PN 113, state on, key 02000000000000000000000000000000
ubuntu@box-1 ~$

    As outlined here, encryption was not configured and it is off by default.

  • tcpdump on the physical enp0s8 interface 

    ubuntu@box-2 ~$ sudo tcpdump -nli enp0s8
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on enp0s8, link-type EN10MB (Ethernet), capture size 262144 bytes
23:27:10.072761 08:00:27:ae:4d:62 > 08:00:27:f2:1d:8c, ethertype Unknown (0x88e5), length 130:
    0x0000:  2000 0000 0078 0800 27ae 4d62 0001 0800  .....x..'.Mb....
    0x0010:  4500 0054 7b8d 4000 4001 bb17 0101 0101  E..T{.@.@.......
    0x0020:  0101 0102 0800 c280 0be2 0004 5389 9e57  ............S..W
    0x0030:  0000 0000 6be5 0d00 0000 0000 1011 1213  ....k...........
    0x0040:  1415 1617 1819 1a1b 1c1d 1e1f 2021 2223  .............!"#
    0x0050:  2425 2627 2829 2a2b 2c2d 2e2f 3031 3233  $%&'()+,-./0123
    0x0060:  3435 3637 e209 f7db 1d85 1ea6 7532 d240  4567........u2.@
    0x0070:  a10e 1f8a                                ....
23:27:10.072846 08:00:27:f2:1d:8c > 08:00:27:ae:4d:62, ethertype Unknown (0x88e5), length 130:
    0x0000:  2000 0000 0078 0800 27f2 1d8c 0001 0800  .....x..'.......
    0x0010:  4500 0054 8965 0000 4001 ed3f 0101 0102  E..T.e..@..?....
    0x0020:  0101 0101 0000 ca80 0be2 0004 5389 9e57  ............S..W
    0x0030:  0000 0000 6be5 0d00 0000 0000 1011 1213  ....k...........
    0x0040:  1415 1617 1819 1a1b 1c1d 1e1f 2021 2223  .............!"#
    0x0050:  2425 2627 2829 2a2b 2c2d 2e2f 3031 3233  $%&'()+,-./0123
    0x0060:  3435 3637 e6cc 6ca0 dec9 0520 fd25 b73d  4567..l......%.=
    0x0070:  fc62 f9d1                                .b..


As you can see here, tcpdump command does not understand ethertype 0x88e5 so it does not understand what kind of traffic that is.


To have a better picture, I captured the traffic and opened it with Wireshark - this one knows about the Ethertype 0x88e5 as being MACsec, but still it misses dissectors to interpret the data:

MACsec Packet Capture

I have also included a Total Bytes calculation, starting from an ICMP packet of 64B (the default on Ubuntu) - note that the SecTag has a length of 16B here (though default is 8 bytes, here it includes the optional Secure Channel Identifier (SCI) encoding which adds an extra 8 bytes).

As always, I uploaded the packet capture and you can view it here.

Enabling Encryption

As indicated above, by default MACsec performs data integrity and authentication but no encryption. In our scenario, to enable encryption on the macsec interface, use the following command:

sudo ip link set macsec0 type macsec encrypt on

Since neither tcpdump nor Wireshark are not (yet!) capable of dissecting the MACsec payload, the packet captures with encrypt off and encrypt on look almost identical - you cannot say which one has encrypted data (since in both cases data is not dissected). But the SecTag contains a signal, the "E" bit, to indicate if encryption is on or off - see below:

MACsec Encrypt OFF vs ON


Another good command is ip -s macsec show that contains individual counters for each type of protection: integrity-only (encrypt off) and encryption (encrypt on): 

ubuntu@box-2 ~$ ip -s macsec show
5: macsec0: protect on validate strict sc off sa off encrypt off send_sci on end_station off scb off replay off
    cipher suite: GCM-AES-128, using ICV length 16
    TXSC: 01008c1df2270008 on SA 0
    stats: OutPktsUntagged InPktsUntagged OutPktsTooLong InPktsNoTag InPktsBadTag InPktsUnknownSCI InPktsNoSCI InPktsOverrun
                         0              0              0           0            0                0           0             0
    stats: OutOctetsProtected OutOctetsEncrypted OutPktsProtected OutPktsEncrypted
                       150247                  0        227442534                0
        0: PN 150347, state on, key 02000000000000000000000000000000
           OutPktsProtected OutPktsEncrypted
                     150247                0
    RXSC: 0100624dae270008, state on
    stats: InOctetsValidated InOctetsDecrypted InPktsUnchecked InPktsDelayed InPktsOK InPktsInvalid InPktsLate InPktsNotValid InPktsNotUsingSA InPktsUnusedSA
                     6855406                 0               0             0    68431             0          0              0                0              0
        0: PN 68539, state on, key 01000000000000000000000000000000
           InPktsOK InPktsInvalid InPktsNotValid InPktsNotUsingSA InPktsUnusedSA
              68431             0              0                0              0
ubuntu@box-2 ~$


Although initially my plan was to include another usecase - GRE over MACsec - I will leave this for the next post since it seems that I have the tendency to write very long articles.


As always, thank you for your interest and comments!