grout-frr - Man Page

FRR dataplane plugin for grout

Description

The dplane_grout zebra module replaces the Linux kernel as the forwarding plane for FRR (Free Range Routing). Once loaded, all routing decisions made by FRR daemons (bgpd, ospfd, ospf6d, isisd, ...) are programmed directly into grout instead of the kernel networking stack.

The plugin maintains multiple connections to the grout API socket:

A control channel for pushing route, nexthop, address, MAC and VTEP updates.
An event channel for receiving interface and address notifications from grout.
A separate event channel for receiving route and nexthop notifications.

At startup (or after a grout restart), the plugin performs a full synchronization of all grout interfaces, addresses, FDB entries, nexthops and routes into zebra's RIB. During normal operation, updates flow bidirectionally: FRR pushes learned routes to grout, and grout notifies FRR of link and address changes.

Supported operations

The following zebra dataplane operations are handled:

IPv4/IPv6 route install, update and delete
Nexthop install, update and delete
IP address install and uninstall
MAC/FDB install and delete
VXLAN flood VTEP add and delete
Neighbor install, update and delete (used for EVPN RMAC caching)
SRv6 encap source address

EVPN support

The plugin supports EVPN with VXLAN overlays, including L2 VNI (type-2 MAC/IP and type-3 flood routes), L3 VNI (type-5 IP prefix routes with symmetric IRB) and mixed L2+L3 VNI coexistence within the same VRF.

In grout, L3 VNI VXLAN interfaces use a "VRF mode" that does not require a bridge and SVI as Linux does. The plugin takes care of presenting VXLAN interfaces to FRR in the way its EVPN code expects.

Control Plane Representors

For each grout interface (port, VLAN, bond, bridge, VXLAN), grout creates a TAP device in the host network namespace with the same name. These control plane representors allow FRR daemons to bind sockets (e.g. OSPF multicast, BFD sessions) and exchange packets with the network through grout.

TAP devices operate at L2 (full Ethernet frames). They are configured as NOARP since ARP/ND resolution is handled by grout, not the kernel.

For each VRF, grout creates a TUN device (named after the VRF for the default VRF, or gr-loopN for non-default VRFs). TUN devices operate at L3 (raw IP packets) and act as the catch-all for traffic that does not match a specific TAP. For non-default VRFs, grout also creates a kernel VRF device and enslaves both the TUN and all TAP devices belonging to that VRF.

Routing

IP addresses configured on grout interfaces are mirrored onto the corresponding TAP or TUN devices as host routes (/32 for IPv4, /128 for IPv6). No connected subnet routes are installed.

A default route pointing at the TUN device is installed in each VRF's routing table so that all traffic originated by FRR daemons is funneled into grout for forwarding.

For TAP devices, grout installs per-device policy routing rules (ip rule add oif <tap> lookup 999) with default routes in table 999. This ensures that SO_BINDTODEVICE sockets (used by BFD, OSPF, etc.) are routed through the correct TAP even though no connected subnet routes exist.

Packet flow

Packets from FRR daemons follow the kernel routing table to either the TUN (via the VRF default route) or a TAP (via policy routing). Grout reads these packets from the TUN/TAP file descriptors and injects them into the datapath for forwarding out the physical ports.

In the other direction, packets arriving on DPDK ports that are destined to a local address are delivered to the appropriate TAP or TUN device, where the kernel hands them to the FRR daemon's socket.

Setup

Building the plugin

The plugin is built as a shared library (dplane_grout.so) and installed into FRR's module directory. It requires FRR >= 10.5. The grout build system handles this automatically via meson.

Network namespace

FRR daemons must run in the same network namespace as grout so they can access the control plane representors (TAP/TUN devices) created by grout. The grout.conf systemd drop-in for frr.service is installed automatically and takes care of this.

Loading the plugin

Edit /etc/frr/daemons and add -M dplane_grout to zebra's options:

  zebra_options="  -A 127.0.0.1 -s 90000000 -M dplane_grout"

No other plugin-specific configuration is needed. Routing protocol configuration (bgpd, ospfd, etc.) follows standard FRR conventions.

Socket path

The plugin connects to grout via the path given by the GROUT_SOCK_PATH environment variable, defaulting to /run/grout.sock.

Startup order

Grout should be running before FRR starts. If it is not, the plugin retries the connection every second until grout becomes available. Once connected, the plugin performs a full synchronization before FRR begins normal operation.

Caveats

VRF backend

Only vrf backend lite is supported. The plugin refuses to start if vrf backend netns is configured and exits immediately.

BGP update-source

Because addresses are installed as host routes on the control plane representors, the kernel cannot automatically pick the correct source address for outgoing TCP sessions. BGP neighbors must be configured with an explicit update-source pointing to the grout local IP:

  router bgp 64512
      neighbor 172.16.0.2 remote-as 64512
      neighbor 172.16.0.2 update-source 172.16.0.1
  exit

This is not needed for protocols that use multicast or link-local addresses (OSPF, OSPFv3) since they bind to the interface directly.