EVPN over MPLS Configuration for Layer-3 VPN Architecture
- Explaining EVPN over MPLS IRB to create a scalable Layer-3 VPN architecture ‹
- Configuring MPLS-based EVPN IRB L3VPN with Segment Routing ‹
- Packet capture MPLS label stack ‹
- Download section ‹
Explaining EVPN over MPLS IRB to create a scalable Layer-3 VPN architecture
The following table describes key aspects of deploying EVPN over MPLS IRB together with Segment Routing, in order to create a Layer-3 VPN architecture. The table draws a comparison between EVPN IRB over Segment Routing versus VPNv4 address-family over LDP/RSVP-TE.
| BGP VPNv4 with LDP/RSVP-TE | EVPN IRB with Segment Routing |
|---|---|
| Multiprotocol BGP VPNv4 provides only Layer-3 VPN services. | MP-BGP with EVPN IRB can provide Layer-2 VPN and Layer-3 VPN services. |
| LDP uses UDP Hello packets for neighbor discovery, and sends (keepalive) Hellos at regular intervals to maintain an adjacency. Meanwhile, RSVP-TE sends Path messages to establish and maintain Traffic Engineering tunnels. Both LDP and RSVP-TE interoperate with the underlying IGP (OSPF or IS-IS) to share protocol information. | Segment Routing uses a stateless core architecture, it leverages only the IGP (OSPF or IS-IS) to signal MPLS label information, and therefore uses less overhead in an MPLS provider core network (no LDP keepalive/no RSVP soft state). There are multiple options for Segment Routing Traffic Engineering including static tunnels, SR policies with BGP color extended community, or with a PCE architecture. |
| LDP and IGP synchronization issue needs to be considered. | There is no separate protocol to advertise labels because the IGP advertises prefix-to-label bindings, as a result there are no synchronization issues. |
| Fast Reroute mechanism possible with RSVP-TE backup tunnels, and also autotunnels can be used to simplify configuration. | Fast Reroute mechanism ensured with TI-LFA, no backup tunnel configuration and no tunnel keepalive mechanism needed, configuration is simpler. Additionally, TI-LFA provides a broader coverage of traffic protection. |
Configuring MPLS-based EVPN IRB L3VPN with Segment Routing
In the following example scenario two sites are connected over an EVPN-based MPLS L3VPN using VRF Gold. The Service Provider uses Segment Routing (SR) with OSPF as the IGP. The transport labels are assigned from the default SR Global Block (SRGB) of 16000 - 23999.
Each provider router advertises its /32 Loopback IP address as the Node Segment Identifier (SID) together with an index offset value to create a unique MPLS label. In other words, R6 uses the Loopback 6.6.6.6 and receives label 16006, R5 has label 16005, R2 has 16002 and so on.
R2#show mpls forwarding-table Local Outgoing Prefix Bytes Label Outgoing Next Hop Label Label or Tunnel Id Switched interface 200 No Label evpn(uc:ifh 0x7, efp 5) \ 0 none point2point 201 No Label evpn(mc:bd 10) 539 none point2point 202 Pop Label 10.1.0.2-A 0 Gi2 10.1.0.2 203 No Label 10.0.1.0/30[V] 0 aggregate/Gold 204 No Label 192.168.1.0/24[V] \ 880 BD10 10.0.1.2 16003 Pop Label 3.3.3.3/32 0 Gi2 10.1.0.2 16004 16004 4.4.4.4/32 0 Gi2 10.1.0.2 « R4 uses SR label 16004 16005 16005 5.5.5.5/32 0 Gi2 10.1.0.2 « R5 uses SR label 16005 16006 16006 6.6.6.6/32 0 Gi2 10.1.0.2 « R6 uses SR label 16006 A - Adjacency SID
Regarding the VPN service delivery, EVPN IRB is configured between the PE routers R2 and R6, together with the BGP Route Reflector R8 deployed for scalability. This means, a Layer-3 Bridge Domain Interface (BDI) is configured on each PE in VRF Gold in order to exchange eBGP IPv4 routes with the CE routers R1 and R7.
The BGP L2VPN EVPN address-family is enabled on each PE and on the RR. Importantly, BGP VPNv4 is not used here. The MPLS VPN label range is defined on each PE, so PE R2 assigns VPN service labels between 200 - 299, and PE R6 uses 600 - 699.
Configuration:
R2 (PE)
R2#show run | sec ^mpls mpls label range 200 299 R2#show run | sec ^vrf vrf definition Gold rd 65100:1 ! address-family ipv4 route-target export 65100:1 stitching route-target import 65100:2 stitching exit-address-family R2#show run | sec ^segment segment-routing mpls ! connected-prefix-sid-map address-family ipv4 2.2.2.2/32 index 2 range 1 exit-address-family ! R2#show run int Lo10 | sec int interface Loopback10 ip address 2.2.2.2 255.255.255.255 R2#show run int Gi1 | sec int interface GigabitEthernet1 description ** to CE R1 ** no ip address negotiation auto no mop enabled no mop sysid service instance 5 ethernet encapsulation untagged ! R2#show run int Gi2 | sec int interface GigabitEthernet2 description ** to R3 ** ip address 10.1.0.1 255.255.255.252 ip ospf network point-to-point negotiation auto no mop enabled no mop sysid R2#show run int BDI 10 | sec int interface BDI10 description ** to CE R1 in VRF Gold ** vrf forwarding Gold ip address 10.0.1.1 255.255.255.252 no mop enabled no mop sysid R2#show run | sec ^l2vpn l2vpn evpn replication-type ingress mpls label mode per-ce router-id Loopback10 l2vpn evpn instance 100 vlan-based R2#show run | sec ^bridge-domain bridge-domain 10 member GigabitEthernet1 service-instance 5 member evpn-instance 100 R2#show run | sec ^router router ospf 10 router-id 2.2.2.2 segment-routing area 0 mpls segment-routing mpls network 2.2.2.2 0.0.0.0 area 0 network 10.1.0.0 0.0.0.3 area 0 router bgp 65100 bgp router-id 2.2.2.2 bgp log-neighbor-changes neighbor 8.8.8.8 remote-as 65100 neighbor 8.8.8.8 update-source Loopback10 ! address-family l2vpn evpn neighbor 8.8.8.8 activate neighbor 8.8.8.8 send-community both exit-address-family ! address-family ipv4 vrf Gold advertise l2vpn evpn network 10.0.1.0 mask 255.255.255.252 neighbor 10.0.1.2 remote-as 65001 neighbor 10.0.1.2 activate exit-address-family
R6 (PE)
R6#show run | sec ^mpls mpls label range 600 699 R6#show run | sec ^vrf vrf definition Gold rd 65100:2 ! address-family ipv4 route-target export 65100:2 stitching route-target import 65100:1 stitching exit-address-family R6#show run | sec ^segment segment-routing mpls ! connected-prefix-sid-map address-family ipv4 6.6.6.6/32 index 6 range 1 exit-address-family ! R6#show run int Lo10 | sec int interface Loopback10 ip address 6.6.6.6 255.255.255.255 R6#show run int Gi1 | sec int interface GigabitEthernet1 description ** to CE R7 ** no ip address negotiation auto no mop enabled no mop sysid service instance 5 ethernet encapsulation untagged ! R6#show run int Gi2 | sec int interface GigabitEthernet2 description ** to R5 ** ip address 10.4.0.1 255.255.255.252 ip ospf network point-to-point negotiation auto no mop enabled no mop sysid R6#show run int BDI 10 | sec int interface BDI10 description ** to CE R7 in VRF Gold ** vrf forwarding Gold ip address 10.0.2.1 255.255.255.252 no mop enabled no mop sysid R6#show run | sec ^l2vpn l2vpn evpn replication-type ingress mpls label mode per-ce router-id Loopback10 l2vpn evpn instance 100 vlan-based R6#show run | sec ^bridge-domain bridge-domain 10 member GigabitEthernet1 service-instance 5 member evpn-instance 100 R6#show run | sec ^router router ospf 10 router-id 6.6.6.6 segment-routing area 0 mpls segment-routing mpls network 6.6.6.6 0.0.0.0 area 0 network 10.4.0.0 0.0.0.3 area 0 router bgp 65100 bgp router-id 6.6.6.6 bgp log-neighbor-changes neighbor 8.8.8.8 remote-as 65100 neighbor 8.8.8.8 update-source Loopback10 ! address-family l2vpn evpn neighbor 8.8.8.8 activate neighbor 8.8.8.8 send-community both exit-address-family ! address-family ipv4 vrf Gold advertise l2vpn evpn network 10.0.2.0 mask 255.255.255.252 neighbor 10.0.2.2 remote-as 65002 neighbor 10.0.2.2 activate exit-address-family
R8 (BGP RR)
R8#show run int Lo10 | sec int interface Loopback10 ip address 8.8.8.8 255.255.255.255 R8#show run int Gi0/0 | sec int interface GigabitEthernet0/0 description ** to R4 ** ip address 10.5.0.2 255.255.255.252 ip ospf network point-to-point duplex auto speed auto media-type rj45 R8#show run | sec ^router router ospf 10 router-id 8.8.8.8 network 8.8.8.8 0.0.0.0 area 0 network 10.5.0.0 0.0.0.3 area 0 router bgp 65100 bgp router-id 8.8.8.8 bgp log-neighbor-changes neighbor 2.2.2.2 remote-as 65100 neighbor 2.2.2.2 update-source Loopback10 neighbor 6.6.6.6 remote-as 65100 neighbor 6.6.6.6 update-source Loopback10 ! address-family l2vpn evpn neighbor 2.2.2.2 activate neighbor 2.2.2.2 send-community both neighbor 2.2.2.2 route-reflector-client neighbor 6.6.6.6 activate neighbor 6.6.6.6 send-community both neighbor 6.6.6.6 route-reflector-client exit-address-family
R3
R3#show run | sec ^segment segment-routing mpls ! connected-prefix-sid-map address-family ipv4 3.3.3.3/32 index 3 range 1 exit-address-family ! R3#show run int Lo10 | sec int interface Loopback10 ip address 3.3.3.3 255.255.255.255 R3#show run int Gi1 | sec int interface GigabitEthernet1 description ** to PE R2 ** ip address 10.1.0.2 255.255.255.252 ip ospf network point-to-point negotiation auto no mop enabled no mop sysid R3#show run int Gi2 | sec int interface GigabitEthernet2 description ** to R4 ** ip address 10.2.0.1 255.255.255.252 ip ospf network point-to-point negotiation auto no mop enabled no mop sysid R3#show run | sec ^router router ospf 10 router-id 3.3.3.3 segment-routing area 0 mpls segment-routing mpls network 3.3.3.3 0.0.0.0 area 0 network 10.1.0.0 0.0.0.3 area 0 network 10.2.0.0 0.0.0.3 area 0
R4
R4#show run | sec ^segment segment-routing mpls ! connected-prefix-sid-map address-family ipv4 4.4.4.4/32 index 4 range 1 exit-address-family ! R4#show run int Lo10 | sec int interface Loopback10 ip address 4.4.4.4 255.255.255.255 R4#show run int Gi1 | sec int interface GigabitEthernet1 description ** to R3 ** ip address 10.2.0.2 255.255.255.252 ip ospf network point-to-point negotiation auto no mop enabled no mop sysid R4#show run int Gi2 | sec int interface GigabitEthernet2 description ** to R5 ** ip address 10.3.0.1 255.255.255.252 ip ospf network point-to-point negotiation auto no mop enabled no mop sysid R4#show run int Gi3 | sec int interface GigabitEthernet3 description ** to BGP RR R8 ** ip address 10.5.0.1 255.255.255.252 ip ospf network point-to-point negotiation auto no mop enabled no mop sysid R4#show run | sec ^router router ospf 10 router-id 4.4.4.4 segment-routing area 0 mpls segment-routing mpls network 4.4.4.4 0.0.0.0 area 0 network 10.2.0.0 0.0.0.3 area 0 network 10.3.0.0 0.0.0.3 area 0 network 10.5.0.0 0.0.0.3 area 0
R1 (CE)
R1#show run | sec ^vrf vrf definition Gold rd 1:1 ! address-family ipv4 exit-address-family R1#show run int Gi0/0 | sec int interface GigabitEthernet0/0 description ** to Host1 ** no ip address duplex auto speed auto media-type rj45 R1#show run int Gi0/0.1 | sec int interface GigabitEthernet0/0.1 description ** to Host1 in VRF Gold (VLAN 1) ** encapsulation dot1Q 1 native vrf forwarding Gold ip address 192.168.1.2 255.255.255.0 R1#show run int Gi0/1 | sec int interface GigabitEthernet0/1 description ** to PE R2 ** no ip address duplex auto speed auto media-type rj45 R1#show run int Gi0/1.10 | sec int interface GigabitEthernet0/1.10 description ** to PE R2 in VRF Gold ** encapsulation dot1Q 1 native vrf forwarding Gold ip address 10.0.1.2 255.255.255.252 R1#show run | sec ^router router bgp 65001 bgp router-id 1.1.1.1 bgp log-neighbor-changes ! address-family ipv4 vrf Gold network 192.168.1.0 neighbor 10.0.1.1 remote-as 65100 neighbor 10.0.1.1 activate exit-address-family
R7 (CE)
R7#show run | sec ^vrf vrf definition Gold rd 1:1 ! address-family ipv4 exit-address-family R7#show run int Gi0/0 | sec int interface GigabitEthernet0/0 description ** to Host2 ** no ip address duplex auto speed auto media-type rj45 R7#show run int Gi0/0.1 | sec int interface GigabitEthernet0/0.1 description ** to Host2 in VRF Gold (VLAN 1) ** encapsulation dot1Q 1 native vrf forwarding Gold ip address 192.168.2.2 255.255.255.0 R7#show run int Gi0/1 | sec int interface GigabitEthernet0/1 description ** to PE R6 ** no ip address duplex auto speed auto media-type rj45 R7#show run int Gi0/1.10 | sec int interface GigabitEthernet0/1.10 description ** to PE R6 in VRF Gold ** encapsulation dot1Q 1 native vrf forwarding Gold ip address 10.0.2.2 255.255.255.252 R7#show run | sec ^router router bgp 65002 bgp router-id 7.7.7.7 bgp log-neighbor-changes ! address-family ipv4 vrf Gold network 192.168.2.0 neighbor 10.0.2.1 remote-as 65100 neighbor 10.0.2.1 activate exit-address-family
Host1
Host1#show run int Gi0/0 | sec int interface GigabitEthernet0/0 description ** to CE R1 ** ip address 192.168.1.1 255.255.255.0 duplex auto speed auto media-type rj45 Host1#show run | sec ^ip route ip route 0.0.0.0 0.0.0.0 192.168.1.2
Host1#trace 192.168.2.1 probe 1 Type escape sequence to abort. Tracing the route to 192.168.2.1 VRF info: (vrf in name/id, vrf out name/id) 1 192.168.1.2 2 msec 2 10.0.1.1 23 msec 3 10.1.0.2 [MPLS: Labels 16006/604 Exp 0] 90 msec « MPLS label stack with transport SR label 16006 4 10.2.0.2 [MPLS: Labels 16006/604 Exp 0] 91 msec 5 10.3.0.2 [MPLS: Labels 16006/604 Exp 0] 90 msec 6 10.0.2.1 [MPLS: Label 604 Exp 0] 68 msec « PE R6 assigns the VPN service label 604 7 10.0.2.2 90 msec 8 192.168.2.1 89 msec Host2#trace 192.168.1.1 probe 1 Type escape sequence to abort. Tracing the route to 192.168.1.1 VRF info: (vrf in name/id, vrf out name/id) 1 192.168.2.2 2 msec 2 10.0.2.1 24 msec 3 10.4.0.2 [MPLS: Labels 16002/204 Exp 0] 91 msec 4 10.3.0.1 [MPLS: Labels 16002/204 Exp 0] 91 msec 5 10.2.0.1 [MPLS: Labels 16002/204 Exp 0] 90 msec 6 10.0.1.1 [MPLS: Label 204 Exp 0] 72 msec 7 10.0.1.2 92 msec 8 192.168.1.1 91 msec
As visible from the above outputs, Host1 and Host2 can communicate with each other using MPLS label switching. The label stack consists of a transport label assigned by Segment Routing from the Global Block (16000 - 23999), and a VPN service label which is assigned from a preconfigured range (PE R2 200 - 299 and PE R6 600 - 699).
The following outputs are taken from PE R2 and show the EVPN Route Type 5 which is used to advertise IP prefixes.
R2#show ip bgp l2vpn evpn all sum | beg Ne Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd 8.8.8.8 4 65100 108 105 11 0 0 01:28:33 4 « PE R2 receives four EVPN routes from RR R8 R2#show ip bgp l2vpn evpn rd 65100:2 | beg Ne Network Next Hop Metric LocPrf Weight Path Route Distinguisher: 65100:2 *>i [5][65100:2][0][24][192.168.2.0]/17 « EVPN Type 5 IP Prefix Route received from the RR 6.6.6.6 0 100 0 65002 i *>i [5][65100:2][0][30][10.0.2.0]/17 6.6.6.6 0 100 0 i R2#show ip bgp l2vpn evpn route-type 5 0 192.168.2.0 24 BGP routing table entry for [5][65100:2][0][24][192.168.2.0]/17, version 8 « Site 2 prefix advertised in EVPN Route Type 5 Paths: (1 available, best #1, table EVPN-BGP-Table) Flag: 0x100 Not advertised to any peer Refresh Epoch 1 65002 6.6.6.6 (metric 5) (via default) from 8.8.8.8 (8.8.8.8) Origin IGP, metric 0, localpref 100, valid, internal, best EVPN ESI: 00000000000000000000, Gateway Address: 0.0.0.0, VNI Label 0, MPLS VPN Label 604 « VPN service label assigned by PE R6 Extended Community: RT:65100:2 Originator: 6.6.6.6, Cluster list: 8.8.8.8 « EVPN route originator is PE R6 rx pathid: 0, tx pathid: 0x0 net: 0x7FBA8E83CB78, path: 0x7FBA5E0717D0, pathext: 0x7FBA5E125DD8 flags: net: 0x100, path: 0x3, pathext: 0xA1 attribute: 0x7FBA5E061550, ref: 2 Updated on May 11 2024 15:28:29 UTC
In this example topology, each Service Provider core router advertises a Segment Routing label, however the design can be simplified by making only the PE routers advertise SR labels. In other words, only two labels (one label per PE router) can make an EVPN MPLS L3VPN work.
Also, if the VRF-aware PE - CE link needs to be assigned to a VLAN, this can be done on the BDI interface using the following command.
PE-Router#show run int BDI10 | sec int interface BDI10 description ** to CE-Router in VRF Gold ** vrf forwarding Gold ip address 10.0.1.1 255.255.255.252 encapsulation dot1Q 10 no mop enabled no mop sysid
Packet capture MPLS label stack
The following capture shows an ICMP packet sent from Host1 (Site 1) to Host2 (Site 2) with an MPLS label stack. The top label (transport label) is assigned by Segment Routing on R6. The bottom label (VPN service label) is also assigned by R6, however it is advertised using EVPN IRB.
The following packet capture shows a BGP Update message carrying the EVPN IP Prefix route for Site 2 (192.168.2.0/24), and including the VPN service label 603. This Update is sent from the BGP Route Reflector R8, however it is originated on the PE router R6. Upon receiving this Update, PE R2 can use the VPN service label to reach the correct remote site VRF Gold interface (BDI 10) on the PE R6.
Furthermore, it is visible that the BGP Update message uses the Segment Routing transport label 16002 to reach the PE router R2 from the Route Reflector R8. The SR transport label is inserted between the Ethernet header and the IP header of the displayed packet.
