10.4. Configuring an EVPN Service With CLI

This section provides examples for configuring EVPN multihoming for VPLS services using the command line interface:

10.4.1. EVPN All-Active Multihoming Configuration Example

This section shows a configuration example for three 7705 SAR PEs, with the following assumptions.

  1. PE-1 and PE-2 are multihomed to CE-12, which uses a LAG to connect to the network. CE-12 is connected to LAG SAPs configured in an all-active multihoming Ethernet segment.
  2. PE-3 is a remote PE that performs aliasing for traffic destined for the CE-12.

The following configuration excerpt applies to a VPLS-1 on PE-1 and PE-2 and includes the corresponding ethernet-segment and lag commands.

A:PE1# configure lag 1 
A:PE1>config>lag# info 
----------------------------------------------
        mode access
        encap-type dot1q
        port 1/1/2 
        lacp active administrative-key 1 system-id 00:00:00:00:69:72 
        no shutdown
----------------------------------------------
 
A:PE1>config>lag# /configure service system bgp-evpn 
A:PE1>config>service>system>bgp-evpn# info 
----------------------------------------------
                route-distinguisher 192.0.2.69:0
                ethernet-segment "ESI-71" create
                    esi 0x01000000007100000001
                    es-activation-timer 10
                    service-carving
                        mode auto
                    exit
                    multi-homing all-active
                    lag 1
                    no shutdown
                exit
----------------------------------------------
 
A:PE1>config>service>system>bgp-evpn# /configure service vpls 1 
A:PE1>config>service>vpls# info 
----------------------------------------------
            bgp
            exit
            bgp-evpn
                cfm-mac-advertisement
                evi 1
                exit
                mpls
                    ingress-replication-bum-label
                    auto-bind-tunnel
                        resolution any
                    exit
                    no shutdown
                exit
            exit
            stp
                shutdown
            exit
            sap lag-1:1 create
            exit
            no shutdown
----------------------------------------------
A:PE2# configure lag 1 
A:PE2>config>lag# info 
----------------------------------------------
        mode access
        encap-type dot1q
        port 1/1/3 
        lacp active administrative-key 1 system-id 00:00:00:00:69:72 
        no shutdown
----------------------------------------------
 
A:PE2>config>lag# /configure service system bgp-evpn 
A:PE2>config>service>system>bgp-evpn# info 
----------------------------------------------
                route-distinguisher 192.0.2.72:0
                ethernet-segment "ESI-71" create
                    esi 0x01000000007100000001
                    es-activation-timer 10
                    service-carving
                        mode auto
                    exit
                    multi-homing all-active
                    lag 1
                    no shutdown
                exit
----------------------------------------------
 
A:PE2>config>service>system>bgp-evpn# /configure service vpls 1 
A:PE2>config>service>vpls# info 
----------------------------------------------
            bgp
            exit
            bgp-evpn
                cfm-mac-advertisement
                evi 1
                exit
                mpls
                    ingress-replication-bum-label
                    auto-bind-tunnel
                        resolution any
                    exit
                    no shutdown
                exit
            exit
            stp
                shutdown
            exit
            sap lag-1:1 create
            exit
            no shutdown
----------------------------------------------

The configuration on the remote PE (PE-3), which supports aliasing to PE-1 and PE-2 is shown below. PE-3 does not have an Ethernet segment configured. It only requires the VPLS-1 configuration and ECMP > 1 in order to perform aliasing.

*A:PE3>config>service>vpls# info 
----------------------------------------------
            bgp
            exit
            bgp-evpn
                cfm-mac-advertisement
                evi 1
                exit
                mpls
                    ingress-replication-bum-label
                    ecmp 4
                    auto-bind-tunnel
                        resolution any
                    exit
                    no shutdown
                exit
            exit
            stp
                shutdown
            exit
            sap 1/1/1:1 create
            exit
            spoke-sdp 4:13 create
                no shutdown
            exit
            no shutdown
----------------------------------------------

10.4.2. EVPN Single-Active Multihoming Configuration Example

To use single-active multihoming on PE-1 and PE-2 instead of all-active multihoming, make the following modifications to the configuration of the EVPN All-Active Multihoming Configuration Example:

  1. change the LAG configuration to single-active
    CE-12 will now be configured with two different LAGs; therefore, the admin-key, system-id, and system-priority must be different on PE-1 and PE-2.
  2. change the Ethernet segment configuration to single-active

No changes are needed at the service level on any of the three PEs.

The differences between single-active and all-active multihoming are highlighted in bold in the following configuration excerpts:

A:PE1# configure lag 1 
A:PE1>config>lag# info 
----------------------------------------------
        mode access
        encap-type dot1q
        port 1/1/2 
        lacp active administrative-key 1 system-id 00:00:00:00:69:69 
        no shutdown
----------------------------------------------
 
A:PE1>config>lag# /configure service system bgp-evpn 
A:PE1>config>service>system>bgp-evpn# info 
----------------------------------------------
                route-distinguisher 192.0.2.69:0
                ethernet-segment "ESI-71" create
                    esi 0x01000000007100000001
                    es-activation-timer 10
                    service-carving
                        mode auto
                    exit
                    multi-homing single-active
                    lag 1
                    no shutdown
                exit
----------------------------------------------
 
A:PE2# configure lag 1 
A:PE2>config>lag# info 
----------------------------------------------
        mode access
        encap-type dot1q
        port 1/1/3 
        lacp active administrative-key 1 system-id 00:00:00:00:72:72 
        no shutdown
----------------------------------------------
 
A:PE2>config>lag# /configure service system bgp-evpn 
A:PE2>config>service>system>bgp-evpn# info 
----------------------------------------------
                route-distinguisher 192.0.2.72:0
                ethernet-segment "ESI-71" create
                    esi 0x01000000007100000001
                    es-activation-timer 10
                    service-carving
                        mode auto
                    exit
                    multi-homing single-active
                    lag 1
                    no shutdown
                exit
----------------------------------------------

10.4.3. EVPN-MPLS r-VPLS Configuration Examples

This section contains the following configuration examples:

EVPN can be used as the unified control plane VPN technology, not only for providing Layer 2 connectivity, but also for Layer 3 (inter-subnet forwarding). EVPN for MPLS tunnels, along with multihoming and passive VRRP, provides efficient Layer 2 or Layer 3 connectivity to distributed hosts and routers.

10.4.3.1. EVPN-MPLS r-VPLS Without Multihoming

The first scenario describes r-VPLS support including IP route advertisement (BGP-EVPN route type 5) with EVPN tunnel interfaces without multihoming. VPLS 101 does not have a connected host, but the linked VPRN has SAP 1/2/1:10. Figure 175 shows an example of topology used for r-VPLS with an EVPN tunnel but without multihoming. IP prefixes are advertised.

Figure 175:  r-VPLS With EVPN Tunnel, Without Multihoming 

The initial configuration includes the following:

  1. cards, MDAs, ports
  2. router interface between PE-2 and PE-3
  3. IS-IS (or OSPF)
  4. LDP enabled on the router interface between PE-2 and PE-3

BGP is configured for address family EVPN on PE-2 and PE-3. The BGP configuration on PE-2 is as follows. The BGP configuration on PE-3 is similar.

# on PE-2:
configure
    router
        autonomous-system 64500
        bgp
            family evpn
            vpn-apply-import
            vpn-apply-export
            enable-peer-tracking
            rapid-withdrawal
            rapid-update evpn
            group "internal"
                peer-as 64500
                neighbor 192.0.2.3
                exit
            exit
        exit

The CEs are connected to SAP 1/2/1:10 in VPRN 10. r-VPLS 101 is bound to VPRN 10 and VPRN 10 has a dedicated interface “int-evi-101” for the EVPN tunnel. In general, if only one route target (RT) is used for import and export in the EVPN-VPLS, it is best to add the EVI and have the route distinguisher (RD) and RT auto-derived from the EVI; this simplifies the configuration and reduces the chance of errors. The service configuration on PE-2 is as follows:

# on PE-2:
configure
    service
        vprn 10 name "VPRN 10" customer 1 create
            route-distinguisher 192.0.2.2:10
            vrf-target target:64500:10
            interface "int-PE-2-CE-20" create
                address 172.16.2.1/24
                sap 1/2/1:10 create
                exit
            exit
            interface "int-evi-101" create
                vpls "evi-101"
                    evpn-tunnel
                exit
            exit
            no shutdown
        exit
        vpls 101 name "evi-101" customer 1 create
            allow-ip-int-bind
            exit
            bgp              // RD and RT are not manually configured in BGP context
            exit
            bgp-evpn
                ip-route-advertisement
                evi 101      // RD and RT will be auto-derived from the EVI
                mpls bgp 1
                    auto-bind-tunnel
                        resolution any
                    exit
                    no shutdown
                exit
            exit
            no shutdown
        exit

In the preceding configuration:

  1. the allow-ip-int-binding command is required so that r-VPLS 101 can be bound to VPRN 10
  2. the service name is required and the configured name “evi-101” must match the name in the VPRN 10 VPLS interface. The service name is configured at service creation time.
  3. the VPRN 10 VPLS interface is configured with the keyword evpn-tunnel. This configuration has the advantage of not having to allocate IP addresses to the r-VPLS interfaces; however, it cannot be used when the r-VPLS interface has local SAPs.

The configuration is similar on PE-3. The RD must be different on PE-2 and PE-3; this is automatically the case when the RD is auto-derived from the configured EVI, as in the example. The RD on PE-2 is 192.0.2.2:101; on PE-3, the RD is 192.0.2.3:101.

PE-3 receives the following BGP-EVPN IP prefix route for prefix 172.16.2.0/24 from PE-2:

34 2019/09/27 12:21:38.100 UTC MINOR: DEBUG #2001 Base Peer 1: 192.0.2.2
"Peer 1: 192.0.2.2: UPDATE
Peer 1: 192.0.2.2 - Received BGP UPDATE:
    Withdrawn Length = 0
    Total Path Attr Length = 97
    Flag: 0x90 Type: 14 Len: 45 Multiprotocol Reachable NLRI:
        Address Family EVPN
        NextHop len 4 NextHop 192.0.2.2
        Type: EVPN-IP-Prefix Len: 34 RD: 192.0.2.2:101, tag: 0, 
                             ip_prefix: 172.16.2.0/24 gw_ip 0.0.0.0 Label: 8388464 
    Flag: 0x40 Type: 1 Len: 1 Origin: 0
    Flag: 0x40 Type: 2 Len: 0 AS Path:
    Flag: 0x80 Type: 4 Len: 4 MED: 0
    Flag: 0x40 Type: 5 Len: 4 Local Preference: 100
    Flag: 0xc0 Type: 16 Len: 24 Extended Community:
        target:64500:101
        mac-nh:04:0b:ff:ff:ff:a2
        bgp-tunnel-encap:MPLS
"

GW IP 0.0.0.0 is an indication that an EVPN tunnel is in use. With EVPN tunnels, no IRB IP address needs to be configured in the VPRN. EVPN tunnels make provisioning easier to automate and save IP addresses from the tenant IP space.

The BGP tunnel encapsulation is MPLS, but the MPLS label in the debug message is not the same as in the service, because the router strips the extra four lowest bits to get the 20-bit MPLS label. In the debug message, the label is 8388464. This is because the debug message is shown before the router can parse the label field and see if it corresponds to an MPLS label (20 bits). The MPLS label is calculated by dividing the label value by 24 (16), as follows: 8388464/16 = 524279.

The MAC next-hop extended community 04:0b:ff:ff:ff:a2 is the MAC address of the interface “int-evi-101” in VPRN 10 on PE-2, as follows:

*A:PE-2# show service id 10 interface "int-evi-101" detail | match MAC 
MACSec           : N/A
MAC Address      : 04:0b:ff:ff:ff:a2    Mac Accounting    : Disabled

The routing table for VPRN 10 on PE-3 contains the route for prefix 172.16.2.0/24 as the BGP-EVPN route with next-hop “int-evi-101” and interface name “ET-04:0b:ff:ff:ff:a2” (ET stands for EVPN Tunnel), as follows:

*A:PE-3# show router 10 route-table
 
===============================================================================
Route Table (Service: 10)
===============================================================================
Dest Prefix[Flags]                            Type    Proto     Age        Pref
      Next Hop[Interface Name]                                    Metric   
-------------------------------------------------------------------------------
172.16.2.0/24                                 Remote  BGP EVPN  00h06m45s  169
       int-evi-101 (ET-04:0b:ff:ff:ff:a2)                           0
172.16.3.0/24                                 Local   Local     00h06m48s  0
       int-PE-3-CE-30                                               0
-------------------------------------------------------------------------------
No. of Routes: 2
Flags: n = Number of times nexthop is repeated
       B = BGP backup route available
       L = LFA nexthop available
       S = Sticky ECMP requested
===============================================================================

The forwarding database (FDB) for VPLS 101 on PE-3 shows an entry for MAC address 04:0b:ff:ff:ff:a2 that is learned via EVPN. The MAC address is static (S) and protected (P). The MPLS label is 524279.

*A:PE-3# show service id 101 fdb detail 
 
===============================================================================
Forwarding Database, Service 101
===============================================================================
ServId     MAC               Source-Identifier       Type     Last Change
            Transport:Tnl-Id                         Age      
-------------------------------------------------------------------------------
101        04:0b:ff:ff:ff:a2 eMpls:                  EvpnS:P  09/27/19 12:55:59
                             192.0.2.2:524279
           ldp:65538
101        04:0d:ff:ff:ff:a2 cpm                     Intf     09/27/19 12:55:57
-------------------------------------------------------------------------------
No. of MAC Entries: 2
-------------------------------------------------------------------------------
Legend:  L=Learned O=Oam P=Protected-MAC C=Conditional S=Static Lf=Leaf
===============================================================================

When the CEs have IPv6 addresses, the VPRN configuration is similar on the PEs, but the ipv6 context must be enabled on the EVPN tunnel interface so that the router can advertise and process BGP-EVPN type 5 routes with IPv6 prefixes. The configuration of VPLS is identical for IPv4 and IPv6.

# on PE-2:
configure
    service
        vprn 16 name "VPRN 16" customer 1 create
            route-distinguisher 192.0.2.2:16
            vrf-target target:64500:16
            interface "int-PE-2-CE-26" create
                ipv6
                    address 2001:db8:16::2:1/120 
                exit
                sap 1/2/1:16 create
                exit
            exit
            interface "int-evi-106" create
                ipv6
                exit
                vpls "evi-106"
                    evpn-tunnel
                exit
            exit
            no shutdown
        exit
        vpls 106 name "evi-106" customer 1 create
            allow-ip-int-bind
            exit
            bgp
            exit
            bgp-evpn
                ip-route-advertisement
                evi 106
                mpls bgp 1
                    auto-bind-tunnel
                        resolution any
                    exit
                    no shutdown
                exit
            exit
            no shutdown
        exit

When advertising IPv6 prefixes, the GW IP field for route type 5 is always populated with the IPv6 address of the r-VPLS interface. In this example, because no specific IPv6 global address is configured, the GW IP will be populated with the auto-created link local address. The following BGP update is received by PE-3 for IPv6 prefix 2001:db8:16::2:0/120:

# on PE-3:
36 2019/09/27 12:21:38.123 UTC MINOR: DEBUG #2001 Base Peer 1: 192.0.2.2
"Peer 1: 192.0.2.2: UPDATE
Peer 1: 192.0.2.2 - Received BGP UPDATE:
    Withdrawn Length = 0
    Total Path Attr Length = 113
    Flag: 0x90 Type: 14 Len: 69 Multiprotocol Reachable NLRI:
        Address Family EVPN
        NextHop len 4 NextHop 192.0.2.2
        Type: EVPN-IP-Prefix Len: 58 RD: 192.0.2.2:106, tag: 0, 
         ip_prefix: 2001:db8:16::2:0/120 gw_ip fe80::60b:1ff:fe02:1 Label: 8388448 
    Flag: 0x40 Type: 1 Len: 1 Origin: 0
    Flag: 0x40 Type: 2 Len: 0 AS Path:
    Flag: 0x80 Type: 4 Len: 4 MED: 0
    Flag: 0x40 Type: 5 Len: 4 Local Preference: 100
    Flag: 0xc0 Type: 16 Len: 16 Extended Community:
        target:64500:106
        bgp-tunnel-encap:MPLS
"

The IPv6 route table on PE-3 is as follows:

*A:PE-3# show router 16 route-table ipv6 
 
===============================================================================
IPv6 Route Table (Service: 16)
===============================================================================
Dest Prefix[Flags]                            Type    Proto     Age        Pref
      Next Hop[Interface Name]                                    Metric   
-------------------------------------------------------------------------------
2001:db8:16::2:0/120                          Remote  BGP EVPN  00h17m24s  169
       fe80::60b:1ff:fe02:1-"int-evi-106"                          0
2001:db8:16::3:0/120                          Local   Local     00h17m26s  0
       int-PE-3-CE-36                                               0
-------------------------------------------------------------------------------
No. of Routes: 2

10.4.3.2. EVPN-MPLS r-VPLS With All-Active Multihoming

Figure 176 shows an example of the topology with all-active multihoming Ethernet segment (ES) “ESI-23”.

Figure 176:  EVPN-MPLS r-VPLS With All-Active Multihoming ES 

BGP is configured between PE-2, PE-3, and PE-4 for address family EVPN. The configuration on PE-2 is as follows:

# on PE-2:
configure
    router
        autonomous-system 64500
        bgp
            family evpn
            vpn-apply-import
            vpn-apply-export
            enable-peer-tracking
            rapid-withdrawal
            rapid-update evpn
            group "internal"
                peer-as 64500
                neighbor 192.0.2.3
                exit
                neighbor 192.0.2.4
                exit
            exit
        exit

All-active multihoming Ethernet segment “ESI-23” is configured on PE-2 and PE-3 as follows:

configure
    service
        system
            bgp-evpn
                ethernet-segment "ESI-23" create
                    esi 01:00:00:00:00:23:00:00:00:01
                    es-activation-timer 3
                    service-carving
                        mode auto
                    exit
                    multi-homing all-active
                    lag 1
                    no shutdown
                exit

The following services are configured on the PEs:

  1. VPRN 20 has interfaces bound to VPLS 200 and VPLS 202. On PE-4, VPRN 20 also has an interface bound to VPLS 203.
  2. VPLS 200 is configured as an EVPN tunnel that connects the PEs.
  3. VPLS 202 and VPLS 203 have attachment circuits to CEs.

The services are configured on PE-2 as follows. The configuration on PE-3 and PE-4 is similar.

# on PE-2:
configure
    service
        vprn 20 name "VPRN 20" customer 1 create
            route-distinguisher 192.0.2.2:20
            vrf-target target:64500:20
            interface "int-evi-202" create
                address 172.16.20.2/24
                mac 00:ca:fe:00:02:02
                vrrp 1 passive
                    backup 172.16.20.254
                    ping-reply
                    traceroute-reply
                exit
                ipv6
                    address 2001:db8:16::20:2/120 
                    link-local-address fe80::16:20:2
                    vrrp 1 passive
                        backup fe80::16:20:fe
                        ping-reply
                        traceroute-reply
                    exit
                exit
                vpls "evi-202"
                exit
            exit
            interface "int-evi-200" create
                ipv6
                exit
                vpls "evi-200"
                    evpn-tunnel
                exit
            exit
            router-advertisement
                interface "int-evi-202"
                    use-virtual-mac
                    no shutdown
                exit
            exit
            no shutdown
        exit
        vpls 200 name "evi-200" customer 1 create
            allow-ip-int-bind
            exit
            bgp
            exit
            bgp-evpn
                ip-route-advertisement
                evi 200
                mpls bgp 1
                    auto-bind-tunnel
                        resolution any
                    exit
                    no shutdown
                exit
            exit
            no shutdown
        exit
        vpls 202 name "evi-202" customer 1 create
            allow-ip-int-bind
            exit
            bgp
            exit
            bgp-evpn
                evi 202
                mpls bgp 1
                    auto-bind-tunnel
                        resolution any
                    exit
                    no shutdown
                exit
            exit
            sap lag-1:20 create
            exit
            no shutdown
        exit

The IPv6 VRRP backup address is in the same subnet as the link local address of the interface “int-evi-202”. The IPv6 address can be set as preferred. Also for IPv6, router advertisement must be enabled and configured to use the virtual MAC address.

10.4.3.2.1. Passive VRRP

EVI 202 is configured as an r-VPLS interface with passive VRRP. A passive VRRP VRID instance suppresses the transmission and reception of keepalive messages. All PEs configured with passive VRRP become VRRP masters and take ownership of the virtual IP and MAC addresses.

Each individual r-VPLS interface has a different MAC/IP address on each PE. The MAC/IP addresses for “int-evi-202” on PE-2 are MAC 00:ca:fe:00:02:02 and IP 172.16.20.2/24 for IPv4 and the same MAC address with IPv6 2001:db8:16::20:2 and fe80::16:20:2. However, the r-VPLS interfaces on all PEs share the same VRID 1 and backup IP address 172.16.20.254, so the same vMAC/vIP 00:00:5e:00:01:01/172.16.20.254 and vMAC/vIP 00:00:5e:00:02:01/ fe80::16:20:fe are advertised by all PEs. PE-2 advertises the following EVPN MAC routes:

82 2019/09/27 12:20:38.600 UTC MINOR: DEBUG #2001 Base Peer 1: 192.0.2.3
"Peer 1: 192.0.2.3: UPDATE
Peer 1: 192.0.2.3 - Send BGP UPDATE:
    Withdrawn Length = 0
    Total Path Attr Length = 292
    Flag: 0x90 Type: 14 Len: 240 Multiprotocol Reachable NLRI:
        Address Family EVPN
        NextHop len 4 NextHop 192.0.2.2
        Type: EVPN-MAC Len: 49 RD: 192.0.2.2:202 ESI: ESI-0, tag: 0, mac len: 48 
          mac: 00:00:5e:00:02:01, IP len: 16, IP: fe80::16:20:fe, label1: 8388384 
        Type: EVPN-MAC Len: 37 RD: 192.0.2.2:202 ESI: ESI-0, tag: 0, mac len: 48
          mac: 00:00:5e:00:01:01, IP len: 4, IP: 172.16.20.254, label1: 8388384 
        Type: EVPN-MAC Len: 49 RD: 192.0.2.2:202 ESI: ESI-0, tag: 0, mac len: 48
          mac: 00:ca:fe:00:02:02, IP len: 16, IP: fe80::16:20:2, label1: 8388384 
        Type: EVPN-MAC Len: 49 RD: 192.0.2.2:202 ESI: ESI-0, tag: 0, mac len: 48
          mac: 00:ca:fe:00:02:02, IP len: 16, IP: 2001:db8:16::20:2, label1: 8388384
        Type: EVPN-MAC Len: 37 RD: 192.0.2.2:202 ESI: ESI-0, tag: 0, mac len: 48
          mac: 00:ca:fe:00:02:02, IP len: 4, IP: 172.16.20.2, label1: 8388384 
    Flag: 0x40 Type: 1 Len: 1 Origin: 0
    Flag: 0x40 Type: 2 Len: 0 AS Path:
    Flag: 0x80 Type: 4 Len: 4 MED: 0
    Flag: 0x40 Type: 5 Len: 4 Local Preference: 100
    Flag: 0xc0 Type: 16 Len: 24 Extended Community:
        target:64500:202
        bgp-tunnel-encap:MPLS
        mac-mobility:Seq:0/Static
"

The three PEs advertise the same (anycast) vMAC/vIP in EVI 202 as protected, but each PE keeps its own MAC entry in the FDB. The following FDB output shows that the source identifier for vMAC 00:00:5e:00:01:01 and vMAC 00:00:5e:00:02:01 is the CPM. These two vMAC entries with source identifier CPM are seen on all PEs.

*A:PE-2# show service id 202 fdb detail 
 
===============================================================================
Forwarding Database, Service 202
===============================================================================
ServId     MAC               Source-Identifier       Type     Last Change
            Transport:Tnl-Id                         Age      
-------------------------------------------------------------------------------
202        00:00:01:00:00:11 sap:lag-1:20            L/90     09/27/19 12:00:35
202        00:00:01:00:00:16 sap:lag-1:20            L/90     09/27/19 12:00:36
202        00:00:04:00:00:41 eMpls:                  Evpn     09/27/19 11:57:24
                             192.0.2.4:524279
           ldp:65539
202        00:00:5e:00:01:01 cpm                     Intf     09/27/19 12:20:19
202        00:00:5e:00:02:01 cpm                     Intf     09/27/19 12:20:19
202        00:ca:fe:00:02:02 cpm                     Intf     09/27/19 11:56:56
202        00:ca:fe:00:02:03 eMpls:                  EvpnS:P  09/27/19 11:57:12
                             192.0.2.3:524274
           ldp:65537
202        00:ca:fe:00:02:04 eMpls:                  EvpnS:P  09/27/19 11:57:23
                             192.0.2.4:524279
           ldp:65539
-------------------------------------------------------------------------------
No. of MAC Entries: 8
-------------------------------------------------------------------------------
Legend:  L=Learned O=Oam P=Protected-MAC C=Conditional S=Static Lf=Leaf
===============================================================================

The interface MAC 00:ca:fe:00:02:02 is local, so it also has the CPM as source identifier. MAC 00:ca:fe:00:02:03 is the r-VPLS interface MAC for PE-3 and it is learned via EVPN-MPLS (eMpls) as static (S) and protected (P). MAC address 00:ca:fe:00:02:04 on PE-4 is also static and protected.

PE-4 sends the following IP prefix route (BGP-EVPN route type 5) for prefix 172.16.23.0/24 to the other PEs:

35 2019/09/27 12:20:38.600 UTC MINOR: DEBUG #2001 Base Peer 1: 192.0.2.2
"Peer 1: 192.0.2.2: UPDATE
Peer 1: 192.0.2.2 - Send BGP UPDATE:
    Withdrawn Length = 0
    Total Path Attr Length = 97
    Flag: 0x90 Type: 14 Len: 45 Multiprotocol Reachable NLRI:
        Address Family EVPN
        NextHop len 4 NextHop 192.0.2.4
        Type: EVPN-IP-Prefix Len: 34 RD: 192.0.2.4:200, tag: 0,
                             ip_prefix: 172.16.23.0/24 gw_ip 0.0.0.0 Label: 8388384 
    Flag: 0x40 Type: 1 Len: 1 Origin: 0
    Flag: 0x40 Type: 2 Len: 0 AS Path:
    Flag: 0x80 Type: 4 Len: 4 MED: 0
    Flag: 0x40 Type: 5 Len: 4 Local Preference: 100
    Flag: 0xc0 Type: 16 Len: 24 Extended Community:
        target:64500:200
        mac-nh:04:0f:ff:00:00:05
        bgp-tunnel-encap:MPLS
"

The IP prefixes are advertised with the next hop equal to the EVPN-tunnel GW MAC “int-evi-200”, as follows:

*A:PE-4# show router 20 interface "int-evi-200" detail | match MAC 
MACSec           : N/A
MAC Address      : 04:0f:ff:00:00:05    Mac Accounting    : Disabled

The routing table for VPRN 20 on PE-2 contains IP prefix 172.16.23.0/24 with next hop 04:0f:ff:00:00:05, as follows:

*A:PE-2# show router 20 route-table
 
===============================================================================
Route Table (Service: 20)
===============================================================================
Dest Prefix[Flags]                            Type    Proto     Age        Pref
      Next Hop[Interface Name]                                    Metric   
-------------------------------------------------------------------------------
172.16.20.0/24                                Local   Local     00h01m07s  0
       int-evi-202                                                  0
172.16.23.0/24                                Remote  BGP EVPN  00h00m48s  169
       int-evi-200 (ET-04:0f:ff:00:00:05)                           0
-------------------------------------------------------------------------------
No. of Routes: 2

The following IPv6 routing table for VPRN 20 on PE-2 contains prefix 2001:db8:16::23:0/120, which has also been advertised by PE-4. The next hop is again “int-evi-200”, only this time the link local IPv6 address is displayed (GW IP) instead of the MAC address. The next hop is the GW IP value for route type 5, as long as it is a non-zero value. When the GW IP address is 0, route type 5 is expected to contain a mac-nh extended community. The MAC encoded in the extended community is used as the next hop in that case.

*A:PE-2# show router 20 route-table ipv6
 
===============================================================================
IPv6 Route Table (Service: 20)
===============================================================================
Dest Prefix[Flags]                            Type    Proto     Age        Pref
      Next Hop[Interface Name]                                    Metric   
-------------------------------------------------------------------------------
2001:db8:16::20:0/120                         Local   Local     00h01m06s  0
       int-evi-202                                                  0
2001:db8:16::23:0/120                         Remote  BGP EVPN  00h00m47s  169
       fe80::a3:a899:473e:c489 -"int-evi-200"                        0
-------------------------------------------------------------------------------
No. of Routes: 2

The EVPN tunnel service VPLS 200 has all the MAC addresses of the EVPN interfaces within VPRN 20 as static (S) and protected (P), as follows:

*A:PE-2# show service id "evi-200" fdb detail 
 
===============================================================================
Forwarding Database, Service 200
===============================================================================
ServId     MAC               Source-Identifier       Type     Last Change
            Transport:Tnl-Id                         Age      
-------------------------------------------------------------------------------
200        04:0b:ff:00:00:05 cpm                     Intf     09/27/19 12:20:31
200        04:0d:ff:00:00:05 eMpls:                  EvpnS:P  09/27/19 12:20:39
                             192.0.2.3:524275
           ldp:65537
200        04:0f:ff:00:00:05 eMpls:                  EvpnS:P  09/27/19 12:20:51
                             192.0.2.4:524280
           ldp:65539
-------------------------------------------------------------------------------
No. of MAC Entries: 3
-------------------------------------------------------------------------------
Legend:  L=Learned O=Oam P=Protected-MAC C=Conditional S=Static Lf=Leaf
===============================================================================

The VRRP instance in each PE is master, as follows:

*A:PE-2# show router 20 vrrp instance 
 
===============================================================================
VRRP Instances
===============================================================================
Interface Name                   VR Id Own Adm  State       Base Pri   Msg Int
                                 IP        Opr  Pol Id      InUse Pri  Inh Int
-------------------------------------------------------------------------------
int-evi-202                      1     No  Up   Master       100       1
                                 IPv4      Up   n/a         100        No
  Backup Addr: 172.16.20.254                                            
int-evi-202                      1     No  Up   Master       100       1
                                 IPv6      Up   n/a         100        Yes
  Backup Addr: fe80::16:20:fe
-------------------------------------------------------------------------------
Instances : 2
===============================================================================
*A:PE-3# show router 20 vrrp instance 
 
===============================================================================
VRRP Instances
===============================================================================
Interface Name                   VR Id Own Adm  State       Base Pri   Msg Int
                                 IP        Opr  Pol Id      InUse Pri  Inh Int
-------------------------------------------------------------------------------
int-evi-202                      1     No  Up   Master       100       1
                                 IPv4      Up   n/a         100        No
  Backup Addr: 172.16.20.254                                            
int-evi-202                      1     No  Up   Master       100       1
                                 IPv6      Up   n/a         100        Yes
  Backup Addr: fe80::16:20:fe
-------------------------------------------------------------------------------
Instances : 2
===============================================================================
*A:PE-4# show router 20 vrrp instance 
 
===============================================================================
VRRP Instances
===============================================================================
Interface Name                   VR Id Own Adm  State       Base Pri   Msg Int
                                 IP        Opr  Pol Id      InUse Pri  Inh Int
-------------------------------------------------------------------------------
int-evi-202                      1     No  Up   Master       100       1
                                 IPv4      Up   n/a         100        No
  Backup Addr: 172.16.20.254                                            
int-evi-203                      2     No  Up   Master       100       1
                                 IPv4      Up   n/a         100        No
  Backup Addr: 172.16.23.254                                            
int-evi-202                      1     No  Up   Master       100       1
                                 IPv6      Up   n/a         100        Yes
  Backup Addr: fe80::16:20:fe
int-evi-203                      2     No  Up   Master       100       1
                                 IPv6      Up   n/a         100        Yes
  Backup Addr: fe80::16:23:fe
-------------------------------------------------------------------------------
Instances : 4
===============================================================================

10.4.3.2.2. Operation

On PE-4, VPRN 20 has one interface bound to VPLS 202 and another interface bound to VPLS 203. CE-41 is attached to VPLS 202 and CE-43 is attached to VPLS 203. When ping messages are sent from CE-41 to CE-43, or vice versa, the messages go via VPRN 20, which has routes to both CEs, as follows:

*A:PE-4# show router 20 route-table 
 
===============================================================================
Route Table (Service: 20)
===============================================================================
Dest Prefix[Flags]                            Type    Proto     Age        Pref
      Next Hop[Interface Name]                                    Metric   
-------------------------------------------------------------------------------
172.16.20.0/24                                Local   Local     04h25m52s  0
       int-evi-202                                                  0
172.16.23.0/24                                Local   Local     04h25m51s  0
       int-evi-203                                                  0
-------------------------------------------------------------------------------
No. of Routes: 2
*A:PE-4# show router 20 route-table ipv6 
 
===============================================================================
IPv6 Route Table (Service: 20)
===============================================================================
Dest Prefix[Flags]                            Type    Proto     Age        Pref
      Next Hop[Interface Name]                                    Metric   
-------------------------------------------------------------------------------
2001:db8:16::20:0/120                         Local   Local     00h00m50s  0
       int-evi-202                                                  0
2001:db8:16::23:0/120                         Local   Local     00h00m50s  0
       int-evi-203                                                  0
-------------------------------------------------------------------------------
No. of Routes: 2

When traffic is sent between CE-11 and CE-41, which are both associated with VPLS 202, the forwarding is done by VPLS and not via the VPRN. The FDB for VPLS 202 on PE-2 is as follows:

*A:PE-2# show service id 202 fdb detail 
 
===============================================================================
Forwarding Database, Service 202
===============================================================================
ServId     MAC               Source-Identifier       Type     Last Change
            Transport:Tnl-Id                         Age      
-------------------------------------------------------------------------------
202        00:00:01:00:00:11 sap:lag-1:20            L/90     09/27/19 12:20:43
202        00:00:01:00:00:16 sap:lag-1:20            L/90     09/27/19 12:20:49
202        00:00:04:00:00:41 eMpls:                  Evpn     09/27/19 12:20:38
                             192.0.2.4:524275
           ldp:65539
202        00:00:5e:00:01:01 cpm                     Intf     09/27/19 12:20:19
202        00:00:5e:00:02:01 cpm                     Intf     09/27/19 12:20:19
202        00:ca:fe:00:02:02 cpm                     Intf     09/27/19 12:20:18
202        00:ca:fe:00:02:03 eMpls:                  EvpnS:P  09/27/19 12:20:26
                             192.0.2.3:524274
           ldp:65537
202        00:ca:fe:00:02:04 eMpls:                  EvpnS:P  09/27/19 12:20:37
                             192.0.2.4:524275
           ldp:65539
-------------------------------------------------------------------------------
No. of MAC Entries: 8
-------------------------------------------------------------------------------
Legend:  L=Learned O=Oam P=Protected-MAC C=Conditional S=Static Lf=Leaf
===============================================================================

MAC 00:00:01:00:00:11 corresponds to CE-11 and is learned on SAP lag-1:20 on PE-2 and advertised via an EVPN MAC route to the BGP peers. MAC 00:00:04:00:00:41 corresponds to CE-41 and was advertised via an EVPN MAC route from PE-4, where the MAC was learned on SAP 1/2/1:41 of VPLS 202, as shown in the following FDB:

*A:PE-4# show service id 202 fdb detail 
 
===============================================================================
Forwarding Database, Service 202
===============================================================================
ServId     MAC               Source-Identifier       Type     Last Change
            Transport:Tnl-Id                         Age      
-------------------------------------------------------------------------------
202        00:00:01:00:00:11 eES:                    Evpn     09/27/19 12:20:04
                             01:00:00:00:00:23:00:00:00:01
202        00:00:01:00:00:16 eES:                    Evpn     09/27/19 12:20:10
                             01:00:00:00:00:23:00:00:00:01
202        00:00:04:00:00:41 sap:1/2/1:41            L/0      09/27/19 12:19:59
202        00:00:5e:00:01:01 cpm                     Intf     09/27/19 12:19:58
202        00:00:5e:00:02:01 cpm                     Intf     09/27/19 12:19:58
202        00:ca:fe:00:02:02 eMpls:                  EvpnS:P  09/27/19 12:19:59
                             192.0.2.2:524274
           ldp:65537
202        00:ca:fe:00:02:03 eMpls:                  EvpnS:P  09/27/19 12:19:59
                             192.0.2.3:524274
           ldp:65539
202        00:ca:fe:00:02:04 cpm                     Intf     09/27/19 12:19:58
-------------------------------------------------------------------------------
No. of MAC Entries: 8
-------------------------------------------------------------------------------
Legend:  L=Learned O=Oam P=Protected-MAC C=Conditional S=Static Lf=Leaf
===============================================================================

The MAC address of CE-43 is not present in the FDB of VPLS 202. The FDB of VPLS 203 shows the MAC address of CE-43, but not of CE-41. Traffic between these two VPLS services goes via the VPRN and cannot use Layer 2 forwarding.

*A:PE-4# show service id 203 fdb detail 
 
===============================================================================
Forwarding Database, Service 203
===============================================================================
ServId     MAC               Source-Identifier       Type     Last Change
            Transport:Tnl-Id                         Age      
-------------------------------------------------------------------------------
203        00:00:04:00:00:43 sap:1/2/1:43            L/0      09/27/19 12:20:32
203        00:00:5e:00:01:02 cpm                     Intf     09/27/19 12:20:16
203        00:00:5e:00:02:02 cpm                     Intf     09/27/19 12:20:16
203        00:ca:fe:00:23:04 cpm                     Intf     09/27/19 12:20:16
-------------------------------------------------------------------------------
No. of MAC Entries: 4
-------------------------------------------------------------------------------
Legend:  L=Learned O=Oam P=Protected-MAC C=Conditional S=Static Lf=Leaf
===============================================================================

10.4.3.3. EVPN-MPLS r-VPLS With Single-Active Multihoming

Figure 177 shows an example of topology with single-active multihoming ES “ESI-23”. The difference between this figure and Figure 176 is that in Figure 177 the ES is single-active and SDPs are used instead of a LAG.

Figure 177:  EVPN-MPLS r-VPLS With Single-Active Multihoming 

The configuration is modified as follows:

  1. LAG 1 is removed from MTU-1, PE-2, and PE-3
  2. network interfaces are configured between MTU-1 and PE-2/PE-3 with IS-IS and LDP enabled
  3. SDPs are configured
  4. ES “ESI-23” is redefined as single-active multihoming. The SDP is associated with this ES.
  5. VPLS 202 on PE-2 and PE-3 no longer has a SAP, but has a spoke-SDP instead
  6. no changes are required on VPRN 20 or VPLS 200

The service configuration on PE-2 is as follows. The configuration on PE-3 is similar. No changes are required on PE-4.

*A:PE-2# configure service 
*A:PE-2>config>service# info 
----------------------------------------------
        system
            bgp-evpn
                ethernet-segment "ESI-23"  create
                    esi 01:00:00:00:00:23:00:00:00:01
                    es-activation-timer 3
                    service-carving
                        mode auto
                    exit
                    multi-homing single-active
                    sdp 21
                    no shutdown
                exit
            exit
        exit
---snip---
        sdp 21 mpls create
            far-end 192.0.2.1
            ldp
            keep-alive
                shutdown
            exit
            no shutdown
        exit
---snip---
        vprn 20 name "VPRN 20" customer 1 create
            route-distinguisher 192.0.2.2:20
            vrf-target target:64500:20
            interface "int-evi-202" create
                address 172.16.20.2/24
                mac 00:ca:fe:00:02:02
                vrrp 1 passive
                    backup 172.16.20.254
                    ping-reply
                    traceroute-reply 
                exit
                ipv6
                    address 2001:db8:16::20:2/120 
                    link-local-address fe80::16:20:2 dad-disable
                    vrrp 1 passive
                        backup fe80::16:20:fe
                        ping-reply
                        traceroute-reply
                    exit
                exit
                vpls "evi-202"
                exit
            exit
            interface "int-evi-200" create
                ipv6
                exit
                vpls "evi-200"
                    evpn-tunnel
                exit
            exit
            router-advertisement
                interface "int-evi-202"
                    use-virtual-mac
                    no shutdown
                exit
            exit
            no shutdown
        exit
        vpls 200 name "evi-200" customer 1 create
            allow-ip-int-bind
            exit
            bgp
            exit
            bgp-evpn
                ip-route-advertisement
                evi 200
                mpls bgp 1
                    auto-bind-tunnel
                        resolution any
                    exit
                    no shutdown
                exit
            exit
            stp
                shutdown
            exit
            no shutdown
        exit
        vpls 202 name "evi-202" customer 1 create
            allow-ip-int-bind
            exit
            bgp
            exit
            bgp-evpn
                evi 202
                mpls bgp 1
                    auto-bind-tunnel
                        resolution any
                    exit
                    no shutdown
                exit
            exit
            stp
                shutdown
            exit
            spoke-sdp 21:20 create
                no shutdown
            exit
            no shutdown
        exit

PE-2 is the designated forwarder (DF) in the single-active ES, as shown in the following output:

*A:PE-2# show service id 202 ethernet-segment 
No sap entries
===============================================================================
SDP Ethernet-Segment Information
===============================================================================
SDP                   Eth-Seg                          Status
-------------------------------------------------------------------------------
21:20                 ESI-23                           DF
===============================================================================
*A:PE-3# show service id 202 ethernet-segment 
No sap entries
===============================================================================
SDP Ethernet-Segment Information
===============================================================================
SDP                   Eth-Seg                          Status
-------------------------------------------------------------------------------
31:20                 ESI-23                           NDF
===============================================================================

When traffic is sent between CE-11 and CE-41, the FDB on PE-2 is as follows, where MAC address 00:00:01:00:00:11 corresponds to CE-11 and is learned on spoke-SDP 21:20, and MAC address 00:00:04:00:00:41 corresponds to CE-41 and his advertised by PE-4 in an EVPN-MAC route.

*A:PE-2# show service id 202 fdb detail 
 
===============================================================================
Forwarding Database, Service 202
===============================================================================
ServId     MAC               Source-Identifier       Type     Last Change
            Transport:Tnl-Id                         Age      
-------------------------------------------------------------------------------
202        00:00:01:00:00:11 sdp:21:20               L/30     09/27/19 12:24:05
202        00:00:01:00:00:16 sdp:21:20               L/30     09/27/19 12:24:10
202        00:00:04:00:00:41 eMpls:                  Evpn     09/27/19 12:20:38
                             192.0.2.4:524275
           ldp:65539
202        00:00:5e:00:01:01 cpm                     Intf     09/27/19 12:20:19
202        00:00:5e:00:02:01 cpm                     Intf     09/27/19 12:20:19
202        00:ca:fe:00:02:02 cpm                     Intf     09/27/19 12:20:18
202        00:ca:fe:00:02:03 eMpls:                  EvpnS:P  09/27/19 12:20:26
                             192.0.2.3:524274
           ldp:65537
202        00:ca:fe:00:02:04 eMpls:                  EvpnS:P  09/27/19 12:20:37
                             192.0.2.4:524275
           ldp:65539
-------------------------------------------------------------------------------
No. of MAC Entries: 8
-------------------------------------------------------------------------------
Legend:  L=Learned O=Oam P=Protected-MAC C=Conditional S=Static Lf=Leaf
===============================================================================

When the SDP between MTU-1 and DF PE-2 goes down, traffic from CE-41 to CE-11 is forwarded by PE-4 to DF PE-2. PE-2 cannot forward the packets to CE-11 directly and will forward the packets to its ES peer PE-3. PE-3 will forward to CE-11 even if the MAC SA matches its own vMAC. Virtual MACs bypass the r-VPLS interface protection, so traffic can be forwarded between the PEs without being dropped.