4. Ethernet Virtual Private Networks

This chapter provides information about Ethernet Virtual Private Networks (EVPN) for 7210 SAS-Mxp.

4.1. EVPN Applications

EVPN, as described in RFC 7432, BGP MPLS-Based Ethernet VPN, is an IETF technology that uses a new BGP address family and allows Virtual Private LAN Services (VPLS) to operate in a similar manner to IP-VPNs, in which the MAC addresses and information to set up flooding trees are distributed by BGP.

EVPN is designed to fill the gaps of traditional L2VPN technologies, such as VPLS. The main objective of EVPN is to build E-LAN services similar to IP-VPNs defined in RFC 4364, while supporting MAC learning in the control plane (distributed using multi-protocol BGP (MP-BGP)), efficient multi-destination traffic delivery, and single-active/active-active multi-homing.

EVPN can be used as the control plane for different data plane encapsulations. The Nokia implementation supports EVPN for MPLS tunnels (EVPN-MPLS), where PEs are connected by any type of MPLS tunnel. EVPN-MPLS is generally used as an evolution for VPLS services. The EVPN-MPLS functionality is standardized in RFC 7432.

4.1.1. EVPN for MPLS Tunnels in E-LAN Services

Figure 42 shows the use of EVPN for MPLS tunnels on the 7210 SAS. In this example, EVPN is used as the control plane for E-LAN services.

Note:

Figure 42 shows generic EVPN capabilities. It does not imply support on all 7210 SAS platforms referenced in this guide. For more information about the supported EVPN capabilities for the platforms referenced in this guide, see the sections in this chapter.

Figure 42:  EVPN for MPLS in VPLS Services 

Service providers that offer E-LAN services request EVPN for its multi-homing capabilities and to leverage the optimization EVPN provides.

EVPN supports single-active multi-homing (per-service load-balancing multi-homing). Although VPLS already supports single-active multi-homing, EVPN single-active multi-homing is seen as the superior technology because of its mass-withdrawal capabilities to speed up convergence in scaled environments.

EVPN technology provides significant benefits, including:

  1. IP-VPN-like operation and control for E-LAN services
  2. reduction and (in some cases) suppression of the broadcast, unknown unicast, and multicast (BUM) traffic in the network
  3. simple provision and management
  4. new set of tools to control the distribution of MAC addresses and ARP entries in the network
  5. potential to use single unified control-plane for both L2 VPN services and L3 VPN services
  6. superior multi-homing capabilities

The SR OS EVPN-MPLS implementation is compliant with RFC 7432.

4.2. EVPN for MPLS Tunnels

This section provides information about EVPN for MPLS tunnels (EVPN-MPLS).

4.2.1. BGP-EVPN Control Plane for MPLS Tunnels

Table 51 lists the supported EVPN routes and their usage in the 7210 SAS EVPN-MPLS.

Table 51:  EVPN Routes and Usage 

EVPN Route

Usage

EVPN-MPLS 7210 SAS Support

Type 1 — Ethernet auto-discovery route (A-D)

Mass-withdraw, ESI labels, aliasing 1

Yes 1

Type 2 — MAC/IP advertisement route

MAC/IP advertisement, IP advertisement for ARP resolution

Yes

Type 3 — Inclusive multicast Ethernet Tag route

Flooding tree setup (BUM flooding)

Yes

Type 4 — Ethernet segment (ES) route

ES discovery and DF election

Yes

    Note:

  1. ESI labels and aliasing are not supported on 7210 SAS platforms.

RFC 7432 describes the BGP-EVPN control plane for MPLS tunnels. If EVPN multi-homing is not required, two route types are needed to set up a basic EVPN Instance (EVI): MAC/IP Advertisement and the Inclusive Multicast Ethernet Tag routes. If multi-homing is required, the ES and the Auto-Discovery routes are also needed.

When EVPN multi-homing is enabled in the system, two additional routes are required. Figure 43 shows the fields in routes type 1 and 4 and their associated extended communities.

Figure 43:  EVPN Routes Type 1 and 4  

4.2.1.1. EVPN Route Type 3 — Inclusive Multicast Ethernet Tag Route

The 7210 SAS router generates this route type for setting up the flooding tree (BUM flooding) for a specified VPLS service. The received inclusive multicast routes add entries to the VPLS flood list. When BGP-EVPN MPLS is enabled, the standard supports ingress replication, p2mp mLDP, and composite tunnels as tunnel types in route type 3. On the 7210 SAS, only ingress replication is supported.

4.2.1.2. EVPN Route Type 2 — MAC/IP Advertisement Route

The 7210 SAS router generates this route type for advertising MAC addresses (and IP addresses if proxy-ARP/proxy-ND is enabled). The router generates MAC advertisement routes for the following entities:

  1. learned MACs on SAPs, if the mac-advertisement command is enabled
  2. conditional static MACs, if the mac-advertisement command is enabled
  3. learned MACs on spoke-SDPs, if the mac-advertisement command is enabled
Note:

  1. The unknown-mac-route command is not supported for EVPN-MPLS services.
  2. Proxy-ARP and proxy-ND commands are not supported for IP/MAC associations learned over spoke-SDP.

The route type 2 generated by a router uses the following fields and values:

  1. Route Distinguisher
    This is taken from the route distinguisher (RD) of the VPLS service within the BGP context. The RD can be configured or derived from the bgp-evpn evi value.
  2. Ethernet Segment Identifier (ESI)
    This is zero (0) for MACs learned from single-homed CEs and different from zero for MACs learned from multi-homed CEs.
  3. Ethernet tag ID
    This is zero (0).
  4. MAC address length
    This is always 48.
  5. MAC address
    This is learned or statically configured.
  6. IP address and IP address length
    1. This is the IP address associated with the MAC being advertised with a length of 32 (or 128 for IPv6).
    2. In general, any MAC route without IP has IPL (IP length) = 0 and the IP is omitted.
    3. When received, any IPL value not equal to zero, 32, or 128 discards the route.
  7. MPLS Label 1
    This carries the MPLS label allocated by the system to the VPLS service. The label value is encoded in the high-order 20 bits of the field and is the same label used in type 3 routes for the same service, unless bgp-evpn mpls ingress-replication-bum-label is configured in the service.
  8. MPLS Label 2
    This is 0.
  9. MAC mobility extended community
    This is used for signaling the sequence number in case of mac moves and the “sticky” bit in case of advertising conditional static MACs. If a MAC route is received with a MAC mobility of ext-community, the sequence number and the “sticky” bit are considered for the route selection.

4.2.1.3. EVPN Route Type 1 — Ethernet Auto-Discovery Route

The 7210 SAS router generates this route type to advertise for multi-homing functions. The system can generate two types of Ethernet Auto-Discovery (AD) routes:

  1. Ethernet AD route per-ESI
  2. Ethernet AD route per-EVI

The Ethernet AD per-ESI route uses the following fields and values:

  1. Route Distinguisher
    This is taken from the service-level RD.
  2. Ethernet Segment Identifier (ESI)
    This contains a 10-byte identifier as configured in the system for a specified ethernet-segment.
  3. Ethernet tag ID
    This value, MAX-ET (0xFFFFFFFF), is reserved and used only for AD routes per ESI.
  4. MPLS label
    This is zero (0).
  5. ESI label extended community
    This includes the single-active bit (0 for all-active and 1 for single-active) and ESI label for all-active multi-homing split-horizon. The 7210 SAS always sets the single-active bit to “1”, as long as single-active is supported. In addition, 7210 SAS does not allocate and send the ESI label for single-active multi-homing.
  6. Route-target extended community
    This is taken from the service level route target (RT).

The system can send only a separate Ethernet AD per-ESI route per service.

The Ethernet AD per-EVI route uses the following fields and values:

  1. Route Distinguisher
    This is taken from the service level RD.
  2. Ethernet Segment Identifier (ESI)
    This contains a 10-byte identifier, as configured in the system for a specified ethernet-segment.
  3. Ethernet tag ID
    This is zero (0).
  4. MPLS label
    This encodes the unicast label allocated for the service (high-order 20 bits).
  5. Route-target extended community
    This is taken from the service level RT.
Note:

The AD per-EVI route is not sent with the ESI label Extended Community.

4.2.1.4. EVPN Route Type 4 — ES route

The 7210 SAS router generates this route type for multi-homing ES discovery and DF (Designated Forwarder) election.

  1. Route Distinguisher
    This is taken from the system level RD.
  2. Ethernet Segment Identifier (ESI)
    This contains a 10-byte identifier, as configured in the system for a specified ethernet-segment.
  3. ES-import route-target community
    This value is automatically derived from the MAC address portion of the ESI.
    This extended community is treated as an RT and is supported by RT-constraint (route-target BGP family).

4.2.1.5. BGP Tunnel Encapsulation Extended Community

The following routes are sent with the RFC 5512 BGP Encapsulation Extended Community:

  1. MAC/IP
  2. Inclusive Multicast Ethernet Tag
  3. AD per-EVI routes

ES routes and AD per-ESI routes are not sent with this extended community.

The router processes MPLS encapsulation: 10, the BGP tunnel encapsulation tunnel value registered by IANA for RFC 5512. Any other tunnel value makes the route “treat-as-withdraw”.

If the encapsulation value is MPLS, the BGP validates the high-order 20 bits of the label field, ignoring the low-order 4 bits.

If the encapsulation extended community (as defined in RFC 5512) is not present in a received route, BGP treats the route as an MPLS. On the 7210 SAS, only MPLS encapsulation is supported.

4.2.2. EVPN for MPLS Tunnels in VPLS Services

EVPN can be used in MPLS networks where provider edge (PE) routers are interconnected through any type of MPLS tunnel, including RSVP-TE, LDP, BGP, Segment Routing IS-IS, or Segment Routing OSPF. As with VPRN services, the tunnel selection for a VPLS service (with BGP-EVPN MPLS enabled) is based on the auto-bind-tunnel command.

The EVPN-MPLS VPLS service uses a regular VPLS service where EVPN-MPLS “bindings” can coexist with SAPs. The following is a sample configuration output that shows a VPLS service with EVPN-MPLS.

*A:PE-1>config>service>vpls# info
----------------------------------------------
description "evpn-mpls-service"
bgp
 
bgp-evpn
         evi 10
         mpls
                 auto-bind-tunnel resolution any
                 no shutdown
 
sap 1/1/1:1 create
exit
-------------------------------------------------

First configure a bgp-evpn context as mpls. In addition, the minimum set of commands that must be configured to set up the EVPN-MPLS instance are the evi and the auto-bind-tunnel resolution commands.

Note:

Ensure that the EVI and the system IP are configured before executing the configure>service/vpls>bgp-evpn>mpls>no shutdown command.

The evi value, which is the EVPN instance (EVI) identifier, is unique in the system. It is used by the service-carving algorithm for multi-homing (if configured) and for auto-deriving RT and RDs in EVPN-MPLS services.

If the evi value is not specified, the value is zero and no RD or RTs are auto-derived from it. If it is specified, and no other RD or RT are configured in the service, the following applies:

  1. the RD is derived from: system_ip:evi
  2. the RT is derived from: autonomous-system:evi
Note:

When vsi-import and vsi-export polices are configured, the RT must be configured in the policies, and those values take precedence over the auto-derived RTs. The operational RT for a service is displayed by the show service id svc-id bgp command output. If the bgp-ad>vpls-id command is configured in the service, the vpls-id-derived RT takes precedence over the evi-derived RT.

When the evi command is configured, a config>service>vpls>bgp node (even empty) is required to output correct information using the show service id 1 bgp and show service system bgp-route-distinguisher commands.

The configuration of an EVI is enforced for EVPN services with SAPs in an ethernet-segment. See EVPN Multi-Homing in VPLS Services for more information about ESs.

The following options are specific to EVPN-MPLS and are configured in the config>service>vpls>bgp-evpn>mpls context:

  1. control-word
    Enable or disable the control-word command to guarantee interoperability to other vendors. In accordance with RFC 7432, this command is required to avoid frame disordering.
  2. auto-bind-tunnel
    This command is used to select the type of MPLS transport tunnel used for a specific instance. The command is used in the same way as in VPRN services. See auto-bind-tunnel for more information.
    For BGP-EVPN MPLS, bgp must be explicitly added to the resolution-filter in EVPN (BGP is implicit in VPRNs).
  3. force-vlan-vc-forwarding
    This command allows the system to preserve the VLAN ID and pBits of the service-delimiting qtag in a new tag added in the customer frame before sending the frame to the EVPN core.
    Note:

    On the 7210 SAS-Mxp, the force-vlan-vc-forwarding command is always set to false. The user cannot enable this command by configuring it to true.

  4. split-horizon-group
    This command associates a user-created split-horizon group to all the EVPN-MPLS destinations. See EVPN and VPLS Integration for more information.
  5. ingress-replication-bum-label
    When this command is enabled, it allows the PE to advertise a label for BUM traffic (inclusive multicast routes) that is different from the label advertised for unicast traffic (with the MAC/IP routes). This is useful to avoid potential transient packet duplication in all-active multi-homing.
    Note:

    On the 7210 SAS, because all-active multi-homing is not supported by default, the ingress-replication-bum-label command is disabled. The user has the option to enable this command.

In addition to the preceding options, the following bgp-evpn commands are also available for EVPN-MPLS services:

  1. [no] mac-advertisement
  2. mac-duplication and settings

When EVPN-MPLS is established among some PEs in the network, EVPN unicast and multicast “bindings” to the remote EVPN destinations are created on each PE. A specified ingress PE creates the following:

  1. a unicast EVPN-MPLS destination binding to a remote egress PE, as soon as a MAC/IP route is received from that egress PE
  2. a multicast EVPN-MPLS destination binding to a remote egress PE, only if the egress PE advertises an inclusive multicast Ethernet tag route with a BUM label (only possible if the egress PE is configured with ingress-replication-bum-label)

These bindings, as well as the MACs learned on them, can be checked using the show commands in the following output example, where the remote PE(192.0.2.69) is configured with no ingress-replication-bum-label and PE(192.0.2.70) is configured with ingress-replication-bum-label. As a result, the device has a single EVPN-MPLS destination binding to PE(192.0.2.69) and two bindings (unicast and multicast) to PE(192.0.2.70). The following is a sample configuration output.

*A:Dut# show service id 1 evpn-mpls  
 
===============================================================================
BGP EVPN-MPLS Dest
===============================================================================
TEP Address     Egr Label     Num. MACs   Mcast           Last Change
                 Transport                                
-------------------------------------------------------------------------------
192.0.2.69      262118        1           Yes             06/11/2015 19:59:03
                ldp                                        
192.0.2.70      262139        0           Yes             06/11/2015 19:59:03
                ldp                                        
192.0.2.70      262140        1           No              06/11/2015 19:59:03
                ldp                                        
192.0.2.72      262140        0           Yes             06/11/2015 19:59:03
                ldp                                        
192.0.2.72      262141        1           No              06/11/2015 19:59:03
                ldp                                        
192.0.2.73      262139        0           Yes             06/11/2015 19:59:03
                ldp                                        
192.0.2.254     262142        0           Yes             06/11/2015 19:59:03
                bgp                                        
-------------------------------------------------------------------------------
Number of entries : 7
-------------------------------------------------------------------------------
===============================================================================
 
*A:Dut# show service id 1 fdb detail 
 
===============================================================================
Forwarding Database, Service 1
===============================================================================
ServId    MAC               Source-Identifier        Type     Last Change
                                                     Age      
-------------------------------------------------------------------------------
1         00:ca:fe:ca:fe:69 eMpls:                   EvpnS    06/11/15 21:53:48
                            192.0.2.69:262118
1         00:ca:fe:ca:fe:70 eMpls:                   EvpnS    06/11/15 19:59:57
                            192.0.2.70:262140
1         00:ca:fe:ca:fe:72 eMpls:                   EvpnS    06/11/15 19:59:57
                            192.0.2.72:262141
-------------------------------------------------------------------------------
No. of MAC Entries: 3
-------------------------------------------------------------------------------
Legend:  L=Learned O=Oam P=Protected-MAC C=Conditional S=Static
===============================================================================

4.2.2.1. EVPN and VPLS Integration

In accordance with draft-ietf-bess-evpn-vpls-seamless-integ, the 7210 SAS EVPN implementation supports the integration of EVPN-MPLS and VPLS to the same network within the same service. Because EVPN is not deployed in greenfield deployments, this feature is useful for facilitating the integration between both technologies and for migrating VPLS services to EVPN-MPLS.

The following behavior enables the integration of EVPN and SDP-bindings in the same VPLS network.

  1. Systems with EVPN endpoints and SDP-bindings to the same far-end bring down the SDP-bindings.
    1. The router allows the establishment of an EVPN endpoint and an SDP-binding to the same far-end, but the SDP-binding is kept operationally down. Only the EVPN endpoint is operationally up. This applies to spoke-SDPs (manual and BGP-AD).
    2. If an EVPN endpoint to a specified far-end exists and a spoke-SDP establishment is attempted, the spoke-SDP is set up but is kept down with an operational flag, indicating that there is an EVPN route to the same far-end.
    3. If an spoke-SDP exists and a valid or used EVPN route arrives, the EVPN endpoint is set up and the spoke-SDP is brought down with an operational flag indicating that there is an EVPN route to the same far-end.
    4. In the case of an SDP-binding and EVPN endpoint to different far-end IPs on the same remote PE, both links are up. This may occur if the SDP-binding is terminated in an IPv4 address that is different from the system address where the EVPN endpoint is terminated.
  2. Users can add spoke-SDPs and all the EVPN-MPLS endpoints in the same split-horizon group (SHG).
    1. The following CLI command is added under the bgp-evpn>mpls context so that the EVPN-MPLS endpoints can be added to a split-horizon group: bgp-evpn mpls [no] split-horizon-group group-name.
    2. The bgp-evpn mpls split-horizon-group command must reference a user-configured split-horizon group. User-configured split-horizon groups can be configured within the service context. The same group-name can be associated with SAPs, spoke-SDPs, pw-template-bindings, and EVPN-MPLS endpoints.
    3. If the bgp-evpn mpls split-horizon-group command is not used, the default split-horizon group (that contains all the EVPN endpoints) is still used but cannot be referred to using SAPs/spoke-SDPs.
  3. The system disables the advertisement of MACs learned on spoke-SDPs or SAPs that are part of an EVPN split-horizon group.
    1. When the SAPs or spoke-SDPs (manual or BGP-AD-discovered) are configured within the same split-horizon group as the EVPN endpoints, MAC addresses are still learned on them, but they are not advertised in EVPN.
    2. The preceding statement is also true if proxy-ARP/proxy-ND is enabled and an IP-to-MAC pair is learned on a SAP that belongs to the EVPN split-horizon group.
    3. The SAPs added to an EVPN split-horizon group should not be part of any EVPN multi-homed ES. If that occurs, the PE still advertises the AD per-EVI route for the SAP, attracting EVPN traffic that could not possibly be forwarded to that SAP.

Figure 44 shows an example of EVPN-VPLS integration.

Figure 44:  EVPN-VPLS Integration 

The following is a sample configuration for PE1, PE5, and PE2 from the EVPN-VPLS integration example in Figure 44.

*A:PE1>config>service# info 
----------------------------------------------
pw-template 1 create
vpls 1 customer 1 create
  split-horizon-group "SHG-1" create 
  bgp
    route-target target:65000:1
    pw-template-binding 1 split-horizon-group SHG-1 
  bgp-ad
    no shutdown
    vpls-id 65000:1
  bgp-evpn
    evi 1
    mpls
      no shutdown
      split-horizon-group SHG-1
  spoke-sdp 12:1 create
  exit
  sap 1/1/1:1 create
  exit
 
*A:PE5>config>service# info 
----------------------------------------------
pw-template 1 create
vpls 1 customer 1 create
  bgp
    route-target target:65000:1
    pw-template-binding 1 split-horizon-group SHG-1 # auto-created SHG
  bgp-ad
    no shutdown
    vpls-id 65000:1
  spoke-sdp 52:1 create
  exit
 
*A:PE2>config>service# info 
----------------------------------------------
vpls 1 customer 1 create
  end-point CORE create
    no suppress-standby-signaling
  spoke-sdp 21:1 end-point CORE
    precedence primary
  spoke-sdp 25:1 end-point CORE

The following applies to the configuration described in the preceding example:

  1. PE1, PE3, and PE4 have BGP-EVPN and BGP-AD enabled in VPLS-1. PE5 has BGP-AD enabled and PE2 has active/standby spoke-SDPs to PE1 and PE5. In this configuration:
    1. PE1, PE3, and PE4 attempt to establish BGP-AD spoke-SDPs, but they are kept operationally down as long as there are EVPN endpoints active among them
    2. BGP-AD spoke-SDPs and EVPN endpoints are instantiated within the same split-horizon group, for example, SHG-1
    3. manual spoke-SDPs from PE1 and PE5 to PE2 are not part of SHG-1
  2. EVPN MAC advertisements
    1. MACs learned on FEC128 spoke-SDPs are advertised normally in EVPN
    2. MACs learned on FEC129 spoke-SDPs are not advertised in EVPN (because they are part of SHG-1, which is the split-horizon group used for bgp-evpn>mpls). This prevents any data plane MACs learned on the SHG from being advertised in EVPN.
  3. BUM traffic operation on PE1
    1. when CE1 sends BUM traffic, PE1 floods to all the active bindings
    2. when CE2 sends BUM traffic, PE2 sends it to PE1 (active spoke-SDP) and PE1 floods to all the bindings and SAPs
    3. when CE5 sends BUM traffic, PE5 floods to the three EVPN PEs. PE1 floods to the active spoke-SDP and SAPs, never to the EVPN PEs because they are part of the same SHG.

4.2.2.2. Auto-Derived RD in Services with Multiple BGP Families

Multiple BGP families and protocols can be enabled at the same time in a VPLS. When BGP-EVPN is enabled, BGP-AD is also supported. A single RD is used per service and not per BGP family or protocol.

The following rules apply.

  1. The VPLS RD is selected based on the following precedence:
    1. manual RD always takes precedence when configured
    2. if no manual-rd configuration, the RD is derived from the bgp-ad>vpls-id
    3. if no manual-rd or vpls-id configuration exists, the RD is derived from the bgp-evpn>evi
    4. if no manual-rd, vpls-id, or evi configuration exists, there is no RD and the service fails
  2. The selected RD (see the preceding selection criteria) is displayed by the Oper Route Dist field of the show service id bgp command.
  3. The service supports dynamic RD changes; for example, the CLI allows the vpls-id to be updated dynamically, even if it is used to auto-derive the service RD for bgp-ad.
    Note:

    When the RD changes, the active routes for that VPLS are withdrawn and re-advertised with the new RD.

  4. If one of the mechanisms to derive the RD for a specified service is removed from the configuration, the system selects a new RD based on the preceding rules. For example, if the vpls-id is removed from the configuration, the routes are withdrawn, the new RD is selected from the EVI, and the routes re-advertised with the new RD. See Auto-Derived RD in Services with Multiple BGP Families for more information about rules governing the RD selection.
    Note:

    The reconfiguration fails if the new RD already exists in a different VPLS or Epipe service.

  5. Because the vpls-id takes precedence over the evi when the RD is auto-derived, the existing RD is not affected when evpn is added to an existing bgp-ad service. This is important to support BGP-AD to EVPN migration.

4.2.3. EVPN Multi-Homing in VPLS Services

EVPN multi-homing implementation is based on the concept of the Ethernet Segment (ES). An ES is a logical structure that can be defined in one or more PEs and identifies the CE (or access network) multi-homed to the EVPN PEs. An ES is associated with port or LAG objects and shared by all the services defined on those objects. On the 7210 SAS, only the following service objects are allowed to be configured as an ES: port and LAG.

Each ES has a unique Ethernet Segment Identifier (ESI) that is 10 bytes long and is manually configured in the router.

Note:

Because the esi command is advertised in the control plane to all the PEs in the EVPN network, it is important to ensure that the 10-byte esi value is unique throughout the entire network. Single-homed CEs are assumed to be connected to an ES with esi = 0 (single-homed ESs are not explicitly configured).

4.2.4. EVPN All-Active Multi-Homing

Note:

The 7210 SAS supports only single-active multi-homing. All-active multi-homing is not supported. References to all-active multi-homing in this section are provided for completeness of this description only and are not intended to imply support on the 7210 SAS.

In accordance with RFC 7432, all-active multi-homing is only supported on access LAG SAPs, and it is mandatory to configure the CE with a LAG to avoid duplicated packets to the network. LACP is optional.

When PE3 installs MAC1 in the Forwarding Database (FDB), it associates MAC1 not only with the advertising PE (PE1), but also with all the PEs advertising the same esi (ESI2) for the service. In this example, PE1 and PE2 advertise an AD per-EVI route for ESI2; therefore, PE3 installs the two next-hops associated with MAC1.

To enable aliasing, configure ECMP greater than 1 in the bgp-evpn>mpls context.

4.2.4.1. All-Active Multi-Homing Procedures

The following procedures are implemented in SR OS to provide all-active multi-homing for a specified ES:

  1. Designated Forwarder Election
  2. Split-Horizon
  3. Aliasing
Note:

The 7210 SAS only supports two PEs per ES for all-active multi-homing.

4.2.4.1.1. Designated Forwarder Election

Using Designated Forwarder (DF) election in EVPN all-active multi-homing prevents duplicate packets on the multi-homed CE. The DF election procedure elects one DF PE per ESI per service; the rest of the PEs are non-DF for the ESI and service. Only the DF forwards BUM traffic from the EVPN network toward the ES SAPs (the multi-homed CE). The non-DF PEs do not forward BUM traffic to the local ES SAPs.

Figure 45 shows the need for DF election in all-active multi-homing.

Figure 45:  DF Election 
Note:

BUM traffic from the CE to the network and known unicast traffic in any direction is allowed on both the DF and non-DF PEs.

4.2.4.1.2. Split-Horizon

The EVPN split-horizon procedure ensures that BUM traffic originated by the multi-homed PE and sent from the non-DF to the DF is not replicated back to the CE in the form of echoed packets. To avoid echoed packets, the non-DF (PE1) sends all the BUM packets to the DF (PE2) with an indication of the source ES. That indication is the ESI Label (ESI2 in Figure 46), previously signaled by PE2 in the AD per-ESI route for the ES. When it receives an EVPN packet (after the EVPN label lookup), PE2 finds the ESI label that identifies its local ES ESI2. The BUM packet is replicated to other local CEs but not to the ESI2 SAP.

Figure 46 shows the EVPN split-horizon concept for all-active multi-homing.

Figure 46:  Split-Horizon 

4.2.4.1.3. Aliasing

Figure 47 shows the EVPN aliasing procedure for all-active multi-homing. Because CE2 is multi-homed to PE1 and PE2 using an all-active ES, “aliasing” is the procedure by which PE3 can load-balance the known unicast traffic between PE1 and PE2, even if the destination MAC address is only advertised by PE1.

Figure 47:  Aliasing  

4.2.4.2. All-Active Multi-Homing Service Model

The following is a sample output of the PE1 configuration that provides all-active multi-homing to the CE2 shown in Figure 47.

*A:PE1>config>lag(1)# info  
---------------------------------------------- 
  mode access 
  encap-type dot1q 
  port 1/1/2 
  lacp active administrative-key 1 system-id 00:00:00:00:00:22 
  no shutdown 
 
*A:PE1>config>service>system>bgp-evpn# info  
---------------------------------------------- 
  route-distinguisher 10.1.1.1:0 
  ethernet-segment "ESI2" create 
    esi 01:12:12:12:12:12:12:12:12:12 
    multi-homing all-active 
    service-carving 
      mode auto 
    lag 1  
    no shutdown 
 
*A:PE1>config>redundancy>evpn-multi-homing# info 
---------------------------------------------- 
    boot-timer 120 
    es-activation-timer 10 
 
*A:PE1>config>service>vpls# info  
----------------------------------------------
  description "evpn-mpls-service with all-active multihoming" 
  bgp 
  bgp-evpn 
    evi 10 
    mpls 
      no shutdown 
      auto-bind-tunnel resolution any 
      ingress-replication-bum-label 
  sap lag-1:1 create  
  exit 

In the same way, PE2 is configured as follows:

*A:PE1>config>lag(1)# info  
---------------------------------------------- 
  mode access 
  encap-type dot1q 
  port 1/1/1 
  lacp active administrative-key 1 system-id 00:00:00:00:00:22 
  no shutdown 
 
*A:PE1>config>service>system>bgp-evpn# info  
---------------------------------------------- 
  route-distinguisher 10.1.1.1:0 
  ethernet-segment "ESI12" create 
    esi 01:12:12:12:12:12:12:12:12:12 
    multi-homing all-active 
    service-carving 
      mode auto 
    lag 1  
    no shutdown 
 
*A:PE1>config>redundancy>evpn-multi-homing# info  
---------------------------------------------- 
    boot-timer 120 
    es-activation-timer 10 
 
*A:PE1>config>service>vpls# info  
----------------------------------------------
  description "evpn-mpls-service with all-active multihoming" 
  bgp 
    route-distinguisher 65001:60 
    route-target target:65000:60 
  bgp-evpn 
    evi 10 
    mpls 
      no shutdown 
      auto-bind-tunnel resolution any 
  sap lag-1:1 create  
  exit 

The following considerations apply when the all-active multi-homing procedure is enabled.

  1. The ethernet-segment command must be configured with a name and a 10-byte esi using the config service system bgp-evpn ethernet-segment es_name create and config service system bgp-evpn ethernet-segment esi value commands.
  2. When configuring the esi, the system enforces that the 6 high-order octets after the type are not zero, which ensures that the auto-derived route-target for the ES route is not zero). In addition, the entire ESI value must be unique in the system.
  3. Only a LAG can be associated with the all-active ES. LAG is used exclusively for EVPN multi-homing. Other LAG ports in the system can continue to be used for MC-LAG and other services.
  4. When the LAG is configured on PE1 and PE2, the same admin-key, system-priority, and system-id must be configured on both PEs so that CE2 can respond as though it is connected to the same system.
  5. Only one SAP per service can be part of the same ethernet-segment.

4.2.4.3. ES Discovery and DF Election Procedures

The ES discovery and DF election are implemented in three logical steps, as shown in Figure 48.

Figure 48:  ES Discovery and DF Election 

4.2.4.3.1. Step 1 — ES Advertisement and Discovery

ES ESI-1 is configured with all the required parameters, as described in All-Active Multi-Homing Service Model. When ethernet-segment no shutdown is executed, PE1 and PE2 advertise an ES route for ESI-1. They both include the route-target auto-derived from the MAC portion of the configured ESI. If the route-target address family is configured in the network, this allows the RR to keep the dissemination of the ES routes under control.

In addition to the ES route, PE1 and PE2 advertise AD per-ESI routes and AD per-EVI routes.

  1. AD per-ESI routes announces the ES capabilities, including the mode (single-active or all-active) and the ESI label for split horizon.
  2. AD per-EVI routes are advertised so that PE3 knows what services (EVIs) are associated with the ESI. These routes are used by PE3 for its aliasing and backup procedures.

4.2.4.3.2. Step 2 — DF Election

When ES routes exchange between PE1 and PE2 is complete, both run the DF election for all the services in the ethernet-segment.

PE1 and PE2 elect a Designated Forwarder (DF) per ESI service. The default DF election mechanism in the SR OS is service-carving (as per RFC 7432). The following applies when the mechanism is enabled on a specified PE.

  1. An ordered list of PE IPs where ESI-1 resides is built. The IPs are derived from the origin IP fields of all the ES routes received for ESI-1, as well as the local system address. The lowest IP is considered ordinal “0” in the list.
  2. The local IP can only be considered a “candidate” after successful ethernet-segment no shutdown for a specified service.
    Note:

    The remote PE IPs must be present in the local PE RTM so that they can participate in the DF election.

  3. A PE only considers a specified remote IP address as candidate for the DF election algorithm for a specified service if, as well as the ES route, the corresponding AD routes per-ESI and per-EVI for that PE have been received and properly activated.
  4. All remote PEs that receive the AD per-ES routes (for example, PE3) interpret ESI-1 as all-active if all the PEs send their AD per-ES routes with the single-active bit = 0. Otherwise, if at least one PE sends an AD route per-ESI with the single-active flag set or the local ESI configuration is single-active, the ESI behaves as single-active.
  5. An es-activation-timer can be configured at the redundancy>bgp-evpn-multi-homing>es-activation-timer level or at the service>system>bgp-evpn>eth-seg>es-activation-timer level. This timer, which is 3 seconds by default, delays the transition from non-DF to DF for a specified service after the DF election has run.
    1. This use of the es-activation-timer is different from zero and minimizes the risks of loops and packet duplication due to “transient” multiple DFs.
    2. The same es-activation-timer should be configured in all PEs that are part of the same ESI. It is up to the user to configure either a long timer to minimize the risks of loops/duplication or even es-activation-timer=0 to speed up the convergence for non-DF to DF transitions. When the user configures a specific value, the value configured at the ES level supersedes the configured global value.
  6. The DF election is triggered by the following events.
    1. The config service system bgp-evpn eth-seg no shutdown command triggers the DF election for all the services in the ESI.
    2. Reception of a new update or withdrawal of an ES route (containing an ESI configured locally) triggers the DF election for all the services in the ESI.
    3. Reception of a new update or withdrawal of an AD per-ES route (containing an ESI configured locally) triggers the DF election for all the services associated with the list of route-targets received along with the route.
    4. Reception of a new update of an AD per-ES route with a change in the ESI-label extended community (single-active bit or MPLS label) triggers the DF election for all the services associated with the list of route-targets received along with the route.
    5. Reception of a new update or withdrawal of an AD route per-EVI (containing an ESI configured locally) triggers the DF election for that service.
  1. When the PE boots up, the boot-timer allows the necessary time for the control plane protocols to come up before bringing up the ES and running the DF algorithm. The boot-timer is configured at the system level, using the config redundancy bgp-evpn-multi-homing boot-timer command, and should use a value that is long enough to allow the node (with any cards, if available) to boot up and BGP sessions to come up before exchanging ES routes and running the DF election for each EVI/ISID.
    1. The system does not advertise ES routes until the boot timer expires. This guarantees that the peer ES PEs do not run the DF election until the PE is ready to become the DF, if it needs to.
    2. The following show command displays the configured boot-timer and the remaining timer, if the system is still in boot-stage.
A:PE1# show redundancy bgp-evpn-multi-homing 
===============================================================================
Redundancy BGP EVPN Multi-homing Information
===============================================================================
Boot-Timer              : 10 secs                 
Boot-Timer Remaining    : 0 secs                  
ES Activation Timer     : 3 secs                  
===============================================================================
  1. When service-carving mode auto is configured (default mode), the DF election algorithm runs the function [V(evi) mod N(peers) = i(ordinal)] to identify the DF for a specified service and ESI, as described in the following example:
    1. As shown in Figure 48, PE1 and PE2 are configured with ESI-1. Given that V(10) mod N(2) = 0, PE1 are elected DF for VPLS-10 (because its IP address is lower than PE2's and it is the first PE in the candidate list).
      Note:

      The algorithm uses the configured evi in the service and not the service-id. The evi for a service must match in all PEs that are part of the ESI. This guarantees that the election algorithm is consistent across all PEs of the ESI. The evi must be always configured in a service with SAPs that are created in an ES.

  2. A service-carving command is supported to manually configure the EVI identifiers for which the PE is primary: service-carving mode manual/manual evi start-evi to end-evi. The following considerations apply.
    1. The system is the PE forwarding/multicasting traffic for the evi identifiers included in the configuration. The PE is secondary (non-DF) for the non-specified evi identifiers.
    2. If a range is configured but service-carving is not mode manual, the range has no effect.
    3. Only two PEs are supported when service-carving mode manual is configured. If manual mode is configured for a third PE for an ESI, the two non-primary PEs remain non-DF regardless of the primary status.
    4. For example, as shown in Figure 48: if PE1 is configured with service-carving manual evi 1 to 100 and PE2 with service-carving manual evi 101 to 200, PE1 is the primary PE for service VPLS 10 and PE2 the secondary PE.
  3. If service-carving is disabled, the lowest originator IP wins the election for a specified service and ESI. Use the config service system bgp-evpn eth-seg service-carving mode off command to disable service-carving.

The following sample configuration output shows the ethernet-segment configuration and DF status for all EVIs configured in the ethernet-segment.

*A:Dut-B# /show service system bgp-evpn ethernet-segment name "eslag1" all 
===============================================================================
Service Ethernet Segment
===============================================================================
Name                    : eslag1
Admin State             : Enabled            Oper State         : Up
ESI                     : 00:bc:01:00:00:00:00:00:00:01
Multi-homing            : allActive          Oper Multi-homing  : allActive
Lag Id                  : 1                  
ES Activation Timer     : 3 secs (default)   
Exp/Imp Route-Target    : target:bc:01:00:00:00:00
Svc Carving             : auto               
ES SHG Label            : 131070             
===============================================================================
===============================================================================
EVI Information 
===============================================================================
EVI                 SvcId               Actv Timer Rem      DF
-------------------------------------------------------------------------------
1                   1                   0                   no
-------------------------------------------------------------------------------
Number of entries: 1
===============================================================================
-------------------------------------------------------------------------------
DF Candidate list
-------------------------------------------------------------------------------
EVI                                     DF Address
-------------------------------------------------------------------------------
1                                       10.20.1.2
1                                       10.20.1.3
-------------------------------------------------------------------------------
Number of entries: 2
-------------------------------------------------------------------------------
------------------------------------------------------------------------------- 

4.2.4.3.3. Step 3 — DF and Non-DF Service Behavior

Based on the result of the DF election or the manual service-carving, the control plane on the non-DF (PE1) instructs the data path to remove the LAG SAP (associated with the ESI) from the default flooding list for BM traffic (unknown unicast traffic may still be sent if the EVI label is a unicast label and the source MAC address is not associated to the ESI). On PE1 and PE2, both LAG SAPs learn the same MAC address (coming from the CE).

For example, in the following sample configuration output, 00:ca:ca:ba:ce:03 is learned on both PE1 and PE2 access LAG (on ESI-1). However, PE1 learns the MAC as “Learned” whereas PE2 learns it as “Evpn”. This is because CE2 hashes the traffic for that source MAC to PE1. And PE2 learns the MAC through EVPN but associates the MAC to the ESI SAP, because the MAC belongs to the ESI.

*A:PE1# show service id 1 fdb detail 
===============================================================================
Forwarding Database, Service 1
===============================================================================
ServId    MAC               Source-Identifier        Type     Last Change
                                                     Age      
-------------------------------------------------------------------------------
1         00:ca:ca:ba:ce:03 sap:lag-1:1              L/0      06/11/15 00:14:47
1         00:ca:fe:ca:fe:70 eMpls:                   EvpnS    06/11/15 00:09:06
                            192.0.2.70:262140
1         00:ca:fe:ca:fe:72 eMpls:                   EvpnS    06/11/15 00:09:39
                            192.0.2.72:262141
-------------------------------------------------------------------------------
No. of MAC Entries: 3
-------------------------------------------------------------------------------
Legend:  L=Learned O=Oam P=Protected-MAC C=Conditional S=Static
===============================================================================
 
*A:PE2# show service id 1 fdb detail 
===============================================================================
Forwarding Database, Service 1
===============================================================================
ServId    MAC               Source-Identifier        Type     Last Change
                                                     Age      
-------------------------------------------------------------------------------
1         00:ca:ca:ba:ce:03 sap:lag-1:1              Evpn     06/11/15 00:14:47
1         00:ca:fe:ca:fe:69 eMpls:                   EvpnS    06/11/15 00:09:40
                            192.0.2.69:262141
1         00:ca:fe:ca:fe:70 eMpls:                   EvpnS    06/11/15 00:09:40
                            192.0.2.70:262140
-------------------------------------------------------------------------------
No. of MAC Entries: 3
-------------------------------------------------------------------------------
Legend:  L=Learned O=Oam P=Protected-MAC C=Conditional S=Static
===============================================================================

When PE1 (non-DF) and PE2 (DF) exchange BUM packets for evi 1, the packets are sent including the ESI label at the bottom of the stack (in both directions). The ESI label advertised by each PE for ESI-1 can be displayed using the following command.

*A:PE1# show service system bgp-evpn ethernet-segment name "ESI-1" 
===============================================================================
Service Ethernet Segment
===============================================================================
Name                    : ESI-1
Admin State             : Up                 Oper State         : Up
ESI                     : 01:00:00:00:00:71:00:00:00:01
Multi-homing            : allActive          Oper Multi-homing  : allActive
Lag Id                  : 1                  
ES Activation Timer     : 0 secs             
Exp/Imp Route-Target    : target:00:00:00:00:71:00
 
Svc Carving             : auto               
ES SHG Label            : 262142             
===============================================================================
*A:PE2# show service system bgp-evpn ethernet-segment name "ESI-1" 
 
===============================================================================
Service Ethernet Segment
===============================================================================
Name                    : ESI-1
Admin State             : Up                 Oper State         : Up
ESI                     : 01:00:00:00:00:71:00:00:00:01
Multi-homing            : allActive          Oper Multi-homing  : allActive
Lag Id                  : 1                  
ES Activation Timer     : 20 secs            
Exp/Imp Route-Target    : target:00:00:00:00:71:00
 
Svc Carving             : auto               
ES SHG Label            : 262142             
===============================================================================

4.2.4.4. Aliasing

As shown in the example in Figure 48, if the service configuration on PE3 has ECMP > 1, PE3 adds PE1 and PE2 to the list of next-hops for ESI-1. As soon as PE3 receives a MAC for ESI-1, it starts load-balancing between PE1 and PE2 the flows to the remote ESI CE.

The following is a sample configuration output that shows the FDB in PE3.

Note:

mac 00:ca:ca:ba:ce:03 is associated with the ethernet-segment eES:01:00:00:00:00:71:00:00:00:01 (esi configured on PE1 and PE2 for ESI-1).

*A:PE3# show service id 1 fdb detail 
===============================================================================
Forwarding Database, Service 1
===============================================================================
ServId    MAC               Source-Identifier        Type     Last Change
                                                     Age      
-------------------------------------------------------------------------------
1         00:ca:ca:ba:ce:03 eES:                     Evpn     06/11/15 00:14:47
                            01:00:00:00:00:71:00:00:00:01
1         00:ca:fe:ca:fe:69 eMpls:                   EvpnS    06/11/15 00:09:18
                            192.0.2.69:262141
1         00:ca:fe:ca:fe:70 eMpls:                   EvpnS    06/11/15 00:09:18
                            192.0.2.70:262140
1         00:ca:fe:ca:fe:72 eMpls:                   EvpnS    06/11/15 00:09:39
                            192.0.2.72:262141
-------------------------------------------------------------------------------
No. of MAC Entries: 4
-------------------------------------------------------------------------------
Legend:  L=Learned O=Oam P=Protected-MAC C=Conditional S=Static
===============================================================================

The following is a sample configuration output that shows all the EVPN-MPLS destination bindings on PE3, including the ES destination bindings.

Note:

The ethernet-segment eES:01:00:00:00:00:71:00:00:00:01 is resolved to PE1 and PE2 addresses.

*A:Dut-B# /show service id 1 evpn-mpls 
===============================================================================
BGP EVPN-MPLS Dest
===============================================================================
TEP Address     Egr Label     Num. MACs   Mcast           Last Change
                 Transport                                
-------------------------------------------------------------------------------
10.20.1.3       131069        0           Yes             02/02/2014 15:29:40
                rsvp                                       
10.20.1.4       131069        0           Yes             02/02/2014 15:29:33
                rsvp                                       
10.20.1.5       131059        0           Yes             02/02/2014 15:29:42
                rsvp                                       
-------------------------------------------------------------------------------
Number of entries : 3
-------------------------------------------------------------------------------
===============================================================================
===============================================================================
BGP EVPN-MPLS Ethernet Segment Dest
===============================================================================
Eth SegId                       Num. Macs               Last Change
-------------------------------------------------------------------------------
00:de:01:00:00:00:00:00:00:01   1                       02/02/2014 15:47:04
-------------------------------------------------------------------------------
Number of entries: 1
------------------------------------------------------------------------------- 

PE3 performs aliasing for all the MACs associated with that ESI. This is possible because PE1 is configured with ECMP parameter >1. The following is a sample configuration output.

*A:PE3>config>service>vpls# info 
----------------------------------------------
            bgp
            exit
            bgp-evpn
                evi 1
                mpls
                    ecmp 4
                    auto-bind-tunnel
                        resolution any
                    exit
                    no shutdown
                exit
            exit
            proxy-arp
                shutdown
            exit
            stp
                shutdown
            exit
            sap 1/1/1:2 create
            exit
            no shutdown

4.2.4.5. Network Failures and Convergence for All-Active Multi-Homing

Figure 49 shows the behavior on the remote PEs (PE3) when there is an ethernet-segment failure.

Figure 49:  All-Active Multi-Homing ES Failure 

The following steps describe the unicast traffic behavior on PE3.

  1. PE3 can only forward MAC DA = CE2 to both PE1 and PE2 when the MAC advertisement route from PE1 (or PE2) and the set of Ethernet AD per-ES routes and Ethernet AD per-EVI routes from PE1 and PE2 are active at PE3.
  2. In case of a failure between CE2 and PE2, PE2 withdraws its set of Ethernet AD routes and ES route, and PE3 forwards traffic destined for CE2 to PE1 only. PE3 does not need to wait for the withdrawal of the individual MAC.
  3. The same handling is used if the failure was at PE1.
  4. If after step 2, PE2 withdraws its MAC advertisement route, PE3 treats traffic to MAC DA = CE2 as unknown unicast, unless PE1 has previously advertised the MAC.

For BUM traffic, the following events trigger a DF election on a PE, and only the DF forwards BUM traffic after the esi-activation-timer expires (if there was a transition from non-DF to DF):

  1. reception of ES route update (local ES shutdown/no shutdown or remote route)
  2. new AD-ES route update/withdraw
  3. new AD-EVI route update/withdraw
  4. local ES port/SAP/service shutdown
  5. service carving range change (affecting the EVI)
  6. multi-homing mode change (single/all active to all/single-active)

4.2.4.6. Logical Failures on ESs and Blackholes

Specific “failure scenarios” in the network can trigger effects. Figure 50 shows some of these scenarios.

Figure 50:  Blackhole Caused by SAP/SVC Shutdown 

If an individual VPLS service is shutdown in PE1 (the example is also valid for PE2), the corresponding LAG SAP goes operationally down. This event triggers the withdrawal of the AD per-EVI route for that SAP. PE3 removes PE1 from its list of aliased next-hops, and PE2 takes over as DF (if it was not the DF already). However, this does not prevent the network from black-holing the traffic that CE2 “hashes” to the link to PE1. Because traffic sent from CE2 to PE2 or traffic from the rest of the CEs to CE2 is unaffected, the situation is not easily detected on the CE.

The same result occurs if the ES SAP is administratively shutdown instead of the service.

Note:

When the bgp-evpn mpls shutdown command is executed, the SAP associated with the ES goes operationally down (StandbyforMHprotocol). If no other SAPs or SDP-bindings are configured in the service, the service also goes operationally down. However, if other SAPs or SDP-bindings are present, the service remains operationally up.

4.2.4.7. Transient Issues Due to MAC Route Delays

Figure 51 shows scenarios that may cause potential transient issues in the network.

Figure 51:  Transient Issues Caused by “slow” MAC Learning 

In Figure 51, the scenario on the left shows an example of transient packet duplication caused by delay in PE3 to learn MAC1.

In an all-active multi-homing scenario, if a specified MAC address (for example, MAC1), is not yet learned in a remote PE (for example, PE3), but it is known in the two PEs of the ES (for example, PE1 and PE2), the latter PEs might send duplicated packets to the CE.

Configuring ingress-replication-bum-label in PE1 and PE2 resolves the issue. PE1 and PE2 know that the received packet is an unknown unicast packet; consequently, the NDF (PE1) does not send the packets to the CE, which prevents transient and duplication.

In Figure 51, the scenario on the right shows an example of transient blackhole caused by delay in PE1 to learn MAC1.

In an all-active multi-homing scenario, MAC1 is known in PE3 and aliasing is applied to MAC1. However, MAC1 is not yet known in PE1, which is the NDF for the ES. If PE3 hashing picks up PE1 as the destination of the aliased MAC1, the packets are blackholed. To resolve this issue, unknown unicast traffic that arrives with a unicast label should not be blocked on the NDF. If PE1 and PE2 are configured using ingress-replication-bum-label, PE3 sends unknown unicast packets with a BUM label and known unicast with a unicast label. In the latter case, PE1 considers it safe to forward the frame to the CE, even if it is unknown unicast.

Note:

This is a transient issue that is resolved as soon as MAC1 is learned in PE1 and the frames are forwarded as known unicast.

4.2.5. EVPN Single-Active Multi-Homing

The 7210 SAS SR OS supports only single-active multi-homing on access LAG SAPs and regular SAPs for a specified VPLS service. For LAG SAPs, the CE is configured with a different LAG to each PE in the ES (in contrast to a single LAG in an all-active multi-homing).

The following SR OS procedures support EVPN single-active multi-homing for a specified ES:

  1. DF election
    The DF election in single-active multi-homing determines the forwarding for BUM traffic from the EVPN network to the ES CE. DF election also determines the forwarding of any traffic (unicast or BUM) in any direction (to or from the CE).
  2. backup PE
    In single-active multi-homing, the remote PEs do not perform aliasing to the PEs in the ES. The remote PEs identify the DF based on the MAC routes and send the unicast flows for the ES to the PE in the DF. The remote PEs also program a backup PE as an alternative next-hop for the remote ESI in case of failure. This is in accordance with the Backup PE procedure, defined in RFC 7432.
    Figure 52 shows an example backup PE for PE3.
    Figure 52:  Backup PE 

4.2.5.1. Single-Active Multi-Homing Service Model

The following example shows a PE1 configuration that provides single-active multi-homing to CE2, as shown in Figure 52.

*A:PE1>config>service>system>bgp-evpn# info 
----------------------------------------------
  route-distinguisher 10.1.1.1:0
  ethernet-segment "ESI2" create
    esi 01:12:12:12:12:12:12:12:12:12
    multi-homing single-active no-esi-label
    service-carving
      mode auto 
    lag 1
    no shutdown
 
*A:PE1>config>redundancy>evpn-multi-homing# info 
----------------------------------------------
    boot-timer 120
    es-activation-timer 10
 
*A:PE1>config>service>vpls# info 
----------------------------------------------
  description "evpn-mpls-service with single-active multihoming"
  bgp
  bgp-evpn
    evi 10
    mpls
      no shutdown
      auto-bind-tunnel resolution any
  lag 1:1 create 
  exit
 

The PE2 example configuration for this scenario is as follows.

*A:PE1>config>service>system>bgp-evpn# info 
----------------------------------------------
  route-distinguisher 10.1.1.1:0
  ethernet-segment "ESI2" create
    esi 01:12:12:12:12:12:12:12:12:12
    multi-homing single-active no-esi-label
    service-carving
    lag 2 
    no shutdown
 
*A:PE1>config>redundancy>evpn-multi-homing# info 
----------------------------------------------
    boot-timer 120
    es-activation-timer 10
 
*A:PE1>config>service>vpls# info 
----------------------------------------------
  description "evpn-mpls-service with single-active multihoming"
  bgp
  bgp-evpn
    evi 10
    mpls
      no shutdown
      auto-bind-tunnel resolution any
  lag 2:1 create 
  exit
 

In single-active multi-homing, the non-DF PEs for a specified ESI block unicast and BUM traffic in both directions (upstream and downstream) on the object associated with the ESI. Otherwise, single-active multi-homing is similar to all-active multi-homing with the following differences.

  1. The ethernet-segment is configured for single-active: service>system>bgp-evpn>eth-seg>multi-homing single-active.
  2. The advertisement of the ESI-label in an AD per-ESI is optional in standards for single-active ESs. Use the service system bgp-evpn eth-seg multi-homing single-active no-esi-label command to control the ESI label advertisement. By default on the 7210 SAS, the ESI label is not used for single-active ESs, and there is no option available to enable the use of the ESI label.
    Note:

    The 7210 SAS ignores the ESI label received from an EVPN peer, which means that BUM traffic sent by the 7210 SAS to a peer DF node is always sent without the ESI label advertised by the DF. On the 7210 SAS-Mxp, only multi-homing single-active no-esi-label can be configured.

  3. For single-active multi-homing, the ES can be associated with a port or lag-id, as shown in Figure 52, where:
    1. port is used for single-active SAP redundancy without the need for LAG
    2. lag is used for single-active LAG redundancy
      Note:

      For a LAG configured with single-active homing, the LAG parameters key, system-id, and system-priority must be different on the PEs that are part of the ES.

  4. For single-active multi-homing, when the PE is non-DF for the service, the SAPs on the ethernet-segment are down and show StandByForMHProtocol as the reason.
  5. From a service perspective, single-active multi-homing can provide redundancy to CEs (MHD, Multi-Homed Devices) with the following setup:
    1. LAG with or without LACP
      In this case, the multi-homed ports on the CE are part of the different LAGs (a LAG per multi-homed PE is used in the CE).
    2. regular Ethernet 802.1q/ad ports
      In this case, the multi-homed ports on the CE/network are not part of any LAG.

4.2.5.2. ES and DF Election Procedures

In all-active multi-homing, the non-DF keeps the SAP up, although it removes it from the default flooding list. In the single-active multi-homing implementation, the non-DF brings the SAP operationally down. For more information, see ES Discovery and DF Election Procedures.

The following show command output is an example status of the single-active ESI-7413 in the non-DF.

*A:Dut-B# show service system bgp-evpn ethernet-segment name "eslag1" 
===============================================================================
Service Ethernet Segment
===============================================================================
Name                    : eslag1
Admin State             : Enabled            Oper State         : Up
ESI                     : 00:bc:01:00:00:00:00:00:00:01
Multi-homing            : singleActiveNoEsi* Oper Multi-homing  : singleActive*
Lag Id                  : 1                  
ES Activation Timer     : 3 secs (default)   
Exp/Imp Route-Target    : target:bc:01:00:00:00:00
Svc Carving             : auto               
ES SHG Label            : N/A                
===============================================================================
* indicates that the corresponding row element may have been truncated.
===============================================================================
*A:Dut-D# /show service system bgp-evpn ethernet-segment name "eslag1" evi 1 
===============================================================================
EVI DF and Candidate List
===============================================================================
EVI           SvcId         Actv Timer Rem      DF  DF Last Change
-------------------------------------------------------------------------------
1             1             0                   no  03/13/2000 11:43:16
===============================================================================
===============================================================================
DF Candidates                           Time Added
-------------------------------------------------------------------------------
10.20.1.4                               03/13/2000 12:00:30
10.20.1.5                               03/13/2000 11:43:16
-------------------------------------------------------------------------------
Number of entries: 2
===============================================================================
*A:Dut-D#        

4.2.5.3. Backup PE Function

In the example in Figure 52, the remote PE3 imports AD routes per ESI where the single-active flag is set. PE3 interprets the ethernet-segment as single-active if at least one PE sends an AD route per-ESI with the single-active flag set. MACs for a specified service and ESI are learned from a single PE, that is, the DF for that <ESI, EVI>.

The remote PE installs a single EVPN-MPLS destination (TEP, label) for a received MAC address and a backup next-hop to the PE for which the AD routes per-ESI and per-EVI are received. For example, in the following show command sample output, 00:ca:ca:ba:ca:06 is associated with the remote ethernet-segment eES 01:74:13:00:74:13:00:00:74:13. This ES is resolved to PE(192.0.2.73), which is the DF on the ES.

*A:PE3# show service id 1 fdb detail 
===============================================================================
Forwarding Database, Service 1
===============================================================================
ServId    MAC               Source-Identifier        Type     Last Change
                                                     Age      
-------------------------------------------------------------------------------
1         00:ca:ca:ba:ca:02 sap:1/1/1:2              L/0      06/12/15 00:33:39
1         00:ca:ca:ba:ca:06 eES:                     Evpn     06/12/15 00:33:39
                            01:74:13:00:74:13:00:00:74:13
1         00:ca:fe:ca:fe:69 eMpls:                   EvpnS    06/11/15 21:53:47
                            192.0.2.69:262118
1         00:ca:fe:ca:fe:70 eMpls:                   EvpnS    06/11/15 19:59:57
                            192.0.2.70:262140
1         00:ca:fe:ca:fe:72 eMpls:                   EvpnS    06/11/15 19:59:57
                            192.0.2.72:262141
-------------------------------------------------------------------------------
No. of MAC Entries: 5
-------------------------------------------------------------------------------
Legend:  L=Learned O=Oam P=Protected-MAC C=Conditional S=Static
===============================================================================
*A:Dut-D# /show service id 1 evpn-mpls 
===============================================================================
BGP EVPN-MPLS Dest
===============================================================================
TEP Address     Egr Label     Num. MACs   Mcast           Last Change
                 Transport                                
-------------------------------------------------------------------------------
10.20.1.2       131061        0           Yes             03/13/2000 11:26:29
                rsvp                                       
10.20.1.2       131062        1           No              03/13/2000 12:10:04
                rsvp                                       
10.20.1.5       131061        0           Yes             03/13/2000 11:18:23
                rsvp                                       
-------------------------------------------------------------------------------
Number of entries : 3
-------------------------------------------------------------------------------
===============================================================================
===============================================================================
BGP EVPN-MPLS Ethernet Segment Dest
===============================================================================
Eth SegId                       Num. Macs               Last Change
-------------------------------------------------------------------------------
00:de:01:00:00:00:00:00:00:01   453                     03/13/2000 12:10:14
-------------------------------------------------------------------------------
Number of entries: 1
-------------------------------------------------------------------------------
===============================================================================
*A:Dut-D# /show service id 1 evpn-mpls esi 00:de:01:00:00:00:00:00:00:01 
===============================================================================
BGP EVPN-MPLS Ethernet Segment Dest
===============================================================================
Eth SegId                       Num. Macs               Last Change
-------------------------------------------------------------------------------
00:de:01:00:00:00:00:00:00:01   453                     03/13/2000 12:10:14
===============================================================================
===============================================================================
BGP EVPN-MPLS Dest TEP Info
===============================================================================
TEP Address              Egr Label                Last Change
                         Transport                
-------------------------------------------------------------------------------

If PE3 sees only two single-active PEs in the same ESI, the second PE is the backup PE. Upon receiving an AD per-ES/per-EVI route withdrawal for the ESI from the primary PE, PE3 starts sending the unicast traffic to the backup PE immediately.

If PE3 receives AD routes for the same ESI and EVI from more than two PEs, the PE does not install any backup route in the data path. Upon receiving an AD per-ES/per-EVI route withdrawal for the ESI, it flushes the MACs associated with the ESI.

Note:

On the 7210 SAS, an ES can be multi-homed to up to two PEs.

4.2.5.4. Network Failures and Convergence for Single-Active Multi-Homing

Figure 53 shows an example of remote PE (PE3) behavior when there is an ethernet-segment failure.

Figure 53:  Single-Active Multi-Homing ES Failure 

The following steps list the behavior of the remote PE3 for unicast traffic.

  1. PE3 forwards MAC DA = CE2 to PE2 when the MAC advertisement route came from PE2 and the set of Ethernet AD per-ES routes and Ethernet AD per-EVI routes from PE1 and PE2 are active at PE3.
  2. If there is a failure between CE2 and PE2, PE2 withdraws its set of Ethernet AD and ES routes. PE3 does not need to wait for the withdrawal of the individual MAC, and immediately forwards the traffic destined for CE2 to PE1 (the backup PE) only.
  3. After the (2) PE2 withdraws its MAC advertisement route, PE3 treats traffic to MAC DA = CE2 as unknown unicast, unless the MAC has been previously advertised by PE1.

A DF election on PE1 is also triggered. A DF election is triggered by the same events as all-active multi-homing. In this case, the DF forwards traffic to CE2 when the esi-activation-timer expires; the timer is triggered when a transition from non-DF to DF occurs.

4.3. General EVPN Topics

This section provides information about general topics related to EVPN.

Note:

Hash labels (that is, the Flow Aware Transport label (RFC 6391)) are not supported with 7210 SAS EVPN VPLS services.

4.3.1. ARP and ND Snooping and Proxy Support

VPLS services support proxy-Address Resolution Protocol (proxy-ARP) and proxy-Neighbor Discovery (proxy-ND) functions cannot be enabled or disabled per service. When enabled, the config>service>system>evpn-proxy-arp-nd command populates the corresponding proxy-ARP or proxy-ND table with IP-to-MAC entries learned from the following sources:

  1. EVPN-received IP-to-MAC entries
  2. user-configured static IP-to-MAC entries
  3. snooped dynamic IP-to-MAC entries (learned from ARP, GARP, or NA messages received on local SAPs; snooped dynamic IP-to-MAC entries on spoke-SDP bindings are not supported)

In addition, any ingress ARP or ND frame on a SAP are intercepted and processed. The system answers ARP requests and Neighbor Solicitation messages if the requested IP address is present in the proxy table.

Figure 54 shows an example proxy-ARP usage in an EVPN network. Proxy-ND functions in a similar way. The MAC address notation in the diagram is shortened for readability.

Figure 54:  Proxy-ARP Example Usage in an EVPN Network 

In Figure 54, PE1 is configured as follows:

*A:Dut-B>config>service>system# info 
----------------------------------------------
 evpn-proxy-arp-nd
----------------------------------------------
*A:Dut-B>config>service>vpls# info 
----------------------------------------------
            description "Vpls 1 "
            service-mtu 1400
            split-horizon-group "vpls1" create
                description "Default description for SHG vpls1"
            exit
            bgp
                route-distinguisher auto-rd
                route-target export target:100:1 import target:100:1
                pw-template-binding 100
                exit
            exit
            bgp-ad
                shutdown
                vpls-id 1:1
            exit
            bgp-evpn
                evi 1
                mpls
                    split-horizon-group "vpls1"
                    ingress-replication-bum-label
                    auto-bind-tunnel
                        resolution-filter
                            ldp
                        exit
                        resolution filter
                    exit
                    no shutdown
                exit
            exit
            stp
                shutdown
            exit
            sap lag-1:1 create
                description "Default sap description for service id 1"
                no shutdown
            exit
            proxy-arp
                age-time 600
                send-refresh 200      
                dup-detect window 3 num-moves 3 hold-down max anti-spoof-
mac 00:aa:aa:aa:aa:aa
                dynamic-arp-populate
                no shutdown
            exit
            no shutdown
----------------------------------------------
*A:Dut-B>config>service>vpls#

Figure 54 shows the following steps, assuming proxy-ARP is no shutdown on PE1 and PE2, and the tables are empty.

  1. ISP-A sends ARP-request for 10.10.10.3.
  2. PE1 learns the MAC 00:01 in the FDB as usual and advertises it in EVPN without any IP. Optionally if the MAC is configured as a Cstatic MAC, it is advertised as a protected MAC to other PEs with the sticky bit set.
  3. The ARP-request is sent to the CPM, where it is handled as follows.
    1. An ARP entry (IP 10.1'MAC 00:01) is populated into the proxy-ARP table.
    2. EVPN advertises MAC 00:01 and IP 10.1 in EVPN with the same SEQ number and protected bit as the previous route-type 2 for MAC 00:01.
    3. A GARP is also issued to other SAPs/SDP-bindings (assuming they are not in the same split-horizon group as the source). If the garp-flood-evpn command is enabled, the GARP message is also sent to the EVPN network.
    4. The original ARP-request can still be flooded to the EVPN or not based on the unknown-arp-request-flood-evpn command.
  4. Assuming PE1 was configured with unknown-arp-request-flood-evpn, the ARP-request is flooded to PE2 and delivered to ISP-B. ISP-B replies with its MAC in the ARP-reply. The ARP-reply is finally delivered to ISP-A.
  5. PE2 learns MAC 00:01 in the FDB and the entry 10.1'00:01 in the proxy-ARP table, based on the EVPN advertisements.
  6. When ISP-B replies with its MAC in the ARP-reply, the MAC is handled as follows.
    1. MAC 00:03 is learned in FDB at PE2 and advertised in EVPN.
    2. MAC 00:03 and IP 10.3 are learned in the proxy-ARP table and advertised in EVPN with the same SEQ number as the previous MAC route.
    3. ARP-reply is unicasted to MAC 00:01.
  7. EVPN advertisements are used to populate PE1's FDB (MAC 00:03) and proxy-ARP (IP 10.3 to MAC 00:03) tables as mentioned in 5.

From this point onward, the PEs reply to any ARP-request for 00:01 or 00:03 without the need for flooding the message in the EVPN network. By replying to known ARP-requests and Neighbor Solicitations, the PEs help to significantly reduce the flooding in the network.

Use the following commands to customize proxy-ARP/proxy-ND behavior:

  1. dynamic-arp-populate and dynamic-nd-populate
    These commands enable the addition of dynamic entries to the proxy-ARP or proxy-ND table (disabled by default). When executed, the system populates proxy-ARP/proxy-ND entries from snooped GARP/ARP/NA messages on SAPs/SDP-bindings, in addition to the entries coming from EVPN (if EVPN is enabled). These entries are shown as dynamic.
  2. static ipv4-address mac-address, static ipv4-address mac-address, and static ipv6-address mac-address {host | router}
    These commands configure static entries to be added to the table.
    Note:

    A static IP-to-MAC entry requires the addition of the MAC address to the FDB as either learned or CStatic (conditional static mac) in order to become active (Status active).

  3. age-time seconds
    This command specifies the aging timer per proxy-ARP/proxy-ND entry. When the aging expires, the entry is flushed. The age is reset when a new ARP/GARP/NA for the same IP-to-MAC is received.
  4. send-refresh seconds
    If this command is enabled, the system sends ARP-request or Neighbor Solicitation (NS) messages at the configured time, which enables the owner of the IP to reply and, therefore, refresh its IP-to-MAC (proxy-ARP entry) and MAC (FDB entry).
  5. table-size table-size
    This command enables the user to limit the number of entries learned on a specified service. By default, the table-size limit is 250.
    Flooding unknown ARP-requests, NS messages, or unsolicited GARPs and NA messages in an EVPN network can be configured using the following commands:
    1. proxy-arp [no] unknown-arp-request-flood-evpn
    2. proxy-arp [no] garp-flood-evpn
    3. proxy-nd [no] unknown-ns-flood-evpn
    4. proxy-nd [no] host-unsolicited-na-flood-evpn
    5. proxy-nd [no] router-unsolicited-na-flood-evpn
  6. dup-detect [anti-spoof-mac mac-address] window minutes num-moves count hold-down minutes | max
    This command enables a mechanism that detects duplicate IPs and ARP/ND spoofing attacks. The following is a summary of the dup-detect command mechanism.
    1. Attempts (relevant to dynamic and EVPN entry types) to add the same IP (different MAC) are monitored for window minutes value and when the count value is reached within the configured window, the proxy-ARP/proxy-ND entry for the IP is suspected and marked as duplicate. An alarm is also triggered.
    2. The condition is cleared when hold-down time expires (max does not expire) or a clear command is issued.
    3. If the anti-spoof-mac command is configured, the proxy-ARP or proxy-ND offending entry's MAC is replaced by the configured mac-address and advertised in an unsolicited GARP/NA for local SAP or SDP-bindings and in EVPN to remote PEs.
    4. This mechanism assumes that the same anti-spoof-mac is configured in all PEs for the service, and that traffic with destination anti-spoof-mac received on SAPs/SDP-bindings is dropped. An ingress MAC filter must be configured to drop traffic to the anti-spoof-mac.

Table 52 shows the combinations that produce a Status = Active proxy-ARP entry in the table. The system only replies to proxy-ARP requests for active entries. Any other combination result in a Status = inActv entry. If the service is not active, the proxy-ARP entries are not active, regardless of the FDB entries

Note:

A static entry is active in the FDB even when the service is down.

Table 52:  Proxy-ARP Entry Combinations 

Proxy-ARP Entry Type

FDB Entry Type (for the same MAC)

Dynamic

learned

Static

CStatic

EVPN

EVPN, EVPNS with matching ESI

Duplicate

—

When proxy-ARP or proxy-ND is enabled on services with multi-homed ESs, a proxy-ARP entry type “EVPN” might be associated with a “learned” FDB entry because the CE can send traffic for the same MAC to all the multi-homed PEs in the ES. In such cases, the entry is inactive, in accordance with Table 52.

4.3.1.1. Proxy-ARP/ND Periodic Refresh, Unsolicited Refresh, and Confirm-Messages

When proxy-ARP or proxy-ND is enabled, the system starts populating the proxy table and responding to ARP-requests or NS messages. To keep the active IP-to-MAC entries alive and ensure that all the host/routers in the service update their ARP/ND caches, the system may generate the following three types of ARP/ND messages for a specified IP-to-MAC entry:

  1. periodic refresh messages (ARP-requests or NS for a specified IP)
    These messages are activated by the send-refresh command and their objective is to keep the existing FDB and proxy-ARP/ND entries alive, in order to minimize EVPN withdrawals and re-advertisements.
  2. unsolicited refresh messages (unsolicited GARP or NA messages)
    These messages are sent by the system when a new entry is learned or updated. Their objective is to update the attached host/router caches.
  3. confirm messages (unicast ARP-requests or unicast NS messages)
    These messages are sent by the system when a new MAC is learned for an existing IP. The objective of the confirm messages is to verify that a specified IP has moved to a different part of the network and is associated with the new MAC. If the IP has not moved, it forces the owners of the duplicate IP to reply and triggers dup-detect.

4.3.1.2. Proxy-ND and the Router Flag in Neighbor Advertisement Messages

RFC 4861 describes the use of the (R) or “Router” flag in NA messages as follows:

  1. a node capable of routing IPv6 packets must reply to NS messages with NA messages where the R flag is set (R=1)
  2. hosts must reply with NA messages where R=0

The use of the R flag in NA messages impacts how the hosts select their default gateways when sending packets “off-link”. Therefore, it is important that the proxy-ND function on the 7210 SAS meet one of the following criteria:

  1. provide the appropriate R flag information in proxy-ND NA replies.
  2. flood the received NA messages if it cannot provide the appropriate R flag when replying

Because of the use of the R flag, the procedure for learning proxy-ND entries and replying to NS messages differs from the procedures for proxy-ARP in IPv4: the router or host flag is added to each entry, and that determines the flag to use when responding to a NS.

4.3.1.3. Procedure to Add the R Flag to a Specified Entry

The procedure to add the R flag to a specified entry is as follows.

  1. Dynamic entries are learned based on received NA messages. The R flag is also learned and added to the proxy-ND entry so that the appropriate R flag is used in response to NS requests for a specified IP.
  2. Static entries are configured as host or router as per the [no] static ip-address ieee-address {host | router} command.
  3. EVPN entries are learned from BGP and the evpn-nd-advertise {host | router} the R flag added to them.
  4. In addition, the evpn-nd-advertise {host | router} command indicates what static and dynamic IP-to-MAC entries the system advertises in EVPN. If evpn-nd-advertise router is configured, the system should flood the received unsolicited NA messages for hosts. This is controlled by the [no] host-unsolicited-na-flood-evpn command. The opposite is also recommended, so that the evpn-nd-advertise host is configured using the router-unsolicited-na-flood-evpn command.

4.3.1.4. Configuration Guidelines for Proxy-ARP and Proxy-ND

On the 7210 SAS, users can enable or disable proxy-ARP and proxy-ND commands for all EVPN services configured on the node; however, the option to enable or disable proxy-ARP or proxy-ND per service is not available.

Use the following syntax to enable or disable proxy-ARP or proxy-ND capability per node.

CLI Syntax:
configure>service>system>[no]evpn-proxy-arp-nd

If the per-node evpn-proxy-arp-nd command is disabled, it is not possible to enable the proxy-arp or proxy-nd command per service, and proxy-arp or proxy-nd is disabled for all EVPN services on the node.

When the evpn-proxy-arp-nd command is enabled, the user must run the following sequence of commands to disable the per-node proxy-arp and proxy-nd commands, if required.

  1. Run the config>service>system>no evpn-proxy-arp-nd command.
  2. Run the config>service>vpls>no proxy-arp command.
  3. Run the config>service>vpls>no proxy-nd command.

The following example shows the command usage to enable proxy-arp and proxy-nd for all services, with all parameters set to default values:

--------------------------------------------------------------------------
configure
  service
    system
       evpn-proxy-arp-nd
    exit
  exit
  
  service 
    vpls 100
       proxy-arp
          no age-time
          dup-detect window 3 num-moves 5 hold-down 9
          no dynamic-arp-populate
          ...  (so on for other parameters supported under proxy-arp)
          no shutdowm
                exit
       
       proxy-nd
          no age-time
          dup-detect window 3 num-moves 5 hold-down 9
          no dynamic-nd-populate
          ...  (so on for other parameters supported under proxy-nd)
          no shutdowm
                exit
         exit
  exit
--------------------------------------------------------------------------

When the user runs the config>service>system>evpn-proxy-arp-nd command, the software automatically runs a no shutdown command for proxy-arp and proxy-nd for all services. Similarly, when the user runs the config>service>system>no evpn-proxy-arp-nd command, the software automatically runs a shutdown command for proxy-arp and proxy-nd for all services.

When the evpn-proxy-arp-nd command is enabled, the proxy-arp or proxy-nd command is enabled for all services and cannot be disabled for individual services. Configuring service-specific proxy-arp or proxy-nd command parameters is supported only when the evpn-proxy-arp-nd command is enabled.

4.3.1.4.1. Proxy-ARP and Proxy-ND Support for Spoke-SDP Bindings

The 7210 SAS does not support proxy-ARP and proxy-ND on spoke-SDP bindings. ARP or ND packets received on spoke-SDP bindings are not identified or sent to the CPU for further processing.

When using spoke-SDP bindings on the 7210 SAS, Nokia recommends that users disable proxy-ARP and proxy-ND functionality on all PEs belonging to the EVPN service, or enable the forwarding of ARP or ND packets over EVPN bindings on all PEs that belong to the EVPN service, so that ARP or ND packets are flooded throughout the EVPN instance.

4.3.2. BGP-EVPN MAC Mobility

EVPN defines a mechanism to allow the smooth mobility of MAC addresses from one CE/NVE to another. The 7210 SAS supports this procedure and the MAC mobility extended community in MAC advertisement routes.

  1. The router honors and generates the Sequence (SEQ) number in the MAC mobility extended community for MAC moves.
  2. When a MAC is EVPN-learned and it is attempted to be learned locally, a BGP update is sent with SEQ number changed to “previous SEQ”+1 (exception: mac-duplication detect num-moves value is reached).
  3. A SEQ number = zero or no mac mobility ext-community are interpreted as sequence zero.
  4. In case of mobility, the following MAC selection procedure is followed.
    1. If a PE has two or more active remote EVPN routes for the same MAC, the highest SEQ number is selected. The tie-breaker is the lowest IP (BGP NH IP).
    2. If a PE has two or more active EVPN routes and it is the originator of one of them, the highest SEQ number is selected. The tie-breaker is the lowest IP (BGP NH IP of the remote route is compared to the local system address).
Note:

When EVPN multi-homing is used in EVPN-MPLS, the ESI is compared to determine whether a MAC received from two different PEs should be processed within the context of MAC mobility or multi-homing. Two MAC routes that are associated with the same remote or local ESI but different PEs are considered reachable through all those PEs. Mobility procedures are not triggered if the MAC route still belongs to the same ESI.

4.3.3. BGP-EVPN MAC Duplication

EVPN defines a mechanism to protect the EVPN service from control plane churn as a result of loops or accidental duplicated MAC addresses. The 7210 SAS supports an enhanced version of this procedure, which is described in this section.

In a scenario where two or more hosts are misconfigured using the same (duplicate) MAC address, the duplicate MAC address is learned by the PEs in the VPLS. As a result, the traffic originating from the hosts triggers continuous MAC moves among the PEs attached to the hosts. It is important to recognize such situations and avoid incrementing the sequence number (in the MAC Mobility attribute) to infinity.

To remedy accidentally duplicated MAC addresses, a router that detects a MAC mobility event through local learning starts a window in-minutes timer (the default value is 3). If the configured num-moves num value is detected before the timer expires (the default value is 5), the router concludes that a duplicate MAC situation has occurred and sends a trap message to alert the operator. Use the show service id svc-id bgp-evpn command to display the MAC addresses. The following is a sample configuration output.

10 2014/01/14 01:00:22.91 UTC MINOR: SVCMGR #2331 Base 
"VPLS Service 1 has MAC(s) detected as duplicates by EVPN mac-
duplication detection."
# show service id 1 bgp-evpn 
===============================================================================
BGP EVPN Table
===============================================================================
MAC Advertisement  : Enabled            Unknown MAC Route  : Disabled
MPLS Admin Status : Enabled            Creation Origin    : manual
MAC Dup Detn Moves : 5                  MAC Dup Detn Window: 3
MAC Dup Detn Retry : 9                  Number of Dup MACs : 1
-------------------------------------------------------------------------------
Detected Duplicate MAC Addresses             Time Detected
-------------------------------------------------------------------------------
00:00:00:00:00:12                            01/14/2014 01:00:23
-------------------------------------------------------------------------------
===============================================================================

After a duplicate MAC address is detected, the router stops sending and processing BGP MAC advertisement routes for that MAC address until one of the following occurs.

  1. The MAC is flushed because of a local event (SAP or SDP-binding associated with the MAC fails) or the reception of a remote update with better SEQ number (because of a MAC flush at the remote router).
  2. The retry in-minutes timer expires, which flushes the MAC and restarts the process.
Note:

The other routers in the VPLS instance forward the traffic for the duplicate MAC address to the router advertising the best route for the MAC.

The values of num-moves and window can be configured for different environments. In scenarios where BGP rapid-update EVPN is configured, the operator should configure a shorter window timer than scenarios where BGP updates are sent per the configured min-route-advertisement interval, which is the default.

The preceding MAC duplication parameters can be configured per VPLS service under the bgp-evpn mac-duplication context. The following is a sample configuration output.

A:Dut-B>config>service>vpls>bgp-evpn# info 
----------------------------------------------
                evi 1
                mac-duplication
                    detect num-moves 5 window 2
                    retry 10
                exit
                mpls
                    split-horizon-group "vpls1"
                    ingress-replication-bum-label
                    auto-bind-tunnel
                        resolution-filter
                            ldp
                        exit
                        resolution filter
                    exit
                    no shutdown
                exit
----------------------------------------------

4.3.4. Conditional Static MAC and Protection

In RFC 7432, the MAC Mobility Extended Community section defines the use of the sticky bit to signal static MAC addresses. These addresses must be protected to prevent attempts to dynamically learn them in a different place in the EVPN-MPLS VPLS service.

Note:

On the 7210 SAS, the conditional static MACs are not protected using MAC-protect functionality. A Cstatic MAC is advertised to other PEs with the sticky bit set so that it is prevented from being learned dynamically at a different place in the EVPN-MPLS VPLS service. MAC frames whose source MAC address matches the statically configured MAC address are forwarded based on destination MAC address lookup and are not dropped.

In the 7210 SAS, any conditional static MAC address that is defined in an EVPN-MPLS VPLS service is advertised by BGP-EVPN as a static address (that is, with the sticky bit set). The following is a sample output that shows the configuration of a conditional static MAC.

*A:Dut-B>config>service>vpls# info 
----------------------------------------------
            description "evpn mpls service "
……….
            sap lag-1:1 create
                description "Default sap description for service id 1"
                no shutdown
            exit
            static-mac
                mac 00:ca:ca:ca:ca:00 create sap lag-1:1 monitor fwd-status
            exit
A:Dut-C# show router bgp routes evpn mac hunt mac-address 00:ca:ca:ca:ca:00 
……..
===============================================================================
BGP EVPN MAC Routes
===============================================================================
-------------------------------------------------------------------------------
RIB In Entries
-------------------------------------------------------------------------------
Network        : n/a
Nexthop        : 10.20.1.2
From           : 10.20.1.2
Res. Nexthop   : 10.10.3.2
Local Pref.    : 100                    Interface Name : ip-10.10.3.3
Aggregator AS  : None                   Aggregator     : None
Atomic Aggr.   : Not Atomic             MED            : 0
AIGP Metric    : None                   IGP Cost       : 400
Connector      : None
Community      : target:100:1 bgp-tunnel-encap:MPLS
                 mac-mobility:Seq:0/Static
Cluster        : No Cluster Members
Originator Id  : None                   Peer Router Id : 10.20.1.2
Flags          : Used  Valid  Best  IGP  
Route Source   : Internal
AS-Path        : No As-Path
EVPN type      : MAC                    
ESI            : 00:bc:01:00:00:00:00:00:00:01
Tag            : 0                      
IP Address     : n/a
Route Dist.    : 2.2.2.2:1              
Mac Address    : 00:ca:ca:ca:ca:00      
MPLS Label1    : LABEL 131056           MPLS Label2    : n/a
Route Tag      : 0                      
Neighbor-AS    : n/a
Orig Validation: N/A                    
Add Paths Send : Default                
Last Modified  : 00h02m02s              
 
-------------------------------------------------------------------------------
RIB Out Entries
-------------------------------------------------------------------------------
-------------------------------------------------------------------------------
Routes : 1
===============================================================================

4.3.5. BGP and EVPN Route Selection for EVPN Routes

When two or more EVPN routes are received at a PE, BGP route selection typically takes place when the route key or the routes are equal. When the route key is different, but the PE has to make a selection (for example, the same MAC is advertised in two routes with different RDs), BGP hands over the routes to EVPN and the EVPN application performs the selection.

EVPN and BGP selection criteria are as follows:

  1. EVPN route selection for MAC routes
    When two or more routes with the same mac-length/mac but different route key are received, BGP transfers the routes to EVPN. EVPN selects the route based on the following tie-breaking order:
    1. conditional static MACs (local protected MACs)
    2. EVPN static MACs (remote protected MACs)
    3. data plane learned MACs (regular learning on SAPs/SDP-bindings)
    4. EVPN MACs with higher SEQ number
    5. lowest IP (next-hop IP of the EVPN NLRI)
    6. lowest Ethernet tag (that is zero for MPLS)
    7. lowest RD
  2. BGP route selection for MAC routes with the same route-key
    The priority order is as follows:
    1. EVPN static MACs (remote protected MACs)
    2. EVPN MACs with higher sequence number
    3. regular BGP selection (local-pref, aigp metric, shortest as-path, …, lowest IP)
  3. BGP route selection for the rest of the EVPN routes follows regular BGP selection
Note:

If BGP runs through the selection criteria and a specified and valid EVPN route is not selected in favor of another EVPN route, the non-selected route is displayed by the show router bgp routes evpn evpn-type detail command with a tie-breaker reason.

4.3.6. Interaction of EVPN and Other Features

This section describes the interaction of EVPN with other features.

4.3.6.1. Interaction of EVPN-MPLS with Existing VPLS Features

When enabling existing VPLS features in an EVPN-MPLS enabled service, the following considerations apply.

  1. EVPN-MPLS is only supported in regular VPLS. Other VPLS types, such as m-vpls, are not supported with EVPN-MPLS.
  2. In general, no router-generated control packets are sent to the EVPN destination bindings, except for proxy-ARP/proxy-ND confirm messages for EVPN-MPLS.
  3. For xSTP and M-VPLS services, the following applies.
    1. xSTP can be configured in BGP-EVPN services. BPDUs are not sent over the EVPN bindings.
    2. BGP-EVPN is blocked in M-VPLS services; however, a different M-VPLS service can manage a SAP or spoke-SDP in a BGP-EVPN-enabled service.
  4. For BGP-EVPN-enabled VPLS services, mac-move can be used in SAPs/SDP-bindings; however, the MACs being learned through BGP-EVPN are not considered.
    Note:

    MAC duplication already provides protection against MAC moves between EVPN and SAPs/SDP-bindings.

  5. The disable-learning command and other FDB-related tools only work for data-plane-learned MAC addresses.
  6. MAC OAM tools (mac-ping, mac-trace, mac-populate, mac-purge, and cpe-ping) are not supported for BGP-EVPN services.
  7. SAPs that belong to a specified ES but are configured on non-BGP-EVPN-MPLS-enabled VPLS or Epipe services are kept down using the StandByForMHProtocol flag.
  8. CPE ping is not supported on EVPN services.
  9. Other features not supported in conjunction with BGP-EVPN are:
    1. endpoints and attributes
    2. BPDU translation
    3. L2PT termination
    4. MAC-pinning
    5. IGMP-snooping in VPLS services when BGP-EVPN MPLS is enabled (in the service)
    6. ETH-CFM (MEPs, vMEPs, MIPs)

4.3.7. Routing Policies for BGP EVPN Routes

Routing policies match on specific fields when importing or exporting EVPN routes. These matching fields are the following:

  1. communities (comm-val), extended communities (ext-comm), and large communities (large-comm)
  2. well-known communities (well-known-comm); no-export | no-export-subconfed | no-advertise
  3. family EVPN
  4. protocol BGP-VPN (this term also matches VPN-IPv4/6 routes)
  5. BGP attributes that are applicable to EVPN routes (such as AS-path, local-preference, next-hop)