4.1.1. EVPN for MPLS Tunnels in E-LAN Services

Figure 42 shows the use of EVPN for MPLS tunnels on the 7210 SAS. In this example, EVPN is used as the control plane for E-LAN services.

Figure 42:  EVPN for MPLS in VPLS Services 

Service providers that offer E-LAN services request EVPN for its multi-homing capabilities and to leverage the optimization EVPN provides.

EVPN supports all-active multi-homing (per-flow load-balancing multi-homing) and single-active multi-homing (per-service load-balancing multi-homing). Although VPLS already supports single-active multi-homing, EVPN single-active multi-homing is deemed the superior technology because of its mass-withdrawal capabilities to speed up convergence in scaled environments.

EVPN technology provides significant benefits, including:

The SR OS EVPN-MPLS implementation is compliant with RFC 7432.

4.2.1. BGP-EVPN Control Plane for MPLS Tunnels

Table 34 lists all the EVPN routes supported in 7210 SAS SR OS and their usage in EVPN-MPLS.

RFC 7432 describes the BGP-EVPN control plane for MPLS tunnels. If EVPN multi-homing is not required, two route types are needed to set up a basic EVPN Instance (EVI): MAC/IP Advertisement and the Inclusive Multicast Ethernet Tag routes. If multi-homing is required, the ES and the Auto-Discovery routes are also needed.

The following route values are used for EVPN-MPLS services:

EVPN Route Type 3 — Inclusive Multicast Ethernet Tag Route

Route type 3 is used for setting up the flooding tree (BUM flooding) for a specified VPLS service. The received inclusive multicast routes add entries to the VPLS flood list. When BGP-EVPN MPLS is enabled, the standard supports ingress replication, p2mp mLDP, and composite tunnels as tunnel types in route type 3. On the 7210 SAS, only ingress replication is supported.

EVPN Route Type 2 — MAC/IP Advertisement Route

The node generates this route type for advertising MAC addresses (and IP addresses if proxy-ARP/proxy-ND is enabled). The router generates MAC advertisement routes for the following entities:

The route type 2 generated by a router uses the following fields and values:

When EVPN multi-homing is enabled in the system, two additional routes are required. Figure 43 shows the fields in routes type 1 and 4 and their associated extended communities.

Figure 43:  EVPN Routes Type 1 and 4  

EVPN Route Type 1 — Ethernet Auto-Discovery Route

The 7210 SAS router generates this route type for advertising for multi-homing functions. The system can generate two types of Ethernet Auto-Discovery (AD) routes:

The Ethernet AD per-ESI route generated by a router uses the following fields and values:

The system can send only a separate Ethernet AD per-ESI route per service.

The Ethernet AD per-EVI route generated by a router uses the following fields and values:

Note: The AD per-EVI route is not sent with the ESI label Extended Community.

EVPN Route Type 4 — ES route

The router generates this route type for multi-homing ES discovery and DF (Designated Forwarder) election.

RFC 5512 — BGP Tunnel Encapsulation Extended Community

The following routes are sent with the RFC 5512 BGP Encapsulation Extended Community: MAC/IP, Inclusive Multicast Ethernet Tag, and AD per-EVI routes. ES and AD per-ESI routes are not sent with this Extended Community.

The router processes the following BGP tunnel encapsulation tunnel value registered by IANA for RFC 5512: MPLS encapsulation: 10.

Any other tunnel value makes the route “treat-as-withdraw”.

If the encapsulation value is MPLS, the BGP validates the high-order 20 bits of the label field, ignoring the low-order 4 bits.

If the encapsulation extended community (as defined in RFC 5512) is not present in a received route, BGP treats the route as an MPLS. On the 7210 SAS, only MPLS encapsulation is supported.

4.2.2. EVPN for MPLS Tunnels in VPLS Services

EVPN can be used in MPLS networks where provider edge (PE) routers are interconnected through any type of MPLS tunnel, including RSVP-TE, LDP, BGP, Segment Routing IS-IS, or Segment Routing OSPF. As with VPRN services, the tunnel selection for a VPLS service (with BGP-EVPN MPLS enabled) is based on the auto-bind-tunnel command.

The EVPN-MPLS VPLS service uses a regular VPLS service where EVPN-MPLS “bindings” can coexist with SAPs. The following is a sample configuration output that shows a VPLS service with EVPN-MPLS.

First configure a bgp-evpn context as mpls no shutdown. In addition to the mpls no shutdown command, the minimum set of commands that must be configured to set up the EVPN-MPLS instance are the evi and the auto-bind-tunnel resolution commands.

Note: The EVI and the system IP must be configured before executing the configure>service/vpls>bgp-evpn>mpls>no shutdown command.

The evi value is the EVPN instance (EVI) identifier and is unique in the system. It is used for the service-carving algorithm for multi-homing (if configured) and for auto-deriving route-target and route-distinguishers in EVPN-MPLS services.

If the evi value is not specified, the value is zero and no route-distinguisher or route-targets are auto-derived from it. If it is specified, and no other route-distinguisher or route-target are configured in the service, the following applies:

Note: When vsi-import and export polices are configured, the route-target must be configured in the policies, and those values take precedence over the auto-derived route-targets. The operational route-target for a service is displayed by the show service id svc-id bgp command output. If the bgp-ad>vpls-id command is configured in the service, the vpls-id-derived route-target takes precedence over the evi-derived route-target.

When the evi command is configured, a config>service>vpls>bgp node (even empty) is required to output correct information using the show service id 1 bgp and show service system bgp-route-distinguisher commands.

The configuration of an EVI is enforced for EVPN services with SAPs in an ethernet-segment. See EVPN Multi-Homing in VPLS Services for more information about ESs.

The following options are specific to EVPN-MPLS and are configured in the config>service>vpls>bgp-evpn>mpls context:

In addition to the preceding options, the following bgp-evpn commands are also available for EVPN-MPLS services:

When EVPN-MPLS is established among some PEs in the network, EVPN unicast and multicast “bindings” to the remote EVPN destinations are created on each PE. A specified ingress PE creates the following:

These bindings, as well as the MACs learned on them, can be checked using the show commands in the following output example, where the remote PE(192.0.2.69) is configured with no ingress-replication-bum-label and PE(192.0.2.70) is configured with ingress-replication-bum-label. As a result, the device has a single EVPN-MPLS destination binding to PE(192.0.2.69) and two bindings (unicast and multicast) to PE(192.0.2.70). The following is a sample configuration output.

4.2.2.1. EVPN and VPLS Integration

In accordance with draft-ietf-bess-evpn-vpls-seamless-integ, the 7210 SAS EVPN implementation allows EVPN-MPLS and VPLS to be integrated to the same network within the same service. Because EVPN is not deployed in green-field networks, this feature is useful for facilitating the integration between both technologies and for migrating VPLS services to EVPN-MPLS.

The following behavior enables the integration of EVPN and SDP-bindings in the same VPLS network.

1. Systems with EVPN endpoints and SDP-bindings to the same far-end bring down the SDP-bindings.
   1. The router allows the establishment of an EVPN endpoint and an SDP-binding to the same far-end, but the SDP-binding is kept operationally down. Only the EVPN endpoint is operationally up. This applies to spoke-SDPs (manual and BGP-AD) and mesh-SDPs.
   2. If an EVPN endpoint to a specified far-end exists and a spoke-SDP establishment is attempted, the spoke-SDP is set up but is kept down with an operational flag, indicating that there is an EVPN route to the same far-end.
   3. If an spoke-SDP exists and a valid or used EVPN route arrives, the EVPN endpoint is set up and the spoke-SDP is brought down with an operational flag indicating that there is an EVPN route to the same far-end.
   4. In the case of an SDP-binding and EVPN endpoint to different far-end IPs on the same remote PE, both links are up. This may occur if the SDP-binding is terminated in an IPv4 address that is different from the system address where the EVPN endpoint is terminated.
2. Users can add spoke-SDPs and all the EVPN-MPLS endpoints in the same split-horizon group (SHG).
   1. The following CLI command is added under the bgp-evpn>mpls context so that the EVPN-MPLS endpoints can be added to a split-horizon group: bgp-evpn mpls [no] split-horizon-group group-name.
   2. The bgp-evpn mpls split-horizon-group command must reference a user-configured split-horizon group. User-configured split-horizon groups can be configured within the service context. The same group-name can be associated with SAPs, spoke-SDPs, pw-template-bindings, and EVPN-MPLS endpoints.
   3. If the bgp-evpn mpls split-horizon-group command is not used, the default split-horizon group (that contains all the EVPN endpoints) is still used but cannot be referred to using SAPs/spoke-SDPs.
3. The system disables the advertisement of MACs learned on spoke-SDPs or SAPs that are part of an EVPN split-horizon group.
   1. When the SAPs or spoke-SDPs (manual or BGP-AD-discovered) are configured within the same split-horizon group as the EVPN endpoints, MAC addresses are still learned on them, but they are not advertised in EVPN.
   2. The preceding statement is also true if proxy-ARP/proxy-ND is enabled and an IP-to-MAC pair is learned on a SAP or SDP-binding that belongs to the EVPN split-horizon group.
   3. The SAPs added to an EVPN split-horizon group should not be part of any EVPN multi-homed ES. If that occurs, the PE still advertises the AD per-EVI route for the SAP, attracting EVPN traffic that could not possibly be forwarded to that SAP.

Figure 44 shows an example of EVPN-VPLS integration.

Figure 44:  EVPN-VPLS Integration 

The following is a sample configuration output of PE1, PE5, and PE2.

*A:PE1>config>service# info 

----------------------------------------------

pw-template 1 create

vpls 1 customer 1 create

  split-horizon-group "SHG-1" create 

  bgp

    route-target target:65000:1

    pw-template-binding 1 split-horizon-group SHG-1 

  bgp-ad

    no shutdown

    vpls-id 65000:1

  bgp-evpn

    evi 1

    mpls

      no shutdown

      split-horizon-group SHG-1

  spoke-sdp 12:1 create

  exit

  sap 1/1/1:1 create

  exit

 

*A:PE5>config>service# info 

----------------------------------------------

pw-template 1 create

vpls 1 customer 1 create

  bgp

    route-target target:65000:1

    pw-template-binding 1 split-horizon-group SHG-1 # auto-created SHG

  bgp-ad

    no shutdown

    vpls-id 65000:1

  spoke-sdp 52:1 create

  exit

 

*A:PE2>config>service# info 

----------------------------------------------

vpls 1 customer 1 create

  end-point CORE create

    no suppress-standby-signaling

  spoke-sdp 21:1 end-point CORE

    precedence primary

  spoke-sdp 25:1 end-point CORE

The following applies to the configuration described in the preceding example.

1. PE1, PE3, and PE4 have BGP-EVPN and BGP-AD enabled in VPLS-1. PE5 has BGP-AD enabled and PE2 has active/standby spoke-SDPs to PE1 and PE5. In this configuration:
   1. PE1, PE3, and PE4 attempt to establish BGP-AD spoke-SDPs, but they are kept operationally down as long as there are EVPN endpoints active among them.
   2. BGP-AD spoke-SDPs and EVPN endpoints are instantiated within the same split-horizon group, for example, SHG-1.
   3. Manual spoke-SDPs from PE1 and PE5 to PE2 are not part of SHG-1.
2. EVPN MAC advertisements
   1. MACs learned on FEC128 spoke-SDPs are advertised normally in EVPN.
   2. MACs learned on FEC129 spoke-SDPs are not advertised in EVPN (because they are part of SHG-1, which is the split-horizon group used for bgp-evpn>mpls). This prevents any data plane MACs learned on the SHG from being advertised in EVPN.
3. BUM traffic operation on PE1
   1. When CE1 sends BUM traffic, PE1 floods to all the active bindings.
   2. When CE2 sends BUM traffic, PE2 sends it to PE1 (active spoke-SDP) and PE1 floods to all the bindings and SAPs.
   3. When CE5 sends BUM traffic, PE5 floods to the three EVPN PEs. PE1 floods to the active spoke-SDP and SAPs, never to the EVPN PEs because they are part of the same SHG.

4.2.3. EVPN Multi-Homing in VPLS Services

EVPN multi-homing implementation is based on the concept of the Ethernet Segment (ES). An ES is a logical structure that can be defined in one or more PEs and identifies the CE (or access network) multi-homed to the EVPN PEs. An ES is associated with port or LAG objects and shared by all the services defined on those objects. On the 7210 SAS, only the following service objects are allowed to be configured as an ES: port and LAG.

Each ES has a unique Ethernet Segment Identifier (ESI) that is 10 bytes long and is manually configured in the router.

Note: Because the esi command is advertised in the control plane to all the PEs in the EVPN network, it is important to ensure that the 10-byte esi value is unique throughout the entire network. Single-homed CEs are assumed to be connected to an ES with esi = 0 (single-homed ESs are not explicitly configured).

4.2.4. EVPN All-Active Multi-Homing

In accordance with RFC 7432, all-active multi-homing is only supported on access LAG SAPs, and it is mandatory to configure the CE with a LAG to avoid duplicated packets to the network. LACP is optional.

When PE3 installs MAC1 in the Forwarding Database (FDB), it associates MAC1 not only with the advertising PE (PE1), but also with all the PEs advertising the same esi (ESI2) for the service. In this example, PE1 and PE2 advertise an AD per-EVI route for ESI2; therefore, PE3 installs the two next-hops associated with MAC1.

To enable aliasing, configure ECMP greater than 1 in the bgp-evpn>mpls context.

4.2.4.1. All-Active Multi-Homing Procedures

The following procedures are implemented in SR OS to provide all-active multi-homing for a specified ES:

1. Designated Forwarder Election
2. Split-Horizon
3. Aliasing

Note: The 7210 SAS only supports two PEs per ES for all-active multi-homing.

4.2.4.1.1. Designated Forwarder Election

Using Designated Forwarder (DF) election in EVPN all-active multi-homing prevents duplicate packets on the multi-homed CE. The DF election procedure elects one DF PE per ESI per service; the rest of the PEs are non-DF for the ESI and service. Only the DF forwards BUM traffic from the EVPN network toward the ES SAPs (the multi-homed CE). The non-DF PEs do not forward BUM traffic to the local ES SAPs.

Figure 45 shows the need for DF election in all-active multi-homing.

Figure 45:  DF Election 

Note: BUM traffic from the CE to the network and known unicast traffic in any direction is allowed on both the DF and non-DF PEs.

4.2.4.1.2. Split-Horizon

The EVPN split-horizon procedure ensures that BUM traffic originated by the multi-homed PE and sent from the non-DF to the DF is not replicated back to the CE in the form of echoed packets. To avoid echoed packets, the non-DF (PE1) sends all the BUM packets to the DF (PE2) with an indication of the source ES. That indication is the ESI Label (ESI2 in Figure 46), previously signaled by PE2 in the AD per-ESI route for the ES. When it receives an EVPN packet (after the EVPN label lookup), PE2 finds the ESI label that identifies its local ES ESI2. The BUM packet is replicated to other local CEs but not to the ESI2 SAP.

Figure 46 shows the EVPN split-horizon concept for all-active multi-homing.

Figure 46:  Split-Horizon 

4.2.4.1.3. Aliasing

Figure 47 shows the EVPN aliasing procedure for all-active multi-homing. Because CE2 is multi-homed to PE1 and PE2 using an all-active ES, “aliasing” is the procedure by which PE3 can load-balance the known unicast traffic between PE1 and PE2, even if the destination MAC address is only advertised by PE1.

Figure 47:  Aliasing  

4.2.4.3. ES Discovery and DF Election Procedures

The ES discovery and DF election are implemented in three logical steps, as shown in Figure 48.

Figure 48:  ES Discovery and DF Election 

Step 1 — ES Advertisement and Discovery

ES ESI-1 is configured with all the required parameters, as described in All-Active Multi-Homing Service Model. When ethernet-segment no shutdown is executed, PE1 and PE2 advertise an ES route for ESI-1. They both include the route-target auto-derived from the MAC portion of the configured ESI. If the route-target address family is configured in the network, this allows the RR to keep the dissemination of the ES routes under control.

In addition to the ES route, PE1 and PE2 advertise AD per-ESI routes and AD per-EVI routes.

1. AD per-ESI routes announces the ES capabilities, including the mode (single-active or all-active) and the ESI label for split horizon.
2. AD per-EVI routes are advertised so that PE3 knows what services (EVIs) are associated with the ESI. These routes are used by PE3 for its aliasing and backup procedures.

Step 2 — DF Election

When ES routes exchange between PE1 and PE2 is complete, both run the DF election for all the services in the ethernet-segment.

PE1 and PE2 elect a Designated Forwarder (DF) per ESI service. The default DF election mechanism in the SR OS is service-carving (as per RFC 7432). The following applies when the mechanism is enabled on a specified PE.

1. An ordered list of PE IPs where ESI-1 resides is built. The IPs are derived from the origin IP fields of all the ES routes received for ESI-1, as well as the local system address. The lowest IP is considered ordinal “0” in the list.
2. The local IP can only be considered a “candidate” after successful ethernet-segment no shutdown for a specified service.

   Note: The remote PE IPs must be present in the local PE RTM so that they can participate in the DF election.

3. A PE only considers a specified remote IP address as candidate for the DF election algorithm for a specified service if, as well as the ES route, the corresponding AD routes per-ESI and per-EVI for that PE have been received and properly activated.
4. All remote PEs that receive the AD per-ES routes (for example, PE3) interpret ESI-1 as all-active if all the PEs send their AD per-ES routes with the single-active bit = 0. Otherwise, if at least one PE sends an AD route per-ESI with the single-active flag set or the local ESI configuration is single-active, the ESI behaves as single-active.
5. An es-activation-timer can be configured at the redundancy>bgp-evpn-multi-homing>es-activation-timer level or at the service>system>bgp-evpn>eth-seg>es-activation-timer level. This timer, which is 3 seconds by default, delays the transition from non-DF to DF for a specified service after the DF election has run.
   1. This use of the es-activation-timer is different from zero and minimizes the risks of loops and packet duplication due to “transient” multiple DFs.
   2. The same es-activation-timer should be configured in all PEs that are part of the same ESI. It is up to the user to configure either a long timer to minimize the risks of loops/duplication or even es-activation-timer=0 to speed up the convergence for non-DF to DF transitions. When the user configures a specific value, the value configured at the ES level supersedes the configured global value.
6. The DF election is triggered by the following events.
   1. The config service system bgp-evpn eth-seg no shutdown command triggers the DF election for all the services in the ESI.
   2. Reception of a new update or withdrawal of an ES route (containing an ESI configured locally) triggers the DF election for all the services in the ESI.
   3. Reception of a new update or withdrawal of an AD per-ES route (containing an ESI configured locally) triggers the DF election for all the services associated with the list of route-targets received along with the route.
   4. Reception of a new update of an AD per-ES route with a change in the ESI-label extended community (single-active bit or MPLS label) triggers the DF election for all the services associated with the list of route-targets received along with the route.
   5. Reception of a new update or withdrawal of an AD route per-EVI (containing an ESI configured locally) triggers the DF election for that service.

1. When the PE boots up, the boot-timer allows the necessary time for the control plane protocols to come up before bringing up the ES and running the DF algorithm. The boot-timer is configured at the system level, using the config redundancy bgp-evpn-multi-homing boot-timer command, and should use a value that is long enough to allow the node (with any cards, if available) to boot up and BGP sessions to come up before exchanging ES routes and running the DF election for each EVI/ISID.
   1. The system does not advertise ES routes until the boot timer expires. This guarantees that the peer ES PEs do not run the DF election until the PE is ready to become the DF, if it needs to.
   2. The following show command displays the configured boot-timer and the remaining timer, if the system is still in boot-stage.

A:PE1# show redundancy bgp-evpn-multi-homing 

===============================================================================

Redundancy BGP EVPN Multi-homing Information

===============================================================================

Boot-Timer              : 10 secs                 

Boot-Timer Remaining    : 0 secs                  

ES Activation Timer     : 3 secs                  

===============================================================================

1. When service-carving mode auto is configured (default mode), the DF election algorithm runs the function [V(evi) mod N(peers) = i(ordinal)] to identify the DF for a specified service and ESI, as described in the following example:
   1. As shown in Figure 48, PE1 and PE2 are configured with ESI-1. Given that V(10) mod N(2) = 0, PE1 are elected DF for VPLS-10 (because its IP address is lower than PE2's and it is the first PE in the candidate list).

      Note: The algorithm uses the configured evi in the service and not the service-id. The evi for a service must match in all PEs that are part of the ESI. This guarantees that the election algorithm is consistent across all PEs of the ESI. The evi must be always configured in a service with SAPs that are created in an ES.

2. A service-carving command is supported to manually configure the EVI identifiers for which the PE is primary: service-carving mode manual/manual evi start-evi to end-evi. The following considerations apply.
   1. The system is the PE forwarding/multicasting traffic for the evi identifiers included in the configuration. The PE is secondary (non-DF) for the non-specified evi identifiers.
   2. If a range is configured but service-carving is not mode manual, the range has no effect.
   3. Only two PEs are supported when service-carving mode manual is configured. If manual mode is configured for a third PE for an ESI, the two non-primary PEs remain non-DF regardless of the primary status.
   4. For example, as shown in Figure 48: if PE1 is configured with service-carving manual evi 1 to 100 and PE2 with service-carving manual evi 101 to 200, PE1 is the primary PE for service VPLS 10 and PE2 the secondary PE.
3. If service-carving is disabled, the lowest originator IP wins the election for a specified service and ESI. Use the config service system bgp-evpn eth-seg service-carving mode off command to disable service-carving.

The following sample configuration output shows the ethernet-segment configuration and DF status for all EVIs configured in the ethernet-segment.

*A:Dut-B# /show service system bgp-evpn ethernet-segment name "eslag1" all 

===============================================================================

Service Ethernet Segment

===============================================================================

Name                    : eslag1

Admin State             : Enabled            Oper State         : Up

ESI                     : 00:bc:01:00:00:00:00:00:00:01

Multi-homing            : allActive          Oper Multi-homing  : allActive

Lag Id                  : 1                  

ES Activation Timer     : 3 secs (default)   

Exp/Imp Route-Target    : target:bc:01:00:00:00:00

Svc Carving             : auto               

ES SHG Label            : 131070             

===============================================================================

===============================================================================

EVI Information 

===============================================================================

EVI                 SvcId               Actv Timer Rem      DF

-------------------------------------------------------------------------------

1                   1                   0                   no

-------------------------------------------------------------------------------

Number of entries: 1

===============================================================================

-------------------------------------------------------------------------------

DF Candidate list

-------------------------------------------------------------------------------

EVI                                     DF Address

-------------------------------------------------------------------------------

1                                       10.20.1.2

1                                       10.20.1.3

-------------------------------------------------------------------------------

Number of entries: 2

-------------------------------------------------------------------------------

------------------------------------------------------------------------------- 

Step 3 — DF and Non-DF Service Behavior

Based on the result of the DF election or the manual service-carving, the control plane on the non-DF (PE1) instructs the data path to remove the LAG SAP (associated with the ESI) from the default flooding list for BM traffic (unknown unicast traffic may still be sent if the EVI label is a unicast label and the source MAC address is not associated to the ESI). On PE1 and PE2, both LAG SAPs learn the same MAC address (coming from the CE).

For example, in the following sample configuration output, 00:ca:ca:ba:ce:03 is learned on both PE1 and PE2 access LAG (on ESI-1). However, PE1 learns the MAC as “Learned” whereas PE2 learns it as “Evpn”. This is because CE2 hashes the traffic for that source MAC to PE1. And PE2 learns the MAC through EVPN but associates the MAC to the ESI SAP, because the MAC belongs to the ESI.

*A:PE1# show service id 1 fdb detail 

===============================================================================

Forwarding Database, Service 1

===============================================================================

ServId    MAC               Source-Identifier        Type     Last Change

                                                     Age      

-------------------------------------------------------------------------------

1         00:ca:ca:ba:ce:03 sap:lag-1:1              L/0      06/11/15 00:14:47

1         00:ca:fe:ca:fe:70 eMpls:                   EvpnS    06/11/15 00:09:06

                            192.0.2.70:262140

1         00:ca:fe:ca:fe:72 eMpls:                   EvpnS    06/11/15 00:09:39

                            192.0.2.72:262141

-------------------------------------------------------------------------------

No. of MAC Entries: 3

-------------------------------------------------------------------------------

Legend:  L=Learned O=Oam P=Protected-MAC C=Conditional S=Static

===============================================================================

 

*A:PE2# show service id 1 fdb detail 

===============================================================================

Forwarding Database, Service 1

===============================================================================

ServId    MAC               Source-Identifier        Type     Last Change

                                                     Age      

-------------------------------------------------------------------------------

1         00:ca:ca:ba:ce:03 sap:lag-1:1              Evpn     06/11/15 00:14:47

1         00:ca:fe:ca:fe:69 eMpls:                   EvpnS    06/11/15 00:09:40

                            192.0.2.69:262141

1         00:ca:fe:ca:fe:70 eMpls:                   EvpnS    06/11/15 00:09:40

                            192.0.2.70:262140

-------------------------------------------------------------------------------

No. of MAC Entries: 3

-------------------------------------------------------------------------------

Legend:  L=Learned O=Oam P=Protected-MAC C=Conditional S=Static

===============================================================================

When PE1 (non-DF) and PE2 (DF) exchange BUM packets for evi 1, the packets are sent including the ESI label at the bottom of the stack (in both directions). The ESI label advertised by each PE for ESI-1 can be displayed using the following command.

*A:PE1# show service system bgp-evpn ethernet-segment name "ESI-1" 

===============================================================================

Service Ethernet Segment

===============================================================================

Name                    : ESI-1

Admin State             : Up                 Oper State         : Up

ESI                     : 01:00:00:00:00:71:00:00:00:01

Multi-homing            : allActive          Oper Multi-homing  : allActive

Lag Id                  : 1                  

ES Activation Timer     : 0 secs             

Exp/Imp Route-Target    : target:00:00:00:00:71:00

 

Svc Carving             : auto               

ES SHG Label            : 262142             

===============================================================================

*A:PE2# show service system bgp-evpn ethernet-segment name "ESI-1" 

 

===============================================================================

Service Ethernet Segment

===============================================================================

Name                    : ESI-1

Admin State             : Up                 Oper State         : Up

ESI                     : 01:00:00:00:00:71:00:00:00:01

Multi-homing            : allActive          Oper Multi-homing  : allActive

Lag Id                  : 1                  

ES Activation Timer     : 20 secs            

Exp/Imp Route-Target    : target:00:00:00:00:71:00

 

Svc Carving             : auto               

ES SHG Label            : 262142             

===============================================================================

4.2.4.4. Aliasing

As shown in the example in Figure 48, if the service configuration on PE3 has ECMP > 1, PE3 adds PE1 and PE2 to the list of next-hops for ESI-1. As soon as PE3 receives a MAC for ESI-1, it starts load-balancing between PE1 and PE2 the flows to the remote ESI CE.

The following is a sample configuration output that shows the FDB in PE3.

Note: mac 00:ca:ca:ba:ce:03 is associated with the ethernet-segment eES:01:00:00:00:00:71:00:00:00:01 (esi configured on PE1 and PE2 for ESI-1).

*A:PE3# show service id 1 fdb detail 

===============================================================================

Forwarding Database, Service 1

===============================================================================

ServId    MAC               Source-Identifier        Type     Last Change

                                                     Age      

-------------------------------------------------------------------------------

1         00:ca:ca:ba:ce:03 eES:                     Evpn     06/11/15 00:14:47

                            01:00:00:00:00:71:00:00:00:01

1         00:ca:fe:ca:fe:69 eMpls:                   EvpnS    06/11/15 00:09:18

                            192.0.2.69:262141

1         00:ca:fe:ca:fe:70 eMpls:                   EvpnS    06/11/15 00:09:18

                            192.0.2.70:262140

1         00:ca:fe:ca:fe:72 eMpls:                   EvpnS    06/11/15 00:09:39

                            192.0.2.72:262141

-------------------------------------------------------------------------------

No. of MAC Entries: 4

-------------------------------------------------------------------------------

Legend:  L=Learned O=Oam P=Protected-MAC C=Conditional S=Static

===============================================================================

The following is a sample configuration output that shows all the EVPN-MPLS destination bindings on PE3, including the ES destination bindings.

Note: The ethernet-segment eES:01:00:00:00:00:71:00:00:00:01 is resolved to PE1 and PE2 addresses.

*A:Dut-B# /show service id 1 evpn-mpls 

===============================================================================

BGP EVPN-MPLS Dest

===============================================================================

TEP Address     Egr Label     Num. MACs   Mcast           Last Change

                 Transport                                

-------------------------------------------------------------------------------

10.20.1.3       131069        0           Yes             02/02/2014 15:29:40

                rsvp                                       

10.20.1.4       131069        0           Yes             02/02/2014 15:29:33

                rsvp                                       

10.20.1.5       131059        0           Yes             02/02/2014 15:29:42

                rsvp                                       

-------------------------------------------------------------------------------

Number of entries : 3

-------------------------------------------------------------------------------

===============================================================================

===============================================================================

BGP EVPN-MPLS Ethernet Segment Dest

===============================================================================

Eth SegId                       Num. Macs               Last Change

-------------------------------------------------------------------------------

00:de:01:00:00:00:00:00:00:01   1                       02/02/2014 15:47:04

-------------------------------------------------------------------------------

Number of entries: 1

------------------------------------------------------------------------------- 

PE3 performs aliasing for all the MACs associated with that ESI. This is possible because PE1 is configured with ECMP parameter >1. The following is a sample configuration output.

*A:PE3>config>service>vpls# info 

----------------------------------------------

            bgp

            exit

            bgp-evpn

                evi 1

                mpls

                    ecmp 4

                    auto-bind-tunnel

                        resolution any

                    exit

                    no shutdown

                exit

            exit

            proxy-arp

                shutdown

            exit

            stp

                shutdown

            exit

            sap 1/1/1:2 create

            exit

            no shutdown

4.2.4.5. Network Failures and Convergence for All-Active Multi-Homing

Figure 49 shows the behavior on the remote PEs (PE3) when there is an ethernet-segment failure.

Figure 49:  All-Active Multi-Homing ES Failure 

The following steps describe the unicast traffic behavior on PE3.

1. PE3 can only forward MAC DA = CE2 to both PE1 and PE2 when the MAC advertisement route from PE1 (or PE2) and the set of Ethernet AD per-ES routes and Ethernet AD per-EVI routes from PE1 and PE2 are active at PE3.
2. In case of a failure between CE2 and PE2, PE2 withdraws its set of Ethernet AD and ES routes, and PE3 forwards traffic destined for CE2 to PE1 only. PE3 does not need to wait for the withdrawal of the individual MAC.
3. The same handling is used if the failure was at PE1.
4. If after step 2, PE2 withdraws its MAC advertisement route, PE3 treats traffic to MAC DA = CE2 as unknown unicast, unless PE1 has previously advertised the MAC.

For BUM traffic, the following events trigger a DF election on a PE, and only the DF forwards BUM traffic after the esi-activation-timer expires (if there was a transition from non-DF to DF):

1. reception of ES route update (local ES shutdown/no shutdown or remote route)
2. new AD-ES route update/withdraw
3. new AD-EVI route update/withdraw
4. local ES port/SAP/service shutdown
5. service carving range change (affecting the EVI)
6. multi-homing mode change (single/all active to all/single-active)

4.2.4.6. Logical Failures on ESs and Blackholes

Specific “failure scenarios” in the network can trigger effects. Figure 50 shows some of these scenarios.

Figure 50:  Blackhole Caused by SAP/SVC Shutdown 

If an individual VPLS service is shutdown in PE1 (the example is also valid for PE2), the corresponding LAG SAP goes operationally down. This event triggers the withdrawal of the AD per-EVI route for that SAP. PE3 removes PE1 from its list of aliased next-hops, and PE2 takes over as DF (if it was not the DF already). However, this does not prevent the network from black-holing the traffic that CE2 “hashes” to the link to PE1. Because traffic sent from CE2 to PE2 or traffic from the rest of the CEs to CE2 is unaffected, the situation is not easily detected on the CE.

The same result occurs if the ES SAP is administratively shutdown instead of the service.

Note: When the bgp-evpn mpls shutdown command is executed, the SAP associated with the ES goes operationally down (StandbyforMHprotocol). If no other SAPs or SDP-bindings are configured in the service, the service also goes operationally down. However, if other SAPs or SDP-bindings are present, the service remains operationally up.

4.2.4.7. Transient Issues Due to MAC Route Delays

Figure 51 shows scenarios that may cause potential transient issues in the network.

Figure 51:  Transient Issues Caused by “slow” MAC Learning 

In Figure 51, the scenario on the left shows an example of transient packet duplication caused by delay in PE3 to learn MAC1.

In an all-active multi-homing scenario, if a specified MAC address (for example, MAC1), is not yet learned in a remote PE (for example, PE3), but it is known in the two PEs of the ES (for example, PE1 and PE2), the latter PEs might send duplicated packets to the CE.

Configuring ingress-replication-bum-label in PE1 and PE2 resolves the issue. PE1 and PE2 know that the received packet is an unknown unicast packet; consequently, the NDF (PE1) does not send the packets to the CE, which prevents transient and duplication.

Note: On the 7210 SAS, this command is required for multi-homing; however, it does not prevent the NDF from forwarding known unicast traffic to the multi-homed CE.

In Figure 51, the scenario on the right shows an example of transient blackhole caused by delay in PE1 to learn MAC1.

In an all-active multi-homing scenario, MAC1 is known in PE3 and aliasing is applied to MAC1. However, MAC1 is not yet known in PE1, which is the NDF for the ES. If PE3 hashing picks up PE1 as the destination of the aliased MAC1, the packets are blackholed. To resolve this issue, unknown unicast traffic that arrives with a unicast label should not be blocked on the NDF. If PE1 and PE2 are configured using ingress-replication-bum-label, PE3 sends unknown unicast packets with a BUM label and known unicast with a unicast label. In the latter case, PE1 considers it safe to forward the frame to the CE, even if it is unknown unicast.

Note: This is a transient issue that is resolved as soon as MAC1 is learned in PE1 and the frames are forwarded as known unicast.

4.2.5. EVPN Single-Active Multi-Homing

The 7210 SAS SR OS supports single-active multi-homing on access LAG SAPs and regular SAPs for a specified VPLS service. For LAG SAPs, the CE is configured with a different LAG to each PE in the ES (in contrast to a single LAG in an all-active multi-homing).

The following SR OS procedures support EVPN single-active multi-homing for a specified ES:

4.2.5.3. Backup PE Function

In the example in Figure 52, the remote PE3 imports AD routes per ESI where the single-active flag is set. PE3 interprets the ethernet-segment as single-active if at least one PE sends an AD route per-ESI with the single-active flag set. MACs for a specified service and ESI are learned from a single PE, that is, the DF for that <ESI, EVI>.

The remote PE installs a single EVPN-MPLS destination (TEP, label) for a received MAC address and a backup next-hop to the PE for which the AD routes per-ESI and per-EVI are received. For example, in the following show command sample output, 00:ca:ca:ba:ca:06 is associated with the remote ethernet-segment eES 01:74:13:00:74:13:00:00:74:13. This ES is resolved to PE(192.0.2.73), which is the DF on the ES.

*A:PE3# show service id 1 fdb detail 

===============================================================================

Forwarding Database, Service 1

===============================================================================

ServId    MAC               Source-Identifier        Type     Last Change

                                                     Age      

-------------------------------------------------------------------------------

1         00:ca:ca:ba:ca:02 sap:1/1/1:2              L/0      06/12/15 00:33:39

1         00:ca:ca:ba:ca:06 eES:                     Evpn     06/12/15 00:33:39

                            01:74:13:00:74:13:00:00:74:13

1         00:ca:fe:ca:fe:69 eMpls:                   EvpnS    06/11/15 21:53:47

                            192.0.2.69:262118

1         00:ca:fe:ca:fe:70 eMpls:                   EvpnS    06/11/15 19:59:57

                            192.0.2.70:262140

1         00:ca:fe:ca:fe:72 eMpls:                   EvpnS    06/11/15 19:59:57

                            192.0.2.72:262141

-------------------------------------------------------------------------------

No. of MAC Entries: 5

-------------------------------------------------------------------------------

Legend:  L=Learned O=Oam P=Protected-MAC C=Conditional S=Static

===============================================================================

*A:Dut-D# /show service id 1 evpn-mpls 

===============================================================================

BGP EVPN-MPLS Dest

===============================================================================

TEP Address     Egr Label     Num. MACs   Mcast           Last Change

                 Transport                                

-------------------------------------------------------------------------------

10.20.1.2       131061        0           Yes             03/13/2000 11:26:29

                rsvp                                       

10.20.1.2       131062        1           No              03/13/2000 12:10:04

                rsvp                                       

10.20.1.5       131061        0           Yes             03/13/2000 11:18:23

                rsvp                                       

-------------------------------------------------------------------------------

Number of entries : 3

-------------------------------------------------------------------------------

===============================================================================

===============================================================================

BGP EVPN-MPLS Ethernet Segment Dest

===============================================================================

Eth SegId                       Num. Macs               Last Change

-------------------------------------------------------------------------------

00:de:01:00:00:00:00:00:00:01   453                     03/13/2000 12:10:14

-------------------------------------------------------------------------------

Number of entries: 1

-------------------------------------------------------------------------------

===============================================================================

*A:Dut-D# /show service id 1 evpn-mpls esi 00:de:01:00:00:00:00:00:00:01 

===============================================================================

BGP EVPN-MPLS Ethernet Segment Dest

===============================================================================

Eth SegId                       Num. Macs               Last Change

-------------------------------------------------------------------------------

00:de:01:00:00:00:00:00:00:01   453                     03/13/2000 12:10:14

===============================================================================

===============================================================================

BGP EVPN-MPLS Dest TEP Info

===============================================================================

TEP Address              Egr Label                Last Change

                         Transport                

-------------------------------------------------------------------------------

If PE3 sees only two single-active PEs in the same ESI, the second PE is the backup PE. Upon receiving an AD per-ES/per-EVI route withdrawal for the ESI from the primary PE, PE3 starts sending the unicast traffic to the backup PE immediately.

If PE3 receives AD routes for the same ESI and EVI from more than two PEs, the PE does not install any backup route in the data path. Upon receiving an AD per-ES/per-EVI route withdrawal for the ESI, it flushes the MACs associated with the ESI.

Note: On the 7210 SAS, an ES can be multi-homed to up to two PEs.

4.2.5.4. Network Failures and Convergence for Single-Active Multi-Homing

Figure 53 shows an example of remote PE (PE3) behavior when there is an ethernet-segment failure.

Figure 53:  Single-Active Multi-Homing ES Failure 

The following steps list the behavior of the remote PE3 for unicast traffic.

1. PE3 forwards MAC DA = CE2 to PE2 when the MAC advertisement route came from PE2 and the set of Ethernet AD per-ES routes and Ethernet AD per-EVI routes from PE1 and PE2 are active at PE3.
2. If there is a failure between CE2 and PE2, PE2 withdraws its set of Ethernet AD and ES routes. PE3 does not need to wait for the withdrawal of the individual MAC, and immediately forwards the traffic destined for CE2 to PE1 (the backup PE) only.
3. After the (2) PE2 withdraws its MAC advertisement route, PE3 treats traffic to MAC DA = CE2 as unknown unicast, unless the MAC has been previously advertised by PE1.

A DF election on PE1 is also triggered. A DF election is triggered by the same events as all-active multi-homing. In this case, the DF forwards traffic to CE2 when the esi-activation-timer expires; the timer is triggered when a transition from non-DF to DF occurs.

4.3.1. ARP and ND Snooping and Proxy Support

VPLS services support proxy-Address Resolution Protocol (proxy-ARP) and proxy-Neighbor Discovery (proxy-ND) functions that can be enabled or disabled independently per service. When enabled (proxy-ARP/proxy-ND no shutdown), the system populates the corresponding proxy-ARP or proxy-ND table with IP-to-MAC entries learned from the following sources:

In addition, any ingress ARP or ND frame on a SAP or SDP-binding are intercepted and processed. The system answers ARP requests and Neighbor Solicitation messages if the requested IP address is present in the proxy table.

Figure 54 shows an example proxy-ARP usage in an EVPN network. Proxy-ND functions in a similar way. The MAC address notation in the diagram is shortened for readability.

Figure 54:  Proxy-ARP Example Usage in an EVPN Network 

In Figure 54, PE1 is configured as follows:

Figure 54 shows the following steps, assuming proxy-ARP is no shutdown on PE1 and PE2, and the tables are empty.

From this point onward, the PEs reply to any ARP-request for 00:01 or 00:03 without the need for flooding the message in the EVPN network. By replying to known ARP-requests and Neighbor Solicitations, the PEs help to significantly reduce the flooding in the network.

Use the following commands to customize proxy-ARP/proxy-ND behavior:

Table 35 shows the combinations that produce a Status = Active proxy-ARP entry in the table. The system only replies to proxy-ARP requests for active entries. Any other combination result in a Status = inActv entry. If the service is not active, the proxy-ARP entries are not active, regardless of the FDB entries

Note: A static entry is active in the FDB even when the service is down.

When proxy-ARP or proxy-ND is enabled on services with all-active multi-homed ESs, a proxy-ARP entry type “EVPN” might be associated with a “learned” FDB entry because the CE can send traffic for the same MAC to all the multi-homed PEs in the ES. In such cases, the entry is inactive, in accordance with Table 35.

4.3.2. BGP-EVPN MAC Mobility

EVPN defines a mechanism to allow the smooth mobility of MAC addresses from one CE/NVE to another. The 7210 SAS supports this procedure and the MAC mobility extended community in MAC advertisement routes.

Note: When EVPN multi-homing is used in EVPN-MPLS, the ESI is compared to determine whether a MAC received from two different PEs should be processed within the context of MAC mobility or multi-homing. Two MAC routes that are associated with the same remote or local ESI but different PEs are considered reachable through all those PEs. Mobility procedures are not triggered if the MAC route still belongs to the same ESI.

4.3.3. BGP-EVPN MAC Duplication

EVPN defines a mechanism to protect the EVPN service from control plane churn as a result of loops or accidental duplicated MAC addresses. The 7210 SAS supports an enhanced version of this procedure, which is described in this section.

In a scenario where two or more hosts are misconfigured using the same (duplicate) MAC address, the duplicate MAC address is learned by the PEs in the VPLS. As a result, the traffic originating from the hosts triggers continuous MAC moves among the PEs attached to the hosts. It is important to recognize such situations and avoid incrementing the sequence number (in the MAC Mobility attribute) to infinity.

To remedy accidentally duplicated MAC addresses, a router that detects a MAC mobility event through local learning starts a window in-minutes timer (the default value is 3). If the configured num-moves num value is detected before the timer expires (the default value is 5), the router concludes that a duplicate MAC situation has occurred and sends a trap message to alert the operator. Use the show service id svc-id bgp-evpn command to display the MAC addresses. The following is a sample configuration output.

After a duplicate MAC address is detected, the router stops sending and processing BGP MAC advertisement routes for that MAC address until one of the following occurs.

Note: The other routers in the VPLS instance forward the traffic for the duplicate MAC address to the router advertising the best route for the MAC.

The values of num-moves and window can be configured for different environments. In scenarios where BGP rapid-update EVPN is configured, the operator should configure a shorter window timer than scenarios where BGP updates are sent per the configured min-route-advertisement interval, which is the default.

The preceding MAC duplication parameters can be configured per VPLS service under the bgp-evpn mac-duplication context. The following is a sample configuration output.

4.3.4. Conditional Static MAC and Protection

In RFC 7432, the MAC Mobility Extended Community section defines the use of the sticky bit to signal static MAC addresses. These addresses must be protected to prevent attempts to dynamically learn them in a different place in the EVPN-MPLS VPLS service.

Note: On the 7210 SAS, the conditional static MACs are not protected using MAC-protect functionality. A Cstatic MAC is advertised to other PEs with the sticky bit set so that it is prevented from being learned dynamically at a different place in the EVPN-MPLS VPLS service. MAC frames whose source MAC address matches the statically configured MAC address are forwarded based on destination MAC address lookup and are not dropped.

In the 7210 SAS, any conditional static MAC address that is defined in an EVPN-MPLS VPLS service is advertised by BGP-EVPN as a static address (that is, with the sticky bit set). The following is a sample output that shows the configuration of a conditional static MAC.

4.3.5. BGP and EVPN Route Selection for EVPN Routes

When two or more EVPN routes are received at a PE, BGP route selection typically takes place when the route key or the routes are equal. When the route key is different, but the PE has to make a selection (for example, the same MAC is advertised in two routes with different RDs), BGP hands over the routes to EVPN and the EVPN application performs the selection.

EVPN and BGP selection criteria are as follows:

Note: If BGP runs through the selection criteria and a specified and valid EVPN route is not selected in favor of another EVPN route, the non-selected route is displayed by the show router bgp routes evpn evpn-type detail command with a tie-breaker reason.

4.3.6. Interaction of EVPN and Other Features

This section describes the interaction of EVPN with other features.

4.3.7. Routing Policies for BGP EVPN Routes

Routing policies match on specific fields when importing or exporting EVPN routes. These matching fields are the following: