In some DC Gateway use cases, static VXLAN must be used to connect DC switches that do not support EVPN to the WAN so that a tenant subnet can be extended to the WAN. For those cases, the DC Gateway is configured with VPLS services that include a static VXLAN instance and a BGP-EVPN instance in the same service. The following combinations are supported in the same VPLS/R-VPLS service:
-
two static VXLAN instances
-
one static VXLAN instance and an EVPN-VXLAN instance
-
one static VXLAN instance and an EVPN-MPLS instance
When a static VXLAN instance coexists with EVPN-MPLS in the same VPLS/R-VPLS service, the VXLAN instance can be associated with a network-interconnect-vxlan ES if VXLAN uses instance 1. Both single-active and all-active multi-homing modes are supported as follows:
-
In single-active mode, the following behavior is for a VXLAN binding associated with the ES on the NDF:
-
TX (Transmission to VXLAN) - no MACs are learned against the binding, and the binding is removed from the default multicast list.
-
RX (Reception from VXLAN) - the RX state is down for the binding.
-
-
In all-active mode, the following behavior is for the NDF:
-
On TX - the binding is kept in the default multicast list, but only forwards the unknown-unicast traffic.
-
On RX, the behavior is determined by the command rx-discard-on-ndf {bm | bum | none} where:
-
The option bm is the default option, discards Broadcast and Multicast traffic, and allows Unicast (known and unknown).
-
The option bum discards any BUM frame on the NDF reception.
-
The option none does not discard any BUM frame on the NDF reception.
-
-
The use of the rx-discard-on-ndf options is illustrated in the following cases.
Use Case 1: Static VXLAN with anycast VTEPs and all-active ES
This use case, which is illustrated in Figure 1, works only for all-active I-ESs.
In this use case, the DGWs use anycast VTEPs, that is, PE1 has a single egress VTEP configured to the DGWs, for example, 12.12.12.12. Normally, PE1 finds ECMP paths to send the traffic to both DGWs. However, because a specified BUM flow can be sent to either the DF or the NDF (but not to both at the same time), the DGWs must be configured with the following option so that BUM is not discarded on the NDF:
rx-discard-on-ndf none
Similar to any LAG-like scenario at the access, the access CE load balances the traffic to the multi-homed PEs, but a specific flow is only sent to one of these PEs. With the option none, the BUM traffic on RX is accepted, and there are no duplicate packets or black-holed packets.
Use Case 2: Static VXLAN with non-anycast VTEPs
This use case, which is illustrated in the following figure, works with single or all-active multi-homing.
In this case, the DCGWs use different VTEPs, for example 1.1.1.1 and 2.2.2.2 respectively. PE1 has two separate egress VTEPs to the DCGWs. Therefore, PE1 sends BUM flows to both DCGWs at the same time. Concerning all-active multi-homing, if the default option for rx-discard-on-ndf is configured, PE2 and PE3 receive duplicate unknown unicast packets from PE1 (because the default option accepts unknown unicast on the RX of the NDF). So, the DCGWs must be configured with the following option:
rx-discard-on-ndf bum
Any use case in which the access PE sends BUM flows to all multi-homed PEs, including the NDF, is similar to Figure 2. BUM traffic must be blocked on the NDF’s RX to avoid duplicate unicast packets.
For single-active multi-homing, the rx-discard-on-ndf is irrelevant because BUM and known unicast are always discarded on the NDF.
Also, when non-anycast VTEPs are used on DCGWs, the following can be stated:
-
MAC addresses learned on one DGW and advertised in EVPN, are not learned on the redundant DGW through EVPN, based on the presence of a local ES in the route. Figure 2, shows a scenario in which the MAC of VM can be advertised by DCGW1, but not learned by DCGW2.
-
As a result of the above behavior and because PE2 known unicast to M1 can be aliased to DGW2, when traffic to M1 gets to DGW2, it is flooded because M1 is unknown. DGW2 floods to all the static bindings, as well as local SAPs.
-
ESI-label filtering, and no VXLAN bind between DCGWs, avoid loops for BUM traffic sent from the DF.
When a static VXLAN instance coexists with EVPN-VXLAN in the same VPLS or R-VPLS service, no VXLAN instance should be associated with an all-active network-interconnect-vxlan ES. This is because when multi-homing is used with an EVPN-VXLAN core, the non-DF PE always discards unknown unicast traffic to the static VXLAN instance (this is not the case with EVPN-MPLS if the unknown traffic has a BUM label) and traffic blackholes may occur. This is discussed in the following example:
-
Consider the example in Figure 2 I-ES Multi-Homing – static VXLAN with non-anycast VTEPs, only replacing EVPN-MPLS by EVPN-VXLAN in the WAN network.
-
Consider the PE2 has learned VM’s MAC via ES-1 EVPN destination. Because of the regular aliasing procedures, PE2 may send unicast traffic with destination VM to DGW1, which is the non-DF for I-ES 1.
-
Because EVPN-VXLAN is used in the WAN instead of EVPN-MPLS, when the traffic gets to DCGW1, it is dropped if the VM’s MAC is not learned on DGW1, creating a blackhole for the flow. If the I-ES had used EVPN-MPLS in the WAN, DCGW1 would have flooded to the static VXLAN binds and no blackhole would have occurred.
Because of the behavior illustrated above, when a static VXLAN instance coexists with an EVPN-VXLAN instance in the same VPLS/R-VPLS service, redundancy based on all-active I-ES is not recommended and single-active or an anycast solution without I-ES should be used instead. Anycast solutions are discussed in Anycast Redundant Solution for Multi-Instance EVPN-VXLAN Services, only with a static VXLAN instance in instance 1 instead of EVPN-VXLAN in this case.