2.1.1. Interfaces

Nokia routers use different types of interfaces for various functions. Interfaces must be configured with parameters such as the interface type (network and system) and address. A port is not associated with a system interface. An interface can be associated with the system (loopback address).

2.1.1.6. QoS Policy Propagation Using BGP (QPPB)

This section describes QPPB as it applies to VPRN, IES, and router interfaces. Refer to the “Internet Enhanced Service” section in the 7450 ESS, 7750 SR, 7950 XRS, and VSR Layer 3 Services Guide: IES and VPRN and the “IP Router Configuration” section in the 7450 ESS, 7750 SR, 7950 XRS, and VSR Router Configuration Guide.

QoS policy propagation using BGP (QPPB) is a feature that allows a route to be installed in the routing table with a forwarding-class and priority so that packets matching the route can receive the associated QoS. The forwarding-class and priority associated with a BGP route are set using BGP import route policies. This feature is called QPPB, even though the feature name refers to BGP specifically. On SR OS, QPPB is supported for BGP (IPv4, IPv6, VPN-IPv4, VPN-IPv6), RIP, and static routes.

SAP ingress and network QoS policies can achieve the same result as QPPB (for example, by assigning a packet arriving on an IP interface to a specific forwarding-class and priority/profile, based on the source address or destination address of the packet). However, the effort involved in creating the QoS policies, keeping them up-to-date, and applying them across many nodes is much greater than with QPPB. In a typical application of QPPB, a BGP route is advertised with a BGP community attribute that conveys a specific QoS. Routers that receive the advertisement accept the route into their routing table and set the forwarding-class and priority of the route from the community attribute.

2.1.1.6.1. QPPB Applications

There are two typical applications of QPPB:

1. Coordination of QoS policies between different administrative domains
2. Traffic differentiation within a single domain, based on route characteristics

2.1.1.6.2. Inter-AS Coordination of QoS Policies

The operator of an administrative domain “A” can use QPPB to signal to a peer administrative domain “B” that traffic sent to certain prefixes advertised by domain A should receive a specific QoS treatment in domain B. For example, an ASBR of domain A can advertise a prefix to domain B and include a BGP community attribute with the route. The community value implies a specific QoS treatment, as agreed by the two domains (in their peering agreement or service level agreement, for example). When the ASBR and other routers in domain B accept and install the route for that prefix into their routing table, they apply a QoS policy on selected interfaces that classifies traffic toward that prefix into the QoS class implied by the BGP community value.

QPPB may also be used to request that traffic sourced from specific networks receive appropriate QoS handling in downstream nodes that may span different administrative domains. This can be achieved by advertising the source prefix with a BGP community, as described. However, in this case, other approaches are equally valid, such as marking the DSCP or other CoS fields based on the source IP address, so that downstream domains can take action based on a common understanding of the QoS treatment implied by different DSCP values.

In the preceding examples, coordination of QoS policies using QPPB could be between a business customer and their IP VPN service provider, or between one service provider and another.

2.1.1.6.3. Traffic Differentiation Based on Route Characteristics

A network operator might need to provide differentiated service to specific traffic flows within its network, and these traffic flows can be identified with known routes. For example, the operator of an ISP network might need to give priority to traffic originating in a specific ASN (the ASN of a content provider offering over-the-top services to the ISP’s customers), following a specific AS_PATH, or destined for a specific next-hop (remaining on-net vs. off-net).

Figure 1 shows an example of an ISP that has an agreement with the content provider managing AS300 to provide traffic sourced and terminating within AS300 with differentiated service appropriate to the content being transported. In this example, ASBR1 and ASBR2 mark the DSCP of packets terminating and sourced, respectively, in AS300 so that other nodes within the ISP’s network do not need to rely on QPPB to determine the correct forwarding-class to use for the traffic. The DSCP or other CoS markings could be left unchanged in the ISP’s network and QPPB used on every node.

Figure 1:  Use of QPPB to Differentiate Traffic in an ISP Network 

2.1.1.7. QPPB

There are two main aspects of the QPPB feature:

1. The ability to associate a forwarding-class and priority with specific routes in the routing table.
2. The ability to classify an IP packet arriving on a specific IP interface to the forwarding-class and priority associated with the route that best matches the packet.

2.1.1.7.1. Associating an FC and Priority with a Route

This feature uses the fc command in the route-policy hierarchy to set the forwarding class and, optionally, the priority associated with routes accepted by a route-policy entry. The command has the following structure:

CLI Syntax:

fc fc-name [priority {low | high}]

The use of the fc command is shown by the following example:

config>router>policy-options

    begin

    community gold members 300:100

    policy-statement qppb_policy

        entry 10

            from

                protocol bgp

                community gold

            exit

            action accept

                fc h1 priority high

            exit

        exit

    exit

    commit

 

The fc command is supported with all existing from and to match conditions in a route policy entry, with any action other than reject, and with next-entry, next-policy, and accept actions. If a next-entry or next-policy action results in multiple matching entries, then the last entry with a QPPB action determines the forwarding class and priority.

A route policy that includes the fc command in one or more entries can be used in any import or export policy, but the fc command has no effect except in the following types of policies:

1. VRF import policies:
   1. config>service>vprn>vrf-import
2. BGP import policies:
   1. config>router>bgp>import
   2. config>router>bgp>group>import
   3. config>router>bgp>group>neighbor>import
   4. config>service>vprn>bgp>import
   5. config>service>vprn>bgp>group>import
   6. config>service>vprn>bgp>group>neighbor>import
3. RIP import policies:
   1. config>router>rip>import
   2. config>router>rip>group>import
   3. config>router>rip>group>neighbor>import
   4. config>service>vprn>rip>import
   5. config>service>vprn>rip>group>import
   6. config>service>vprn>rip>group>neighbor>import

As shown, QPPB route policies support routes learned from RIP and BGP neighbors of a VPRN, as well as for routes learned from RIP and BGP neighbors of the base/global routing instance.

QPPB is supported for BGP routes belonging to any of the following address families:

1. IPv4 (AFI=1, SAFI=1)
2. IPv6 (AFI=2, SAFI=1)
3. VPN-IPv4 (AFI=1, SAFI=128)
4. VPN-IPv6 (AFI=2, SAFI=128)

A VPN-IP route may match both a VRF import policy entry and a BGP import policy entry (if vpn-apply-import is configured in the base router BGP instance). In this case, the VRF import policy is applied first, then the BGP import policy, so the QPPB QoS is based on the BGP import policy entry.

This feature also provides the ability to associate a forwarding-class and, optionally, priority with IPv4 and IPv6 static routes. This is achieved by specifying the forwarding-class within the static-route-entry next-hop or indirect context.

Priority is optional when specifying the forwarding class of a static route, but when configured it can only be deleted and returned to unspecified by deleting the entire static route.

2.1.1.7.2. Displaying QoS Information Associated with Routes

The following commands are enhanced to show the forwarding-class and priority associated with the displayed routes:

1. show router route-table
2. show router fib
3. show router bgp routes
4. show router rip database
5. show router static-route

This feature uses a qos keyword with the show>router>route-table command. When this option is specified, the output includes an additional line per route entry that displays the forwarding class and priority of the route. If a route has no fc and priority information, the third line is blank. The following CLI shows an example:

show router route-table [family] [ip-prefix[/prefix-length]] [longer | exact] [protocol protocol-name] qos

An example output of this command is as follows:

A:Dut-A# show router route-table 10.1.5.0/24 qos

===============================================================================

Route Table (Router: Base)

===============================================================================

Dest Prefix                                   Type    Proto    Age         Pref

       Next Hop[Interface Name]                                     Metric

       QoS

-------------------------------------------------------------------------------

10.1.5.0/24                                   Remote  BGP      15h32m52s   0

       PE1_to_PE2                                                   0

       h1, high

-------------------------------------------------------------------------------

No. of Routes: 1

===============================================================================

A:Dut-A#

2.1.1.7.3. Enabling QPPB on an IP interface

To enable QoS classification of ingress IP packets on an interface based on the QoS information associated with the routes that best match the packets, configure the qos-route-lookup command in the IP interface. The qos-route-lookup command has parameters to indicate whether the QoS result is based on lookup of the source or destination IP address in every packet. There are separate qos-route-lookup commands for the IPv4 and IPv6 packets on an interface, which allows QPPB to be enabled for IPv4 only, IPv6 only, or both IPv4 and IPv6. Currently, QPPB based on a source IP address is not supported for IPv6 packets or for ingress subscriber management traffic on a group interface.

The qos-route-lookup command is supported on the following types of IP interfaces:

1. base router network interfaces (config>router>interface)
2. VPRN SAP and spoke SDP interfaces (config>service>vprn>interface)
3. VPRN group-interfaces (config>service>vprn>sub-if>grp-if)
4. IES SAP and spoke SDP interfaces (config>service>ies>interface)
5. IES group-interfaces (config>service>ies>sub-if>grp-if)

When the qos-route-lookup command with the destination parameter is applied to an IP interface and the destination address of an incoming IP packet matches a route with QoS information, the packet is classified to the fc and priority associated with that route. The command overrides the FC and priority/profile determined from the SAP ingress or network QoS policy associated with the IP interface (see section 5.7 for more information). If the destination address of the incoming packet matches a route with no QoS information, the fc and priority of the packet remain as determined by the sap-ingress or network qos policy.

Similarly, when the qos-route-lookup command with the source parameter is applied to an IP interface and the source address of an incoming IP packet matches a route with QoS information, the packet is classified to the FC and priority associated with that route. The command overrides the FC and priority/profile determined from the SAP ingress or network QoS policy associated with the IP interface. If the source address of the incoming packet matches a route with no QoS information, the FC and priority of the packet remain as determined by the SAP ingress or network QoS policy.

Currently, QPPB is not supported for ingress MPLS traffic on network interfaces or on CsC PE’-CE’ interfaces (config>service>vprn>nw-if).

Note: QPPB based on a source IP address is not supported for ingress subscriber management traffic on a group interface.

2.1.1.7.4. QPPB When Next-Hops are Resolved by QPPB Routes

In some cases (IP VPN inter-AS model C, Carrier Supporting Carrier, indirect static routes, and so on), an IPv4 or IPv6 packet may arrive on a QPPB-enabled interface and match a route A1 whose next-hop N1 is resolved by a route A2 with next-hop N2. Similarly, N2 is resolved by a route A3 with next-hop N3, and so on. The QPPB result is based only on the forwarding-class and priority of route A1. If A1 does not have a forwarding-class and priority association, the QoS classification is not based on QPPB, even if routes A2, A3, and so on, have forwarding-class and priority associations.

2.1.1.7.5. QPPB and Multiple Paths to a Destination

When ECMP is enabled, some routes may have multiple equal-cost next-hops in the forwarding table. When an IP packet matches such a route, the next-hop selection is typically based on a hash algorithm that tries to load balance traffic across all the next-hops while keeping all packets of a flow on the same path. The QPPB configuration model described in Associating an FC and Priority with a Route allows different QoS information to be associated with the different ECMP next-hops of a route. The forwarding-class and priority of a packet matching an ECMP route is based on the next-hop used to forward the packet.

When Edge PIC [1] is enabled, some BGP routes may have a backup next-hop in the forwarding table, as well as the one or more primary next-hops representing the equal-cost best paths allowed by the ECMP/multipath configuration. When an IP packet matches such a route, a reachable primary next-hop is selected (based on the hash result) but if all the primary next-hops are unreachable, the backup next-hop is used. The QPPB configuration model described in Associating an FC and Priority with a Route allows the forwarding-class and priority associated with the backup path to be different from the QoS characteristics of the equal-cost best paths. The forwarding class and priority of a packet forwarded on the backup path is based on the fc and priority of the backup route.

2.1.1.7.6. QPPB and Policy-Based Routing

When an IPv4 or IPv6 packet with destination address arrives on an interface with both QPPB and policy-based-routing enabled:

1. There is no QPPB classification if the IP filter action redirects the packet to a directly connected interface, even if the destination address is matched by a route with a forwarding-class and priority.
2. QPPB classification is based on the forwarding-class and priority of the route matching IP address Y if the IP filter action redirects the packet to the indirect next-hop IP address Y, even if the destination address is matched by a route with a forwarding-class and priority.

2.1.2. Router ID

The router ID, a 32-bit number, uniquely identifies the router within an autonomous system (AS) (see Autonomous Systems (AS)). In protocols such as OSPF, routing information is exchanged between areas—groups of networks that share routing information. It can be set to be the same as the loopback address. The router ID is used by both OSPF and BGP routing protocols in the routing table manager instance.

There are several ways to obtain the router ID. On each router, the router ID can be obtained in the following ways.

2.1.3. Autonomous Systems (AS)

Networks can be grouped into areas. An area is a collection of network segments within an AS that have been administratively assigned to the same group. An area’s topology is concealed from the rest of the AS, which results in a significant reduction in routing traffic.

Routing in the AS takes place on two levels, depending on whether the source and destination of a packet reside in the same area (intra-area routing) or different areas (inter-area routing). In intra-area routing, the packet is routed solely on information obtained within the area; no routing information obtained from outside the area can be used. This protects intra-area routing from the injection of bad routing information.

Routers that belong to more than one area are called area border routers. All routers in an AS do not have an identical topological database. An area border router has a separate topological database for each area it is connected to. Two routers, which are not area border routers, belonging to the same area, have identical area topological databases.

Autonomous systems share routing information, such as routes to each destination and information about the route or AS path, with other ASs using BGP. Routing tables contain lists of next hops, reachable addresses, and associated path cost metrics to each router. BGP uses the information and path attributes to compile a network topology.

2.1.4. Confederations

Configuring confederations is optional and should only be implemented to reduce the IBGP mesh inside an AS. An AS can be logically divided into smaller groupings called sub-confederations and then assigned a confederation ID (similar to an autonomous system number). Each sub-confederation has fully meshed IBGP and connections to other ASs outside of the confederation.

The sub-confederations have EBGP-type peers to other sub-confederations within the confederation. They exchange routing information as if they were using IBGP. Parameter values such as next hop, metric, and local preference settings are preserved. The confederation appears and behaves like a single AS.

Confederations have the following characteristics:

To migrate from a non-confederation configuration to a confederation configuration requires a major topology change and configuration modifications on each participating router. Setting BGP policies to select an optimal path through a confederation requires other BGP modifications.

There are no default confederations. Router confederations must be explicitly created. Figure 2 shows an example of a confederation configuration.

Figure 2:  Confederation Configuration 

2.1.5. Proxy ARP

Proxy ARP is the technique in which a router answers ARP requests intended for another node. The router appears to be present on the same network as the “real” node that is the target of the ARP and takes responsibility for routing packets to the “real” destination. Proxy ARP can help nodes on a subnet reach remote subnets without configuring routing or a default gateway.

Typical routers only support proxy ARP for directly attached networks; the router is targeted to support proxy ARP for all known networks in the routing instance where the virtual interface proxy ARP is configured.

To support DSLAM and other edge-like environments, proxy ARP supports policies that allow the provider to configure prefix lists that determine for which target networks proxy ARP will be attempted and prefix lists that determine for which source hosts proxy ARP will be attempted.

Also, the proxy ARP implementation will support the ability to respond for other hosts within the local subnet domain. This is needed in environments such as DSL where multiple hosts are in the same subnet but can not reach each other directly.

Static ARP is used when a Nokia router needs to know about a device on an interface that cannot or does not respond to ARP requests. The configuration can state that, if it has a packet with a specific IP address, to send it to the corresponding ARP address. Use proxy ARP so the router responds to ARP requests on behalf of another device.

2.1.6. Exporting an Inactive BGP Route from a VPRN

The export-inactive-bgp command under config>service>vprn provides an IP VPN configuration option that allows the best BGP route learned by a VPRN to be exported as a VPN-IP route even when that BGP route is inactive due to the presence of a more preferred BGP-VPN route from another PE. This “best-external” type of route advertisement is useful in active/standby multi-homing scenarios because it can ensure that all PEs have knowledge of the backup path provided by the standby PE.

2.1.7. DHCP Relay

Refer to the 7450 ESS, 7750 SR, and VSR Triple Play Service Delivery Architecture Guide for information about DHCP relay and support, as well as configuration examples.

2.1.8. Internet Protocol Versions

The -TiMOS implements IP routing functionality, providing support for IP version 4 (IPv4) and IP version 6 (IPv6). IP version 6 (RFC 1883, Internet Protocol, Version 6 (IPv6)) is a version of the Internet Protocol designed as a successor to IP version 4 (IPv4) (RFC-791, Internet Protocol). The changes from IPv4 to IPv6 affect the following categories:

Figure 3:  IPv6 Header Format 

2.1.8.1. IPv6 Address Format

IPv6 uses a 128-bit address, as opposed to the IPv4 32-bit address. Unlike IPv4 addresses, which use the dotted-decimal format, with each octet assigned a decimal value from 0 to 255, IPv6 addresses use the colon-hexadecimal format X:X:X:X:X:X:X:X, where each X is a 16-bit section of the 128-bit address. For example:

2001:0db8:0000:0000:0000:0000:0000:0000

Leading zeros must be omitted from each block in the address. A series of zeros can be replaced with a double colon. For example:

2001:db8::

The double colon can only be used once in an address.

The IPv6 prefix is the part of the IPv6 address that represents the network identifier, which appears at the beginning of the address. The IPv6 prefix length, which begins with a forward slash (/), shows how many bits of the address make up the network identifier. For example, the address 2001:db8:8086:6502::1/64 means that the first 64 bits of the address represent the network identifier; the remaining 64 bits represent the node identifier.

Note: IPv6 addresses and prefixes are displayed according to RFC 5952, A Recommendation for IPv6 Address Text Representation.

2.1.8.6. IPv6 Provider Edge Router over MPLS (6PE)

6PE allows IPv6 domains to communicate with each other over an IPv4 MPLS core network. Because forwarding is based on MPLS labels, backbone infrastructure upgrades and core router re-configuration is not required in this architecture. 6PE is a cost-effective solution for IPv6 deployment.

Figure 9:  Example of a 6PE Topology within One AS 

2.1.8.6.1. 6PE Control Plane Support

The 6PE MP-BGP routers support:

1. IPv4 and IPv6 dual-stack
2. MP-BGP to exchange IPv6 reachability information:
   1. The 6PE routers exchange IPv6 reachability information using MP-BGP (AFI 2, SAFI 4).
   2. An IPv4 address of the 6PE router is encoded as an IPv4-mapped IPv6 address in the BGP next-hop field. This is usually the IPv4 system address.
   3. The 6PE router binds MPLS labels to the IPv6 prefixes it advertises. SR OS routers advertise the IPv6 explicit null (value 2) in advertised 6PE routes but accept any arbitrary label from peers.
3. The most preferred tunnel to the BGP next-hop allowed by the 6PE resolution filter (config>router>bgp>next-hop-resolution>labeled-routes>transport-tunnel>family label-ipv6>resolution-filter) is used to tunnel the traffic to the remote 6PE router.

2.1.8.6.2. 6PE Data Plane Support

The ingress 6PE router can push two or more MPLS labels to send the packets to the egress 6PE router. The top labels are associated with resolving the transport tunnels. The bottom label is advertised in MP-BGP by the remote 6PE router. Typically, the IPv6 explicit null (value 2) label is used, but any arbitrary value can be received when the remote 6PE router is not an SR OS router.

The egress 6PE router pops the top transport labels. When the IPv6 explicit null label is exposed, the egress 6PE router knows that an IPv6 packet is encapsulated. It pops the IPv6 explicit null label and performs an IPv6 route lookup to find the next hop for the IPv6 packet.

2.1.9. Static Route Resolution Using Tunnels

The user can forward packets of a static route to an indirect next-hop over a tunnel programmed in TTM by configuring the following static route tunnel binding command:

If tunnel-next-hop context is configured and resolution is set to disabled, the binding to tunnel is removed and resolution resumes in RTM to IP next-hops.

If resolution is set to any, any supported tunnel type in static route context will be selected following TTM preference.

The following tunnel types are supported in a static route context: LDP, RSVP-TE, Segment Routing (SR) Shortest Path, and Segment Routing Traffic Engineering (SR-TE):

If one or more explicit tunnel types are specified using the resolution-filter option, only these tunnel types will be selected again following the TTM preference.

The user must set resolution to filter to activate the list of tunnel-types configured under resolution-filter.

If disallow-igp is enabled, the static route will not be activated using IP next-hops in RTM if no tunnel next-hops are found in TTM.

2.2.1. Weighted Load Balancing IGP, BGP, and Static Route Prefix Packets over IGP Shortcut

2.2.2. Weighted Load Balancing for 6PE

ECMP-like spraying for BGP labeled IPv6 packets (6PE) is controlled using the config>router>ecmp max-ecmp-routes command, where max-ecmp-routes represents the maximum number of RSVP tunnels in the set representing equal-cost paths to the BGP next hop.

Weighted ECMP behavior, where the load-balancing weight of the RSVP tunnel is considered in the packet spraying behavior, is configured using the config>router>bgp>next-hop-resolution>weighted-ecmp command. Weighted ECMP is disabled by default.

2.3.1. Feature Configuration

The user enables CBF over IGP shortcuts using the following command:

A:Reno 194# configure>router$ class-forwarding

All FCs are mapped to set 1 as soon as the policy is created. The user can make changes to the mapping of FCs as required. An FC, which is not added to the class-forwarding policy, is thus always mapped to set 1. At most, an FC can be mapped to a single forwarding set. One or more FCs can map to the same set. The user can indicate the initial default set by including the default-set option.

The default forwarding set is used to forward packets of any FC in cases where all LSPs of the forwarding set the FC maps to become operationally down. The router uses the user-configured default set as the initial default set. Otherwise, the router elects the lowest numbered set as the default forwarding set in a class-forwarding policy. When the last LSP in a default forwarding set goes into an operationally down state, the router designates the next lowest-numbered set as the new default forwarding set.

A mapping to a class-forwarding policy and set is added to the existing CBF configuration of an RSVP-TE LSP and to the LSP template of an RSVP-TE auto-LSP. The following commands perform this function.

A:Reno 194# config>router>mpls>lsp>class-forwarding$ forwarding-set policy policy-name set set-id <1..4>

A:Reno 194# config>router>mpls>lsp-template>class-forwarding$ forwarding-set policy policy-name set set-id <1..4>

An MPLS LSP can map only to a single class-forwarding policy and forwarding set. Multiple LSPs can map to the same policy and set. If they form an ECMP set, from the IGP shortcut perspective, packets of the FCs mapped to this set will be sprayed over these LSPs based on a modulo operation of the output of the hash routine on the packet's headers and the number of LSPs in the set.

2.3.2. Feature Behavior

When a BGP IPv4 or IPv6 prefix is resolved to a BGP next-hop, consisting of up to 64 resolved next-hops (LSPs and IP links), the default behavior of the data path is to spray the packets over the entire ECMP set using a modulo operation of the number of resolved next-hops in the ECMP set and the output of the hash on the packet header fields.

Both the CBF feature in LDP-over-RSVP and this CBF feature over IGP IPv4 shortcuts make use of the CBF class-forwarding policy. IGP always passes the CBF information populated by MPLS for each LSP used as a tunnel next-hop by an IGP prefix. The new CBF information is checked for consistency. If more than a single class-forwarding policy exists in the tunnel next-hops of a IGP prefix, IGP removes the new CBF information from all the corresponding tunnels and the behavior will be as if there were no CBF info.

When the CBF feature is enabled (class-forwarding option, enabled under config>router context), each application (BGP, CPM, or LDP), when looking up a prefix in RTM, will find up to 64 IP and tunnel next-hops. This lookup is split into three subsets:

The BGP application performs a lookup in RTM for a prefix matching each BGP next-hop of a prefix. The BGP application selects tunnels belonging to the class-forwarding sets in Subset 2 and for each BGP next-hop of a prefix. The remaining tunnels, with no CBF configuration or with the older CBF information and the IP next-hops, are still programmed to IOM. However, BGP and the data path will use them only when all the class-forwarding sets are not available as explained below.

The SR OS implements a hierarchical ECMP architecture for BGP prefixes in the data path. The first level is the ECMP at the BGP next-hop level. The second level is ECMP at the resolved next-hop (IP or tunnel next-hop) level. The CBF feature is independently applied to the set of resolved tunnel next-hops of each BGP next-hop of a prefix. The user must make sure that the sets of LSPs that are used as IGP shortcuts to reach each of the BGP next-hops have the appropriate FC mappings.

The following procedures are enforced in the CBF feature.

The following describes the fallback behavior in data path of the CBF feature.

2.3.3. Feature Limitations

The following are the limitations of the CBF feature.

2.3.4. Data Path Support

When a packet of a BGP IPv4 or IPv6 prefix is received, the data path uses the FC that the packet was classified into to look up the forwarding set ID. The data path then performs a modulo operation on the tunnel next-hops of this set ID, to select the one next-hop for forwarding the packet. Therefore, packets matching an FC are only sprayed over the ECMP tunnel-next-hops of the set ID this FC maps to.

Both the BGP or CPM application and IOM use the same algorithm for failover and default class-forwarding set determination, as described in Feature Behavior and illustrated in Example Configuration and Default CBF Set Election.

If MPLS deletes an LSP from a specified set ID, the IOM handles failover within the same set ID. The IOM reprograms the data path to spray packets of the impacted FCs over the remaining tunnel next-hops of the set ID.

Similarly, the IOM handles failover between class-forwarding sets when MPLS deletes the last LSP in a set ID. The IOM reprograms the data path to spray packets of the impacted FCs over the tunnel next-hops of the failover set ID. In both cases, the failover does not make use of the uniform failover procedure; however, if an LSP activated its FRR backup path, it remains in the set ID and continues to forward traffic of the mapped FCs.

Finally, BGP updates the set IDs, used to reach a BGP next-hop, any time IGP updates the information in the RTM.

2.3.5. Example Configuration and Default CBF Set Election

Assume the following user configuration.

Figure 10:  Default Forwarding Set Election 

2.4.1. BFD Control Packet

The base BFD specification does not specify the encapsulation type to be used for sending BFD control packets. Instead, use the appropriate encapsulation type for the medium and network. The encapsulation for BFD over IPv4 and IPv6 networks is specified in draft-ietf-bfd-v4v6-1hop-04.txt, BFD for IPv4 and IPv6 (Single Hop). This specification requires that BFD control packets be sent over UDP with a destination port number of 3784 and the source port number must be within the range 49152 to 65535.

Also, the TTL of all transmitted BFD packets must have an IP TTL of 255. All BFD packets received must have an IP TTL of 255 if authentication is not enabled. If authentication is enabled, the IP TTL should be 255, but can still be processed if it is not (assuming the packet passes the enabled authentication mechanism).

If multiple BFD sessions exist between two nodes, the BFD discriminator is used to de-multiplex the BFD control packet to the appropriate BFD session.

2.4.2. Control Packet Format

The BFD control packet has two sections: a mandatory section and an optional authentication section.

Figure 11:  Mandatory Frame Format 

2.4.3. BFD for RSVP-TE

BFD will notify RSVP-TE if the BFD session goes down, in addition to notifying other configured BFD enabled protocols (for example, OSPF, IS-IS, and PIM). This notification will then be used by RSVP-TE to begin the reconvergence process. This greatly accelerates the overall RSVP-TE response to network failures.

All encapsulation types supporting IPv4 and IPv6 are supported because all BFD packets are carried in IPv4 and IPv6 packets; this includes Frame Relay and ATM.

BFD is supported on the following interfaces:

The following interfaces are supported only on the 7750 SR and 7450 ESS:

2.4.4. Echo Support

Echo support for BFD calls for the support of the echo function within BFD. By supporting BFD echo, the router loops back received BFD echo messages to the original sender based on the destination IP address in the packet.

The echo function is useful when the local router does not have sufficient CPU power to handle a periodic polling rate at a high frequency. Therefore, it relies on the echo sender to send a high rate of BFD echo messages through the receiver node, which is only processed by the receiver’s forwarding path. This allows the echo sender to send BFD echo packets at any rate.

SR OS does not support the sending of echo requests, only the response to echo requests.

2.4.5. BFD Support for BGP

This feature allows BGP peers to be associated with the BFD session. If the BFD session fails, BGP peering will also be torn down.

2.4.6. Centralized BFD

The following applications of centralized BFD require BFD to run on the SF/CPM.

2.4.6.1. IES Over Spoke SDP

One application for a central BFD implementation is so BFD can be supported over spoke SDPs used to inter-connect IES or VPRN interfaces. When there are spoke SDPs for inter-connections over an MPLS network between two routers, BFD is used to speed up failure detections between nodes so re-convergence of unicast and multicast routing information can begin as quickly as possible.

The MPLS LSP associated with the spoke SDP can enter or egress from multiple interfaces on the router. BFD for these types of interfaces cannot exist on the IOMXCM by itself.

Figure 12:  BFD for IES/VPRN over Spoke SDP 

2.4.6.2. BFD Over LAG and VSM Interfaces

A second application for a central BFD implementation is so BFD can be supported over LAG or VSM interface. This is useful where BFD is not used for link failure detection, but for node failure detection. In this application, the BFD session can run between the IP interfaces associated with the LAG or VSM interface, but there is only one session between the two nodes. There is no requirement for the message flow to across a certain link, or VSM, to get to the remote node.

Figure 13:  BFD Over LAG and VSM Interfaces  

2.4.7. Aggregate Next Hop

This feature adds the ability to configure an indirect next-hop for aggregate routes. The indirect next-hop specifies where packets will be forwarded if they match the aggregate route, but is not a more-specific route in the IP forwarding table.

2.4.8. Invalidate Next-Hop Based on ARP/Neighbor Cache State

This feature invalidates next-hop entries for static routes when the next-hop is no longer reachable on directly connected interfaces. This invalidation is based on ARP and Neighbor Cache state information.

When a next-hop is detected as no longer reachable due to ARP/Neighbor Cache expiry, the route’s next-hop is set as unreachable to prevent the SR from sending continuous ARPs/Neighbor Solicitations triggered by traffic destined for the static route prefix. When the next-hop is detected as reachable via ARP or Neighbor Advertisements, the state of the next-hop is set back to valid.

2.4.9. LDP Shortcut for IGP Route Resolution

This feature enables you to forward user IP packets and specified control IP packets using LDP shortcuts over all network interfaces in the system that participate in the IS-IS and OSPF routing protocols. The default is to disable the LDP shortcut across all interfaces in the system.

config>router>ldp-shortcut [ipv4] [ipv6]

2.6.1. Feature Configuration

The user defines a subnet for the termination of GRE packets by applying the gre-termination command to a numbered network IP interface, including a loopback interface, using the config>router>interface>gre-termination command.

The following rules apply to termination of IP-over-GRE and MPLS-over-GRE on a user-defined subnet.

2.6.2. MPLS-over-GRE and IP-over-GRE Termination Function

When a GRE packet is received over any network IP interface, the router checks if destination address matches the system interface address (exact match) or the GRE termination subnet (Longest Prefix Match). The router then processes the packet according to the following criteria:

This feature supports GRE/IPv4 encapsulation when the payload is MPLS, IPv4, or IPv6.

All MPLS egress LER and LSR features associated with the processed label are supported.

2.6.3. Outgoing Packet Ethertype Setting and TTL Handling in MPLS-over-GRE Termination

The router sets the Ethertype field value of the outgoing packet according to the following criteria.

The router sets the TTL of the outgoing packet as per the behavior of a PHP LSR:

2.6.4. Ethertype Setting and TTL Handling in IP-over-GRE Termination

The router sets the Ethertype field value of the outgoing packet to IPv4 or IPv6 value when the GRE protocol field value in the incoming packet is IPv4 or IPv6 respectively.

The router checks and decrements the TTL field of the inner IPv4 or IPv6 header and ignores the TTL of the outer IPv4 header.

2.6.5. LER and LSR Hashing Support

When the router removes the GRE encapsulation, pops one or more labels including the Bottom-of-Stack (BoS) label, it acts as a LER. The exposed packet will be forwarded in the global routing table or in a service context. The LAG/ECMP hashing of the packet when forwarded follow the procedures of that specific forwarding context. Refer to LAG and ECMP Hashing in the 7450 ESS, 7750 SR, 7950 XRS, and VSR Interface Configuration Guide.

When the router removes the GRE encapsulation, pops one or more labels and then swaps a label, it acts as an LSR. The LSR hashing for packets of a MPLS-over-GRE SDP or tunnel terminating on the GRE subnet follows a new procedure which is enabled automatically and overrides the LSR hashing option enabled on the incoming network IP interface (lsr-load-balancing {lbl-only | lbl-ip | ip-only | eth-encap-ip | lbl-ip-l4-teid}). For more details, refer to LSR Hashing of MPLS-over-GRE Encapsulated Packet in section Changing Default Per Flow Hashing Inputs of the 7450 ESS, 7750 SR, 7950 XRS, and VSR Interface Configuration Guide.

2.7.1. Sample GRE Tunnel Configurations

Public interface example:

Private interface example:

2.8.1. NGE Domains

An NGE domain is a group of nodes and router interfaces forming a network that uses a single key group to create a security domain. NGE domains are created when router interface encryption is enabled on router interfaces that need to participate in the NGE domain. The NSP NFM-P assists operators in managing the nodes and interfaces that participate in the NGE domain. See the NSP NFM-P User Guide for more information.

Figure 16 shows various traffic types crossing an NGE domain.

Figure 16:  NGE Domain Transit 

In Figure 16, nodes A, B, C, and D have router interfaces configured with router interface encryption enabled. Traffic is encrypted when entering the NGE domain using the key group configured on the router interface and is decrypted when exiting the NGE domain. Traffic may traverse multiple hops before exiting the NGE domain, yet decryption only occurs on the final node when the traffic exits the NGE domain.

Various traffic types are supported and encrypted when entering the NGE domain, as illustrated by the following items on node A in Figure 16:

GRE-MPLS- or MPLSoUDP-based service traffic consists of Layer 3 packets, and router interface NGE is not applied to these types of packets. Instead, service-level NGE is used for encryption to avoid double-encrypting these packets and impacting throughput and latencies. The two types of GRE-MPLS or MPLSoUDP packets that can enter the NGE domain are illustrated by items 4 and 5 in Figure 16.

Creating an NGE domain from the NSP NFM-P requires the operator to determine the type of NGE domain being managed. This will indicate whether NGE gateway nodes are required to manage the NGE domain, and other operational considerations. The two types of NGE domains are:

2.8.1.1. Private IP/MPLS Network NGE Domain

One type of NGE domain is a private IP/MPLS network, as shown in Figure 17.

Figure 17:  Private IP/MPLS Network NGE Domain 

In a private IP/MPLS network NGE domain, all interfaces are owned by the operator and there is no intermediary service provider needed to interconnect nodes. Each interface is a point-to-point private link between private nodes. When a new node is added to this type of NGE domain (node D in Figure 17), the links that connect node D to the existing nodes in the NGE domain (nodes A, B, and C) must be enabled with NGE router interface encryption. Links from the new node to the existing nodes are enabled one at a time. The NSP NFM-P provides tools that simplify adding nodes to the NGE domain and enabling NGE on their associated interfaces. In this type of NGE domain, each interface is a direct link between two nodes and is not used to communicate with multiple nodes over a broadcast medium offered by an intermediary network. Also, there are no NGE gateway nodes required between the NSP NFM-P and new nodes entering the NGE domain.

2.8.1.2. Private Over Intermediary Network NGE Domain

The other type of NGE domain is a private IP/MPLS network that traverses an intermediary network NGE domain; the intermediary network is used to interconnect nodes in the NGE domain using a multipoint-to-multipoint service. The intermediary network is typically a service provider network that provides a private IP VPN service or a private VPLS service used to interconnect a private network that does not mimic point-to-point links as described in the Private IP/MPLS Network NGE Domain section.

This type of NGE domain is shown in Figure 18.

Figure 18:  Private Over Intermediary Network NGE Domain 

Private over intermediary network NGE domains have nodes with links that connect to a service provider network where a single link can communicate with multiple nodes over a Layer 3 service such as a VPRN. In Figure 18, node A has NGE enabled on its interface with the service provider and uses that single interface to communicate with nodes B and C, and eventually with node D when node D has been added to the NGE domain. This type of NGE domain requires the recognition of NGE gateway nodes that allow the NSP NFM-P to reach new nodes that enter the domain. Node C is designated as a gateway node.

When node D is added to the NGE domain, it must first have the NGE domain key group downloaded to it from the NSP NFM-P. The NSP NFM-P creates an NGE exception ACL on the gateway node, C, to allow communication with node D using SNMPv3 and SSH through the NGE domain. After the key group is downloaded, the NSP NFM-P enables router interface encryption on node D’s interface with the service provider and node D is now able to participate in the NGE domain. The NSP NFM-P automatically removes the IP exception ACL from node C when node D enters the NGE domain.

See Router Interface NGE Domain Concepts for more information.

2.8.2. Router Interface NGE Domain Concepts

An NGE domain is a group of nodes whose router interfaces in the base routing context (GRT) are enabled for router interface NGE. An interface without router interface NGE enabled is considered to be outside the NGE domain. NGE domains use only one key group when the domain is created; however, two key groups may be active at once if some links within the NGE domain are in transition from one key group to the other.

Figure 19 illustrates the NGE domain concept. Table 6 describes the three configuration scenarios inside the NGE domain.

Figure 19:  Inside and Outside NGE Domains 

A router interface is considered to be inside the NGE domain when it has been configured with group-encryption on the interface. When group-encryption is configured on the interface, the router can receive unencrypted packets or NGE-encrypted packets from any configured key group on the router, but any other type of IPSec-formatted packet is not allowed. If an IPSec-formatted packet is received on an interface that has group-encryption enabled, it will not pass NGE authentication and will be dropped. Therefore, IPSec packets cannot exist within the NGE domain without first being converted to NGE packets. This conversion requirement delineates the boundary of the NGE domain and other IPSec services.

When NGE router interface encryption is enabled and only an outbound key group is configured, the interface can receive unencrypted packets or NGE-encrypted packets from any configured key group on the router. All outbound packets are encrypted using the outbound key group if the packet was not already encrypted further upstream in the network.

When NGE router interface encryption has been configured with both an inbound and outbound key group, only NGE packets encrypted with the key group security association can be sent and received over the interface.

When there is no NGE router interface encryption, the interface is considered outside the NGE domain where NGE is not applied.

Refer to the “NGE Packet Overhead and MTU Considerations” section in the 7450 ESS, 7750 SR, 7950 XRS, and VSR Services Overview Guide for MTU information related to enabling NGE on a router interface.

2.8.3. GRE-MPLS and MPLSoUDP Packets Inside the NGE Domain

NGE router interface encryption is never applied to GRE-MPLS or MPLSoUDP packets, for example:

GRE-MPLS and MPLSoUDP packets that enter the NGE domain or transit the NGE domain are forwarded as is.

Because these GRE-MPLS and MPLS-oUDP packets provide transport for MPLS-based services, they already use the NGE services-based encryption techniques for MPLS, such as SDP or VPRN-based encryption. To avoid double encryption, the packets are left in clear text when entering an NGE domain or crossing intermediate nodes in the NGE domain, and are forwarded as needed when exiting an NGE domain.

2.8.4. EVPN-VXLAN Tunnels and Services

NGE router interface encryption does not differentiate between EVPN-VXLAN tunnels and other L3 traffic, and therefore encrypts all EVPN-VXLAN traffic that egresses the node.

For received encrypted EVPN-VXLAN packets, if the VXLAN tunnel terminates on the node (that is, the destination IP is for a VTEP on this node), then the NGE packet is decrypted and the EVPN-VXLAN traffic is processed as if NGE encryption never took place.

2.8.5. Router Encryption Exceptions using ACLs

In some cases, Layer 3 packets may need to cross the NGE domain in clear text, such as when an NGE-enabled router needs to peer with a non-NGE-capable router to exchange routing information. This can be accomplished by using a router interface NGE exception filter applied on the router interface for the required direction, inbound or outbound.

Figure 20 shows the use of a router interface NGE exception filter.

Figure 20:  Router Interface NGE Exception Filter Example 

The inbound or outbound exception filter is used to allow specific packet flows through the NGE domain in clear text, where there is an explicit inbound and outbound key group configured on the interface. The behavior of the exception filter for each router interface configuration is as follows:

2.8.6. IPSec Packets Crossing an NGE Domain

IPSec packets can cross the NGE domain because they are still considered Layer 3 packets. To avoid confusion between the security association used in an IPSec packet and the one used in a router interface NGE packet, the router will always apply NGE to any IPSec packet that traverses the NGE domain.

IPSec packets that originate from a router within the NGE domain are not allowed to enter the NGE domain. The only exception to this restriction is OSPFv3 packets.

Figure 21 shows how IPSec packets can transit an NGE domain.

Figure 21:  IPSec Packets Transiting an NGE Domain 

An IPSec packet enters the router from outside the NGE domain. When the router determines that the egress interface to route the packet is inside an NGE domain, it will select an NGE router interface with one of the following configurations.

OSPFv3 IPSec support also uses IPSec transport mode packets. These packets originate from the CPM, which is considered outside the NGE domain; however, the above rules for encapsulating the packets with an NGE ESP apply and allow these packets to successfully transit the NGE domain.

2.8.7. Multicast Packets Traversing the NGE Domain

Multicast packets that traverse an NGE domain can be categorized into two main scenarios:

Figure 22 shows these scenarios.

Figure 22:  Processing Multicast Packets 

Multicast packets received from outside the NGE domain (Scenario 1) are processed similarly to multicast packets received from inside the NGE domain (Scenarios 2a and 2b).

The processing rule is that multicast packets are always forwarded as clear text over the fabric. This means that for Scenario 2b, when a multicast packet is received on an encryption-capable interface and is NGE-encrypted, the packet is always decrypted first so that it can be processed in the same way as packets in Scenarios 1 and 2a.

On egress, the following scenarios apply:

2.8.8. Assigning Key Groups to Router Interfaces

Assigning key groups to router interfaces involves the following three steps:

Step 1 is required so that the router can initialize and differentiate the interface for NGE traffic before accepting or sending NGE packets. This assigns the interface to an NGE domain.

Assigning key groups to a router interface in steps 2 and 3 is similar to assigning key groups to SDPs or VPRN-based services. An outbound key group cannot be configured for a router interface without first enabling group-encryption.

When group-encryption is enabled and no inbound key group is configured, the router will accept NGE Layer 3 packets that were encrypted using keys from any security association configured in any key group on the system. If the packet specifies a security association that is not configured in any key group on the node, the packet is dropped.

The outbound key group references the key group to use when traffic egresses the router on the router interface. The inbound key group is used to make sure ingress traffic is using the correct key group on the router interface. If ingress traffic is not using the correct key group, the router counts these packets as errors.

2.8.9. NGE and BFD Support

When NGE is enabled on a router interface, BFD packets that originate from the network processor on the adapter card or from the system are encrypted in the same way as BFD packets that are generated by the CPM.

2.8.10. NGE and ACL Interactions

When NGE is enabled on a router interface, the ACL function is applied as follows:

2.8.11. Router Interface NGE and ICMP Interactions Over the NGE Domain

Typically, ICMP works as expected over an NGE domain when all routers participating in the NGE domain are NGE-capable; this includes running an NGE domain over a private IP/MPLS network. When an ICMP message is required, the NGE packet is decrypted first and the original packet is restored to create a detailed ICMP message using the original packet’s header information.

When the NGE domain crosses a Layer 3 service provider, or crosses over routers that are not NGE-aware, it is not possible to create a detailed ICMP message using the original packet’s information, as the NGE packet protocol is always set to ESP. Furthermore, the NGE router that receives these ICMP messages will drop them because the messages are not NGE-encrypted.

The combination of dropping ICMP messages at the NGE border node and the missing unencrypted packet details in the ICMP information can cause problems with diagnosing network issues.

To help with diagnosing network issues, additional statistics are available on the interface to show whether ICMP messages are being returned from a foreign node. The following statistics are included in the group encryption NGE statistics for an interface:

These statistics are used when clear text ICMP messages are received on an NGE router interface. The Invalid ESP statistics are not used in this situation even though the packet does not have a correct NGE ESP header. If there is no ingress exception ACL configured on the interface to allow the ICMP messages to be forwarded, the messages are counted and dropped.

If more information is required for these ICMP messages, such as source or destination address information, a second ICMP filter can be configured on the interface to allow logging of the ICMP messages. If the original packet information is also required, an egress exception ACL can be configured with the respective source or destination address information, or other criteria, to allow the original packet to enter the NGE domain in clear text and determine which flows are causing the ICMP failures.

2.8.12. 1588v2 Encryption With NGE

If a router interface is enabled for encryption and Layer 3 1588v2 packets are sent, they will be encrypted using NGE. This means that if port timestamping is enabled on a router interface with NGE, the port timestamp is applied to the Layer 3 1588v2 packet using software-based timestamping instead of hardware-based timestamping, and consequently, timing accuracy may degrade. The exact level of timing or synchronization degradation is dependent on many factors, and testing is recommended to measure any impact.

If there is a need to support Layer 3 1588v2 with better accuracy for frequency or better time using port timestamping, an NGE exception ACL is required to keep the Layer 3 1588v2 packets in clear text. The exception ACL must enable UDP packets with destination port 319 to be sent in clear text.