3.1.1. Overview

In order to provide what network engineers call Quality of Service (QoS), the flow of data in the form of packets must be predetermined and resources must be somehow assured for that predetermined flow. Simple routing does not provide a predetermined path for the traffic, and priorities that are described by Class of Service (CoS) coding simply increase the odds of successful transit for one packet over another. There is still no guarantee of service quality. The guarantee of service quality is what distinguishes QoS from CoS. CoS is an element of overall QoS.

By using the traffic management features of the 7705 SAR, network engineers can achieve a QoS for their customers. Multiprotocol Label Switching (MPLS) provides a predetermined path, while policing, shaping, scheduling, and marking features ensure that traffic flows in a predetermined and predictable manner.

There is a need to distinguish between high-priority (that is, mission-critical traffic like signaling) and best-effort traffic priority levels when managing traffic flow. Within these priority levels, it is important to have a second level of prioritization, that is, between a certain volume of traffic that is contracted/needed to be transported, and the amount of traffic that is transported if the system resources allow. Throughout this guide, contracted traffic is referred to as in-profile traffic. Traffic that exceeds the user-configured traffic limits is either serviced using a lower priority or discarded in an appropriate manner to ensure that an overall quality of service is achieved.

The 7705 SAR must be properly configured to provide QoS. To ensure end-to-end QoS, each and every intermediate node together with the egress node must be coherently configured. Proper QoS configuration requires careful end-to-end planning, allocation of appropriate resources and coherent configuration among all the nodes along the path of a given service. Once properly configured, each service provided by the 7705 SAR will be contained within QoS boundaries associated with that service and the general QoS parameters assigned to network links.

The 7705 SAR is designed with QoS mechanisms at both egress and ingress to support different customers and different services per physical interface or card, concurrently and harmoniously (refer to Egress and Ingress Traffic Direction for a definition of egress and ingress traffic). The 7705 SAR has extensive and flexible capabilities to classify, police, shape and mark traffic to make this happen.

Note: The characteristics and nature of traffic flows in the ingress and egress directions are usually totally different. As an example, traffic is usually shaped at egress for pacing purposes and jitter tolerance imposed by the network transport rules, whereas at ingress, traffic is usually policed to ensure it fits into the traffic volumes defined in the Service Level Agreement. Thus, segregation between ingress and egress offers not only the seamless flexibility to address different requirements but as well allows fine-tuning of appropriate parameters in each direction.

The 7705 SAR supports multiple forwarding classes (FCs) and associated class-based queuing. Ingress traffic can be classified to multiple FCs, and the FCs can be flexibly associated with queues. This provides the ability to control the priority and drop priority of a packet while allowing the fine-tuning of bandwidth allocation to individual flows.

Each forwarding class is important only in relation to the other forwarding classes. A forwarding class allows network elements to weigh the relative importance of one packet over another. With such flexible queuing, packets belonging to a specific flow within a service can be preferentially forwarded based on the CoS of a queue. The forwarding decision is based on the forwarding class of the packet, as assigned by the ingress QoS policy defined for the service access point (SAP).

7705 SAR routers use QoS policies to control how QoS is handled at distinct points in the service delivery model within the device. QoS policies act like a template. Once a policy is created, it can be applied to many other similar services and ports. As an example, if there is a group of Node Bs connected to a 7705 SAR node, one QoS policy can be applied to all services of the same type, such as High-Speed Downlink Packet Access (HSDPA) offload services.

There are different types of QoS policies that cater to the different QoS needs at each point in the service delivery model. QoS policies are defined in a global context in the 7705 SAR and only take effect when the policy is applied to a relevant entity.

QoS policies are uniquely identified with a policy ID number or a policy ID name. Policy ID 1 and policy ID “default” are reserved for the default policy, which is used if no policy is explicitly applied.

The different QoS policies within the 7705 SAR can be divided into two main types.

The sections that follow provide an overview of the QoS traffic management performed on the 7705 SAR.

3.1.2. Egress and Ingress Traffic Direction

Throughout this document, the terms ingress and egress, when describing traffic direction, are always defined relative to the fabric. For example:

When combined with the terms access and network, which are port and interface modes, the four traffic directions relative to the fabric are (see Figure 1):

Note: Throughout this guide, the terms access ingress/egress and service ingress/egress are interchangeable. This section (QoS Overview) uses the term access, and the following sections (beginning with QoS Policies Overview) use the term service.

Figure 1:  Egress and Ingress Traffic Direction 

3.1.2.1. Ring Traffic

On the 2-port 10GigE (Ethernet) Adapter card and 2-port 10GigE (Ethernet) module, traffic can flow between the Layer 2 bridging domain and the Layer 3 IP domain (see Figure 2). In the bridging domain, ring traffic flows from one ring port to another, as well as to and from the add/drop port. From the network point of view, traffic from the ring towards the add/drop port and the v-port is considered ingress traffic (drop traffic). Similarly, traffic from the fabric towards the v-port and the add/drop port is considered egress traffic (add traffic).

The 2-port 10GigE (Ethernet) Adapter card and 2-port 10GigE (Ethernet) module function as an add/drop card to a network side 10 Gb/s optical ring. Conceptually, the card and module should be envisioned as having two domains—a Layer 2 bridging domain where the add/drop function operates, and a Layer 3 IP domain where the normal IP processing and IP nodal traffic flows are managed. Ingress and egress traffic flow remains in the context of the nodal fabric. The ring ports are considered to be east-facing and west-facing and are referenced as Port 1 and Port 2. A virtual port (or v-port) provides the interface to the IP domain within the structure of the card or module.

Figure 2:  Ingress and Egress Traffic on a 2-port 10GigE (Ethernet) Adapter Card 

3.1.3. Forwarding Classes

Queues can be created for each forwarding class to determine the manner in which the queue output is scheduled and the type of parameters the queue accepts. The 7705 SAR supports eight forwarding classes per SAP. Table 2 shows the default mapping of these forwarding classes in order of priority, with Network Control having the highest priority.

The traffic flows of different forwarding classes are mapped to the queues. This mapping is user-configurable. Each queue has a unique priority. Packets from high-priority queues are scheduled separately, before packets from low-priority queues. More than one forwarding class can be mapped to a single queue. In such a case, the queue type defaults to the priority of the lowest forwarding class (see Queue Type for more information on queue type). By default, the following logical order is followed:

At access ingress, traffic can be classified as unicast traffic or one of the multipoint traffic types (broadcast, multicast, or unknown (BMU)). After classification, traffic can be assigned to a queue that is configured to support one of the four traffic types, namely:

3.1.4. Scheduling Modes

The scheduler modes available on adapter cards are profiled (2-priority), 4-priority, and 16-priority. Which modes are supported on a particular adapter card depends on whether the adapter card is a first-, second-, or third-generation card.

Note: Throughout the 7705 SAR documentation set, first-, second-, and third-generation Ethernet adapter cards and Ethernet ports on fixed platforms are also referred to as Gen-1, Gen-2, and Gen-3 hardware.

On Gen-3 hardware, 4-priority scheduling mode is the implicit, default scheduling mode and is not user-configurable. Gen-3 platforms with a TDM block support 4-priority scheduling mode. Gen-2 adapter cards support 16-priority and 4-priority scheduling modes. Gen-1 adapter cards support profile (2-priority) and 4-priority scheduling modes.

For details on differences between Gen-1, Gen-2, and Gen-3 hardware related to scheduling mode QoS behavior, see QoS for Gen-3 Adapter Cards and Platforms.

For information on scheduling modes as they apply to traffic direction, refer to the following sections:

3.1.5. Intelligent Discards

Most 7705 SAR systems are susceptible to network processor congestion if the packet rate of small packets received on a node or card exceeds the processing capacity. If a node or card receives a high rate of small packet traffic, the node or card enters overload mode. Prior to the introduction of intelligent discards, when a node or card entered an overload state, the network processor would randomly drop packets.

The “intelligent discards during overload” feature allows the network processor to discard packets according to a preset priority order. In the egress direction, intelligent discards is applied to traffic entering the card from the fabric.

Traffic is discarded in the following order: low-priority out-of-profile user traffic is discarded first, followed by high-priority out-of-profile user traffic, then low-priority in-profile user traffic, high priority in-profile user traffic, and lastly control plane traffic. In the ingress direction, intelligent discards is applied to traffic entering the card from the physical ports. Traffic is discarded in the following order: low-priority user traffic is always discarded first, followed by high-priority user traffic. This order ensures that low-priority user traffic is always the most susceptible to discards.

In the egress direction, the system differentiates between high-priority and low-priority user traffic based on the internal forwarding class and queue-type fabric header markers. In the ingress direction, the system differentiates between high-priority and low-priority user traffic based on packet header bits. Table 3 details the classification of user traffic in the ingress direction.

Intelligent discards during overload ensures priority-based handling of traffic and helps existing traffic management implementations. It does not change how QoS-based classification, buffer management, or scheduling operates on the 7705 SAR. If the node or card is not in overload operation mode, there is no change to the way packets are handled by the network processor.

There are no commands to configure intelligent discards during overload; the feature is automatically enabled on the following cards, modules, and ports:

3.1.6. Buffering

Buffer space is allocated to queues based on the committed buffer space (CBS), the maximum buffer space (MBS) and availability of the resources, and the total amount of buffer space. The CBS and the MBS define the queue depth for a particular queue. The MBS represents the maximum buffer space that is allocated to a particular queue. Whether that much space can actually be allocated depends on buffer usage (that is, the number of other queues and their sizes).

Memory allocation is optimized to guarantee the CBS for each queue. The allocated queue space beyond the CBS is limited by the MBS and depends on the use of buffer space and the guarantees accorded to queues as configured in the CBS.

This section contains information on the following topics:

3.1.6.3. Buffer Unit Allocation and Buffer Chaining

Packetization buffers and queues are supported in the packet memory of each adapter card or platform. All adapter cards and platforms allocate a fixed space for each buffer. The 7705 SAR supports two buffer sizes: 512 bytes or 2304 bytes, depending on the type of adapter card or platform.

The adapter cards and platforms that support a buffer size of 2304 bytes do not support buffer chaining (see the description below) and only allow a 1-to-1 correspondence of packets to buffers.

The adapter cards and platforms that support a buffer of size of 512 bytes use a method called buffer chaining to process packets that are larger than 512 bytes. To accommodate packets that are larger than 512 bytes, these adapter cards or platforms divide the packet dynamically into a series of concatenated 512-byte buffers. An internal 64-byte header is prepended to the packet, so only 448 bytes of buffer space is available for customer traffic in the first buffer. The remaining customer traffic is split among concatenated 512-byte buffers.

Note: The 8-port Ethernet Adapter card (version 2) uses an additional 64 bytes at the end of the first buffer for packets larger than 452 bytes (448 bytes plus 4 bytes for FCS); for example, when 449 bytes need to be buffered, the available space in the first buffer is 384 bytes, whereas all other adapter cards that support buffer chaining will have 448 bytes of buffer space available in the first buffer.

Table 4 shows the supported buffer sizes on the 7705 SAR adapter cards and platforms. If a version number or variant is not specified, this implies all versions of the adapter card or variants of the platform. Adapter cards and platforms that support buffer chaining have 512 byte buffer size (“Yes”); those that do not support buffer chaining have 2304 byte buffer size (“No”).

Table 4:  Buffer Support on Adapter Cards and Platforms 

Adapter Card or Platform	Buffer Space per Card/Platform (Mbytes)	Buffer Chaining Support
2-port 10GigE (Ethernet) Adapter card	268 201 (for L2 bridging domain)	Yes Yes (each buffer unit is 768 bytes)
2-port 10GigE (Ethernet) module	201 (for L2 bridging domain)	Yes (each buffer unit is 768 bytes)
2-port OC3/STM1 Channelized Adapter card	310	No
4-port OC3/STM1 / 1-port OC12/STM4 Adapter card	217	Yes
4-port OC3/STM1 Clear Channel Adapter card	352	No
4-port DS3/E3 Adapter card	280	No
6-port E&M Adapter card	38	No
6-port FXS Adapter card	38	No
6-port Ethernet 10Gbps Adapter card	1177	Yes
8-port Ethernet Adapter card	156	Yes
8-port FXO Adapter card	38	No
8-port Gigabit Ethernet Adapter card	268	Yes
8-port Voice & Teleprotection card	38	No
10-port 1GigE/1-port 10GigE X-Adapter card	537	Yes
12-port Serial Data Interface card, version 1	38	No
12-port Serial Data Interface card, version 2	268	Yes
16-port T1/E1 ASAP Adapter card	38	No
32-port T1/E1 ASAP Adapter card	57	No
Integrated Services card	268	Yes
Packet Microwave Adapter card	268	Yes
7705 SAR-A	268	Yes
7705 SAR-Ax	268	Yes
7705 SAR-H	268	Yes
7705 SAR-Hc	268	Yes
7705 SAR-M	268	Yes
7705 SAR-W	268	Yes
7705 SAR-Wx	268	Yes
7705 SAR-X (Ethernet ports) 1	1177	Yes
7705 SAR-X (TDM ports) 1	46	Yes

Note:

1. The 7705 SAR-X has three buffer pools. Each block of ports (MDA) has its own buffer pool.

3.1.6.3.1. Advantages of Buffer Chaining

Buffer chaining offers improved efficiency, which is especially evident when smaller packet sizes are transmitted. For example, to queue a 64-byte packet, a card with a fixed buffer of 2304 bytes allocates 2304 bytes, whereas a card with a fixed buffer of 512 bytes allocates only 512 bytes. To queue a 1280-byte packet, a card with a fixed buffer of 2304 bytes allocates 2304 bytes, whereas a card with a fixed buffer of 512 bytes allocates only 1536 bytes (that is, 512 bytes × 3 buffers).

3.1.7. Per-SAP Aggregate Shapers (H-QoS) On Gen-2 Hardware

This section contains overview information as well as information on the following topics:

This section provides information on per-SAP aggregate shapers for Gen-2 adapter cards and platforms. For information on Gen-3 adapter cards and platforms, see QoS for Gen-3 Adapter Cards and Platforms.

Hierarchical QoS (H-QoS) provides the 7705 SAR with the ability to shape traffic on a per-SAP basis for traffic from up to eight CoS queues associated with that SAP.

On Gen-2 hardware, the per-SAP aggregate shapers apply to access ingress and access egress traffic and operate in addition to the 16-priority scheduler, which must be used for per-SAP aggregate shaping.

The 16-priority scheduler acts as a soft policer, servicing the SAP queues in strict priority order, with conforming traffic (less than CIR) serviced prior to non-conforming traffic (between CIR and PIR). The 16-priority scheduler on its own cannot enforce a traffic limit on a per-SAP basis; to do this, per-SAP aggregate shapers are required (see H-QoS Example).

The per-SAP shapers are considered aggregate shapers because they shape traffic from the aggregate of one or more CoS queues assigned to the SAP.

Figure 3 and Figure 4 illustrate per-SAP aggregate shapers for access ingress and access egress, respectively. They indicate how shaped and unshaped SAPs are treated.

H-QoS is not supported on the 4-port SAR-H Fast Ethernet module.

3.1.7.1. Shaped and Unshaped SAPs

Shaped SAPs have user-configured rate limits (PIR and CIR)—called the aggregate rate limit—and must use 16-priority scheduling mode. Unshaped SAPs use default rate limits (PIR is maximum and CIR is 0 kb/s) and can use 4-priority or 16-priority scheduling mode.

Shaped 16-priority SAPs are configured with a PIR and a CIR using the agg-rate-limit command in the config>service>service-type service-id>sap context, where service-type is epipe, ipipe, ies, vprn, or vpls (including routed VPLS). The PIR is set using the agg-rate variable and the CIR is set using the cir-rate variable.

Unshaped 4-priority SAPs are considered unshaped by definition of the default PIR and CIR values (PIR is maximum and CIR is 0 kb/s). Therefore, they do not require any configuration other than to be set to 4-priority scheduling mode.

Unshaped 16-priority SAPs are created when 16-priority scheduling mode is selected, when the default PIR is maximum and the default CIR is 0 kb/s, which are same default settings of a 4-priority SAP. The main reason for preferring unshaped SAPs using 16-priority scheduling over unshaped SAPs using 4-priority scheduling is to have a coherent scheduler behavior (one scheduling model) across all SAPs.

In order for unshaped 4-priority SAPs to compete fairly for bandwidth with 16-priority shaped and unshaped SAPs, a single, aggregate CIR for all the 4-priority SAPs can be configured. This aggregate CIR is applied to all the 4-priority SAPs as a group, not to individual SAPs. In addition, the aggregate CIR is configured differently for access ingress and access egress traffic. On the 7705 SAR-8 and 7705 SAR-18, access ingress is configured in the config>qos>fabric-profile context. On the 7705 SAR-M, 7705 SAR-H, 7705 SAR-Hc, 7705 SAR-A, 7705 SAR-Ax, 7705 SAR-W, and 7705 SAR-Wx, access ingress is configured in the config>system>qos>access-ingress-aggregate-rate context. For all platforms, access egress configuration uses the config>port>ethernet context.

For more information about access ingress scheduling and traffic arbitration from the 16-priority and 4-priority schedulers toward the fabric, see Access Ingress Per-SAP Aggregate Shapers (Access Ingress H-QoS).

Figure 3:  Access Ingress Scheduling for 4-Priority and 16-Priority SAPs (with Per-SAP Aggregate Shapers) 

Figure 4:  Access Egress Scheduling for 4-Priority and 16-Priority SAPs (with Per-SAP Aggregate Shapers) (per Port) 

3.1.7.1.1. Per-SAP Aggregate Shaper Support

The per-SAP aggregate shapers are supported in both access ingress and access egress directions and can be enabled on the following Ethernet access ports:

1. 6-port Ethernet 10Gbps Adapter card
2. 8-port Gigabit Ethernet Adapter card
3. 10-port 1GigE/1-port 10GigE X-Adapter card (10-port GigE mode)
4. Packet Microwave Adapter card
5. 7705 SAR-A
6. 7705 SAR-Ax
7. 7705 SAR-M
   All Ethernet access ports except those on the following modules:
   1. 8-port xDSL module
   2. GPON module
   3. 6-port DSL Combination module (SHDSL ports)
8. 7705 SAR-H
   1. All Ethernet access ports except those on the 4-port SAR-H Fast Ethernet module
9. 7705 SAR-Hc
10. 7705 SAR-W
11. 7705 SAR-Wx
12. 7705 SAR-X

3.1.8. Per-VLAN Network Egress Shapers

This section provides information on per-VLAN network egress shapers for Gen-1 and Gen-2 adapter cards and platforms. For information on Gen-3 adapter cards and platforms, see QoS for Gen-3 Adapter Cards and Platforms.

The 7705 SAR supports a set of eight network egress queues on a per-port or on a per-VLAN basis for network Ethernet ports. Eight unique per-VLAN CoS queues are created for each VLAN when a per-VLAN shaper is enabled. When using per-VLAN shaper mode, in addition to the per-VLAN eight CoS queues, there is a single set of eight queues for hosting traffic from all unshaped VLANs, if any. VLAN shapers are enabled on a per-interface basis (that is, per VLAN) when a network queue policy is assigned to the interface. See Per-VLAN Shaper Support for a list of cards and nodes that support per-VLAN shapers.

On a network port with dot1q encapsulation, shaped and unshaped VLANs can coexist. In such a scenario, each shaped VLAN has its own set of eight CoS queues and is shaped with its own configured dual-rate shaper. The remaining VLANs (that is, the unshaped VLANs) are serviced using the unshaped-if-cir rate, which is configured using the config>port>ethernet>network>egress>unshaped-if-cir command. Assigning a rate to the unshaped VLANs is required for arbitration between the shaped VLANs and the bulk (aggregate) of unshaped VLANs, where each shaped VLAN has its own shaping rate while the aggregate of the unshaped VLANs has a single rate assigned to it.

Per-VLAN shapers are supported on dot1q-encapsulated ports. They are not supported on null- or qinq-encapsulated ports.

Figure 5 illustrates the queuing and scheduling blocks for network egress VLAN traffic.

Note: Due to space limitations in Figure 5, the second-tier, per-VLAN aggregate shapers are represented as a single loop containing the label “per VLAN”, even though they are dual-rate shapers similar to the third-tier network aggregate shaper.

Figure 5:  Network Egress Shaped and Unshaped VLAN Queuing and Scheduling 

3.1.8.1. Shaped and Unshaped VLANs

Shaped VLANs have user-configured rate limits (PIR and CIR)—called the aggregate rate limits—and must use 16-priority scheduling mode. Shaped VLANs operate on a per-interface basis and are enabled after a network queue policy is assigned to the interface. If a VLAN does not have a network queue policy assigned to the interface, it is considered an unshaped VLAN.

Note: On the 8-port Ethernet Adapter card (version 2), profiled scheduling must be used for VLAN shaping. For these cases, fewer shaped VLANs are supported.

To configure a shaped VLAN with aggregate rate limits, use the agg-rate-limit command in the config>router>if>egress context. If the VLAN shaper is not enabled, then the agg-rate-limit settings do not apply. The default aggregate rate limit (PIR) is set to the port egress rate.

Unshaped VLANs use default rate limits (PIR is the maximum possible port rate and CIR is 0 kb/s) and use 16-priority scheduling mode. All unshaped VLANs are classified, queued, buffered, and scheduled into an aggregate flow that gets prepared for third-tier arbitration by a single VLAN aggregate shaper.

In order for the aggregated unshaped VLANs to compete fairly for bandwidth with the shaped VLANs, a single, aggregate CIR for all the unshaped VLANs can be configured using the unshaped-if-cir command. The aggregate CIR is applied to all the unshaped VLANs as a group, not to individual VLANs, and is configured in the config>port>ethernet> network>egress context.

3.1.8.2. Per-VLAN Shaper Support

The following cards and nodes support network egress per-VLAN shapers:

1. 6-port Ethernet 10Gbps Adapter card
2. 8-port Gigabit Ethernet Adapter card
3. 10-port 1GigE/1-port 10GigE X-Adapter card (1-port 10GigE mode and 10-port 1GigE mode)
4. Packet Microwave Adapter card (includes 1+1 redundancy)
5. v-port on the 2-port 10GigE (Ethernet) Adapter card/module
6. 7705 SAR-A
7. 7705 SAR-Ax
8. 7705 SAR-H
9. 7705 SAR-Hc
10. 7705 SAR-M
11. 7705 SAR-W
12. 7705 SAR-Wx
13. 7705 SAR-X
14. 8-port Ethernet Adapter card (version 2) (Gigabit Ethernet ports only; egress-rate cannot be configured)

Note: Per-VLAN network egress shapers are not supported on: Fast Ethernet ports (including ports 9 to 12 on the 7705 SAR-A) Gigabit Ethernet ports in Fast Ethernet mode non-datapath Ethernet ports (for example, the Management port) GPON, xDSL, and SHDSL module slots DSL ports on the 7705 SAR-Wx For the 8-port Ethernet Adapter card, version 2, the following rules apply: To configure per-VLAN shapers, the profile scheduler mode must be in use (that is, rate-based scheduling). Per-VLAN shapers are only supported on Gigabit Ethernet ports (not on Fast Ethernet ports). Setting egress-rate and using per-VLAN shapers are mutually exclusive. If egress-rate is configured, then per-VLAN shapers cannot be configured. If one or more per-VLAN shapers are configured, then egress-rate cannot be enabled. A maximum of eight per-VLAN shapers can be configured on an 8-port Ethernet Adapter card, version 2. On an 8-port Ethernet Adapter card, version 2, installed in an 7705 SAR-18, when a Fast Ethernet port is configured as a network port with profile scheduler mode, the number of per-VLAN shapers supported on the card is reduced by one.

3.1.8.3.  VLAN Shaper Applications

This section describes the following two scenarios:

1. VLAN Shapers for Dual Uplinks
2. VLAN Shapers for Aggregation Site

3.1.8.3.1. VLAN Shapers for Dual Uplinks

One of the main uses of per-VLAN network egress shapers is to enable load balancing across dual uplinks out of a spoke site. Figure 6 represents a typical hub-and-spoke mobile backhaul topology. To ensure high availability through the use of redundancy, a mobile operator invests in dual 7750 SR nodes at the MTSO. Dual 7750 SR nodes at the MTSO offer equipment protection, as well as protection against backhaul link failures.

In this example, the cell site 7705 SAR is dual-homed to 7750 SR_1 and SR_2 at the MTSO, using two disjoint Ethernet virtual connections (EVCs) leased from a transport provider. Typically, the EVCs have the same capacity and operate in an forwarding/standby manner. One of the EVCs—the 7750 SR—transports all the user/mobile traffic to and from the cell site at any given time. The other EVC transports only minor volumes of control plane traffic between network elements (the 7705 SAR and the 7750 SR). Leasing two EVCs with the same capacity and using only one of them actively wastes bandwidth and is expensive (the mobile operator pays for two EVCs with the same capacity).

Mobile operators with increasing volumes of mobile traffic look for ways to utilize both of the EVCs simultaneously, in an active/active manner. In this case, using per-VLAN shapers would ensure that each EVC is loaded up to the leased capacity. Without per-VLAN shapers, the 7705 SAR supports a single per-port shaper, which does not meet the active/active requirement.

1. If the egress rate is set to twice the EVC capacity, then either one of the EVCs can end up with more traffic than its capacity.
2. If the egress rate is set to the EVC capacity, then half of the available bandwidth can never be consumed, which is similar to the 7705 SAR having no per-VLAN egress shapers.

Figure 6:  VLAN Shapers for Dual Uplinks 

3.1.8.3.2. VLAN Shapers for Aggregation Site

Another typical use of per-VLAN shapers at network egress is shown in Figure 7. The figure shows a hub-and-spoke mobile backhaul network where EVCs leased from a transport provider are groomed to a single port, typically a 10-Gigabit Ethernet or a 1-Gigabit Ethernet port, at the hand-off point at the hub site. Traffic from different cell sites is handed off to the aggregation node over a single port, where each cell site is uniquely identified by the VLAN assigned to it.

In the network egress direction of the aggregation node, per-VLAN shaping is required to ensure traffic to different cell sites is shaped at the EVC rate. The EVC for each cell site would typically have a different rate. Therefore, every VLAN feeding into a particular EVC needs to be shaped at its own rate. For example, compare a relatively small cell site (Cell Site-1) at 20 Mb/s rate with a relatively large cell site (Cell Site-2) at 200 Mb/s rate. Without the granularity of per-VLAN shaping, shaping only at the per-port level cannot ensure that an individual EVC does not exceed its capacity.

Figure 7:  VLAN Shapers in Aggregation Site Scenario 

3.1.9. Per-Customer Aggregate Shapers (Multiservice Site) on Gen-2 Hardware

This section provides information on per-customer aggregate shapers for Gen-2 adapter cards and platforms. For information on Gen-3 adapter cards and platforms, see QoS for Gen-3 Adapter Cards and Platforms.

A per-customer aggregate shaper is an aggregate shaper into which multiple SAP aggregate shapers can feed. The SAPs can be shaped at a desired rate called the Multiservice Site (MSS) aggregate rate. At ingress, SAPs that are bound to a per-customer aggregate shaper can span a whole Ethernet MDA meaning that SAPs mapped to the same MSS can reside on any port on a given Ethernet MDA. At egress, SAPs that are bound to a per-customer aggregate shaper can only span a port. Towards the fabric at ingress and towards the port at egress, multiple per-customer aggregate shapers are shaped at their respective configured rates to ensure fair sharing of available bandwidth among different per-customer aggregate shapers. Deep ingress queuing capability ensures that traffic bursts are absorbed rather than dropped. Multi-tier shapers are based on an end-to-end backpressure mechanism that uses the following order (egress is given as an example):

To configure per-customer aggregate shaping, a shaper policy must be created and shaper groups must be created within that shaper policy. For access ingress per-customer aggregate shaping, a shaper policy must be assigned to an Ethernet MDA and SAPs on that Ethernet MDA must be bound to a shaper group within the shaper policy bound to that Ethernet MDA. For access egress per-customer aggregate shaping, a shaper policy must be assigned to a port and SAPs on that port must be bound to a shaper group within the shaper policy bound to that port. The unshaped SAP shaper group within the policy provides the shaper rate for all the unshaped SAPs (4-priority scheduled SAPs). For each shaped SAP, however, an ingress or egress shaper group can be specified. For more information on shaper policies, see Applying a Shaper QoS Policy and Shaper Groups.

The access ingress shaper policy is configured at the MDA level for fixed platforms. The default value for an access ingress shaper policy for each MDA and module is blank, as configured using the no shaper-policy command. On all 7705 SAR fixed platforms (with the exception of the 7705 SAR-X), when no MSS is configured, the existing access ingress aggregate rate is used as the shaper rate for the bulk of access ingress traffic. In order to use MSS, a shaper policy must be assigned to the access ingress interface of one MDA, and the shaper policy change is cascaded to all MDAs and modules in the chassis.

Before the access ingress shaper policy is assigned, the config system qos access-ingress-aggregate-rate 10000000 unshaped-sap-cir max command must be configured. Once a shaper policy is assigned to an access ingress MDA, the values configured using the access-ingress-aggregate-rate command cannot be changed.

On all 7705 SAR fixed platforms (with the exception of the 7705 SAR-X), when a shaper policy is assigned to an Ethernet MDA for access ingress aggregate shaping, it is automatically assigned to all the Ethernet MDAs in that chassis. The shaper group members contained in the shaper policy span all the Ethernet MDAs. SAPs on different Ethernet MDAs configured with the same ingress shaper group will share the shaper group rate.

Once the first MSS is configured, traffic from the designated SAPs is mapped to the MSS and shaped at the configured rate. The remainder of the traffic is then shaped according to the configured unshaped SAP rate. When a second MSS is added, SAPs that are mapped to the second MSS are shaped at the configured rate and traffic is arbitrated between the first MSS, the second MSS and unshaped SAP traffic.

In the access egress direction, the default shaper policy is assigned to each MSS-capable port. Ports that cannot support MSS are assigned a blank value, as configured using the no shaper-policy command. The default egress shaper group is assigned to each egress SAP that supports MSS. If the SAP does not support MSS, the egress SAP is assigned a blank value, as configured using the no shaper-group command.

3.1.9.1. MSS Support

The following cards, modules, and platforms support MSS:

1. 6-port Ethernet 10Gbps Adapter card
2. 8-port Gigabit Ethernet Adapter card
3. 10-port 1GigE/1-port 10GigE X-Adapter card
4. Packet Microwave Adapter card
5. 6-port SAR-M Ethernet module
6. 7705 SAR-A
7. 7705 SAR-Ax
8. 7705 SAR-H
9. 7705 SAR-Hc
10. 7705 SAR-M
11. 7705 SAR-W
12. 7705 SAR-Wx
13. 7705 SAR-X

Note: MSS is not supported on the following: 4-port SAR-H Fast Ethernet module Fast Ethernet ports on the 7705 SAR-A 6-port DSL Combination module 8-port xDSL module GPON module DSL ports on the 7705 SAR-Wx

3.1.10. QoS for Hybrid Ports on Gen-2 Hardware

This section provides information on QoS for hybrid ports on Gen-2 adapter cards and platforms. For information on Gen-3 adapter cards and platforms, see QoS for Gen-3 Adapter Cards and Platforms.

In the ingress direction of a hybrid port, traffic management behavior is the same as it is for access and network ports. See Access Ingress and Network Ingress.

In the egress direction of a hybrid port, access and network aggregate shapers are used to arbitrate between the bulk (aggregate) of access and network traffic flows. As shown in Figure 8, on the access side (above the solid line), both the access egress SAP aggregates (#1) and the unshaped SAP shaper (#2) feed into the access egress aggregate shaper (#3). On the network side (below the solid line), both the per-VLAN shapers (#4) and the unshaped interface shaper (#5) feed into the network egress aggregate shaper (#6). Then, the access and the network aggregate shapers are arbitrated in a dual-rate manner, in accordance with their respective configured committed and peak rates (#7). As a last step, the egress-rate for the port (when configured) applies backpressure to both the access and the network aggregate shapers, which apply backpressure all the way to the FC queues belonging to both access and network traffic.

Note: Due to space limitations in Figure 8, the second-tier, per-SAP and per-VLAN aggregate shapers are represented as a single loop containing the label “per SAP” or “per VLAN”, even though they are dual-rate shapers similar to the third-tier network aggregate shaper. Tiers are labeled at the top of the figure.

As part of the hybrid port traffic management solution, access and network second-tier shapers are bound to access and network aggregate shapers, respectively. The hybrid port egress datapath can be visualized as access and network datapaths that coexist separately up until the access and network aggregate shapers at Tier 3 (#3 and #6).

In Figure 8, the top half is identical to access egress traffic management, where CoS queues (Tier 1) feed into either per-SAP shapers for shaped SAPs (#1) or a single second-tier shaper for all unshaped SAPs (#2). Up to the end of the second-tier, per-SAP aggregate shapers, the access egress datapath is maintained in the same manner as an Ethernet port in access mode. The same logic applies for network egress. The bottom half of the figure shows the datapath from the CoS queues to the per-VLAN shapers, which is identical to the datapath for any other Ethernet port in network mode.

The main difference between hybrid mode and access and network modes is shown when the access and the network traffic is arbitrated towards the port (Tier 3). At this point, a new set of dual-rate shapers (called shaper groups) are introduced: one shaper for the aggregate (bulk) of the access traffic (#3) and another shaper for the aggregate of the network traffic (#6) —to ensure rate-based arbitration among access and network traffic.

Depending on the use and the application, the committed rate for any one mode of flow might need to be fine-tuned to minimize delay, jitter and loss. In addition, through the use of egress-rate limiting, a fourth level of shaping can be achieved.

When egress-rate is configured (under config>port>ethernet), the following events occur:

Figure 8:  Hybrid Port Egress Shapers and Schedulers on Gen-2 Hardware 

3.1.11. QoS for Gen-3 Adapter Cards and Platforms

Third-generation (Gen-3) Ethernet adapter cards and Ethernet ports on Gen-3 platforms support 4-priority scheduling.

The main differences between Gen-3 hardware and Gen-1 and Gen-2 hardware are that on Gen-3 hardware:

See Scheduling Modes for a summary of scheduler mode support. For information on adapter card generations, refer to the “Evolution of Ethernet Adapter Cards, Modules, and Platforms” section in the 7705 SAR Interface Configuration Guide.

Note: From Release 7.0.R6, schedulers on Gen-3 adapter cards and platforms have been updated to better align with the scheduling behavior supported on the 7705 SAR adapter cards and platforms. The updated scheduler mode is called “4-priority” scheduler-mode throughout the CLI. Prior to Release 7.0.R6, the CLI designation was “4-priority-hqos”. In the updated mode of operation, arbitration among different flows at the second-tier aggregate (per-SAP or per-VLAN) and third-tier aggregate (per-customer (MSS)) levels are carried out in a round-robin manner, and scheduling is done on a per-SAP basis.

Caution: Any Gen-3 adapter card or platform running Release 7.0.R6 or later software uses 4-priority scheduling instead of 4-priority-hqos scheduling, which was supported previously. The migration of scheduler mode is automatic with an upgrade and there is no operator action required. As part of the migration, all CIR values at second-tier (per-SAP and per-VLAN) and third-tier (per-customer (MSS)) aggregate shaper levels are set to zero. Operators must exercise caution when performing an upgrade to Release 7.0.R6 or later from a previous Release 7.0 version, and must adjust the affected CIR values in accordance with the needs of their applications as soon as possible. The shapers with affected CIR values are listed later in this section.

Table 5 describes the access, network, and hybrid port scheduling behavior after upgrading to 7705 SAR Release 7.0.R6 (or later) for Gen-3 hardware, and compares it with the scheduling behavior of Gen-2 and Gen-1 hardware.

In summary, with Release 7.0.R6 and later, the following updates to Gen-3 scheduling are implemented:

Note: For network egress traffic when the port is in network mode, there is no change to CIR-based shaping for per-VLAN shapers (that is, PIR-based shaping only, CIR-based shaping is not enabled), and backpressure to the FC queues based on the relative priority among all VLAN-bound IP interfaces is still enabled.

3.1.11.2. 4-Priority Scheduling Behavior on Gen-3 Hardware

For access ingress, access egress, and network ingress traffic, the behavior of 4-priority scheduling on Gen-3 hardware is similar to 4-priority scheduling on Gen-1 and Gen-2 hardware. See Figure 9 (access ingress), Figure 10 (access egress), Figure 11 (network ingress, destination mode), and Figure 12 (network ingress, aggregate mode).

For network egress traffic through a network port on Gen-3 hardware, the behavior of 4-priority scheduling is as follows: traffic priority is determined at the queue-level scheduler, which is based on the queue PIR and CIR and the queue type. The queue-level priority is carried through the various shaping stages and is used by the 4-priority Gen-3 VLAN scheduler at network egress. See Figure 13 and its accompanying description.

For hybrid ports, both access and network egress traffic use 4-priority scheduling that is similar to 4-priority scheduling on Gen-1 and Gen-2 hardware. See Figure 14 and its accompanying description.

Note: The 7705 SAR-X defaults to 4-priority mode and does not support fabric shapers. Traffic from fabric shapers (access and network) are arbitrated in a round-robin manner towards the egress datapath.

In Figure 9, the shaper groups all belong within one shaper policy and only one shaper policy is assigned to an ingress adapter card. Each SAP can be associated with one shaper group. Multiple SAPs can be associated with the same shaper group. All the access ingress traffic flows to the access ingress fabric shaper, in-profile (conforming) traffic first, then out-of-profile (non-conforming) traffic. Network ingress traffic behaves similarly.

The 4-priority schedulers on the Gen-1/Gen-2 and Gen-3 hardware are very similar, except that 4-priority scheduling on Gen-3 hardware is done on a per-SAP basis.

Figure 9:  4-Priority Scheduling at Access Ingress (Gen-3 Hardware) 

Figure 10 shows 4-priority scheduling for access egress on Gen-3 hardware. QoS behavior for access egress is similar to QoS behavior for access ingress.

The 4-priority schedulers on the Gen-1/Gen-2 and Gen-3 hardware are very similar, except that 4-priority scheduling on Gen-3 hardware is done on a per-SAP basis.

Figure 10:  4-Priority Scheduling at Access Egress (Gen-3 Hardware) 

Figure 11 and Figure 12 show network ingress scheduling for per-destination and aggregate modes, which are configured under the fabric-profile command. Traffic arriving on a network port is examined for its destination MDA and directed to the QoS block that sends traffic to the appropriate MDA. There is one set of queues for each block, and an additional set for multipoint traffic.

In Figure 11, there is one per-destination shaper for each destination MDA. In Figure 12, there is a single shaper to handle all the traffic.

Figure 11:  4-Priority Scheduling at Network Ingress (Gen-3 Hardware): Per-Destination Mode 

Figure 12:  4-Priority Scheduling at Network Ingress (Gen-3 Hardware): Aggregate Mode 

Figure 13 shows 4-priority scheduling for Gen-3 hardware at network egress. Queue-level CIR and PIR values and the queue type are determined at queuing and provide the scheduling priority for a given flow across all shapers towards the egress port (#1 in Figure 13). At the per-VLAN aggregate level (#2), only a single rate—the total aggregate rate (PIR)—can be configured; CIR configuration is not supported at the per-VLAN aggregate-shaper level for network egress traffic. All VLANs are aggregated and scheduled by a 4-priority aggregate scheduler (#3). The flow is then fed to the port shaper and processed at the egress rate. In case of congestion, the port shaper provides backpressure, resulting in the buffering of traffic by individual FC queues until the congested state ends.

Note: The behavior of 4-priority scheduling for network egress traffic through a network port differs from the behavior for network egress traffic through a hybrid port. Figure 13 and Figure 14 show the differences.

Figure 13:  4-Priority Scheduling at Network Egress (Gen-3 Hardware) on a Network Port 

Figure 14 shows 4-priority scheduling for Gen-3 hardware where ports are in hybrid mode. The QoS behavior for both access and network egress traffic is similar except that the access egress path includes tier 3, per-customer aggregate shapers. Access and network shapers prepare and present traffic to the port shaper, which arbitrates between access and network traffic.

The 4-priority schedulers on the Gen-1/Gen-2 and Gen-3 hardware are very similar, except that 4-priority scheduling on Gen-3 hardware is done on a per-SAP or a per-VLAN basis (for access egress and network egress, respectively).

Figure 14:  4-Priority Scheduling for Hybrid Port Egress (Gen-3 Hardware) 

3.1.12. QoS on a Ring Adapter Card or Module

This section contains overview information as well as information on the following topics:

Figure 15 shows a simplified diagram of the ports on a 2-port 10GigE (Ethernet) Adapter card (also known as a ring adapter card). The ports can also be conceptualized the same way for a 2-port 10GigE (Ethernet) module. A ring adapter card or module has physical Ethernet ports used for Ethernet bridging in a ring network (labeled Port 1 and Port 2 in the figure). These ports are referred to as the ring ports because they connect to the Ethernet ring. The ring ports operate on the Layer 2 bridging domain side of the ring adapter card or module, as does the add/drop port, which is an internal port on the card or module.

On the Layer 3 IP domain side of a ring adapter card or module, there is a virtual port (v-port) and a fabric port. The v-port is also an internal port. Its function is to help control traffic on the IP domain side of a ring adapter card or module.

To manage ring and add/drop traffic mapping to queues in the L2 bridging domain, a ring type network QoS policy can be configured for the ring at the adapter card level (under the config>card>mda context). To manage ring and add/drop traffic queuing and scheduling in the L2 bridging domain, network queue QoS policies can be configured for the ring ports and the add-drop port.

To manage add/drop traffic classification and remarking in the L3 IP domain, ip-interface type network QoS policies can be configured for router interfaces on the v-port. To manage add/drop traffic queuing and scheduling in the L3 IP domain, network queue QoS policies can be configured for the v-port and at network ingress at the adapter card level (under the config>card>mda context).

Figure 15:  Ports on a 2-port 10GigE (Ethernet) Adapter Card 

3.1.13. QoS for IPSec Traffic

For specific information about QoS for IPSec traffic, refer to the “QoS” section in the “IPSec” chapter in the 7705 SAR Services Guide.

3.1.14. QoS for Network Group Encryption Traffic

The 7705 SAR provides priority and scheduling for traffic into the encryption and decryption engines on nodes that support network group encryption (NGE). This applies to traffic at network ingress or network egress.

For specific information, refer to the “QoS for NGE Traffic” section in the “Network Group Encryption” chapter in the 7705 SAR Services Guide.

3.2.1. Access Ingress Traffic Classification

Traffic classification identifies a traffic flow and maps the packets belonging to that flow to a preset forwarding class, so that the flow can receive the required special treatment. Up to eight forwarding classes are supported for traffic classification. Refer to Table 2 for a list of these forwarding classes.

For TDM channel groups, all of the traffic is mapped to a single forwarding class. Similarly, for ATM VCs, each VC is linked to one forwarding class. On Ethernet ports and VLANs, up to eight forwarding classes can be configured based on 802.1p (dot1p) bits or DSCP bits classification. On PPP/MLPPP, FR (for Ipipes), or cHDLC SAPs, up to eight forwarding classes can be configured based on DSCP bits classification. FR (for Fpipes) and HDLC SAPs are mapped to one forwarding class.

Note: If an Ethernet port is set to null encapsulation, the dot1p value has no meaning and cannot be used for classification purposes. If a port or SAP is set to qinq encapsulation, use the match-qinq-dot1p top | bottom command to indicate which qtag contains the dot1p bits that are used for classification purposes. The match-qinq-dot1p command is found under the config>service context. Refer to the 7705 SAR Services Guide, “VLL Services Command Reference”, for details.

Once the classification takes place, forwarding classes are mapped to queues as described in the sections that follow.

3.2.2. Access Ingress Queues

Once the traffic is classified to different forwarding classes, the next step is to create the ingress queues and bind forwarding classes to these queues.

There is no restriction of a one-to-one association between a forwarding class and a queue. That is, more than one forwarding class can be mapped to the same queue. This capability is beneficial in that it allows a bulk-sum amount of resources to be allocated to traffic flows of a similar nature. For example, in the case of 3G UMTS services, HSDPA and OAM traffic are both considered BE in nature. However, HSDPA traffic can be mapped to a better forwarding class (such as L2) while OAM traffic can remain mapped to a BE forwarding class. But they both can be mapped to a single queue to control the total amount of resources for the aggregate of the two flows.

A large but finite amount of memory is available for the queues. Within this memory space, many queues can be created. The queues are defined by user-configurable parameters. This flexibility and complexity is necessary in order to create services that offer optimal quality of service and is much better than a restrictive and fixed buffer implementation alternative.

Memory allocation is optimized to guarantee the CBS for each queue. The allocated queue space beyond the CBS that is bounded by the MBS depends on the usage of buffer space and existing guarantees to queues (that is, the CBS). The CBS is defaulted to 8 kilobytes (for 512 byte buffer size) or 18 kilobytes (for 2304 byte buffer size) for all access ingress queues on the 7705 SAR. With a small default queue depth (CBS) allocated for each queue, all services at full scale are guaranteed to have buffers for queuing. The default value would need to be altered to meet the requirements of a specific traffic flow or flows.

3.2.3. Access Ingress Queuing and Scheduling

Traffic management on the 7705 SAR uses a packet-based implementation of the dual leaky bucket model. Each queue has a guaranteed space limited with CBS and a maximum depth limited with MBS. New packets are queued as they arrive. Any packet that causes the MBS to be exceeded is discarded.

The packets in the queue are serviced by two different profiled (rate-based) schedulers, the In-Profile and Out-of-Profile schedulers, where CIR traffic is scheduled before PIR traffic. These two schedulers empty the queue continuously.

For 4-priority scheduling, rate-based schedulers (CIR and PIR) are combined with queue-type schedulers (EXP or BE). For 16-priority scheduling, the rate-based schedulers are combined with the strict priority schedulers (CoS-8 queue first to CoS-1 queue last).

Note: For access ingress and egress, the 16-priority schedulers use additional hardware resources and capabilities, which results in increased throughput.

Access ingress scheduling is supported on the adapter cards and ports listed in Table 8. The supported scheduling modes are 4-priority scheduling and 16-priority scheduling. Table 8 shows which scheduling mode each card and port supports at access ingress.

This section also contains information on the following topics:

3.2.3.1. Profiled (Rate-based) Scheduling

Each queue is serviced based on the user-configured CIR and PIR values. If the packets that are collected by a scheduler from a queue are flowing at a rate that is less than or equal to the CIR value, then the packets are scheduled as in-profile. Packets with a flow rate that exceeds the CIR value but is less than the PIR value are scheduled as out-of-profile. Figure 16 depicts this behavior by the “In-Prof” and “Out-Prof” labels. This behavior is comparable to the dual leaky bucket implementation in ATM networks. With in-profile and out-of-profile scheduling, traffic that flows at rates up to the traffic contract (that is, CIR) from all the queues is serviced prior to traffic that flows at rates exceeding the traffic contract. This mode of operation ensures that Service Level Agreements are honored and traffic that is committed to be transported is switched prior to traffic that exceeds the contract agreement.

Note: A profile is an arithmetical analysis of the rates that are permitted for a particular packet flow; therefore, profiled scheduling may also be called rate-based scheduling.

3.2.3.2. Queue-Type Scheduling

As well as profiled scheduling described above, queue-type scheduling is supported at access ingress. Queues are divided into two categories, those that are serviced by the Expedited scheduler and those that are serviced by the Best Effort scheduler.

The Expedited scheduler has precedence over the Best Effort scheduler. Thus, at access ingress, CoS queues that are marked with an Expedited priority are serviced first. Then, the Best Effort marked queues are serviced. In a default configuration, the Expedited scheduler services the following CoS queues before the Best Effort scheduler services the rest:

1. Expedited scheduler: NC, H1, EF, H2
2. Best-Effort scheduler: L1, AF, L2, BE

If a packet with an Expedited forwarding class arrives while a Best Effort marked queue is being serviced, then the Expedited Scheduler will take over and service the Expedited marked CoS queue, as soon as the packet from the Best Effort scheduler is serviced.

The schedulers at access ingress in the 7705 SAR service the group of all Expedited queues exhaustively ahead of the group of all Best Effort queues. This means that all expedited queues have to be empty before any packet from a Best Effort queue is serviced.

Note: There is no user configuration for the schedulers. The operation of the schedulers is described for informational purposes. A user can control the mapping of traffic flows based on classification controls, that is, by mapping forwarding classes to as many as eight CoS queues.

The following basic rules apply to the queue-type scheduling of CoS queues.

1. Queues marked for Expedited scheduling are serviced in a round-robin fashion before any queues marked as Best Effort (in a default configuration, these would be queues CoS-8 through CoS-5).
2. These Expedited queues are serviced exhaustively within the round robin. For example, if in a default configuration there are two packets scheduled for service in both CoS-8 and CoS-5, one packet from CoS-8 is serviced, then one packet from CoS-5 is serviced, and then the scheduler returns back to CoS-8, until all the packets are serviced.
3. Once the Expedited scheduler has serviced all the packets in the queues marked for Expedited scheduling, the Best Effort scheduler starts serving the queues marked as Best Effort. The same principle described in step 2 is followed, until all the packets in the Best Effort queues are serviced.
4. If a packet arrives at any of the queues marked for Expedited scheduling while the scheduler is servicing a packet from a Best Effort queue, the Best Effort scheduler finishes servicing the current packet and then the Expedited scheduler immediately activates to service the packet in the Expedited queue. If there are no other packets to be serviced in any of the Expedited queues, the Best Effort scheduler resumes servicing the packets in the Best Effort queues. If the queues are configured according to the tables and defaults described in this guide, CoS-4 will be scheduled prior to CoS-1 among queues marked as Best Effort.
5. After one cycle is completed across all the queues marked as Best Effort, the next pass is started until all the packets in all the queues marked as Best Effort are serviced, or a packet arrives to a queue marked as Expedited and is serviced as described in step 2.

3.2.4. Access Ingress Per-SAP Aggregate Shapers (Access Ingress H-QoS)

On the 7705 SAR, H-QoS adds second-tier (or second-level), per-SAP aggregate shapers. As shown in Figure 17, traffic ingresses at an Ethernet SAP and is classified and mapped to up to eight different CoS queues on a per-ingress SAP basis. The aggregate rate CIR and PIR values are then used to shape the traffic. The conforming loop (aggregate CIR loop) schedules the packets out of the eight CoS queues in strict priority manner (queue priority CIRs followed by queue priority PIRs). If the aggregate CIR is crossed at any time during the scheduling operation, regardless of the per-queue CIR/PIR configuration, then the aggregate conforming loop for the SAP ends and the aggregate non-conforming loop begins. The aggregate non-conforming loop schedules the packets out of the eight CoS queues in strict priority manner. SAPs sending traffic to the 4-priority scheduler do not have a second-tier per-SAP aggregate shaper unless traffic arbitration is desired, in which case an aggregate CIR for all the 4-priority SAPs can be configured (see Access Ingress Per-SAP Shapers Arbitration). See Per-SAP Aggregate Shapers (H-QoS) On Gen-2 Hardware for general information.

The aggregate rate limit for the per-SAP aggregate shaper is configured in the service context, using the sap>ingress>agg-rate-limit or sap>egress>agg-rate- limit command.

For per-SAP aggregate shaping on Gen-2 adapter cards, the SAP must be scheduled using a 16-priority scheduler.

Note: The default setting for scheduler-mode is 4-priority on Gen-2 adapter cards and platforms. The user must toggle the scheduling mode to 16-priority for a given SAP before SAP aggregate shaper rates (agg-rate-limit) can be configured. Before changing the scheduling mode, the SAP must be shut down.

Figure 17:  Access Ingress Scheduling for 4-Priority and 16-Priority SAPs (with Per-SAP Aggregate Shapers) 

The 16-priority scheduler can be used without setting an aggregate rate limit for the SAP, in which case traffic out of the SAP queues are serviced in strict priority order, the conforming traffic prior to the non-conforming traffic. Using 16-priority schedulers without a configured per-SAP aggregate shaper (PIR = maximum and CIR = 0 kb/s) might be preferred over 4-priority mode for the following reasons:

As indicated in Figure 17, all the traffic leaving from the shaped SAPs must be serviced using 16-priority scheduling mode.

The SAPs without an aggregate rate limit, which are called unshaped SAPs, can be scheduled using either 4-priority or 16-priority mode as one of the following:

The arbitration of access ingress traffic leaving the 4-priority and 16-priority schedulers and continuing towards the fabric is described in the following section.

3.2.4.1. Access Ingress Per-SAP Shapers Arbitration

The 7705 SAR provides per-SAP aggregate shapers for access ingress SAPs. With this feature, both shaped and unshaped SAPs can coexist on the same adapter card. When switching traffic from shaped and unshaped SAPs to the fabric, arbitration is required.

Figure 18 shows how the 7705 SAR arbitrates traffic to the fabric between 4-priority unshaped SAPs, and 16-priority shaped and unshaped SAPs.

All SAPs support configurable CIR and PIR rates on a per-CoS queue basis (per-queue level). In addition, each 16-priority SAP has its own configurable per-SAP aggregate CIR and PIR rates that operate one level above the per-queue rates.

To allow the 4-priority unshaped SAPs to compete for fabric bandwidth with the aggregate CIR rates of the shaped SAPs, the 4-priority unshaped SAPs (as a group) have their own configurable unshaped SAP aggregate CIR rate, which is configured on the 7705 SAR-8 and 7705 SAR-18 under the config>qos>fabric-profile aggregate-mode context using the unshaped-sap-cir parameter. On the 7705 SAR-M, 7705 SAR-H, 7705 SAR-Hc, 7705 SAR-A, 7705 SAR-Ax, 7705 SAR-W, and 7705 SAR-Wx, the CIR rate is configured in the config>system>qos>access-ingress-aggregate-rate context.

The configured CIR and PIR for the 16-priority shaped SAPs dictates committed and uncommitted fabric bandwidth for each of these SAPs. Configuring the unshaped-sap-cir parameter for the group (aggregate) of 4-priority unshaped SAPs ensures that the unshaped SAPs will compete for fabric bandwidth with the aggregate CIR rate of the shaped SAPs. Otherwise, the unshaped SAPs would only be able to send traffic into the fabric after the aggregate CIR rates of all the shaped SAPs were serviced. The 16-priority unshaped SAPs are serviced as if they are non-conforming traffic for 16-priority shaped SAPs.

Figure 18:  Access Ingress per-SAP Arbitration to Fabric  

The aggregate fabric shaper (Figure 18) performs round-robin selection between the 16-priority SAPs (shaped and unshaped) and the 4-priority unshaped SAP aggregate until:

1. the aggregate fabric shaper rate is exceeded
2. the conforming (CIR) traffic for every 16-priority SAP and the 4-priority unshaped SAP aggregate is exceeded
3. the non-conforming traffic for every 16-priority SAP and the 4-priority unshaped SAP aggregate is completed, provided that the aggregate PIR rate is not exceeded

Note: The CLI does not block a fabric profile with an unshaped SAP CIR configuration on an 8-port Ethernet Adapter card, a 6-port Ethernet 10Gbps Adapter card, an ASAP card (16-port T1/E1 ASAP Adapter card or 32-port T1/E1 ASAP Adapter card). However, the unshaped SAP CIR configuration has no effect and is ignored on these cards.

3.2.5. Ingress Shaping to Fabric (Access and Network)

After the traffic is scheduled, it must be sent to the fabric interface. In order to avoid congestion in the fabric and ease the effects of possible bursts, a shaper is implemented on each adapter card.

The shapers smooth out any packet bursts and ease the flow of traffic onto the fabric. The shapers use buffer space on the adapter cards and eliminate the need for large ingress buffers in the fabric.

The ingress to-fabric shapers are user-configurable. For the 7705 SAR-8 and the 7705 SAR-18, the maximum rate depends on a number of factors, including platform, chassis variant, and slot type. See Configurable Ingress Shaping to Fabric (Access and Network) for details. For the 7705 SAR-M, 7705 SAR-H, 7705 SAR-Hc, 7705 SAR-A, 7705 SAR-Ax, 7705 SAR-W, and 7705 SAR-Wx, the shapers can operate at a maximum rate of 5 Gb/s. For the 7705 SAR-X, the shapers are not user-configurable. See Fabric Shaping on the Fixed Platforms (Access and Network) for details.

After the shaping function, all of the traffic is forwarded to the fabric interface in round-robin fashion, one packet at a time, from every access ingress adapter card.

Note: Starting with Release 8.0.R4, unless per-service-hashing is enabled, a 4-byte hash value will be appended to internal overhead for VPLS multicast traffic at ingress. The egress internal hash value is discarded at egress before scheduling. Therefore, shaping rates at access and network ingress and for fabric policies may need to be adjusted accordingly. In addition, the 4-byte internal hash value may be included in any affected statistics counters.

3.2.5.2. LAG SAP Support (Access Only)

Fabric shapers support access ingress traffic being switched from a SAP to another SAP residing on a port that is part of a Link Aggregation Group (LAG). Either the aggregate mode or destination mode can be used for fabric shaping.

When the aggregate mode is used, one aggregate rate sets the rate on all adapter cards. When the destination mode is used, the multipoint shaper is used to set the fabric shaping rate for traffic switched to a LAG SAP.

Note: Even though the multipoint shaper is used to set the fabric shaping rate for traffic switched to a LAG SAP, it is the per-destination unicast counters that are incremented to show the fabric statistics rather than the multipoint counter. Only the fabric statistics of the active port of the LAG are incremented, not the standby port.

3.2.6. Configurable Ingress Shaping to Fabric (Access and Network)

The use of fabric profiles allows the ingress (to the fabric) shapers to be user-configurable for access ingress and network ingress traffic.

For the 7705 SAR-8 and 7705 SAR-18, the maximum rates are:

For information about fabric shapers on the 7705 SAR-M, 7705 SAR-H, 7705 SAR-Hc, 7705 SAR-A, 7705 SAR-Ax, 7705 SAR-W, and 7705 SAR-X, see Fabric Shaping on the Fixed Platforms (Access and Network).

By allowing a rate of 1 Gb/s or higher to be configured from any adapter card to the fabric, the fabric may become congested. Therefore, the collection and display of fabric statistics are provided. These statistics report about the fabric traffic flow and potential discards. Refer to the 7705 SAR Interface Configuration Guide, “Configuring Adapter Card Fabric Statistics”, “Configuration Command Reference”, and “Show, Monitor, Clear, and Debug Command Reference” for information on how to configure, show, and monitor fabric statistics on an adapter card.

The ingress buffers for a card are much larger than the ingress buffers for the fabric; therefore, it is advantageous to use the larger card buffers for ingress shaping. In order to use the ingress card buffers and have much more granular control over traffic, two fabric profile modes are supported, per-destination mode and aggregate mode. Both modes offer shaping towards the fabric from an adapter card, but per-destination shapers offer the maximum flexibility by precisely controlling the amount of traffic to each destination card at a user-defined rate. Aggregate mode is used for simpler deployments, where the amount of traffic flowing to a destination adapter card is not controlled.

The default mode of operation for the 7705 SAR is set to aggregate, and the fixed aggregate rate of 200 Mb/s is set for both access ingress and network ingress traffic. Therefore, in a default configuration, each adapter card can switch up to 200 Mb/s of access ingress and network ingress traffic towards the fabric.

All the switched traffic can be destined for a single adapter card or it can be spread among multiple adapter cards. For higher-bandwidth applications, a network traffic analysis is recommended to determine which shaper rates would best suit the application and traffic patterns of a particular environment.

The to-fabric shapers are provided on the 7705 SAR to ensure adequate use of ingress buffers in case of congestion. With the ingress shapers, the large ingress card buffers can be configured to absorb bursty traffic and pace the traffic for better use of resources.

For example, if the average access ingress traffic bandwidth for an adapter card is 400 Mb/s and the peak bandwidth is 800 Mb/s, the rate of the to-fabric shapers can be configured to be 400 Mb/s. This allows the bursty ingress traffic to be paced by absorbing the bursty traffic after being shaped at 400 Mb/s. The initial burst is absorbed at the adapter card where the bursty traffic ingresses the 7705 SAR. The ingress buffers are used to absorb the burst and the fabric buffers are not exhausted by any single adapter card. The same example applies to network ingress traffic.

Table 9 summarizes the different capabilities offered by the two modes.

Figure 19 and Figure 20 illustrate the functionality of fabric shapers in per-destination mode and aggregate mode, respectively.

Referring to Figure 19, once the per-destination prioritization and scheduling takes place, as described in previous sections in this chapter, the per-destination adapter card shapers take effect. With per destination shapers, the maximum amount of bandwidth that each destination adapter card can receive from the fabric can be controlled. For example, the maximum amount of bandwidth that adapter card 1 can switch to the remaining adapter cards, as well as the amount of bandwidth switched back to adapter card 1, can be configured at a set rate.

Figure 19:  Fabric Shapers in Per Destination Mode 

Figure 20 illustrates the functionality of fabric shapers in aggregate mode. Once the policing, classification, queuing and per-destination based priority queuing takes place, as described in previous sections in this chapter, the aggregate mode adapter card shapers take effect. In aggregate mode, the aggregate of all the access ingress and network ingress traffic is shaped at a user-configured rate regardless of the destination adapter card.

Figure 20:  Fabric Shapers in Aggregate Mode 

Mixing different fabric shaper modes within the same chassis and on the same adapter card is not recommended; however, it is supported. As an example, an 8-port Gigabit Ethernet Adapter card in a 7705 SAR-18 can be configured for aggregate mode for access ingress and for per-destination mode for network ingress. The same chassis can also contain an adapter card (for example, the 8-port Ethernet Adapter card) that is configured for per-destination mode for all traffic. This setup is shown in the following example.

Note: Gen-2 and Gen-3 adapter cards only support aggregate mode fabric shapers for access ingress traffic, regardless of the service types configured. If multipoint services such as IES, VPRN, and VPLS are running on an adapter card, only aggregate mode fabric profile can be configured for the card at access ingress.

3.2.7. Fabric Shaping on the Fixed Platforms (Access and Network)

The 7705 SAR-A, 7705 SAR-Ax, 7705 SAR-M, 7705 SAR-H, 7705 SAR-Hc, 7705 SAR-W, and 7705 SAR-Wx support user-configurable fabric shapers at rates of up to 5 Gb/s for access ingress and network ingress traffic. The fabric interface on these nodes is a shared resource between both traffic types, and one buffer pool serves all MDAs (ports).

These nodes do not support fabric profiles; instead, they have a single aggregate rate limit for restricting access traffic into the fabric and a single aggregate rate limit for restricting network traffic into the fabric. These limits apply to all MDAs. Both access ingress and network ingress traffic can be configured to shaping rates of between 1 kb/s and 5 Gb/s. The default rate for access ingress traffic is 500 Mb/s. The default rate for network ingress traffic is 2 Gb/s. Statistics can be viewed for aggregate access and network traffic flow through the fabric and possible discards.

The 7705 SAR-X fabric shaper rate is not configurable for access ingress or network ingress traffic, and is set to the maximum rate for the platform. There are three buffer pools on the 7705 SAR-X, one for each MDA (block of ports).

3.3.1. Traffic Flow Across the Fabric

The 7705 SAR uses an Ethernet-based fabric. Each packet that is sent to the fabric is equipped with a fabric header that contains its specific CoS requirement. Since all of the packets switched across the fabric are already classified, queued, scheduled and marked according to the desired QoS parameters, each of these packets has been passed through the Traffic Management (TM) block on an adapter card, or the Control and Switching Module (CSM) in the case of control packets. Therefore, each packet arrives at the fabric having been already scheduled for optimal flow. The function of the fabric is to switch each packet through to the appropriate destination adapter card, or CSM in the case of control packets, in an efficient manner.

Since the traffic is shaped at a certain rate by the ingress adapter card (that is, bursts are smoothened by the traffic management function), minimal buffering should be needed on the switch fabric. However, the buffer space allocation and usage is in accordance with the priorities at the ingress adapter card. As is the case with schedulers at the adapter cards, there are two priorities supported on the switch fabric. The switch fabric serves the traffic in the following priority order:

Note: The switch fabric does not support profile scheduling. Since the fabric has a limited buffer space, it is possible for tail drop to occur. Tail drop discards any packet that exceeds the maximum buffer space allocation. The shaping that is performed on the adapter cards helps to prevent or minimize congestion.

3.4.1. BMU Traffic at Network Egress

BMU traffic at network egress is handled in the same way as unicast traffic in terms of scheduling, queuing, or port-level shaping. Both unicast and BMU traffic are mapped to queues as per the FC markings. Traffic from these queues, whether unicast or BMU, is scheduled according to user-configured rates. Port-level shapers treat all the queues identically, regardless of traffic type.

3.4.2. Network Egress Queuing Aggregation

After traffic is switched through the fabric from one or several access ingress adapter cards to a network egress adapter card, queuing-level aggregation on a per-forwarding-class basis is performed on all of the received packets.

An adapter card that is used for network egress can receive—and will likely receive—packets from multiple adapter cards that are configured for access ingress operations, and from the CSM. Adapter cards that are configured for network access permit user configuration of queues and the association of forwarding classes to the queues. These are the same configuration principles that are used for adapter cards that are configured for access ingress connectivity. Like access ingress, more than one forwarding class can share the same queue.

Aggregation of different forwarding classes under queues takes place for each bundle or port. If a port is a member of a bundle, such as a Multilink Point-to-Point Protocol (MLPPP) bundle, then the aggregation and queuing is implemented for the entire bundle. If a port is a standalone port, that is, not a member of bundle, then the queuing takes place for the port.

3.4.3. Network Egress Scheduling

Network egress scheduling is supported on the adapter cards and ports listed in Table 10. The supported scheduling modes are profiled, 4-priority, and 16-priority. Table 10 shows which scheduling mode each card and port supports at network egress.

This section also contains information on the following topics:

3.4.3.1. Network Egress 4-Priority Scheduling

The implementation of network egress scheduling on the cards and ports listed in Table 10 under “4-Priority” is very similar to the scheduling mechanisms used for adapter cards that are configured for access ingress traffic. 4-priority scheduling is a combination of queue-type scheduling (Expedited vs. Best-effort scheduling) and profiled scheduling (rate-based scheduling).

Note: The encapsulation type must be ppp-auto for PPP/MLPPP bundles on the following: T1/E1 ports on the 7705 SAR-A (variants with T1/E1 ports) T1/E1 ports on the 7705 SAR-M (variants with T1/E1 ports) T1/E1 ports on the 7705 SAR-X 16-port T1/E1 ASAP Adapter card 32-port T1/E1 ASAP Adapter card 2-port OC3/STM1 Channelized Adapter card 4-port OC3/STM1 / 1-port OC12/STM4 Adapter card T1/E1 ports on the 4-port T1/E1 and RS-232 Combination module (on 7705 SAR-H)

Packets less than or up to the CIR are scheduled as in-profile. Packets that arrive at rates greater than the CIR, but less than the PIR, are scheduled as out-of-profile. In-profile traffic is exhaustively transmitted from the queues before out-of-profile traffic is transmitted. That is, all of the in-profile packets must be transmitted before any out-of-profile packets are transmitted. In addition, Expedited queues are always scheduled prior to Best Effort queues.

The default configuration of scheduling CoS queues provides a logical and consistent means to manage the traffic priorities. The default configuration is as follows:

1. CoS-8 to CoS-5 Expedited in-profile
2. CoS-4 to CoS-1 Best Effort in-profile
3. CoS-8 to CoS-5 Expedited out-of-profile
4. CoS-4 to CoS-1 Best Effort out-of-profile

Note: Default configuration means that the queues are configured according to the tables and defaults described in this guide. Customers can configure the queues differently.

The order shown below is maintained when scheduling the traffic on the adapter card’s network ports. A strict priority is applied between the four schedulers, and all four schedulers are exhaustive:

1. Expedited in-profile traffic
2. Best Effort in-profile traffic
3. Expedited out-of-profile traffic
4. Best Effort out-of-profile traffic

3.4.3.4. Network Egress 16-Priority Scheduling

The adapter cards and ports that support 16-priority scheduling for network egress traffic are listed in Table 10. This type of scheduling takes into consideration the traffic’s profile type and the priority of the CoS queue that the traffic is coming from.

Packets less than or up to the CIR are scheduled as in-profile. Packets that arrive at rates greater than the CIR, but less than the PIR, are scheduled as out-of-profile. Eight CoS queues in total are available for packets to go through.

In-profile traffic is exhaustively transmitted from the queues, starting with the highest-priority CoS queue. A strict priority is applied between the eight CoS queues. If a packet arrives at a queue of higher priority than the one being serviced, the scheduler services the packet at the higher-priority queue as soon as it finishes servicing the current packet.

Once all the in-profile traffic is transmitted, the out-of-profile traffic is transmitted, still maintaining priority of the queues. If an in-profile packet arrives and the scheduler is servicing an out-of-profile packet, the scheduler finishes servicing the out-of-profile packet and then immediately services the in-profile packet.

The order of priority in the default configuration is as follows:

1. CoS-8 in-profile traffic
2. CoS-7 in-profile traffic
3. CoS-6 in-profile traffic
4. CoS-5 in-profile traffic
5. CoS-4 in-profile traffic
6. CoS-3 in-profile traffic
7. CoS-2 in-profile traffic
8. CoS-1 in-profile traffic
9. CoS-8 out-of-profile traffic
10. CoS-7 out-of-profile traffic
11. CoS-6 out-of-profile traffic
12. CoS-5 out-of-profile traffic
13. CoS-4 out-of-profile traffic
14. CoS-3 out-of-profile traffic
15. CoS-2 out-of-profile traffic
16. CoS-1 out-of-profile traffic

Note: Default configuration means that the queues are configured according to the tables and defaults described in this guide. Customers can configure the queues differently.

3.4.4. Network Egress Shaping

All the network egress traffic is shaped at the bundle or interface rate. An interface may not necessarily correspond directly to a port, and an interface could be a sub-channel of a port. As an example, Fast Ethernet could be the choice of network egress, but the leased bandwidth could still be a fraction of the port speed. In this case, it is possible to shape at the interface rate of 15 Mb/s, for example.

The same also applies to MLPPP bundles. The shaping takes place per MLPPP bundle, and the traffic is shaped at the aggregate rate of the MLPPP bundle.

3.4.5. Network Egress Shaping for Hybrid Ports

Hybrid ports use a third-tier, dual-rate aggregate shaper to provide arbitration between the bulk of access and network egress traffic flows. For details, see QoS for Hybrid Ports on Gen-2 Hardware.

3.4.6. Network Egress Per-VLAN Shapers

Network egress VLAN traffic uses second-tier (or second-level), per-VLAN shapers to prepare network egress traffic for arbitration with the aggregate of the unshaped VLAN shaper. All the shaped VLAN shapers are arbitrated with one unshaped VLAN shaper.

Note: This section applies to Gen-1 and Gen-2 adapter cards. For information on per-VLAN shapers for Gen-3 adapter cards and platforms, such as the 6-port Ethernet 10Gbps Adapter card and the 7705 SAR-X, see Figure 13 in the QoS for Gen-3 Adapter Cards and Platforms section.

As shown in Figure 21, traffic from the fabric flows to one or more VLANs, where it is classified and mapped to up to eight different CoS queues on a per-VLAN basis. The VLANs can be shaped or unshaped. Each shaped VLAN has its own set of CoS queues. The aggregate of unshaped VLANs uses the same set of CoS queues (that is, one set of queues for all unshaped VLANs).

For more information, see Per-VLAN Network Egress Shapers and Shaped and Unshaped VLANs.

Note: Due to space limitations in Figure 21, the second-tier, per-VLAN aggregate shapers are represented as a single loop containing the label “per VLAN”, even though they are dual-rate shapers similar to the third-tier network aggregate shaper.

Figure 21:  Network Egress Shaped and Unshaped VLAN Queuing and Scheduling 

Because the per-VLAN shapers are dual-rate shapers, their aggregate rate CIR and PIR values shape the traffic, as follows.

A shaped VLAN configured with default aggregate rate limits (PIR = maximum and CIR = 0 kb/s) is equivalent to an unshaped VLAN except that its traffic flows through a per-VLAN shaper rather than getting combined with the bulk (aggregate) of the unshaped VLANs. Using a shaped VLAN in this way (default rate limits) might be preferred over using an unshaped VLAN for the following reasons:

The arbitration of shaped and unshaped VLAN traffic at the third-tier shaper is described in the following section.

3.4.6.1. Network Egress Per-VLAN Shapers Arbitration

For shaped VLANs, the configured CIR and PIR limits dictate committed and uncommitted port bandwidth for each of these VLANs. To ensure that the bulk (aggregate) of unshaped VLANs can compete for port bandwidth with the aggregate of CIR rates for the shaped VLANs, the unshaped VLANs (as a group) have their own aggregate CIR rate, which is configured using the unshaped-if-cir command (under the config>port> ethernet>network>egress context). Otherwise, without their own aggregate CIR rate, the unshaped VLANs are only able to send traffic into the port after the aggregate CIR rates of all the shaped VLANs are serviced. Shaped VLANs using default aggregate rate limits (PIR = maximum and CIR = 0 kb/s) are serviced as if they are non-conforming traffic for shaped VLANs.

Note: This section applies to Gen-2 adapter cards. For information on per-VLAN shapers arbitration for Gen-3 adapter cards and platforms, such as the 6-port Ethernet 10Gbps Adapter card and the 7705 SAR-X, see Figure 13 in the QoS for Gen-3 Adapter Cards and Platforms section.

Referring to Figure 21, at the port shaper, conforming (CIR) traffic has priority over non-conforming traffic. The arbitration between the shaped VLANs and unshaped VLANs is handled in the following priority order:

1. committed traffic: the per-VLAN committed rate (CIR) for shaped VLANs as set by the agg-rate-limit command, and the aggregate committed rate for all the unshaped VLANs as set by the unshaped-if-cir command
2. uncommitted traffic: the per-VLAN uncommitted rate (PIR) for shaped VLANs as set by the agg-rate-limit command, and aggregate uncommitted rate for all the unshaped VLANs as set by the unshaped-if-cir command

3.4.7. Network Egress Marking and Re-Marking

The EXP bit settings can be marked at network egress. The EXP bit markings of the forwarding class are used for this purpose. The tunnel and pseudowire EXP bits are marked to the forwarding class value.

The default network egress QoS marking settings are given in Table 24.

3.5.1. Network Ingress Classification

Network ingress traffic originates from a network egress port located on another interworking device, such as a 7710 or 7750 Service Router or another 7705 SAR, and flows from the network toward the fabric in the 7705 SAR.

The ingress MPLS packets can be mapped to forwarding classes based on EXP bits that are part of the headers in the MPLS packets. These EXP bits are used across the network to ensure an end-to-end network-wide QoS offering. With pseudowire services, there are two labels, one for the MPLS tunnel and one for the pseudowire. Mapping is performed using the EXP values from the outer tunnel MPLS label. This ensures that the EXP bit settings, which may have been altered along the path by the tandem label switch routers (LSRs), are used to identify the forwarding class of the encapsulated traffic.

Ingress GRE and IP packets are mapped to forwarding classes based on DSCP bit settings of the IP header. GRE tunnels are not supported for IPv6; therefore, DSCP bit classification of GRE packets is only supported for IPv4. DSCP bit classification of IP packets is supported for both IPv4 and IPv6.

Untrusted traffic uses multi-field classification (MFC), where the traffic is classified based on any IP criteria currently supported by the 7705 SAR filter policies; for example, source and destination IP address, source and destination port, whether or not the packet is fragmented, ICMP code, and TCP state. For information on MFC, refer to the 7705 SAR Router Configuration Guide, “Multi-field Classification (MFC)” and “IP, MAC, and VLAN Filter Entry Commands”.

3.5.2. Network Ingress Queuing

Network ingress traffic can be classified in up to eight different forwarding classes, which are served by 16 queues (eight queues for unicast traffic and eight queues for multicast (BMU) traffic). Each queue serves at least one of the eight forwarding classes that are identified by the incoming EXP bits. These queues are automatically created by the 7705 SAR. Table 11 shows the default network QoS policy for the 16 CoS queues.

The value for CBS and MBS is a percentage of the size of the buffer pool for the adapter card. MBS can be shared across queues, which allows overbooking to occur.

3.5.3. Network Ingress Scheduling

Network ingress scheduling is supported on the adapter cards and ports listed in Table 12. The supported scheduling modes are profiled, 4-priority, and 16-priority. Table 12 shows which scheduling mode each card and port supports at network ingress.

This section also contains information on the following topics:

3.5.3.1. Network Ingress 4-Priority Scheduling

The adapter cards listed in Table 12 under “4-Priority” can receive network ingress traffic. One or more ports on the card is configured for PPP/MLPPP for this purpose.

The implementation of network ingress scheduling on the cards listed in Table 12 under “4-Priority” is very similar to the scheduling mechanisms used for adapter cards that are configured for access ingress traffic. That is, 4-priority scheduling is used (queue-type scheduling combined with profiled scheduling).

Note: The encapsulation type must be ppp-auto for PPP/MLPPP bundles on the following: T1/E1 ports on the 7705 SAR-A (variants with T1/E1 ports) T1/E1 ports on the 7705 SAR-M (variants with T1/E1 ports) T1/E1 ports on the 7705 SAR-X 16-port T1/E1 ASAP Adapter card 32-port T1/E1 ASAP Adapter card 2-port OC3/STM1 Channelized Adapter card 4-port OC3/STM1 / 1-port OC12/STM4 Adapter card T1/E1 ports on the 4-port T1/E1 and RS-232 Combination module (on 7705 SAR-H)

The adapter cards provide sets of eight queues for incoming traffic: 7 sets of queues for the 7705 SAR-8 and 17 sets of queues for the 7705 SAR-18. Each set of queues is specific to a destination adapter card. For the 7705 SAR-8 and 7705 SAR-18 (respectively), 6 and 16 sets of queues are automatically created for each access egress adapter card, plus 1 set of queues for multicast traffic.

There is one additional set of queues for slow-path (control) traffic destined for the CSMs.

The individual queues within each set of queues provide buffer space for traffic isolation based on the CoS values being applied (from the received EXP bits).

All of the network ingress ports of the adapter card share the same sets of queues, which are created automatically.

Once the packets received from the network are mapped to queues, four access ingress-like queue-type and profile (rate-based) schedulers per destination card service the queues in strict priority. The following queue-type and profiled schedulers service the queues in the order listed.

1. Expedited in-profile scheduler
2. Best Effort in-profile scheduler
3. Expedited out-of-profile scheduler
4. Best Effort out-of-profile scheduler

To complete the operation, user-configurable shapers send the traffic into the fabric. See Configurable Ingress Shaping to Fabric (Access and Network) for details. Throughout this operation, each packet retains its individual CoS value.

3.5.4. Network Ingress Shaping to Fabric

After the traffic is scheduled, it must be sent to the fabric interface. In order to avoid congestion in the fabric and ease the effects of possible bursts, a shaper is implemented on each adapter card.

Network ingress shaping to the fabric operates in a similar fashion to access ingress shaping to the fabric. See Ingress Shaping to Fabric (Access and Network) for details.

3.5.5. Configurable Network Ingress Shaping to Fabric

Configuring network ingress shapers to the fabric is similar to configuring access ingress shapers to the fabric.

The ingress to-fabric shapers are user-configurable. For the 7705 SAR-8 and the 7705 SAR-18, the maximum rate depends on a number of factors, including platform, chassis variant, and slot type. See Configurable Ingress Shaping to Fabric (Access and Network) for details.

For information about fabric shapers on the 7705 SAR-M, 7705 SAR-H, 7705 SAR-Hc, 7705 SAR-A, 7705 SAR-Ax, 7705 SAR-W, and 7705 SAR-Wx, see Fabric Shaping on the Fixed Platforms (Access and Network). The 7705 SAR-X does not support configurable network ingress shapers.

3.5.6. Network Fabric Shaping on the Fixed Platforms

The 7705 SAR-A, 7705 SAR-Ax, 7705 SAR-M, 7705 SAR-H, 7705 SAR-Hc,7705 SAR-W, and 7705 SAR-Wx support user-configurable fabric shapers at rates of up to 5 Gb/s for access ingress traffic and network ingress traffic.

On the 7705 SAR-A, 7705 SAR-Ax, 7705 SAR-M, 7705 SAR-H, 7705 SAR-Hc, 7705 SAR-W, and 7705 SAR-Wx, network ingress shapers to the fabric operate similarly to access ingress shapers to the fabric. The 7705 SAR-X does not support configurable network ingress shapers. See Fabric Shaping on the Fixed Platforms (Access and Network) for more information.

3.6.1. Access Egress Queuing and Scheduling

The following sections discuss the queuing and scheduling of access egress traffic, which is traffic that egresses the fabric on the access side:

Access egress scheduling takes place at the native traffic layer. As an example, once the ATM pseudowire payload is delivered from the network ingress to the access egress, the playback of the ATM cells to the appropriate ATM SAP is done according to ATM traffic management specifications.

Access egress scheduling is supported on the adapter cards and ports listed in Table 13. The supported scheduling modes are 4-priority and 16-priority. Table 13 shows which scheduling mode each card and port supports at access egress.

Note: For access ingress and egress, the 16-priority schedulers use additional hardware resources and capabilities, which results in increased throughput.

3.6.2. Access Egress Per-SAP Aggregate Shapers (Access Egress H-QoS)

Per-SAP aggregate shapers in the access egress direction operate in a similar fashion to aggregate shapers for access ingress, except that egress traffic goes through the schedulers to the egress port shaper instead of through the schedulers to the fabric port as in the access ingress case. For information on how access egress and access ingress per-SAP shaping is similar, see Access Ingress Per-SAP Aggregate Shapers (Access Ingress H-QoS). For general information on per-SAP shapers, see Per-SAP Aggregate Shapers (H-QoS) On Gen-2 Hardware.

The arbitration of access egress traffic from the per-SAP aggregate shapers to the schedulers is described in the following section.

3.6.2.1. Access Egress Per-SAP Shapers Arbitration

The arbitration of traffic from 4-priority and 16-priority schedulers towards an access egress port is achieved by configuring a committed aggregate rate limit for the aggregate of all the 4-priority unshaped SAPs. By configuring the 4-priority unshaped SAPs committed aggregate rate, the arbitration between the 16-priority shaped SAPs, 16-priority unshaped SAPs, and 4-priority unshaped SAPs is handled in the following priority order:

1. committed traffic: 16-priority per-SAP agg-rate-limit committed for shaped SAPs and 4-priority aggregate committed rate for all the unshaped SAPs
2. uncommitted traffic: 16-priority per-SAP agg-rate-limit uncommitted for shaped and unshaped SAPs and 4-priority aggregate uncommitted rate for all the unshaped SAPs

Figure 22 illustrates the traffic treatment for a single Ethernet port. It also illustrates that the shaped SAP aggregate CIR rate competes with the unshaped 4-priority aggregate CIR rate for port bandwidth. Once the aggregate CIR rates are satisfied, the shaped SAP aggregate PIR rate competes with the 4-priority PIR rate (always maximum) for port bandwidth.

The egress aggregate CIR rate limit for all the unshaped 4-priority SAPs is configured using the config>port>ethernet>access>egress>unshaped-sap-cir command.

Figure 22:  Access Egress 16-Priority and 4-Priority per-SAP Arbitration for a Single Port 

3.6.3. Access Egress Shaping for Hybrid Ports

Hybrid ports use a third-tier, dual-rate aggregate shaper to provide arbitration between the bulk of access and network egress traffic flows. For details, see QoS for Hybrid Ports on Gen-2 Hardware.

3.6.4. Access Egress for 4-Priority (Gen-3) Scheduling

The adapter cards and ports that support 4-priority (Gen-3) scheduling for access egress traffic are listed in Table 13. See QoS for Gen-3 Adapter Cards and Platforms for details.

3.6.5. Access Egress Marking/Re-Marking

At access egress, where the network-wide QoS boundary is reached, there may be a requirement to mark or re-mark the CoS indicators to match customer requirements. Dot1p and DSCP marking and re-marking is supported at Ethernet access egress.

Note: When dot1p re-marking is needed for a qinq egress SAP, it may be necessary to use the qinq-mark-top-only command to indicate which qtag needs to have its dot1p bits re-marked. The qinq-mark-top-only command is found under the config>service context. Refer to the 7705 SAR Services Guide, 'VLL Services Command Reference”, for details.

Similar to access ingress for Ethernet, DSCP marking or re-marking is supported for untagged, single-tagged, or double-tagged Ethernet frames.

On Ipipe SAPs over an Ethernet VLAN, both dot1p and DSCP marking and re-marking is supported at access egress. On Ipipe SAPs over PPP/MLPPP, DSCP marking and re-marking is supported at access egress. DSCP re-marking is supported for Ipipes using FR or cHDLC SAPS at the access egress.

3.6.6. Packet Byte Offset (PBO)

Packet byte offset (PBO), or internal headerless rate, allows 7705 SAR schedulers to operate on a modified packet size by adding or subtracting a certain number of bytes. The actual packet size remains the same but schedulers take into account the modified size as opposed to the actual size of the packet. One of the main uses of the packet byte offset feature is to allow scheduling, at access ingress, to be carried out on the received packet size without taking into account service (for example, PW, MPLS) or internal overhead. Transport providers who sell bandwidth to customers typically need the 7705 SAR shapers/schedulers to only take into account the received packet size without the added overhead in order to accurately calculate the bandwidth they need to provide to their customers. Packet byte offset addresses this requirement. Another common use is at egress where port shapers can take into account four additional bytes, associated with Ethernet FCS.

Packet byte offset is configured under QoS profiles. Packet size modification might be desired to accommodate inclusion or exclusion of certain headers or even fields of headers during the scheduling operation. The packet size that the schedulers take into account is altered to accommodate or omit the desired number of bytes. Both addition and subtraction options are supported by the packet-byte-offset command. The actual packet size is not modified by the command; only the size used by ingress or egress schedulers is changed. The scheduling rates are affected by the offset, as well as the statistics (accounting) associated with the queue. Packet byte offset does not affect port-level and service-level statistics. It only affects the queue statistics.

When a QoS policy configured with packet byte offset is applied to a SAP or network interface, all the octet counters and statistics operate and report based on the new adjusted value. If configured, per-SAP aggregate shapers and per-customer aggregate shapers also operate on the adjusted packet sizes. The only exceptions to this rule are port shapers. The egress port shapers do not take the adjusted packet size into account, except on the 8-port Ethernet Adapter card, version 2, but operate only on the final packet size.

Note: The fabric shaper, in general, does not take the adjusted packet size into account, except on the 8-port Ethernet Adapter card, version 2.

Table 17 shows PBO support on the 7705 SAR.

3.7.1. Overview

7705 SAR QoS policies are applied on service ingress, service egress, and network interfaces. The service ingress and service egress points may be considered as the network QoS boundaries for the service being provided.

The QoS policies define:

There are several types of QoS policies (see Table 18 for summaries and references to details):

Service ingress QoS policies are applied to the customer-facing Service Access Points (SAPs) and map traffic to forwarding class queues on ingress. The mapping of traffic to queues can be based on combinations of customer QoS marking (dot1p bits and DSCP values). The number of forwarding class queues for ingress traffic and the queue characteristics are defined within the policy. There can be up to eight ingress forwarding class queues in the policy, one for each forwarding class.

Within a service ingress QoS policy, up to three queues per forwarding class can be used for multipoint traffic for multipoint services. Multipoint traffic consists of broadcast, multicast, and unknown (BMU) traffic types. For VPLS, four types of forwarding are supported (which are not to be confused with forwarding classes): unicast, broadcast, multicast, and unknown. The BMU types are flooded to all destinations within the service, while the unicast forwarding type is handled in a point-to-point fashion within the service.

Service ingress QoS policies on the 7705 SAR permits flexible arrangement of these queues. For example, more than one FC can be mapped to a single queue, both unicast and multipoint (BMU) traffic can be mapped to a single queue, or unicast and BMU traffic can be mapped to separate queues. Therefore, customers are not limited to the default configurations that are described in this guide.

Service egress QoS policies are applied to egress SAPs and provide the configurations needed to map forwarding classes to service egress queues. Each service can have up to eight queues configured, since a service may require multiple forwarding classes. A service egress QoS policy also defines how to re-mark dot1p bits and DSCP values of the customer traffic in native format based on the forwarding class of the customer traffic.

Network ingress and egress QoS policies are applied to network interfaces. On ingress for traffic received from the network, the policy maps incoming EXP values to forwarding classes and profile states. On egress, the policy maps forwarding classes and profile states to EXP values for traffic to be transmitted into the network.

On the network side, there are two types of QoS policies: network and network queue (see Table 18). The network type of QoS policy is applied to the network interface under the config>router>interface command and contains the EXP marking rules for both ingress and egress. The network queue type of QoS policy defines all of the internal settings; that is, how the queues, or sets of queues (for ingress), are set up and used per physical port on egress and per adapter card for ingress.

A ring type network policy can be applied to the ring ports and the add/drop port on a ring adapter card. The policy is created under the config>qos>network command, and applied at the adapter card level under the config>card>mda command. The policy maps each dot1p value to a queue and a profile state.

If GRE or IP tunneling is enabled, policy mapping can be set up to use DSCP bits.

Network queue policies are applied on egress to network ports and channels and on ingress to adapter cards. The policies define the forwarding class queue characteristics for these entities.

Service ingress, service egress, and network QoS policies are defined with a scope of either template or exclusive. Template policies can be applied to multiple SAPs or interfaces, whereas exclusive policies can only be applied to a single entity.

One service ingress QoS policy and one service egress QoS policy can be applied to a specific SAP. One network QoS policy can be applied to a specific interface. A network QoS policy defines both ingress and egress behavior. If no QoS policy is explicitly applied to a SAP or network interface, a default QoS policy is applied.

Table 18 provides a summary of the major functions performed by the QoS policies.

3.7.2. Service Ingress QoS Policies

Service ingress QoS policies define ingress service forwarding class queues and map flows to those queues. When a service ingress QoS policy is created, it always has a default ingress traffic queue defined that cannot be deleted. These queues exist within the definition of the policy. The queues only get created when the policy is applied to a SAP.

In the simplest service ingress QoS policy, all traffic is treated as a single flow and mapped to a single queue. The required elements to define a service ingress QoS policy are:

Optional service ingress QoS policy elements include:

Each queue can have unique queue parameters to allow individual policing and rate shaping of the flow mapped to the forwarding class. Figure 23 depicts service traffic being classified into three different forwarding class queues.

Figure 23:  Traffic Queuing Model for Three Queues and Three Classes 

Mapping flows to forwarding classes is controlled by comparing each packet to the match criteria in the QoS policy. The ingress packet classification to forwarding class and enqueuing priority is subject to a classification hierarchy. Each type of classification rule is interpreted with a specific priority in the hierarchy.

Table 19 is given as an example for an Ethernet SAP (that is, a SAP defined over a whole Ethernet port, over a single VLAN, or over QinQ VLANs). It lists the classification rules in the order in which they are evaluated.

The enqueuing priority is specified as part of the classification rule and is set to high or low. The enqueuing priority relates to the forwarding class queue’s high-priority-only allocation, where only packets with a high enqueuing priority are accepted into the queue once the queue’s depth reaches the defined threshold. See High-Priority-Only Buffers.

The mapping of ingress traffic to a forwarding class based on dot1p or DSCP bits is optional. The default service ingress policy is implicitly applied to all SAPs that do not explicitly have another service ingress policy assigned. The characteristics of the default policy are listed in Table 20.

3.7.3. Service Egress QoS Policies

Service egress queues are implemented at the transition from the service network to the service access network. The advantages of per-service queuing before transmission into the access network are:

The subrate capabilities and per-service scheduling control are required to make multiple services per physical port possible. Without egress shaping, it is impossible to support more than one service per port. There is no way to prevent service traffic from bursting to the available port bandwidth and starving other services.

For accounting purposes, per-service statistics can be logged. When statistics from service ingress queues are compared with service egress queues, the ability to conform to per-service QoS requirements within the service network can be measured. The service network statistics are a major asset to network provisioning tools.

Service egress QoS policies define egress service queues and map forwarding class flows to queues. In the simplest service egress QoS policy, all forwarding classes are treated as a single flow and mapped to a single queue.

To define a basic service egress QoS policy, the following are required:

Optional service egress QoS policy elements include:

Each queue in a policy is associated with one or more of the supported forwarding classes. Each queue can have its individual queue parameters, allowing individual rate shaping of the forwarding classes mapped to the queue. More complex service queuing models are supported in the 7705 SAR where each forwarding class is associated with a dedicated queue.

The forwarding class determination per service egress packet is determined at ingress. If the packet ingressed the service on the same 7705 SAR router, the service ingress classification rules determine the forwarding class of the packet. If the packet was received over a service transport tunnel, the forwarding class is marked in the tunnel transport encapsulation.

Service egress QoS policy ID 1 is reserved as the default service egress policy. The default policy cannot be deleted or changed.

The default service egress policy is applied to all SAPs that do not have another service egress policy explicitly assigned. The characteristics of the default policy are listed in Table 21.

3.7.4. MC-MLPPP SAP Egress QoS Policies

SAPs running MC-MLPPP have their own SAP egress QoS policies that differ from standard policies. Unlike standard SAP policies, MC-MLPPP SAP egress policies do not contain queue types, CIR, CIR adaptation rules, or dot1p re-marking.

Standard and MC-MLPPP SAP egress policies can never have the same policy ID except when the policy ID is 1 (default). Standard SAP egress QoS policies cannot be applied to SAPs running MC-MLPPP. Similarly, MC-MLPPP SAP egress QoS policies cannot be applied to standard SAPs. The default policy can be applied to both MC-MLPPP and other SAPs. It will remain the default policy regardless of SAP type.

MC-MLPPP on the 7705 SAR supports scheduling based on multi-class implementation. Instead of the standard profiled queue-type scheduling, an MC-MLPPP encapsulated access port performs class-based traffic servicing.

The four MC-MLPPP classes are scheduled in a strict priority fashion, as shown in Table 22.

For example, if a packet is sent to an MC-MLPPP class 3 queue and all other queues are empty, the 7705 SAR fragments the packet according to the configured fragment size and begins sending the fragments. If a new packet is sent to an MC-MLPPP class 2 queue, the 7705 SAR finishes sending any fragments of the class 3 packet that are on the wire, then holds back the remaining fragments in order to service the higher-priority packet. The fragments of the first packet remain at the top of the class 3 queue. For packets of the same class, MC-MLPPP class queues operate on a first-in, first-out basis.

The user configures the required number of MLPPP classes to use on a bundle. The forwarding class of the packet, as determined by the ingress QoS classification, is used to determine the MLPPP class for the packet. The mapping of forwarding class to MLPPP class is a function of the user-configurable number of MLPPP classes. The default mapping for a 4-class, 3-class, and 2-class MLPPP bundle is shown in Table 23.

If one or more forwarding classes are mapped to a queue, the scheduling priority of the queue is based on the lowest forwarding class mapped to it. For example, if forwarding classes 0 and 7 are mapped to a queue, the queue is serviced by MC-MLPPP class 3 in a 4-class bundle model.

3.7.5. Network and Network Queue QoS Policies

The QoS mechanisms within the 7705 SAR are specialized for the type of traffic on the interface. For customer interfaces, there is service ingress and service egress traffic, and for network interfaces, there is network ingress and network egress traffic.

The 7705 SAR uses QoS policies applied to a SAP for a service or to a network port to define the queuing, queue attributes, and QoS marking/interpretation.

The 7705 SAR supports the following types of network and service QoS policies:

3.7.5.1. Network QoS Policies

Network QoS policies define egress QoS marking and ingress QoS classification for traffic on network interfaces. The 7705 SAR automatically creates egress queues for each of the forwarding classes on network interfaces.

A network QoS policy defines ingress, egress, and ring handling of QoS on the network interface. The following functions are defined:

1. ingress
   1. defines label switched path Experimental bit (LSP EXP) value mappings to forwarding classes
   2. defines DSCP name mappings to forwarding classes
2. egress
   1. defines forwarding class to LSP EXP and dot1p value markings
   2. defines forwarding class to DSCP value markings
3. ring
   1. defines dot1p bit value mappings to queue and profile state

The required elements to be defined in a network QoS policy are:

1. a unique network QoS policy ID
2. egress forwarding class to LSP EXP value mappings for each forwarding class used
3. a default ingress forwarding class and in-profile/out-of-profile state
4. a default queue and in-profile/out-of-profile state for ring type network QoS policy

Optional ip-interface type network QoS policy elements include the LSP EXP value or DSCP name to forwarding class and profile state mappings for all EXP values or DSCP values received. Optional ring type network QoS policy elements include the dot1p bits value to queue and profile state mappings for all dot1p bit values received.

Network policy ID 1 is reserved as the default network QoS policy. The default policy cannot be deleted or changed. The default network QoS policy is applied to all network interfaces and ring ports (for ring adapter cards) that do not have another network QoS policy explicitly assigned.

Table 24, Table 25, Table 26, and Table 27 list the various network QoS policy default mappings.

Table 24 lists the default mapping of forwarding class to LSP EXP values and DSCP names for network egress.

Table 24:  Default Network QoS Policy Egress Marking  

FC-ID	FC Name	FC Label	DiffServ Name	Egress LSP EXP Marking		Egress DSCP Marking
				In-Profile	Out-of-Profile	In-Profile Name	Out-of-Profile Name
7	Network Control	nc	NC2	111 - 7	111 - 7	nc2 111000 - 56	nc2 111000 - 56
6	High-1	h1	NC1	110 - 6	110 - 6	nc1 110000 - 48	nc1 110000 - 48
5	Expedited	ef	EF	101 - 5	101 - 5	ef 101110 - 46	ef 101110 - 46
4	High-2	h2	AF4	100 - 4	100 - 4	af41 100010 - 34	af42 100100 - 36
3	Low-1	l1	AF2	011 - 3	010 - 2	af21 010010 - 18	af22 010100 - 20
2	Assured	af	AF1	011 - 3	010 - 2	af11 001010 - 10	af12 001100 - 12
1	Low-2	l2	CS1	001 - 1	001 - 1	cs1 001000 - 8	cs1 001000 - 8
0	Best Effort	be	BE	000 - 0	000 - 0	be 000000 - 0	be 000000 - 0

For network ingress, Table 25 lists the default mapping of DSCP name to forwarding class and profile state for the default network QoS policy.

Table 25:  Default Network QoS Policy DSCP-to-Forwarding Class Mappings 

Ingress DSCP			Forwarding Class
DSCP Name	DSCP Value	FC ID	Name	Label	Profile State
Default  1		0	Best-Effort	be	Out
ef	101110 - 46	5	Expedited	ef	In
cs1	001000 - 8	1	Low-2	l2	In
nc-1	110000 - 48	6	High-1	h1	In
nc-2	111000 - 56	7	Network Control	nc	In
af11	001010 - 10	2	Assured	af	In
af12	001100 - 12	2	Assured	af	Out
af13	001110 - 14	2	Assured	af	Out
af21	010010 - 18	3	Low-1	l1	In
af22	010100 - 20	3	Low-1	l1	Out
af23	010110 - 22	3	Low-1	l1	Out
af31	011010 - 26	3	Low-1	l1	In
af32	011100 - 28	3	Low-1	l1	Out
af33	011110 - 30	3	Low-1	l1	Out
af41	100010 - 34	4	High-2	h2	In
af42	100100 - 36	4	High-2	h2	Out
af43	100110 - 38	4	High-2	h2	Out

Note:

1. The default forwarding class mapping is used for all DSCP name values for which there is no explicit forwarding class mapping.

Table 26 lists the default mapping of LSP EXP values to forwarding class and profile state for the default network QoS policy.

Table 26:  Default Network QoS Policy LSP EXP-to-Forwarding Class Mappings  

Ingress LSP EXP			Forwarding Class
LSP EXP ID	LSP EXP Value	FC ID	Name	Label	Profile State
Default  1		0	Best-Effort	be	Out
1	001 - 1	1	Low-2	l2	In
2	010 - 2	2	Assured	af	Out
3	011 - 3	2	Assured	af	In
4	100 - 4	4	High-2	h2	In
5	101 - 5	5	Expedited	ef	In
6	110 - 6	6	High-1	h1	In
7	111 - 7	7	Network Control	nc	In

Note:

1. The default forwarding class mapping is used for all LSP EXP values for which there is no explicit forwarding class mapping.

Table 27 lists the default mapping of dot1p values to queue and profile state for the default network QoS policy.

Table 27:  Default Network QoS Policy Dot1p-to-Queue Class Mappings  

Dot1p Value	Queue	Profile State
0	1	Out
1	2	In
2	3	Out
3	3  1	In
4	5	In
5	6	In
6	7	In
7	8	In

Note:

1. The default queue mapping for dot1p values 2 and 3 are both queue 3.

3.7.5.1.1. CoS Marking for Self-generated Traffic

The 7705 SAR is the source of some types of traffic; for example, a link state PDU for sending IS-IS topology updates or an SNMP trap sent to indicate that an event has happened. This type of traffic that is created by the 7705 SAR is considered to be self-generated traffic (SGT). Another example of self-generated traffic is Telnet, but in that application, user commands initiate the sending of the Telnet traffic.

Network operators often have different QoS models throughout their networks and apply different QoS schemes to portions of the networks in order to better accommodate delay, jitter, and loss requirements of different applications. The class of service (DSCP or dot1p) bits of self-generated traffic can be marked on a per-application basis to match the network operator’s QoS scheme. This marking option enhances the ability of the 7705 SAR to match the various requirements of these applications.

The 7705 SAR supports marking self-generated traffic for the base routers and for virtual routers. Refer to “QoS Policies” in the 7705 SAR Services Guide for information on SGT QoS as applied to virtual routers (for VPRN services).

The DSCP and dot1p values of the self-generated traffic, where applicable, are marked in accordance with the values that are configured under the sgt-qos command. In the egress direction, self-generated traffic is forwarded using the egress control queue to ensure premium treatment, unless SGT redirection is configured (see SGT Redirection). PTP (IEEE 1588v2) and SAA-enabled ICMP traffic is forwarded using the CoS queue. The next-hop router uses the DSCP values to classify the traffic accordingly.

Note: IS-IS and ARP traffic are not IP-generated traffic types and are not DSCP-configurable; however, the dot1p bits can be configured in the same way as the DSCP bits. The default setting for the dot1p bits for both types of traffic is 111. For all other applications, the dot1p bits are marked based on the mapped network egress forwarding class.

Table 28 lists various applications and indicates whether they have configurable DSCP or dot1p markings.

Table 28:  Applications and Support for Configurable DSCP or dot1p Markings 

Application	Supported Marking	Default DSCP/dot1p
ARP	dot1p	7
IS-IS	dot1p	7
BGP	DSCP	NC1
DHCP	DSCP	NC1
DNS	DSCP	AF41
FTP	DSCP	AF41
ICMP (ping)	DSCP	BE
IGMP	DSCP	NC1
LDP (T-LDP)	DSCP	NC1
MLD	DSCP	NC1
NDIS	DSCP	NC1
NTP	DSCP	NC1
OSPF	DSCP	NC1
PIM	DSCP	NC1
1588 PTP	DSCP	NC1
RADIUS	DSCP	AF41
RIP	DSCP	NC1
RSVP	DSCP	NC1
SNMP (get, set, etc.)	DSCP	AF41
SNMP trap/log	DSCP	AF41
SSH (SCP)	DSCP	AF41
syslog	DSCP	AF41
TACACS+	DSCP	AF41
Telnet	DSCP	AF41
TFTP	DSCP	AF41
Traceroute	DSCP	BE
VRRP	DSCP	NC1

Note: PTP in the context of SGT QoS is defined as Precision Timing Protocol and is an application in the 7705 SAR. The PTP application name is also used in areas such as event-control and logging. Precision Timing Protocol is defined in IEEE 1588-2008. PTP in the context of IP filters is defined as Performance Transparency Protocol. IP protocols can be used as IP filter match criteria; the match is made on the 8-bit protocol field in the IP header.

3.7.5.1.2. SGT Redirection

The 7705 SAR can be used in deployments where the uplink bandwidth capacity is considerably less than if the router is used for fixed or mobile backhaul applications. However, the 7705 SAR is optimized to operate in environments with megabits per second of uplink capacity for network operations. Therefore, many of the software timers are designed to ensure the fastest possible detection of failures, without considering bandwidth limitations. In deployments with very low bandwidth constraints, the system must also be optimized for effective operation of the routers without any interruption to mission-critical customer traffic.

In lower-bandwidth deployments, SGT can impact mission-critical user traffic such as TDM pseudowire traffic. To minimize the impact on this traffic, SGT can be redirected to a data queue rather than to the high-priority control queue on egress. All SGT applications can be redirected to a data queue, but the type of application must be considered because not all SGT is suitable to be scheduled at a lower priority. SGT applications such as FTP, TFTP, and syslog can be mapped to a lower-priority queue.

Caution: Care must be taken when determining which SGT applications should be moved to data queues, as interrupting traffic flow for applications such as routing protocols (for example, BGP, OSPF, and IS-IS) and MPLS can adversely affect router operation, services, and the network.

As an example, in a scenario where the uplink bandwidth is limited to a fractional E1 link with 2 x DS0 channel groups, downloading software for a new release can disrupt TDM pseudowire traffic, especially if SGT traffic is always serviced first over all other traffic flows. Having the option to map a subset of SGT to data queues will ensure that the mission-critical traffic flows effectively. For example, if FTP traffic is redirected to the best-effort forwarding queue, FTP traffic is then serviced only after all higher-priority traffic is serviced, including network control traffic and TDM pseudowire traffic. This redirection ensures the proper treatment of all traffic types matching the requirements of the network.

Caution: Timeouts for signaling and/or routing protocols can initiate a session teardown. The teardown will not only have local impacts as severe as losing the sole uplink of a node but could also be network-wide if loss of a node is propagated throughout the network. In this scenario, routing protocols, as an example, will rerun on all the nodes within the same area, generating a CPU load and extra control-plane traffic. Such a scenario could cause potential instability across the network or an area of the network. To avoid this scenario, the user must ensure that uplink capacity is enough to transmit all crucial SGT plus the mission-critical user traffic, and SGT redirection should only be used for the non-mission-critical traffic that can tolerate delay and jitter.

Redirection of SGT applications is done using the config>router>sgt-qos> application>fc-queue or config>service>vprn>sgt-qos>application>fc-queue command.

Redirection of the global ping application is not done through the sgt-qos menu hierarchy; this is configured using the fc-queue option in the ping command. Refer to the 7705 SAR OAM and Diagnostics Guide, “OAM and SAA Command Reference”, for details.

SGT redirection is supported on the base router and the virtual routers on ports with Ethernet or PPP/MLPPP encapsulation.

3.7.5.2. Network Queue QoS Policies

Network queue policies define the queue characteristics that are used in determining the scheduling and queuing behavior for a given forwarding class or forwarding classes. Network queue policies are applied on ingress and egress network ports, as well as on the ring ports and the add/drop port on the 2-port 10GigE (Ethernet) Adapter card and 2-port 10GigE (Ethernet) module.

Network queue policies are identified with a unique policy name that conforms to the standard 7705 SAR alphanumeric naming conventions. The policy name is user-configured when the policy is created.

Network queue policies can be configured to use up to 16 queues (8 unicast and 8 multicast). This means that the number of queues can vary. Not all user-created policies will require and use 16 queues; however, the system default network queue policy (named default) does define 16 queues.

The queue characteristics that can be configured on a per-forwarding class basis are:

1. Committed Buffer Size (CBS) as a percentage of the buffer pool
2. Maximum Buffer Size (MBS) as a percentage of the buffer pool
3. High-Priority-Only Buffers as a percentage of MBS
4. Peak Information Rate (PIR) as a percentage of egress port bandwidth
5. Committed Information Rate (CIR) as a percentage of egress port bandwidth

Table 29 describes the default network queue policy definition.

Note: The system default network queue policy cannot be modified or deleted. In the table, the value for Rate in the Definition column is the PIR value.

Table 29:  Default Network Queue Policy Definition  

Forwarding Class	Queue	Definition	Queue	Definition
Network-Control (nc)	8	Rate = 100% CIR = 10% MBS = 2.5% CBS = 0.25% High-Prio-Only = 10%	16	Rate = 100% CIR = 10% MBS = 2.5% CBS = 0.1% High-Prio-Only = 10%
High-1 (h1)	7	Rate = 100% CIR = 10% MBS = 2.5% CBS = 0.25% High-Prio-Only = 10%	15	Rate = 100% CIR = 10% MBS = 2.5% CBS = 0.1% High-Prio-Only = 10%
Expedited (ef)	6	Rate = 100% CIR = 100% MBS = 5% CBS = 0.75% High-Prio-Only = 10%	14	Rate = 100% CIR = 100% MBS = 5% CBS = 0.1% High-Prio-Only = 10%
High-2 (h2)	5	Rate = 100% CIR = 100% MBS = 5% CBS = 0.75% High-Prio-Only = 10%	13	Rate = 100% CIR = 100% MBS = 5% CBS = 0.1% High-Prio-Only = 10%
Low-1 (l1)	4	Rate = 100% CIR = 25% MBS = 2.5% CBS = 0.25% High-Prio-Only = 10%	12	Rate = 100% CIR = 5% MBS = 2.5% CBS = 0.25% High-Prio-Only = 10%
Assured (af)	3	Rate = 100% CIR = 25% MBS = 5% CBS = 0.75% High-Prio-Only = 10%	11	Rate = 100% CIR = 5% MBS = 5% CBS = 0.1% High-Prio-Only = 10%
Low-2 (l2)	2	Rate = 100% CIR = 25% MBS = 5% CBS = 0.25% High-Prio-Only = 10%	10	Rate = 100% CIR = 5% MBS = 5% CBS = 0.1% High-Prio-Only = 10%
Best Effort (be)	1	Rate = 100% CIR = 0% MBS = 5% CBS = 0.1% High-Prio-Only = 10%	9	Rate = 100% CIR = 0% MBS = 5% CBS = 0.1% High-Prio-Only = 10%

3.7.6. Network and Service QoS Queue Parameters

The following queue parameters are provisioned on network and service queues:

3.7.6.4. Adaptation Rule

The schedulers on the network processor can only operate with a finite set of rates. These rates are called the operational rates. The configured rates for PIR and CIR do not necessarily correspond to the operational rates. In order to offer maximum flexibility to the user, the adaptation-rule command can be used to choose how an operational rate is selected based on the configured PIR or CIR rate.

The max parameter causes the network processor to be programmed at an operational rate that is less than the configured PIR or CIR rate by up to 1.0%. The min parameter causes the network processor to be programmed at an operational rate that is greater than the configured PIR or CIR rate by up to 1.0%. The closest parameter causes the network processor to be programmed at an operational rate that is closest to the configured PIR or CIR rate.

A 4-priority scheduler on the network processor of a third-generation (Gen-3) Ethernet adapter card or platform can be programmed at an operational CIR rate that exceeds 1.0% of the configured CIR rate. The PIR rate (that is, the maximum rate for the queue) and the SAP aggregate rates (CIR and PIR), maintain an accuracy of +/- 1.0% of the configured rates.

The average difference between the configured CIR rate and the programmed (operational) CIR rate is as follows:

1. 2.0% for frame sizes that are less than 2049 bytes
2. 4.0% for other frame sizes

Note: The percentages in the above list are averages only; the actual values can be higher.

The Gen-3 network processor PIR rate is programmed to an operational PIR rate that is within 1.0% of the configured rate, which ensures that the FC/CoS queue does not exceed its fair share of the total bandwidth.

3.7.7. Slope Policies (WRED and RED)

As part of 7705 SAR queue management, policies for WRED and/or RED queue management (also known as congestion management or buffer management) to manage the queue depths can be enabled at both access and network ports and associated with both ingress and egress queues. WRED policies can also be enabled on bridged domain (ring) ports.

Without WRED and RED, once a queue reaches its maximum fill size, the queue discards any new packets arriving at the queue (tail drop).

WRED and RED policies prevent a queue from reaching its maximum size by starting random discards once the queue reaches a user-configured threshold value. This avoids the impact of discarding all the new incoming packets. By starting random discards at this threshold, customer devices at an end-system may be adjusted to the available bandwidth.

As an example, TCP has built-in mechanisms to adjust for packet drops. TCP-based flows lower the transmission rate when some of the packets fail to reach the far end. This mode of operation provides a much better way of dealing with congestion than dropping all the packets after the whole queue space is depleted.

The WRED and RED curve algorithms are based on two user-configurable thresholds (minThreshold and maxThreshold) and a discard probability factor (maxDProbability) (see Figure 24). The minThreshold (minT) indicates the level when where discards start and the discard probability is zero. The maxThreshold (maxT) is indicates the level where the discard probability reaches its maximum value. Beyond this the maxT level, all newly arriving packets are discarded. The steepness of the slope between minT and maxT is derived from the maxDProbability (maxDP). Thus, the maxDP indicates the random discard probability at the maxT level.

The main difference between WRED and RED is that with WRED, there can be more than one curve managing the fill rate of the same queue.

WRED slope curves can run against high-priority and low-priority traffic separately for ingress and egress queues. This allows the flexibility to treat low-priority and high-priority traffic differently. WRED slope policies are used to configure the minT, maxT and maxDP values, instead of configuring these thresholds against every queue. It is the slope policies that are then applied to individual queues. Thus, WRED slope policies affect how and when the high-priority and low-priority traffic is discarded within the same queue.

Referring to Figure 24, one WRED slope curve can manage discards on high-priority traffic and another WRED slope curve can manage discards on low-priority traffic. The minT, maxT and maxDP values configured for high-priority and low-priority traffic can be different and can start discarding traffic at different thresholds. Use the start-avr, max-avr, and max-prob commands to set the minThreshold, maxThreshold, and maxDProbabilty values, respectively.

Note: The figure shows a step function at maxT. The maxDP value is the target value entered for the configuration and it partly determines the slope of the weighting curve. At maxT, if the arrival of a new packet will overflow the buffer, the discard probability jumps to 1, which is not the maxDP value. Therefore, a step function exists in this graph.

Figure 24:  WRED for High-Priority and Low-Priority Traffic on the Same Queue 

The formula to calculate the average queue size is:

average queue size = (previous average × (1 – 1/2^TAF)) + (current queue size × 1/2^TAF)

The Time Average Factor (TAF) is the exponential weight factor used in calculating the average queue size. The time_average_factor parameter is not user-configurable and is set to a system-wide default value of 3. By locking TAF to a static value of 3, the average queue size closely tracks the current queue size so that WRED can respond quickly to long queues.

3.7.7.1. WRED MinThreshold and MaxThreshold Computation

CBS is configured in kilobytes through the CLI; MBS is configured in bytes or kilobytes. These configured values are converted to the corresponding number of buffers. The conversion factor is a non-user-configurable, fixed default value that is equal to the system-defined maximum frame size, ensuring that even the largest frames can be hosted in the allocated buffer pools. This type of WRED is called buffer-based WRED.

User-defined minThreshold and maxThreshold values, each defined as a percentage, are also converted to the number of buffers. The minT is converted to the system-minThreshold, and the maxT is converted to the system-maxThreshold.

The system-minT must be the absolute closest value to the minT that satisfies the formula below (2^x means 2 to the exponent x):

      system-maxThreshold – system-minThreshold = 2^x

Note: The 6-port Ethernet 10Gbps Adapter card and the 7705 SAR-X Ethernet ports use payload-based WRED (also called byte-based WRED); see Payload-based WRED. The above “system-minT” calculation does not apply to payload-based WRED. The 7705 SAR-X TDM ports use buffer-based WRED.

3.7.7.2. WRED on Bridging Domain (Ring) Queues

The bridging domain queues support the following DP (discard probability) values: 0% to 10%, 25%, 50%, 75%, and 100%. User-configured values are rounded down to match these DP values.

For example, configuring a DP to be 74% means that the actual value used is 50%.

Note: Tail drop for out-of-profile traffic begins when the queue occupancy of the out-of-profile traffic exceeds that of the committed buffer size.

3.7.7.3. Payload-based WRED

The third-generation Ethernet adapter cards and platforms use payload-based WRED rather than buffer-based WRED (see WRED MinThreshold and MaxThreshold Computation). Payload-based WRED does not count the unused overhead space (empty space in the buffer) when making discard decisions, whereas buffer-based WRED counts the unused overhead space. Payload-based WRED is also referred to as byte-based WRED.

When a queue on an adapter card that uses payload-based WRED reaches its maximum fill (that is, the total byte count exceeds the configured maximum threshold), tail drop begins and operates in the same way as it does on any other adapter card or platform.

With payload-based WRED, the discard decision is based on the number of bytes in the queue instead of the number of buffers in the queue. For example, to accumulate 512 bytes of payload in a queue will take four buffers if the frame size is 128 bytes, but will take one buffer if the frame size is 512 bytes or more. Basing discards on bytes rather than buffers improves the efficient use of queues. In either case, byte- or buffer-based WRED, random discards begin at the minimum threshold (minT) point.

For example, assume a queue has MBS set to 512 kilobytes (converts to 1000 buffers), minT (start-avg) is set to 10% (100 buffers), and maxT (max-avg) is set to 80% (800 buffers). Table 30 shows when discards and tail drop start when payload-based WRED is used.

Table 30:  Payload-Based WRED: Discard and Tail Drop Start  

Frame Size	Discards Start	Tail Drop Starts
128 bytes	400 buffers in the queue (100 x 4)	3200 buffers in the queue (800 x 4)
512 bytes	100 buffers in the queue	800 buffers in the queue
1024 bytes	100 buffers in the queue	800 buffers in the queue

For tail drop, if the high-priority-only threshold is set to 10%:

1. when any frame size is greater than or equal to 512 bytes, tail drop starts after 900 buffers are in use for low-priority traffic

Note: If an adapter card or platform other than a third-generation adapter card or platform is used in the previous example (that is, an adapter card or platform with buffer-based WRED), both WRED and tail drop start after 100 buffers are consumed, because both 128-byte and 512-byte frames fill one buffer (512 bytes). Since tail drop (which is buffer-based) and WRED (which is payload-based) operate differently, it is not recommended that tail drop and WRED be used in the same queue.

3.7.8. ATM Traffic Descriptor Profiles

Traffic descriptor profiles capture the cell arrival pattern for resource allocation. Source traffic descriptors for an ATM connection include at least one of the following:

QoS traffic descriptor profiles are applied on ATM VLL (Apipe) SAPs.

3.7.9. Fabric Profiles

Fabric profiles allow access and network ingress to-fabric shapers to have user-configurable rates of switching throughput from an adapter card towards the fabric.

Two fabric profile modes are supported: per-destination mode and aggregate mode. Both modes offer shaping towards the fabric from an adapter card, but per-destination shapers offer the maximum flexibility by precisely controlling the amount of traffic to each destination card at a user-defined rate.

For the 7705 SAR-8 and the 7705 SAR-18, the maximum rate depends on a number of factors, including platform, chassis variant, and slot type. See Configurable Ingress Shaping to Fabric (Access and Network) for details. For information about fabric shaping on the 7705 SAR-M, 7705 SAR-H, 7705 SAR-Hc, 7705 SAR-A, 7705 SAR-Ax, 7705 SAR-W, and 7705 SAR-Wx, see Fabric Shaping on the Fixed Platforms (Access and Network).

3.7.10. Shaper Policies

Shaper policies define dual-rate shaper parameters that control access or network traffic by providing tier-3 aggregate shaping to:

Note: For network egress traffic on a non-hybrid Gen-3 port, the CIR value of the shaper group is ignored because of the behavior of the 4-priority scheduler on Gen-3 hardware at network egress. For more information, see QoS for Gen-3 Adapter Cards and Platforms.

See Per-SAP Aggregate Shapers (H-QoS) On Gen-2 Hardware and Per-VLAN Network Egress Shapers for details on per-SAP and per-VLAN shapers.

3.7.11. QoS Policy Entities

Services are configured with default QoS policies. Additional policies must be explicitly created and associated. There is one default service ingress QoS policy, one default service egress QoS policy, and one default network QoS policy. Only one ingress QoS policy and one egress QoS policy can be applied to a SAP or network port.

When you create a new QoS policy, default values are provided for most parameters with the exception of the policy ID and queue ID values, descriptions, and the default action queue assignment. Each policy has a scope, default action, a description, and at least one queue. The queue is associated with a forwarding class.

All QoS policy parameters can be configured in the CLI. QoS policies can be applied to the following service types:

QoS policies can be applied to the following network entities:

Default QoS policies treat all traffic with equal priority and allow an equal chance of transmission (Best Effort forwarding class) and an equal chance of being dropped during periods of congestion. QoS prioritizes traffic according to the forwarding class and uses congestion management to control access ingress, access egress, and network traffic with queuing according to priority.

3.8.1. Reference Sources

For information on supported IETF drafts and standards, as well as standard and proprietary MIBs, refer to Standards and Protocol Support.