System management commands allow you to configure basic system management functions such as the system name, the router’s location and coordinates, and Common Language Location Identifier (CLLI) code as well as time zones, Network Time Protocol (NTP), Simple Network Time Protocol (SNTP) properties, CRON and synchronization properties.
On SR OS routers, it is possible to query the DNS server for IPv6 addresses. By default, the DNS names are queried for A-records only (address-preference is IPv4-only). If the address-preference is set to IPv6 first, the DNS server is queried for AAAA-records first, and if there is no successful reply, then A-records.
This section describes system information components.
The system name is the MIB II (RFC 1907, Management Information Base for Version 2 of the Simple Network Management Protocol (SNMPv2)) sysName object. By convention, this text string is the node’s fully-qualified domain name. The system name can be any ASCII-printable text string of up to 64 characters.
The system contact is the MIB II sysContact object. By convention, this text string is a textual identification of the contact person for this managed node, together with information on how to contact this person. The system contact can be any ASCII-printable text string of up to 80 characters.
The system location is the MIB II sysLocation object which is a text string conventionally used to describe the node’s physical location, for example, “Bldg MV-11, 1st Floor, Room 101”. The system location can be any ASCII-printable text string of up to 80 characters.
The system coordinates is the Nokia Chassis MIB tmnxChassisCoordinates object. This text string indicates the Global Positioning System (GPS) coordinates of the location of the chassis.
Two-dimensional GPS positioning offers latitude and longitude information as a four dimensional vector:
<direction, hours, minutes, seconds>
where direction is one of the four basic values: N, S, W, E, hours ranges from 0 to 180 (for latitude) and 0 to 90 for longitude, and minutes and seconds range from 0 to 60.
<W, 122, 56, 89> is an example of longitude and <N, 85, 66, 43> is an example of latitude.
System coordinates can be expressed in different notations, examples include:
The system coordinates can be any ASCII-printable text string up to 80 characters.
Do not configure named objects with a name that starts with “_tmnx_”, or with “_” in general.
A CLLI code string for the device is an 11-character standardized geographic identifier that uniquely identifies the geographic location of places and certain functional categories of equipment unique to the telecommunications industry. The CLLI code is stored in the Nokia Chassis MIB tmnxChassisCLLICode object.
The CLLI code can be any ASCII-printable text string of up to 11 characters.
DNS Security (DNSSEC) Extensions are now implemented in the SR OS, allowing operators to configure DNS behavior of the router to evaluate whether the Authenticated Data bit was set in the response received from the recursive name server and to trust the response, or ignore it.
SR-series routers are equipped with a real-time system clock for time keeping purposes. When set, the system clock always operates on Coordinated Universal Time (UTC), but the SR-series routers OS software has options for local time translation as well as system clock synchronization.
Setting a time zone in SR OS allows for times to be displayed in the local time rather than in UTC. SR OS has both user-defined and system-defined time zones.
A user-defined time zone has a user-assigned name of up to four printable ASCII characters in length and is unique from the system-defined time zones. For user-defined time zones, the offset from UTC is configured as well as any summer time adjustment for the time zone.
SR OS includes multiple commands to control the presentation of times in either UTC or local time zone format. For a CLI session, the environment variable time-display may be set to indicate UTC or local time zone. This setting only affects time strings shown during that specific CLI session. In addition, a global setting of config>system>time>prefer-local-time can be used to control time strings for objects with larger scope that a single CLI session, including the following:
A separate control per log file controls the format of the time strings on the event recorded into the logs (separate from the log filename and header information). Use the config>log>log-id>time-format command to set these time strings.
The SR OS system-defined time zones are listed in Table 9, which includes both time zones with and without summer time correction.
Acronym | Time Zone Name | UTC Offset |
Europe | ||
GMT | Greenwich Mean Time | UTC |
BST | British Summer Time | UTC +1 |
IST | Irish Summer Time | UTC +1* |
WET | Western Europe Time | UTC |
WEST | Western Europe Summer Time | UTC +1 |
CET | Central Europe Time | UTC +1 |
CEST | Central Europe Summer Time | UTC +2 |
EET | Eastern Europe Time | UTC +2 |
EEST | Eastern Europe Summer Time | UTC +3 |
MSK | Moscow Time | UTC +3 |
MSD | Moscow Summer Time | UTC +4 |
US and Canada | ||
AST | Atlantic Standard Time | UTC -4 |
ADT | Atlantic Daylight Time | UTC -3 |
EST | Eastern Standard Time | UTC -5 |
EDT | Eastern Daylight Saving Time | UTC -4 |
ET | Eastern Time | Either as EST or EDT, depending on place and time of year |
CST | Central Standard Time | UTC -6 |
CDT | Central Daylight Saving Time | UTC -5 |
CT | Central Time | Either as CST or CDT, depending on place and time of year |
MST | Mountain Standard Time | UTC -7 |
MDT | Mountain Daylight Saving Time | UTC -6 |
MT | Mountain Time | Either as MST or MDT, depending on place and time of year |
PST | Pacific Standard Time | UTC -8 |
PDT | Pacific Daylight Saving Time | UTC -7 |
PT | Pacific Time | Either as PST or PDT, depending on place and time of year |
HST | Hawaiian Standard Time | UTC -10 |
AKST | Alaska Standard Time | UTC -9 |
AKDT | Alaska Standard Daylight Saving Time | UTC -8 |
Australia | ||
AWST | Western Standard Time (e.g., Perth) | UTC +8 |
ACST | Central Standard Time (e.g., Darwin) | UTC +9.5 |
AEST | Eastern Standard/Summer Time (e.g., Canberra) | UTC +10 |
NTP is the Network Time Protocol defined in RFC 1305, Network Time Protocol (Version 3) Specification, Implementation and Analysis and RFC 5905, Network Time Protocol Version 4: Protocol and Algorithms Specification. It allows for the participating network nodes to keep time more accurately and more importantly they can maintain time in a more synchronized fashion between all participating network nodes.
SR OS uses an NTP process based on a reference build provided by the Network Time Foundation. Nokia strongly recommends that the users review RFC 8633, Network Time Protocol Best Current Practices, when they plan to use NTP with the router. The RFC section “Using Enough Time Sources” indicates that using only two time sources (NTP servers) can introduce instability if they provide conflicting information. To maintain accurate time, Nokia recommends configuring three or more NTP servers.
NTP uses stratum levels to define the number of hops from a reference clock. The reference clock is considered to be a stratum-0 device that is assumed to be accurate with little or no delay. Stratum-0 servers cannot be used in a network. However, they can be directly connected to devices that operate as stratum-1 servers. A stratum-1 server is an NTP server with a directly-connected device that provides Coordinated Universal Time (UTC), such as a GPS or atomic clock.
The higher stratum levels are separated from the stratum-1 server over a network path, thus, a stratum-2 server receives its time over a network link from a stratum-1 server. A stratum-3 server receives its time over a network link from a stratum-2 server.
SR OS routers normally operate as a stratum-2 or higher device. The router relies on an external stratum-1 server to source accurate time into the network. However, SR OS also allows for the use of the local PTP recovered time to be sourced into NTP. In this latter case, the local PTP source appears as a stratum-0 server and SR OS advertises itself as a stratum-1 server. Activation of the PTP source into NTP may impact the network NTP topology because the SR OS router is promoted to stratum-1.
SR OS router runs a single NTP clock which then operates NTP message exchanges with external NTP clocks. Exchanges can be made with external NTP clients, servers, and peers. These exchanges can be through the base, management, or VPRN routing instances.
NTP operates associations between clocks as either client or server, symmetric active and symmetric passive, or broadcast modes. These modes of operation are applied according to which elements are configured on the router. To run server mode, the operator must enable NTP server mode for the base and each desired VPRN routing instance. To run client mode, the operator must configure external servers. If both the local router and remote router are configured with each other as peers, then the router operates in symmetric active mode. If only one side of the association has peering configured, then the modes are symmetric passive. To operate using broadcast mode, interfaces must be configured to transmit as broadcast servers or receive as broadcast clients.
NTP server operation for both unicast and broadcast communication within a VPRN is configured within the VPRN (refer to the NTP Within a VPRN Service section in 7450 ESS, 7750 SR, 7950 XRS, and VSR Layer 3 Services Guide: IES and VPRN).
Note: NTP provides lightweight synchronization across a network for alignment of system time for logging purposes. NTP does not provide the high accuracy time needed for the on-air applications of the mobile base stations. The more recent PTP protocol has been developed for these applications (see Network Synchronization). |
The following NTP elements are supported:
For synchronizing the system clock with outside time sources, the SR OS includes a Simple Network Time Protocol (SNTP) client. As defined in RFC 2030, SNTP Version 4 is an adaptation of the Network Time Protocol (NTP). SNTP typically provides time accuracy within 100 milliseconds of the time source. SNTP can only receive the time from NTP servers; it cannot be used to provide time services to other systems. SNTP is a compact, client-only version of NTP. SNTP does not authenticate traffic.
SNTP can be configured in both unicast client modes (point-to-point) and broadcast client modes (point-to-multipoint). SNTP should be used only at the extremities of the synchronization subnet. SNTP clients should operate only at the highest stratum (leaves) of the subnet and in configurations where no NTP or SNTP client is dependent on another SNTP client for synchronization. SNTP time servers should operate only at the root (stratum 1) of the subnet and then only in configurations where no other source of synchronization other than a reliable radio clock is available. External servers may only be specified using IPv4 addresses.
In the SR OS, the SNTP client can be configured for either broadcast or unicast client mode.
The CRON feature supports periodic and date and time-based scheduling in SR OS. CRON can be used, for example, to schedule Service Assurance Agent (SAA) functions. CRON functionality includes the ability to specify scripts that need to be run, when they are scheduled, including one-time only functionality (one-shot), interval and calendar functions. Scheduled reboots, peer turn ups, service assurance agent tests and more can all be scheduled with CRON, as well as OAM events, such as connectivity checks, or troubleshooting runs.
CRON supports the schedule element. The schedule function configures the type of schedule to run, including one-time only (one-shot), periodic, or calendar-based runs. All runs are determined by month, day of month or weekday, hour, minute, and interval (seconds).
This section discusses the high availability (HA) routing options and features available to service providers that help diminish vulnerability at the network or service provider edge and alleviate the effect of a lengthy outage on IP networks.
High availability is an important feature in service provider routing systems. High availability is gaining momentum due to the unprecedented growth of IP services and applications in service provider networks driven by the demand from the enterprise and residential communities. Downtime can be very costly, and, in addition to lost revenue, customer information and business-critical communications can be lost. High availability is the combination of continuous uptime over long periods (Mean Time Between Failures (MTBF)) and the speed at which failover or recovery occurs (Mean Time To Repair (MTTR)).
The popularity of high availability routing is evident at the network or service provider edge where thousands of connections are hosted and rerouting options around a failed piece of equipment can often be limiting. Or, a single access link exists to a customer because of additional costs for redundant links. As service providers converge business-critical services such as real-time voice (VoIP), video, and VPN applications over their IP networks, high availability becomes much more stringent compared to the requirements for best-effort data. Network and service availability become critical aspects when offering advanced IP services which dictates that IP routers that are used to construct the foundations of these networks be resilient to component and software outages.
For high availability configuration information, see Synchronization and Redundancy.
As more and more critical commercial applications move onto the IP/MPLS networks, providing high availability services becomes increasingly important. This section describes high availability features for routers. Most of these features only apply to routers with two Control Processor Modules (CPM).
The redundancy features enable the duplication of data elements and software functionality to maintain service continuation in case of outages or component failure.
Refer to the 7450 ESS, 7750 SR, and VSR Multiservice Integrated Service Adapter and Extended Services Appliance Guide for information about redundancy for the Integrated Service Adapter (ISA).
Software outages are challenging even when baseline hardware redundancy is in place. There should be a balance to provide high availability routing otherwise router problems typically propagate not only throughout the service provider network, but also externally to other connected networks possibly belonging to other service providers. This could affect customers on a broad scale. Presently, there are several software availability features that contribute to the percentage of time that a router is available to process and forward traffic.
To fully appreciate high availability, you should realize that all routing protocols specify minimum time intervals in which the peer device must receive an acknowledgment before it disconnects the session.
Therefore, router software has to recover faster than the specified time interval to maintain up time.
Features configured on the active device CPM are saved on the standby CPM as well. When the active device CPM fails, these features are brought up on the standby device CPM that takes over the mastership.
Even with modern modular and stable software, the failure of route processor hardware or software can cause the router to reboot or cause other service impacting events. In the best circumstances, failure leads to the initialization of a redundant route processor, which hosts the standby software configuration, to become the active processor. The following options are available.
Component redundancy is critical to reduce MTTR for the system and primarily consists of the following router features:
Router hardware architecture plays a key role in the availability of the system. The principle router architecture styles are centralized and distributed. In these architectures, both active and standby route processors, I/O modules (IOMs) (also called line cards), fans, and power supplies maintain a low MTTR for the routing system.
However, in a centralized architecture, packet processing and forwarding is performed in a central shared route processor and the individual line cards are relatively simple. The cards rely solely on the route processor for routing and forwarding intelligence and, should the centralized route processor fail, there is greater impact to the system overall, as all routing and packet forwarding stops.
In a distributed system, the packet forwarding functionality is situated on each line card. Distributing the forwarding engines off the central route processor and positioning one on each line card lowers the impact of route processor failure as the line cards can continue to forward traffic during an outage.
The distributed system is better suited to enable the convergence of business critical services such as real-time voice (VoIP), Video, and VPN applications over IP networks with superior performance and scalability. The centralized architecture can be prone to performance bottleneck issues and limits service offerings through poor scalability which may lead to customer and service SLA violations.
All service-related statistics are kept during a switchover. Services, SDPs, and SAPs remains up with a minimum loss of forwarded traffic during a CPM switchover.
When there is a switchover and the standby CPM becomes active, the accounting servers are checked and if they are administratively up and capable of coming online (media present, and so on), the standby is brought online and new accounting files are created at that point. Users must manually copy the accounting records from the failed CPM.
In a control plane failure or a forced switchover event, the router continues to forward packets using the existing stale forwarding information. Nonstop forwarding requires clean control plane and data plane separation. Usually the forwarding information is distributed to the IOMs, XCMs and XMAs.
Nonstop forwarding is used to notify peer routers to continue forwarding and receiving packets, even if the route processor (control plane) is not working or is in a switch-over state. Nonstop forwarding requires clean control plane and data plane separation and usually the forwarding information is distributed to the line cards. This method of availability has both advantages and disadvantages. Nonstop forwarding continues to forward packets using the existing stale forwarding information during a failure. This may cause routing loops and black holes, and also requires that surrounding routers adhere to separate extension standards for each protocol. Every router vendor must support protocol extensions for interoperability.
With NSR on the SR-series router devices, routing neighbors are unaware of a routing process fault. If a fault occurs, a reliable and deterministic activity switch to the inactive control complex occurs such that routing topology and reachability are not affected, even in the presence of routing updates. NSR achieves high availability through parallelization by maintaining up to date routing state information, at all times, on the standby route processor. This capability is achieved independently of protocols or protocol extensions, providing a more robust solution than graceful restart protocols between network routers.
The NSR implementation on the SR-series routers supports all routing protocols. NSR makes it possible to keep the existing sessions (BGP, LDP, OSPF, etc.) during a CPM switchover, including support for MPLS signaling protocols. Peers do not see any change.
Protocol extensions are not required. There are no interoperability issues and there is no need to define protocol extensions for every protocol. Unlike nonstop forwarding and graceful restart, the forwarding information in NSR is always up to date, which eliminates possible blackholes or forwarding loops.
Traditionally, addressing high availability issues have been patched through non-stop forwarding solutions. With the implementation of NSR, these limitations are overcome by delivering an intelligent hitless failover solution. This enables a carrier-class foundation for transparent networks, required to support business IP services backed by stringent SLAs. This level of high availability poses a major issue for conventional routers whose architectural design limits or prevents them from implementing NSR.
During a switchover, system control and routing protocol execution are transferred from the active to the standby CPM.
An automatic switchover may occur under the following conditions:
A manual switchover can occur under the following conditions:
Synchronization between the CPMs includes the following:
Configuration and boot-env synchronization are supported in admin>redundancy> synchronize and config>redundancy>synchronize contexts.
If a new standby CPM is inserted into the system, it synchronizes with the active CPM upon a successful boot process.
If the standby CPM is rebooted, it synchronizes with the active CPM upon a successful boot process.
When configuration or state changes occur, an incremental synchronization is conducted from the active CPM to the standby CPM.
If the synchronization fails, the standby does not reboot automatically. The show redundancy synchronization command displays synchronization output information.
If the active and standby are not synchronized for some reason, users can manually synchronize the standby CPM by rebooting the standby by issuing the admin reboot standby command on the active or the standby CPM.
SR-series routers supporting redundancy use a 1:1 redundancy scheme. Redundancy methods facilitate system synchronization between the active and standby Control Processor Modules (CPMs) so they maintain identical operational parameters to prevent inconsistencies in the event of a CPM failure.
When automatic system synchronization is enabled for an entity, any save or delete file operations configured on the primary, secondary or tertiary choices on the active CPM file system are mirrored in the standby CPM file system.
Although software configurations and images can be copied or downloaded from remote locations, synchronization can only occur locally between compact flash drives (cf1:, cf2:, and cf3:).
Synchronization can occur either:
Typically, the first Switch Fabric (SF)/CPM card installed in a redundant SR-series router chassis assumes the role as active, regardless of being inserted in Slot A or B. The next CPM installed in the same chassis then assumes the role as the standby CPM. If two CPM are inserted simultaneously (or almost simultaneously) and are booting at the same time, then preference is given to the CPM installed in Slot A.
If only one CPM is installed in a redundant router device, then it becomes the active CPM regardless of the slot it is installed in.
The active and standby designations can be visually determined by LEDs on the CPM/CCM faceplate. Refer to the appropriate platform Installation Guide for LED indicator details.
The following output shows that the CPM installed in Slot A is acting as the active CPM and the CPM installed in Slot B is acting as the standby.
The following is an example of the 7950 XRS output:
The following console message displays when a CPM boots, sees an active CPM, and becomes the standby CPM:
When an active CPM goes offline (due to reboot, removal, or failure), the standby CPM takes control without rebooting or initializing itself. It is assumed that the CPMs are synchronized, therefore, there is no delay in operability. When the CPM that went offline boots and then comes back online, it becomes the standby CPM.
When the standby CPM comes online, the following output is shown:
The SR OS platform provides a resilient out-of-band (OOB) management Ethernet redundancy mode for system management.
When the management Ethernet port is down on the active CPM, the OOB Ethernet redundancy feature allows the active CPM to use the management Ethernet port of the standby CPM, as shown in Figure 9 and Figure 10.
OOB management Ethernet port redundancy is enabled using the config>redundancy>mgmt-ethernet command.
The persistence feature on the 7750 SR allows information learned through DHCP snooping across reboots to be kept. This information can include data such as the IP address, MAC binding information, lease length information, and ingress SAP information (required for VPLS snooping to identify the ingress interface). This information is referred to as the DHCP lease-state information.
When a DHCP message is snooped, there are steps that make the data persistent in a system with dual CPMs. In systems with only one CPM, only Step 1 applies. In systems with dual CPMs, all steps apply.
A high rate of DHCP renewals can create a load on the compact flash file system when subscriber management and/or DHCP server persistence is enabled. To optimize the access to the Dynamic Data Persistency (DDP) files on the compact flash, a lease-time threshold can be specified that controls the eligibility of a DHCP lease for persistency updates when no other data other than the lease expiry time is to be updated.
When the offered lease time of the DHCP lease is less than the configured threshold, the lease is flagged to skip persistency updates and is installed with its full lease time upon a persistency recovery after a reboot.
The dhcp-leasetime-threshold command controls persistency updates for:
To check if a DHCP relay or proxy lease is flagged to skip persistency updates, use the tools dump persistence submgt record record-key CLI command. When flagged to skip persistency updates, the persistency record output includes “Skip Persistency Updates: true”.
To check if a DHCP server lease is flagged to skip persistency updates, use the tools dump persistence dhcp-server record record-key CLI command. When flagged to skip persistency updates, the persistency record output includes “lease mode : LT” (LT = Lease Time) and a “lease time : …” field. When not flagged to skip persistency updates, the persistency record output includes “lease mode : ET” (ET = Expiry Time) and an “expires : …” field.
This section describes network synchronization capabilities available on SR OS platforms. These capabilities involve multiple approaches to network timing; namely SDH/SONET, Synchronous Ethernet, BITS, and Adaptive clocking and a Precision Time Protocol (PTP) IEEE 1588v2. These features address barriers to entry by:
Network synchronization is commonly distributed in a hierarchical master-slave topology at the physical layer as shown in Figure 11.
The architecture shown in Figure 11 provides the following benefits:
The synchronization network is designed so a clock always receives timing from a clock of equal or higher stratum or quality level. This ensures that if an upstream clock has a fault condition (for example, loses its reference and enters a holdover or free-run state) and begins to drift in frequency, the downstream clock is able to follow it. For greater reliability and robustness, most offices and nodes have at least two synchronization references that can be selected in priority order (such as primary and secondary).
Further levels of resiliency can be provided by designing a capability in the node clock that operates within prescribed network performance specifications without any reference for a specified time-frame. A clock operating in this mode is said to hold the last known state over (or holdover) until the reference lock is once again achieved. Each level in the timing hierarchy is associated with minimum levels of network performance.
Each synchronization capable port can be independently configured to transmit data using the node reference timing or loop timing. In addition, some TDM channels can use adaptive timing.
Transmission of a reference clock through a chain of Ethernet equipment requires that all equipment supports Synchronous Ethernet. A single piece of equipment that is not capable of performing Synchronous Ethernet breaks the chain. Ethernet frames still get through but downstream devices should not use the recovered line timing as it is be traceable to an acceptable stratum source.
The timing subsystem for the platforms has a central clock located on the CPM (motherboard). The timing subsystem performs many of the duties of the network element clock as defined by Telcordia (GR-1244-CORE) and ITU-T G.781.
The system can select from up to three (7950 XRS) or four (7450 ESS and 7750 SR) timing inputs to train the local oscillator. The priority order of these references must be specified. This is a simple ordered list of inputs: {bits, ref1, ref2, ptp}. The CPM clock output shall have the ability to drive the clocking for all line cards in the system. The routers support selection of the node reference using Quality Level (QL) indications. See Figure 12 for a description of the synchronization selection process for the CPM clock.
Note: Not all signals are available on all platforms. |
The recovered clock can derive its timing from any of the following:
The BITS ports accept T1 or E1 signal formats. Some hardware also supports the 2048 kHz signal format. The format must be common between all BITSin and BITSout ports.
All settings of the signal characteristics for the BITS input apply to both ports. When the active CPM considers the BITS input as a possible reference, it first considers the BITS input port on the active CPM or CCM followed by the BITS input port on the standby CPM or CCM in that relative priority order. This relative priority order is in addition to the user-definable ref-order. For example, a ref-order of bits ref1 ref2 would actually be BITS in (active CPM or CCM), followed by BITS in (standby CPM or CCM), followed by ref1, followed by ref2. When ql-selection is enabled, the QL of each BITS input port is viewed independently. The higher QL source is chosen.
When the active CPM considers the SyncE/1588 as a possible reference, the active CPM first considers the SyncE/1588 port on the active CPM or CCM, followed by the SyncE/1588 port on the standby CPM or CCM in that relative priority order. This relative priority order is in addition to the user-definable ref-order. For example, a ref-order of synce ref1 ref2 would actually be SyncE/1588 (active CPM or CCM), followed by SyncE/1588 (standby CPM or CCM), followed by ref1, followed by ref2. When ql-selection is enabled, the QL of each SyncE/1588 input port is viewed independently. The higher QL source is chosen.
The following behavior applies to the platform architecture existing on 7750 SR-7/12/12e, 7750 SR-2s/7s/14s, 7750 SR-1e/2e/3e, 7750 SR-a4/a8, and 7450 ESS-7/12: When the BITS or SyncE port on the standby CPM is an option as input reference into the central clock of the active CPM, a display of the central clock data on the standby CPM indicates that it is locked to its local BITS or SyncE input. This is expected behavior and required to make the BITS input on the standby available to the active CPM as an option for reference selection.
The restrictions on the location for the source-port or source-bits for ref1 and ref2 are listed in Table 10.
Platform | Ref1 Slots | Ref2 Slots | Notes |
7450 ESS-7 | 1 to 2 | 3 to 5 | — |
7450 ESS-12 | 1 to 5 | 6 to 10 | — |
7750 SR-1 | 1 | 1 | Ref1 and ref2 cannot be on the same MDA |
7750 SR-7 | 1 to 2 | 3 to 5 | — |
7750 SR-12 | 1 to 5 | 6 to 10 | — |
7750 SR-12e | 1 to 5 | 6 to 9 | — |
7750 SR-a4 | 1 | 1 | Ref1 and ref2 cannot be on the same MDA. Two CPMs must be installed to allow two references to be used. |
7750 SR-a8 | 1 to 2 | 1 to 2 | Ref1 and ref2 cannot be on the same slot. |
7750 SR-1e | 1 | 1 | Ref1 and ref2 cannot be on the same MDA |
7750 SR-2e | 1 to 2 | 1 to 2 | Ref1 and ref2 cannot be on the same MDA |
7750 SR-3e | 1 to 3 | 1 to 3 | Ref1 and ref2 cannot be on the same MDA |
7750 SR-1s | 1 | 1 | Ref1 and ref2 cannot be on the same MAC chip. Refer to the 7750 SR-1s Installation Guide or use the show datapath command for the mappings. |
7750 SR-2s | 1 to 2 | 1 to 2 | Ref1 and ref2 cannot be on the same slot. |
7750 SR-7s | 1 to 6 | 1 to 6 | Ref1 and ref2 cannot be on the same slot. Slot 6 cannot be used if a CPM has been installed in that slot. |
7750 SR-14s | 1 to 6 | 1 to 6 | Ref1 and ref2 cannot be on the same slot. |
7950 XRS-20 | 1 to 10 | 1 to 10 | Ref1 and ref2 cannot be on the same slot |
7950 XRS-20e | 1 to 10 | 1 to 10 | Ref1 and ref2 cannot be on the same slot |
7950 XRS-40 | 1 to 10 | 1 to 10 | Ref1 and ref2 cannot be on the same slot |
The BITS output ports can be configured to provided either the unfiltered recovered line clock from a line card port or the output of the central clock. The first case would be used if the port was connected to deliver an input reference directly to dedicated timing device in the facility (BITS or SASE device). The second case would be used to test the quality of the clocking used by the router.
When QL selection mode is disabled, then the reversion setting controls when the central clock can re-select a previously failed reference.
The Table 11 shows the selection followed for two reference in both revertive and non-revertive modes:
Status of Reference A | Status of Reference B | Active Reference Non-revertive Case | Active Reference Revertive Case |
OK | OK | A | A |
Failed | OK | B | B |
OK | OK | B | A |
OK | Failed | A | A |
OK | OK | A | A |
Failed | Failed | holdover | holdover |
OK | Failed | A | A |
Failed | Failed | holdover | holdover |
Failed | OK | B | B |
Failed | Failed | holdover | holdover |
OK | OK | A or B | A |
The central clock architecture described above applies to each chassis of the 7950 XRS-40. There is a central clock located on each of the CPMs present in the extension chassis. However, there is no configuration for the central clocks on the CPMs of the extension chassis. The central clocks only use the BITS input ports of the extension chassis for their input reference. It is assumed that the quality of the reference provided into the BITS input ports of the extension chassis CPMs is equal to the quality of the Master chassis central clocks. Refer to the Installation Guide for appropriate physical cabling to support this architecture.
SSM provides a mechanism to allow the synchronization distribution network to both determine the quality level of the clock sourcing a given synchronization trail and to allow a network element to select the best of multiple input synchronization trails. Synchronization Status messages have been defined for various transport protocols including SONET/SDH, T1/E1, and Synchronous Ethernet, for interaction with office clocks, such as BITS or SSUs and embedded network element clocks.
SSM allows equipment to autonomously provision and reconfigure (by reference switching) their synchronization references, while helping to avoid the creation of timing loops. These messages are particularly useful to allow synchronization reconfigurations when timing is distributed in both directions around a ring.
The following sections provide details about the SSM message functionality for different signal types. These functions apply to all platforms that support the given signal type.
DS1 signals can carry an indication of the quality level of the source generating the timing information using the SSM transported within the 1544 kb/s Extended Super Frame (ESF) Data Link (DL) of the signal as specified in Recommendation G.704. No such provision is extended to SF formatted DS1 signals.
The format of the data link messages in ESF frame format is "0xxx xxx0 1111 1111", transmitted rightmost bit first. The six bits denoted "xxx xxx" contain the actual message; some of these messages are reserved for synchronization messaging. It takes 32 frames (such as 4 ms) to transmit all 16 bits of a complete DL.
E1 signals can carry an indication of the quality level of the source generating the timing information using the SSM as specified in Recommendation G.704.
One of the Sa4 to Sa8 bits, (the actual Sa bit is for operator selection), is allocated for Synchronization Status Messages. To prevent ambiguities in pattern recognition, it is necessary to align the first bit (San1) with frame 1 of a G.704 E1 multi-frame.
The numbering of the San (n = 4, 5, 6, 7, 8) bits. A San bit is organized as a 4-bit nibble San1 to San4. San1 is the most significant bit; San4 is the least significant bit.
The message set in San1 to San4 is a copy of the set defined in SDH bits 5 to 8 of byte S1.
The SSM of SDH and SONET interfaces is carried in the S1 byte of the frame overhead. Each frame contains the four bit value of the QL.
DS3/E3 signals are not required to be synchronous. However, it is acceptable for their clocking to be generated from a synchronization source. The 7750 SR and the 7450 ESS permit E3/DS3 physical ports to be specified as a central clock input reference.
DS3/E3 signals do not support an SSM channel. QL-override should be used for these ports if ql-selection is enabled
Traditionally, Ethernet-based networks employ the physical layer transmitter clock to be derived from an inexpensive +/-100ppm crystal oscillator and the receiver locks onto it. There is no need for long term frequency stability because the data is packetized and can be buffered. For the same reason there is no need for consistency between the frequencies of different links. However, you can derive the physical layer transmitter clock from a high quality frequency reference by replacing the crystal with a frequency source traceable to a primary reference clock. This would not affect the operation of any of the Ethernet layers, for which this change would be transparent. The receiver at the far end of the link would lock onto the physical layer clock of the received signal, and thus itself gain access to a highly accurate and stable frequency reference. Then, in a manner analogous to conventional hierarchical master-slave network synchronization, this receiver could lock the transmission clock of its other ports to this frequency reference and a fully time synchronous network could be established.
The advantage of using Synchronous Ethernet, compared with methods that rely on sending timing information in packets over an unclocked physical layer, is that it is not influenced by impairments introduced by the higher levels of the networking technology (packet loss, packet delay variation). Hence, the frequency accuracy and stability may be expected to exceed those of networks with unsynchronized physical layers.
Synchronous Ethernet allows operators to gracefully integrate existing systems and future deployments into conventional industry-standard synchronization hierarchy. The concept behind synchronous Ethernet is analogous to SONET/SDH system timing capabilities. It allows the operator to select any (optical) Ethernet port as a candidate timing reference. The recovered timing from this port is then used to time the system (for example, the CPM locks to this configured reference selection). The operator then could ensure that any of system output would be locked to a stable traceable frequency source.
If the port is a fixed copper Ethernet port and in 1000BASE-T mode of operation, there is a dependency on the 802.3 link timing for the Synchronous Ethernet functionality (refer to ITU-T G.8262). The 802.3 link Master-Slave timing states must align with the desired direction of Synchronous Ethernet timing flow. When a fixed copper Ethernet port is specified as an input reference for the node or when it is removed as an input reference for the node, an 802.3 link auto-negotiation is triggered to ensure the link timing aligns properly.
The SSM of Synchronous Ethernet uses an Ethernet OAM PDU that uses the slow protocol subtype. For a complete description of the format and processing, refer to ITU-T G.8264.
The following clock source quality levels have been identified for the purpose of tracking network timing flow. These levels make up all of the defined network deployment options given in Recommendation G.803 and G.781. The Option I network is a network developed on the original European SDH model; whereas, the Option II network is a network developed on the North American SONET model.
In addition to the QL values received over SSM of an interface, the standards also define additional codes for internal use. These include the following:
There is also an internal quality level of QL-UNKNOWN. This is used to differentiate from a received QL-STU code but is equivalent for the purposes of QL selection.
Table 12 lists the synchronization message coding and source priorities for SSM received.
SSM value received on port | ||||
SDH interface SyncE interface in SDH mode | SONET Interface SyncE interface in SONET mode | E1 interface | T1 interface (ESF) | Internal Relative Quality Level |
0010 (prc) | 0001 (prs) | 0010 (prc) | 00000100 11111111 (prs) | 1 - Best quality |
0000 (stu) | 00001000 11111111 (stu) | 2 | ||
0111 (st2) | 00001100 11111111 (ST2) | 3 | ||
0100 (ssua) | 0100 (tnc) | 0100 (ssua) | 01111000 11111111 (TNC) | 4 |
1101 (st3e) | 01111100 11111111 (ST3E) | 5 | ||
1000 (ssub) | 1000 (ssub) | 6 | ||
1010 (st3/eec2) | 00010000 11111111 (ST3) | 7 | ||
1011 (sec/eec1) | 1011 (sec) | 8 - Lowest quality qualified in QL-enabled mode | ||
1100 (smc) | 00100010 11111111 (smc) | 9 | ||
00101000 11111111 (st4) | 10 | |||
1110 (pno) | 01000000 11111111 (pno) | 11 | ||
1111 (dnu) | 1111 (dus) | 1111 (dnu) | 00110000 11111111 (dus) | 12 |
Any other | Any other | Any other | N/A | 13- QL_INVALID |
14- QL-FAILED | ||||
15 - QL-UNC |
Table 13 lists the synchronization message coding and source priorities for SSM transmitted.
SSM values to be transmitted by interface of type | ||||
Internal Relative Quality Level | SDH interface SyncE interface in SDH mode | SONET Interface SyncE interface in SONET mode | E1 interface | T1 interface (ESF) |
1 - Best quality | 0010 (prc) | 0001 (PRS) | 0010 (prc) | 00000100 11111111 (PRS) |
2 | 0100 (ssua) | 0000 (stu) | 0100 (ssua) | 00001000 11111111 (stu) |
3 | 0100 (ssua) | 0111 (st2) | 0100 (ssua) | 00001100 11111111 (st2) |
4 | 0100 (ssua) | 0100 (tnc) | 0100 (ssua) | 01111000 11111111 (tnc) |
5 | 1000 (ssub) | 1101 (st3e) | 1000 (ssub) | 01111100 11111111 (st3e) |
6 | 1000 (ssub) | 1010 (st3/eec2) | 1000 (ssub) | 00010000 11111111 (st3) |
7 | 1011 (sec/eec1) | 1010 (st3/eec2) | 1011 (sec) | 00010000 11111111 (st3) |
8 - Lowest quality qualified in QL-enabled mode | 1011 (sec/ eec1) | 1100 (smc) | 1011 (sec) | 00100010 11111111 (smc) |
9 | 1111 (dnu) | 1100 (smc) | 1111 (dnu) | 00100010 11111111 (smc) |
10 | 1111 (dnu) | 1111 (dus) | 1111 dnu | 00101000 11111111 (st4) |
11 | 1111 (dnu) | 1110 (pno) | 1111 (dnu) | 01000000 11111111 (pno) |
12 | 1111 (dnu) | 1111 (dus) | 1111 (dnu) | 00110000 11111111 (dus) |
13- QL_INVALID | 1111 (dnu) | 1111 (dus) | 1111 (dnu) | 00110000 11111111 (dus) |
14- QL-FAILED | 1111 (dnu) | 1111 (dus) | 1111 (dnu) | 00110000 11111111 (dus) |
15 - QL-UNC | 1011 (sec/eec1) | 1010 (st3/eec2) | 1011 (sec) | 00010000 11111111 (st3) |
Note: When the internal Quality level is in the range of 9 through 14, the output codes shown in Table 13, only appear if QL selection is disabled. If ql-selection is enabled, then all of these internal states are changed to internal state 15 (Holdover) and the ssm value generated reflects the holdover quality of the internal clock. |
The central clock of the node supports several advanced features of the G.781 standard. These include the specification of a minimum acceptable QL value for the input references, the specification of a minimum acceptable QL value for the BITS output port, the ability to squelch the BITS output signal, and the specification of a Wait To Restore timer for input references. These features allow for more options in the management of the synchronization topology.
Precision Time Protocol (PTP) is a timing-over-packet protocol defined in the IEEE 1588v2 standard 1588 PTP 2008.
PTP may be deployed as an alternative timing-over-packet option to Adaptive Clock Recovery (ACR). PTP provides the capability to synchronize network elements to a Stratum-1 clock or primary reference clock (PRC) traceable frequency source over a network that may or may not be PTP-aware. PTP has several advantages over ACR. It is a standards-based protocol, has lower bandwidth requirements, can transport both frequency and time, and can potentially provide better performance.
Support is provided for an ordinary clock in slave or master mode or a boundary clock. When configured as an ordinary clock master, PTP can only be used for the distribution of a frequency reference, not a time reference. The boundary clock and ordinary clock slave can be used for both frequency and time distribution.
The ordinary clock master, ordinary clock slave, and boundary clock communicate with neighboring IEEE 1588v2 clocks. These neighbor clocks can be ordinary clock masters, ordinary clock slaves, or boundary clocks. The communication can be based on either unicast IPv4 sessions transported through IP interfaces or multicast Ethernet transported through Ethernet ports.
For the unicast IP sessions, the external clocks are labeled 'peers'. There are two types of peers: configured and discovered. An ordinary clock slave or a boundary clock should have configured peers for each PTP neighbor clock from which it might accept synchronization information. The router initiates unicast sessions with all configured peers. An ordinary clock master or boundary clock accepts unicast session requests from external peers. If the peer is not a configured peer, then it is considered a discovered peer. An ordinary clock master or boundary clock can deliver synchronization information toward discovered peers. Figure 13 shows the relationship of various neighbor clocks using unicast IP sessions to communicate with a 7750 SR configured as a boundary clock with two configured peers.
For multicast Ethernet operation, the router shall listen for and transmit PTP messages using the configured multicast MAC address. Neighbor clocks are discovered via the reception of messages through an enabled Ethernet port. An ordinary clock master, ordinary clock slave, and a boundary clock support more than one neighbor PTP clock connecting into a single port. This might be encountered with the deployment of an Ethernet multicast LAN segment between the local clock and the neighbor PTP ports using an End to end transparent clock or an Ethernet switch. The Ethernet switch is not recommended due to the introduction of PDV and the potential degradation of performance but it can be used if appropriate to the application. Figure 14 shows the relationship of various neighbor clocks using multicast Ethernet sessions to a 7750 SR configured as a boundary clock. The 7750 SR has three ports configured for multicast Ethernet communications. Port 1/2/1 of the 7750 SR shows a connection where there are two neighbor clocks connecting to one port of the 7750 SR through an end-to-end transparent clock.
The ordinary clock master, ordinary clock slave, and boundary clock allow for PTP operation over both unicast IPv4 and multicast Ethernet at the same time.
The IEEE 1588v2 standard includes the concept of PTP profiles. These profiles are defined by industry groups or standards bodies that define how IEEE 1588v2 is to be used for a particular application.
Currently, three profiles are supported:
When an ordinary clock slave or a boundary clock receive Announce messages from one or more configured peers or multicast neighbors, it executes a Best Master Clock Algorithm (BMCA) to determine the state of communication between itself and the peers. The system uses the BMCA to create a hierarchical topology allowing the flow of synchronization information from the best source (the Grandmaster clock) out through the network to all boundary and slave clocks. Each profile has a dedicated BMCA.
If the profile setting for the clock is ieee1588-2008, the precedence order for the best master selection algorithm is as follows:
The ordinary clock master, ordinary clock slave, and boundary clock set their local parameters as listed in Table 14:
Parameter | Value |
clockIdentity | Chassis MAC address following the guidelines of 7.5.2.2.2 of IEEE 1588 |
clockClass | 13 — local clock configured as ordinary clock master and is locked to an external reference 14 — local clock configured as ordinary clock master and in holdover after having been locked to an external source 248 — local clock configured as ordinary clock master and is in free run or the router is configured as a boundary clock 255 — local clock configured as ordinary clock slave |
clockAccuracy | FE — unknown |
offsetScaledLogVariance | FFFF — not computed |
If the profile setting for the clock is g8265dot1-2010, the precedence order for the best master selection algorithm is:
The ordinary clock master, ordinary clock slave, and boundary clock set their local parameters as listed in Table 15:
Parameter | Value |
clockClass | 80-110 — value corresponding to the QL out of the central clock as per Table 1/G.8265.1 255 — the clock is configured as ordinary clock slave |
The g8265dot1-2010 profile is for use in an environment with only ordinary clock masters and slaves for frequency distribution.
If the profile setting for the clock is g8275dot1-2014, the precedence order for the best master selection algorithm is very similar to that used with the default profile. It ignores the priority1 parameter, includes a localPriority parameter and includes the ability to force a port to never enter slave state (master-only). The precedence is as follows:
Note: If the two clocks being compared have a clockClass less than 128, then this step is skipped. skipping this step will be the normal case as the vast majority of clocks used as grandmasters advertise clockClass less than 128. |
The ordinary clock master, ordinary clock slave, and boundary clock set their local parameters as listed in Table 16:
Parameter | Value |
clockIdentity | Chassis MAC address following the guidelines of 7.5.2.2.2 of IEEE 1588 |
clockClass | 165 — local clock configured to a boundary clock and the boundary clock was previously locked to a grandmaster with a clock class of 6 248 — local clock configured as boundary clock 255 — local clock configured as ordinary clock slave |
clockAccuracy | FE — unknown |
offsetScaledLogVariance | FFFF — not computed |
There is a limit on the number of external PTP clocks to which the ordinary clock slave or boundary clock requests unicast service (# configured peers) and also a limit to the number of external PTP clocks to which the ordinary clock master or boundary clock grants unicast service (# discovered peers). An association where the boundary clock has a symmetric relationship with another boundary clock (in other words, they both have the other as a configured peer) consumes a request and a grant unicast service in each router.
The number of configured Ethernet ports is not restricted.
There are limits to the maximum transmitted and received event message rates supported in the router. Each unicast IP service established consumes a portion of one of the unicast message limits. Once either limit is reached, additional unicast service requests are refused by sending a grant response with zero in the duration field.
Refer to the scaling guide for the appropriate release for the specific unicast message limits related to PTP.
Multicast messages are not considered when validating the unicast message limit. When multicast messaging on Ethernet ports is enabled, the PTP load needs to be monitored to ensure the load does not exceed the capabilities. There are several commands that can be used for this monitoring:
Because the user cannot control the amount of PTP messages being received over the Ethernet ports, the statistics commands can be used to identify the source of the message load:
Figure 15 shows the unicast negotiation procedure performed between a slave and a peer clock that is selected to be the master clock. The slave clock requests Announce messages from all peer clocks but only request Sync and Delay_Resp messages from the clock selected to be the master clock.
The IEEE 1588v2 standard allows for synchronization of the frequency and time from a master clock to one or more slave clocks over a packet stream. This packet-based synchronization can be over unicast UDP/IPv4 or multicast Ethernet.
As part of the basic synchronization timing computation, a number of event messages are defined for synchronization messaging between the PTP slave port and PTP master port. A one-step or two-step synchronization operation can be used, with the two-step operation requiring a follow-up message after each synchronization message. Ordinary clock master and boundary clock master ports use one-step operation; ordinary clock slave and boundary clock slave ports can accept messages from either one-step or two-step operation master ports.
The IEEE 1588v2 standard includes a mechanism to control the topology for synchronization distribution. The Best Master Clock Algorithm (BMCA) defines the states for the PTP ports on a clock. One port is set into slave state and the other ports are set to master (or passive) states. Ports in slave state recovered synchronization delivered by from an external PTP clock and ports in master state transmit synchronization to toward external PTP clocks.
The basic synchronization timing computation between the PTP slave and PTP master is shown in Figure 16. This figure illustrates the offset of the slave clock referenced to the best master signal during startup.
When using IEEE 1588v2 for distribution of a frequency reference, the slave calculates a message delay from the master to the slave based on the timestamps exchanged. A sequence of these calculated delays contain information of the relative frequencies of the master clock and slave clock but has a noise component related to the packet delay variation (PDV) experienced across the network. The slave must filter the PDV effects so as to extract the relative frequency data and then adjust the slave frequency to align with the master frequency.
When using IEEE 1588v2 for distribution of time, the 7750 SR and 7450 ESS use the four timestamps exchanged using the IEEE 1588v2 messages to determine the offset between the router time base and the external master clock time base. The router determines the offset adjustment and then in between these adjustments, the router maintains the progression of time using the frequency from the central clock of the router. This allows time to be maintained using a BITS input source or a Synchronous Ethernet input source even if the IEEE 1588v2 communications fail. When using IEEE 1588v2 for time distribution, the central clock should at a minimum have a system timing input reference enabled. Figure 17 displays how IEEE 1588v2 is used for time distribution.
Although IEEE 1588v2 can be used on a network that is not PTP-aware, the use of PTP-aware network elements (boundary clocks) within the packet switched network improves synchronization performance by reducing the impact of PDV between the grand master clock and the slave clock. In particular, when IEEE 1588v2 is used to distribute high accuracy time, such as for mobile base station phase requirements, then the network architecture requires the deployment of PTP awareness in every device between the Grandmaster and the mobile base station slave.
In addition, performance is also improved by the removal of any PDV caused by internal queuing within the boundary clock or slave clock. This is accomplished with hardware that is capable of detecting and time stamping the IEEE 1588v2 packets at the Ethernet interface. This capability is referred to as port-based time stamping.
For optimal performance, the 1588 packets should be time-stamped at the ingress and egress. This avoids any possible PDV that might be introduced between the port and the CPM. The ability to timestamp in the interface hardware is provided on a subset of the IMM and MDA assemblies of the routers. Generally, all FP4-based XMA, XMA-s, and MDA-e-XP modules support 1588 port-based timestamping. For other assemblies, contact your Nokia representative to verify the support for 1588 port-based timestamping.
In order for this to operate, the CPM, IOM, IMM, and MDAs must be running firmware that supports this capability. The CPM firmware upgrade occurs automatically when the CPM card software is updated. Since upgrading of IOM, IMM, and MDA firmware is service impacting, this upgrade is not performed automatically on a soft reset of the MDA. The IOM/IMM firmware is upgraded when the IOM/IMM card is hard reset. The MDA firmware is programmed during system initialization, when the MDA is inserted, or when the MDA is hard reset via a clear mda or clear card command. However, when an MDA is soft reset via either a clear card soft command or during a major ISSU, the MDA firmware is not updated.
Port-based timestamping of 1588 packets cannot be used at the same time for Ethernet encapsulation and IP encapsulation on a given port. This means that PTP cannot be configured on an Ethernet port if ptp-hw-assist is already configured on a L3 interface associated with that port. Similarly, ptp-hw-assist cannot be configured on a L3 interface if its associated port is already configured as a PTP port.
For each PTP message type to be exchanged between the router and an external 1588 clock, a unicast session must be established using the unicast negotiation procedures. The router allows the configuration of the message rate to be requested from external 1588 clocks. The router also supports a range of message rates that it grants to requests received from the external 1588 clocks. The IEEE 1588 standard allows the grantor port to respond with a shorter duration than what was in the request. The router can accept such a grant and uses that duration. The router issues a renegotiation before the duration expires.
Table 17 describes the ranges for both the rates that the router can request and grant.
Message Type | Rates Requested by the 7450 ESS, 7750 SR, and 7950 XRS | Rates Granted by the 7450 ESS, 7750 SR, and 7950 XRS | ||
Min | Max | Min | Max | |
Announce | 1 packet every 16 seconds | 8 packets/second | packet every 16 seconds | 8 packets/second |
Sync | 1 packet/second | 64 packet/second | 1 packet/second | 128 packet/second |
Delay_Resp | 1 packet/second | 64 packets/second | 1 packet/second | 128 packets/second |
(Duration) | 300 | 300 | 1 | 1000 |
State and statistics data for each PTP peer are available to assist in the detection of failures or unusual situations.
Traditionally, only clock frequency is required to ensure smooth transmission in a synchronous network. The PTP ordinary clock with slave capability on the router provides another option to reference a Stratum-1 traceable clock across a packet switched network. The recovered clock can be referenced by the internal SSU and distributed to all slots and ports. Figure 18 shows a PTP ordinary slave clock network configuration.
The PTP slave capability is implemented on the CPM, version 3 or later. The IEEE 1588v2 messages can ingress and egress the router on any line interface. Figure 19 shows the operation of an ordinary PTP clock in slave mode.
The router supports the PTP ordinary clock in master mode. Normally, a IEEE 1588v2 grand master is used to support many slaves and boundary clocks in the network. In cases where only a small number of slaves and boundary clocks exist and only frequency is required, a PTP integrated master clock can greatly reduce hardware and management costs to implement PTP across the network. It also provides an opportunity to achieve better performance by placing a master clock closer to the edge of the network, as close to the slave clocks as possible. Figure 20 shows a PTP master clock network configuration.
All packets are routed to their destination via the best route as determined in the route table; see Figure 21. It does not matter which ports are used to ingress and egress these packets (unless port based time stamping is enabled for higher performance).
The router supports boundary clock PTP devices in both master and slave states. IEEE 1588v2 can function across a packet network that is not PTP-aware; however, the performance may be unsatisfactory and unpredictable. PDV across the packet network varies with the number of hops, link speeds, utilization rates, and the inherent behavior of the routers. By using routers with boundary clock functionality in the path between the grand master clock and the slave clock, one long path over many hops is split into multiple shorter segments, allowing better PDV control and improved slave performance. This allows PTP to function as a valid timing option in more network deployments and allows for better scalability and increased robustness in certain topologies, such as rings. Boundary clocks can simultaneously function as a PTP slave of an upstream grand master (ordinary clock) or boundary clock, and as a PTP master of downstream slaves (ordinary clock) and/or boundary clocks, as shown in Figure 22.
In addition, the use of port based timestamping in every network element between the grandmaster and the end slave application is highly recommended for delivering time to meet one microsecond accuracies required of mobile applications.
The router always uses the frequency output of the central clock to maintain the timebase within the router. The PTP reference into the central clock should always be enabled as an option if the router is configured as a boundary clock. This avoids the situation of the router entering holdover while propagating time with 1588.
Note: The ITU-T defined a network architecture for node-by-node time distribution in their Recommendations. These recommendations require that Synchronous Ethernet be used with IEEE 1588 (using the G.8275.1 profile) to meet the target performance. |
The PTP module in the router exists on the CPM. The PTP module on the standby CPM is kept synchronized to the PTP module on the active CPM. All sessions with external PTP peers are maintained over a CPM switchover.
PTP has the potential to provide much more accurate time into the router than can be obtained with NTP. This PTP recovered time can be made available for system time and OAM packet time stamping to improve the accuracies of logged events and OAM delay measurements. The mechanism to activate PTP as the source for these internal time bases is to allocate PTP as a local server into NTP. This permits the NTP time recovery to use PTP as a source for time and then distribute it within the router to system time and the OAM process. This activation also affects the operation of the NTP server within the SR OS. The PTP server appears as NTP stratum 0 server and therefore the SR OS advertises itself as an NTP Stratum 1 server to external peers and clients. This activation may impact the NTP topology.
PTP is supported over direct Ethernet encapsulation (that is, PTP ports) and UDP/IP encapsulation (that is, PTP peers). PTP ports operate below the routing plane. They can be used on appropriate ports irrespective of any type of router interface also on the port. PTP peers operate at the routing plane and have restrictions based on and across the following router instances.
Transmission and reception of PTP messages using PTP peers is supported in the following contexts:
Transmission and reception of PTP messages using PTP peers is not supported in the following contexts:
It is important to note that there is only one PTP clock within the router. All PTP ports and PTP peers communicate into one clock instance. Only one router instance may have PTP peers configured, which means that only that router instance (or PTP port) can run the slave functionality and recover time from an external PTP clock. All other router instances only support the dynamic PTP peers. The PTP process in the router only includes outward server time towards the dynamic PTP peers. The dynamic PTP peers are shared across all router instances. If it is desired to control the number of dynamic peers that can be consumed by a given routing instance, then it must be configured for that routing instance.
The PTP clock in the router monitors the Sync, Follow_Up (if present), and Delay_Resp messages received from external neighbor ports. If a high variation is detected in the network path between the external neighbor port and the local port, that neighbor port is considered unusable (PTSF-unusable as defined in the ITU-T G.8275.1 recommendation). When a neighbor is unusable, all Announce messages from that neighbor are discarded on reception and excluded from the BMCA. If the neighbor is the parent clock to the local clock, the local clock must either select a new parent clock or go into holdover. In addition, any neighbor clock marked as unusable cannot act as the parent to the local PTP clock until underlying condition is investigated and resolved, and the unusable state is cleared. The unusable state is cleared when PTP, PTSF-unusable monitoring, or the local PTP port is administratively disabled, the PTP port is deleted, or the external neighbor port stops sending messages to the node. It can also be cleared by using the appropriate clear command.
On the 7750 SR, the ATM ping OAM loopback feature can be enabled on an ATM SAP for a period of time configured through the interval and the send-count parameters. When the ATM SAP terminates on IES or VPRN services, a failure of the loopback state machine does not bring down the Layer 3 interface. Only receiving AIS/RDI OAM cells or entering the AIS/RDI state brings down the Layer 3 interface.
The ATM ping OAM loopback feature can also be enabled on a continuous basis on an ATM SAP terminating on IES or VPRN services. When the loopback state machine fails, the Layer 3 interface is brought down.
The ATM OAM loopback parameters must first be enabled and configured in the config>system>atm>oam context, and then enabled in the IES or VPRN service interface SAP atm oam context.
Refer to the 7450 ESS, 7750 SR, 7950 XRS, and VSR Layer 3 Services Guide: IES and VPRN for further information. For command descriptions, refer to the 7450 ESS, 7750 SR, 7950 XRS, and VSR Classic CLI Command Reference Guide.
The creation of network interfaces on a QinQ-encapsulated VLAN can be enabled on a system-wide level using the config>system>ip>allow-qinq-network-interface command. When enabled, the egress IOM limits are changed to allow a maximum of 11 MPLS labels instead of 12.
Table 18 lists the allowed and restricted QinQ combinations.
SAP x.0 | SAP x.* | SAP x.y | Nw interface x.0 | Nw interface x.* | Nw interface x.y | SAP *.* | SAP *.NULL | SAP 0.* | Inverse SAP | |
SAP x.0 | x | ✓ | ✓ | x | x | x | ✓ | ✓ | ✓ | x |
SAP x.* | ✓ | x | ✓ | x | x | x | ✓ | ✓ | ✓ | x |
SAP x.z | ✓ | ✓ | ✓ | x | x | ✓ | ✓ | ✓ | ✓ | ✓ |
Nw interface x.0 | x | x | x | x | x | ✓ | ✓ | ✓ | ✓ | x |
Nw interface x.* | x | x | x | x | x | x | ✓ | ✓ | ✓ | x |
Nw interface x.z | x | x | ✓ | ✓ | x | ✓ | ✓ | ✓ | ✓ | x |
SAP *.* | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | x | ✓ | ✓ | ✓ |
SAP *.NULL | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | x | ✓ | x |
SAP 0.* | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | x | x |
Inverse SAP | x | x | ✓ | x | x | x | ✓ | x | x | x |
The IEEE 802.1ab Link Layer Discovery Protocol (LLDP) is a unidirectional protocol that uses the MAC layer to transmit specific information related to the capabilities and status of the local device. Separately from the transmit direction, the LLDP agent can also receive the same kind of information for a remote device which is stored in the related MIBs.
LLDP itself does not contain a mechanism for soliciting specific information from other LLDP agents, nor does it provide a specific means of confirming the receipt of information. LLDP allows the transmitter and the receiver to be separately enabled, making it possible to configure an implementation so the local LLDP agent can either transmit only or receive only, or can transmit and receive LLDP information.
The information fields in each LLDP frame are contained in a LLDP Data Unit (LLDPDU) as a sequence of variable length information elements, that each include type, length, and value fields (known as TLVs), where:
Each LLDPDU contains four mandatory TLVs and can contain optional TLVs as selected by network management:
The chassis ID and the port ID values are concatenated to form a logical identifier that is used by the recipient to identify the sending LLDP agent/port. Both the chassis ID and port ID values can be defined in a number of convenient forms. Once selected however, the chassis ID/port ID value combination remains the same as long as the particular port remains operable.
A non-zero value in the TTL field of the time-to-live TLV tells the receiving LLDP agent how long all information pertaining to this LLDPDU’s identifier is valid so that all the associated information can later be automatically discarded by the receiving LLDP agent if the sender fails to update it in a timely manner. A zero value indicates that any information pertaining to this LLDPDU’s identifier is to be discarded immediately.
A TTL value of zero can be used, for example, to signal that the sending port has initiated a port shutdown procedure.
The end of a LLDPDU TLV marks the end of the LLDPDU.
The IEEE 802.1ab standard defines a protocol that:
Network operators must be able to discover the topology information in order to detect and address network problems and inconsistencies in the configuration. Moreover, standard-based tools can address the complex network scenarios where multiple devices from different vendors are interconnected using Ethernet interfaces.
The example displayed in Figure 24 depicts a MPLS network that uses Ethernet interfaces in the core or as an access/hand off interfaces to connect to different kind of Ethernet enabled devices such as service gateway/routers, QinQ switches, DSLAMs or customer equipment.
IEEE 802.1ab LLDP running on each Ethernet interfaces in between all the above network elements may be used to discover the topology information.
It is now possible to include IP header in the hash routine at an LSR for the purpose of spraying labeled-IPv4 and labeled-IPv6 packets over multiple equal cost paths in ECMP in an LDP LSP and/or over multiple links of a LAG group in all types of LSPs.
A couple of configurable options are supported. The first option is referred to as the Label-IP Hash option and is designated in the CLI as lbl-ip. When enabled, the hash algorithm parses down the label stack and once it hits the bottom of the stack, it checks the next nibble. If the nibble value is four or six then it assumes it is an IPv4 or IPv6 packet. The result of the hash of the label stack, along with the incoming port and system IP address, is fed into another hash along with source and destination address fields in the IP packet’s header. The second option is referred to as IP-only hash and is enabled in CLI by entering the iponly keyword. It operates the same way as the Label-IP Hash method except the hash is performed exclusively on the source and destination address fields in the IP packet header. This method supports both IPv4 and IPv6 payload and operates on packets received on an IP interface on an IOM3-XP/IMM port only.
By default, MPLS packet hashing at an LSR is based on the whole label stack, along with the incoming port and system IP address. This method is referred to as Label-Only Hash option and is enabled in CLI by entering the lbl-only keyword.
The lbl-only, lbl-ip and ip-only hashing options can be configured system-wide and can also be overridden on a per-IP-interface basis.
There are two types of SAS-Sx satellites supported on the 7750 SR:
The following primary tasks must be performed to configure a satellite.
The Ethernet satellite support feature allows a 7210 SAS-Sx or SAS-S chassis to act as a port extension for the 7750 SR host. In this configuration, all configuration and management functions are performed through the host node. Management of the SAS-Sx/SAS-S node is not required when it is configured in an Ethernet satellite operations mode. A direct, non-switched, Ethernet connection between the 7750/7950 host and the 7210 satellite must be provided. The use of active Layer 2 switching devices in the path between the host and satellite is not supported.
Table 19 lists the supported Ethernet satellite chassis.
Chassis Type | Sat-Type String |
7210 SAS-Sx 24-port fiber | es24-1gb-sfp |
7210 SAS-Sx 48-port fiber | es48-1gb-sfp |
7210 SAS-S 24F4SFP+ | es24-sass-1gb-sfp |
7210 SAS-S 48F4SFP+ | es48-sass-1gb-sfp |
7210 SAS-Sx 24-port copper 7210 SAS-S 24-port copper | es24-1gb-tx |
7210 SAS-Sx 48-port copper 7210 SAS-S 48-port copper | es48-1gb-tx |
7210 SAS-Sx 24-port copper + PoE 7210 SAS-S 24-port copper + PoE | es24-1gb-tx |
7210 SAS-Sx 48-port copper + PoE 7210 SAS-S 48-port copper + PoE | es48-1gb-tx |
7210 SAS-Sx 64-port 10GE (CFP) | es64-10gb-sfpp+4-100gb-cfp4 |
7210 SAS-Sx 64-port 10GE + 4-port QSFP28 | es64-10gb-sfpp+4-100gb-qsfp28 |
7210 SAS-Mxp | es24-sasmxp-1gb-sfp |
Note:
|
The SONET/SDH ETR chassis is the only available TDM satellite and can be configured for different modes. Table 20 lists the supported modes of the satellite chassis.
Chassis Type | Sat-Type String |
4 port OC3 | ts4-choc3-sfp |
4 port STM1 | ts4-chstm1-sfp |
1 port OC12 | ts1-choc12-sfp |
1 port STM4 | ts1-chstm4-sfp |
The default type on a supplied TDM satellite is ts4-choc3-sfp. Updating to another type initiates a reboot of the satellite.
The TDM satellite provides CEM functionalities supported on the 7750 SR OC3/OC12 CES MDAs. The satellite is built using the same architecture as the 7705 SAR-8 adapter cards and is designed to transport existing TDM services including:
The following types of synchronization are supported:
To provide a stable frequency from the host to the SONET/SDH satellite, ensure that the host's clock is referenced to a suitable timing source (for example, BITS) and configure Synchronous Ethernet from the host's Ethernet port connecting to the satellite. Copper Ethernet SFPs are not supported because they do not support Synchronous Ethernet.
The TDM satellite is entirely managed through a 7750 SR host system, such as 7750 SR, 7750 SR-a, or 7750 SR-e. As a satellite, no new IP address needs to be assigned. Services on the satellite are configured on the host in the same manner as any ports in a native MDA. The TDM satellite connects to the SR host using a Gigabit Ethernet link, thereby not occupying valuable slots space in the host system. APS is supported across satellites connecting to a single host.
The software repositories define the locations from where the host can obtain software for subcomponents including Ethernet satellites. The software repository is also used to upgrade an existing subcomponent by changing the location of the image to be served to the remote device. The software repositories are not used for management of the host router software, which is managed using the standard procedures described in the SR OS 21.x.Rx Software Release Notes.
Each software repository supports up to three locations to search for the software. A location may be a URL or a directory on a compact flash. When an upgrade operation is initiated, each of the three locations is checked in sequence to locate the required software. The upgrade operation fails if the software is not located in any of the configured locations. The satellite booting operation also fails if the software cannot be located.
At least one software repository must be configured to support a satellite connected to the local host by using the config>system>software-repository CLI tree, as follows.
Caution: Software for TDM satellites and Ethernet satellites should be stored in separate software repositories. There is one file that has the same name for both types of software, that is overwritten if they are placed in the same repository. |
The process to change or upgrade the satellite software consists of the following steps.
After creating the software repositories, configure the satellite. The satellite configuration is required to create a satellite binding to a satellite ID, and to provide additional information that uniquely identifies the satellite chassis, chassis type, and the software repository to be used to boot the remote satellite.
The following parameters can be specified for a satellite.
Use the following format to reference Ethernet satellite client ports:
port esat- sat-id/slotNum/portNum
where:
Use the following format to reference Ethernet satellite uplink port:
port esat- sat-id/1/uplink-id
where:
Use the following format to reference TDM satellite client ports:
port tsat- sat-id/slotNum/portNum.channel
where:
Use the following format to reference TDM satellite uplink port:
port tsat- sat-id/1/u1
where:
Ethernet satellite client ports support all port modes (access, network, and client).
Configuring services associated with satellite client ports is the same as configuring services on local 7750 SR ports, except that satellite client ports are referenced with the syntax for the Ethernet satellite port described above. It is required that a port-scheduler-policy is created to ensure that the 7750 SR is able to shape the traffic for the egress satellite port type and speed.
The local forwarding capability allows traffic to be forwarded between two client satellite ports without going through the SR host, which allows for optimal forwarding by preserving uplink bandwidth.
Figure 25 shows an example of local forwarding.
A local-forward bypass is created by using the following commands to create a local-forward bypass, then associating a set of two satellite access points as endpoints for the local-forward bypass.
Example Configuration:
To configure a local-forward bypass between client ports esat-2/1/1:66 and esat-2/1/50:101, use the following commands:
The port-template command hierarchy allows the creation of a satellite template that reconfigures the port role and uplink association for one or more satellite ports. This template can then be applied to one or more Ethernet satellite instances, in which case those satellites inherit the specified port role and uplink associations.
The port template is necessary when reconfiguring a satellite uplink as a client port for use as part of a local-forward bypass path.
Ports 51 and 52 on the 48xGE + 4x10GE satellite chassis can be reassigned as client ports instead of uplink ports. This provides the flexibility to offer 10GE services from these satellite chassis. These two 10GE ports can be reconfigured as client ports using the port-template configuration commands described above. The port template configuration must be done before SAPs, interfaces, or services can be applied to the associated satellite ports.
Ports 67 and 68 on the 64x10GE + 4x100GE satellites (sat-type es64-10gb-sfpp+4-100gb-cfp4) and connectors 3 and 4 on the 64x10GE+4xQSFP28 (sat-type es64-10gb-sfpp+4-100gb-qsfp28) can be reassigned as client ports instead of uplinks. This provides the flexibility to offer 100GE services from these satellite chassis. These two 100GE ports can be reconfigured as client ports using the port-template configuration commands. The port template must be configured before port topology bindings are configured as well as before SAPs, interfaces, or services can be applied to the associated satellite ports.
On the 7210 SAS-Sx 64x10GE+4x100GE (es64-10gb-sfpp+4-100gb-cfp4) and 64x10GE+4xQSFP28 (sat-type es64-10gb-sfpp+4-100gb-qsfp28) satellite, selected 10GE ports can be reconfigured and used as satellite uplinks to the host router running SR OS.
Up to 16 10GE interfaces can be used as uplinks for the associated satellite. A new satellite template that configures the desired 10GE interfaces as uplinks must be created. In addition, use a port template to specify the uplink association between the remaining client ports and configured uplinks.
Apply the new template to the satellite using the config>system>satellite>eth-sat sat-id>sat-type sat-type>port-template template-name command, where the template-name is the name configured in the port-template context.
This feature requires the 7210 SAS-Sx to be running Release 9.0.R10 or later for the SAS-Sx 64x10GE+4x100GE and 7210 SAS Release 10.0 or later for the 64x10GE+4xqSFP28 satellite.
The following restrictions apply:
The following is an example configuration:
An option in the port-map configuration allows a secondary uplink to be assigned to enable uplink resiliency. A secondary uplink is used to carry the traffic associated with the client port if the primary uplink becomes unavailable. If traffic is switched to the secondary uplink, once the primary uplink becomes available, traffic is reverted to the primary as soon as possible.
The configuration of a secondary uplink is performed on a per-client port basis using the port-map command.
config>system>sat>eth-sat>port-map client-port-id primary primary-uplink-port-id [secondary secondary-uplink-port-id]
config>system>sat>eth-sat>port-map client-port-id system-default
To configure a secondary uplink, after the primary uplink is specified, the secondary keyword should be included, followed by the intended uplink to be used as the secondary uplink.
For example,
The following are basic actions allowed with a single command:
Uplink mapping can be changed, but a client uplink must be maintained throughout the process. For example, client-10 is mapped to uplink-1 (U-1), but must move to uplink-2 (U-2). To do this, add U-2 as the secondary uplink, then swap the primary and secondary, making U-2 the primary uplink for client-10 and switching traffic to U-2. After the switch is complete, remove U-1. U-1 cannot be directly replaced with U-2, as the client port would have no uplink during the switch.
Auto-provisioning is used to provision a node using an external DHCP server and file server. It is used to obtain a configuration file and an image file from an external server using an in-band mechanism. Auto-provisioning is not compatible with an out-of-band management port.
Before using auto-provisioning, the SR OS must be booted up and running the application image. In addition, it needs to have some minimum configuration before the auto-provision script is executed by the operator.
After the auto-provision application is triggered using a tools command, SR OS checks all operationally up ports without IP addresses and send DHCP discovery to these interfaces. The DHCP server needs to be configured with Option 67 and the user must provide the SR OS with the URL of a file server and the corresponding directory for the image.
Figure 26 to Figure 28 describe scenarios in which auto-provisioning are used.
In Figure 26, there is no DHCP relay and all IP addresses are assigned from a single pool.
In Figure 27, there is a DHCP relay which injects the Option 82 as a gateway address. The DHCP server is assigned the IP address from the pool dictated by the gateway address option 82. The DHCP server and HTTP server are in the same subnet. The DHCP offer has option 3 "router" which is used for a default gateway creation on the 7750 SR.
In Figure 28, all components are in different subnets. The DHCP relay adds Option 82 to the DHCP request as the gateway address which is used for pool selection. The DHCP server must add option 3 configured with the gateway address of the HTTP server.
The following are some configuration limits for auto-provisioning:
The following are the DHCP rules in the auto-provisioning stage:
The auto-provisioning command is CLI blocking. All information about the auto-provisioning process is displayed on the CLI and logged.
Auto-provisioning fails for the following reasons:
If the auto-provisioning procedure on this interface fails, then auto-provisioning:
This section contains information to perform administrative tasks.
Whenever configuration changes are made, the modified configuration must be saved so they are not lost when the system is rebooted.
Configuration files are saved by executing explicit command syntax which includes the file URL location to save the configuration file as well as options to save both default and non-default configuration parameters. Boot option file (BOF) parameters specify where the system should search for configuration and image files as well as other operational parameters during system initialization.
For more information about boot option files, refer to the Boot Options section.
Two post-boot configuration extension files are supported and are triggered when either a successful or failed boot configuration file is processed. The boot-bad-exec and boot-good-exec commands specify URLs for the CLI scripts to be run following the completion of the bootup configuration. A URL must be specified or no action is taken.
For example, after a configuration file is successfully loaded, the specified URL can contain a nearly identical configuration file with certain commands enabled or disabled, or particular parameters specified and according to the script which loads that file.
In Time Domain Multiplexed (TDM)-based networks (for example, SONET or SDH circuit- switched networks), the concept of network timing is used to prevent over-run or under-run issues where circuits are groomed (rebundled) and switched. Hardware exists in each node that takes a common clock derived from an internal oscillator, a specific receive interface, or special BITS interface and provides it to each synchronous interface in the system. Usually, each synchronous interface is allowed to choose between using the chassis-provided clock or the clocking recovered from the received signal on the interface. The clocking is used to drive the transmit side of the interface. The appropriate configuration at each node which defines how interface clocking is handled must be considered when designing a network that has a centralized timing source so each interface is operating in a synchronous manner.
The effect of timing on a network is dependent on the nature of the type of traffic carried on the network. With bit-wise synchronous traffic (traditional circuit-based voice or video), non-synchronous transmissions cause a loss of information in the streams affecting performance. With packet-based traffic, the applications expect and handle jitter and latency inherent to packet-based networks. When a packet-based network is used to carry voice or video traffic, the applications use data compression and elasticity buffering to compensate for jitter and latency. The network itself relies on appropriate Quality of Service (QoS) definitions and network provisioning to further minimize the jitter and latency the application may experience.
SR OS supports a power-supply command to configure the type and number of power supplies present in the chassis. The operational status of a power source is always displayed by the LEDs on the Control Processor/Switch Fabric Module (CP/SFM) front panel, but the power supply information must be explicitly configured in order for a power supply alarm to be generated if a power source becomes operationally disabled.
Use the CLI syntax displayed below to configure synchronization components relating to active-to-standby CPM switchover. In redundant systems, synchronization ensures that the active and standby CPMs have identical operational parameters, including the active configuration, CPM, XCM, and IOM images in the event of a failure or reset of the active CPM.
The force-switchover command forces a switchover to the standby CPM card.
To enable automatic synchronization, either the boot-env parameter or the config parameter must be specified. The synchronization occurs when the admin save or bof save commands are executed.
When the boot-env parameter of the synchronize command is specified, the bof.cfg, primary/secondary/tertiary configuration files (.cfg and .ndx), li, and ssh files are automatically synchronized. When the config parameter is specified, only the configuration files are automatically synchronized.
Synchronization also occurs whenever the BOF is modified and when an admin>save command is entered with no filename specified.
The boot-env option enables a synchronization of all the files used in system initialization.
When configuring the system to perform this synchronization, the following occurs:
The config option synchronizes configuration files by copying the files specified in the active CPM BOF file to the same compact flash on the standby CPM.
Both image files (CPM and IOM) on the 7450 ESS must be located in the same directory. Failure to locate and synchronize both images causes an error to be generated.
The admin redundancy synchronize command performs manual CPM synchronizations. The boot-env parameter synchronizes the BOF, image, and configuration files in redundant systems. The config parameter synchronizes only the configuration files in redundant systems.
The force-switchover now command forces an immediate switchover to the standby CPM card.
If the active and standby are not synchronized for some reason, users can manually synchronize the standby CPM by rebooting the standby by issuing the admin reboot standby command on the active or the standby CPM.
SR OS supports multiple Layer 3 router instances. These instances have their own IP addressing spaces and parameters. Router instances are isolated from each other.
The following are the different types of router instances in SR OS:
Some management protocols can use either the base routing instance (in-band) or the management routing instance (out-of-band). A listing of these protocols can be found in the CPM Filter: Protocols and Ports section of the 7450 ESS, 7750 SR, 7950 XRS, and VSR System Management Guide. Unless otherwise stated in the detailed description of the protocol, when the server or client for the protocol is reachable via the management routing instance, those protocol messages use the management interface for the protocol communication.
If BOF is set up with autoconfiguration and the DHCP server provides a general default route such as 0.0.0.0/0, with some protocols (like PCEP, TACACS+, RADIUS, and LDAP), Authentication, Authorization, Accounting (AAA) always prefers OOB over in-band connectivity. This is because these protocols prefer to use the OOB management port first. If a matching route is not found, in-band is attempted. The static route provided by DHCP must be properly set to ensure the correct route preference is made by these protocols.
Figure 29 shows the process to provision basic system parameters.
This section describes system configuration caveats.
To access the CLI, the system must be properly initialized and the boot loader and BOF files successfully executed.