6.1.1. Logging Using the Management VPRN

When a management VPRN is configured, all authentication, authorization, and accounting servers, DNS server, or syslog servers are reachable using this management VPRN. The user can configure a syslog server within the management VPRN, log all SR OS events, and direct the logs to this syslog server. The command config>log>services-all-events service id enables the logging of all events under the management VPRN specified by the service ID. Within the specified VPRN, the command config>service>vprn>log>log-id>from directs the logs from the specified event sources (main, change, security, or debug-trace) to be saved at the log ID destination specified by the config>service>vprn>log>log-id>to command.

6.2.1. Console

Sending events to a console destination means the message will be sent to the system console The console device can be used as an event log destination.

6.2.2. Session

A session destination is a temporary log destination which directs entries to the active telnet or SSH session for the duration of the session. When the session is terminated, for example, when the user logs out, the “to session” configuration is removed. Event logs configured with a session destination are stored in the configuration file but the “to session” part is not stored. Event logs can direct log entries to the session destination.

6.2.3. CLI Logs

A CLI log is a log that outputs log events to a CLI session. An operator can subscribe to a CLI log from within a CLI session using the tools perform log subscribe-to log-id command. The events are sent to the CLI session for the duration of that CLI session (or until an unsubscribe-from command is issued).

6.2.4. Memory Logs

A memory log is a circular buffer. When the log is full, the oldest entry in the log is replaced with the new entry. When a memory log is created, the specific number of entries it can hold can be specified, otherwise it will assume a default size. An event log can send entries to a memory log destination.

6.2.5. Log Files

Log files can be used by both event logs and accounting logs and are stored on the compact flash devices in the file system. I

A log file is identified with a single log file ID, but a log file will generally be composed of a number individual files in the file system. A log file is configured with a rollover parameter, expressed in minutes, which represents the length of time an individual log file should be written to before a new file is created for the relevant log file ID. The rollover time is checked only when an update to the log is performed. Thus, complying to this rule is subject to the incoming rate of the data being logged. For example, if the rate is very low, the actual rollover time may be longer than the configured value.

The retention time for a log file specifies the amount of time the file should be retained on the system based on the creation date and time of the file.

When a log file is created, only the compact flash device for the log file is specified. Log files are created in specific subdirectories with standardized names depending on the type of information stored in the log file.

Event log files are always created in the \log directory on the specified compact flash device. The naming convention for event log files is:

log eeff-timestamp

where:

ee is the event log ID

ff is the log file destination ID

timestamp is the timestamp when the file is created in the form of yyyymmdd-hhmmss where:

yyyy is the four-digit year (for example, 2007)

mm is the two digit number representing the month (for example, 12 for December)

dd is the two digit number representing the day of the month (for example, 03 for the 3rd of the month)

hh is the two digit hour in a 24-hour clock (for example, 04 for 4 a.m.)

mm is the two digit minute (for example, 30 for 30 minutes past the hour)

ss is the two digit second (for example, 14 for 14)

Accounting log files are created in the \act-collect directory on a compact flash device (specifically cf1 or cf2). The naming convention for accounting log files is nearly the same as for log files except the prefix act is used instead of the prefix log. The naming convention for accounting logs is:

act aaff-timestamp.xml.gz

where:

aa is the accounting policy ID

ff is the log file destination ID

timestamp is the timestamp when the file is created in the form of yyyymmdd-hhmmss where:

yyyy is the four-digit year (for example, 2007)

mm is the two digit number representing the month (for example, 12 for December)

dd is the two digit number representing the day of the month (for example, 03 for the 3rd of the month)

hh is the two digit hour in a 24-hour clock (for example, 04 for 4 a.m.)

mm is the two digit minute (for example, 30 for 30 minutes past the hour)

ss is the two digit second (for example, 14 for 14 seconds)

Accounting logs are .xml files created in a compressed format and have a .gz extension.

The \act-collect directory is where active accounting logs are written. When an accounting log is rolled over, the active file is closed and archived in the \act directory before a new active accounting log file created in \act-collect.

When creating a new log file on a Compact Flash disk card, the system will check the amount of free disk space and that amount must be greater than or equal to the lesser of 5.2 MB or 10% of the Compact Flash disk capacity.

6.2.6. SNMP Trap Group

An event log can be configured to send events to SNMP trap receivers by specifying an SNMP trap group destination.

An SNMP trap group can have multiple trap targets. Each trap target can have different operational parameters.

A trap destination has the following properties:

For SNMP traps that will be sent out-of-band through the Management Ethernet port on the SF/CPM, the source IP address of the trap is the IP interface address defined on the Management Ethernet port. For SNMP traps that will be sent in-band, the source IP address of the trap is the system IP address of the router.

Each trap target destination of a trap group receives the identical sequence of events as defined by the log ID and the associated sources and log filter applied.

6.2.7. Syslog

An event log can be configured to send events to one syslog destination. Syslog destinations have the following properties:

Because syslog uses eight severity levels whereas the router uses six internal severity levels, the severity levels are mapped to syslog severities. Table 31 displays the severity level mappings to syslog severities.

The general format of an SR OS syslog message is as follows (see RFC 3164, The BSD Syslog Protocol). The “<” and “>” are informational delimiters to make reading and understanding the format easier and they do not appear in the actual syslog message except as part of the PRI:

<PRI> <HEADER><MSG>

where:

where:

Examples (from different nodes):

default log-prefix (TMNX):

no log-prefix:

log-prefix "test":

6.2.8. NETCONF

A NETCONF log is a log that outputs log events to a NETCONF session as notifications. A NETCONF client can subscribe to a NETCONF log using the configured netconf-stream stream-name for the log in a subscription request. See NETCONF Notifications for more details.

6.3.1. Event Sources

In Figure 15, the event sources are the main categories of events that feed the log manager.

Examples of applications within the system include IP, MPLS, OSPF, CLI, services, and so on. The following example displays a partial sample of the show log applications command output which displays all applications.

6.3.2. Event Control

Event control pre-processes the events generated by applications before the event is passed into the main event stream. Event control assigns a severity to application events and can either forward the event to the main event source or suppress the event. Suppressed events are counted in event control, but these events will not generate log entries as it never reaches the log manager.

Simple event throttling is another method of event control and is configured similarly to the generation and suppression options. See Simple Logger Event Throttling.

Events are assigned a default severity level in the system, but the application event severities can be changed by the user.

Application events contain an event number and description that explains why the event is generated. The event number is unique within an application, but the number can be duplicated in other applications.

The following example, generated by querying event control for application generated events, displays a partial list of event numbers and names.

6.3.3. Log Manager and Event Logs

Events that are forwarded by event control are sent to the log manager. The log manager manages the event logs in the system and the relationships between the log sources, event logs and log destinations, and log filter policies.

An event log has the following properties:

6.3.4. Event Filter Policies

The log manager uses event filter policies to allow fine control over which events are forwarded or dropped based on various criteria. Like other filter policies in the SR OS, filter policies have a default action. The default actions are either:

Filter policies also include a number of filter policy entries that are identified with an entry ID and define specific match criteria and a forward or drop action for the match criteria.

Each entry contains a combination of matching criteria that define the application, message, event number, router, severity, and subject conditions. The entry’s action determines how the packets should be treated if they have met the match criteria.

Entries are evaluated in order from the lowest to the highest entry ID. The first matching event is subject to the forward or drop action for that entry.

Valid operators are displayed in Table 32:

A match criteria entry can include combinations of:

6.3.5. Event Log Entries

Log entries that are forwarded to a destination are formatted in a way appropriate for the specific destination whether it be recorded to a file or sent as an SNMP trap, but log event entries have common elements or properties. All application generated events have the following properties:

The general format for an event in an event log with either a memory, console or file destination is as follows.

The following is an event log example:

The specific elements that compose the general format are described in Table 33.

6.3.6. Simple Logger Event Throttling

Simple event throttling provides a mechanism to protect event receivers from being overloaded when a scenario causes many events to be generated in a very short period of time. A throttling rate, # events/# seconds, can be configured. Specific event types can be configured to be throttled. Once the throttling event limit is exceeded in a throttling interval, any further events of that type cause the dropped events counter to be incremented. Dropped events counts are displayed by the show>log>event-control context. Events are dropped before being sent to one of the logger event collector tasks. There is no record of the details of the dropped events and therefore no way to retrieve event history data lost by this throttling method.

A particular event type can be generated by multiple managed objects within the system. At the point this throttling method is applied the logger application has no information about the managed object that generated the event and cannot distinguish between events generated by object “A” from events generated by object “B”. If the events have the same event-id, they are throttled regardless of the managed object that generated them. It also does not know which events may eventually be logged to destination log-id <n> from events that will be logged to destination log-id <m>.

Throttle rate applies commonly to all event types. It is not configurable for a specific event-type.

A timer task checks for events dropped by throttling when the throttle interval expires. If any events have been dropped, a TIMETRA-SYSTEM-MIB::tmnxTrapDropped notification is sent.

6.3.7. Default System Log

Log 99 is a pre-configured memory-based log which logs events from the main event source (not security, debug, and so on). Log 99 exists by default.

The following example displays the log 99 configuration.

6.3.8. Event Handling System

The Event Handling System (EHS) is a framework that allows operator-defined behavior to be configured on the router. EHS adds user-controlled programmatic exception handling by allowing a CLI script to be executed upon the detection of a log event (the 'trigger'). Regexp style expression matching is available on various fields in the log event to give flexibility in the trigger definition.

EHS handler objects are used to tie together:

EHS, along with CRON, uses the generic SR OS CLI script-control functions for scripts. Any command available in CLI (with some limited exceptions such as 'candidate' commands) can be executed in a script as the result of an EHS handler being triggered. Figure 16 illustrates the relationships between the different configurable objects used by EHS (and CRON).

Figure 16:  EHS Object Relationships 

6.3.8.1. EHS Configuration and Syntax Rules

Complex rules can be configured to match log events as a trigger for an EHS handler.

When a log event is generated in SR OS, it is subject to discard using suppression and throttling (config>log>event-control) before it is evaluated as a trigger for EHS, according to the following.

1. EHS does not trigger on log events that are suppressed through config>log>event-control.
2. EHS does not trigger on log events that are throttled by the logger.

EHS is triggered on log events that are dropped by user-configured log filters that are assigned to individual logs (config>log>filter). The EHS event trigger logic occurs before the distribution of log event streams into individual logs.

Varbinds are variable bindings that represent the variable number of values that are included in a log event.

The common parameters and varbinds for a triggering log event are passed in to the triggered EHS script and can be used within the EHS script as passed-in (dynamic) variables. Passed-in (dynamic) variables are:

1. the common event parameters, for example: appid, name, eventid, severity, subject, and gentime
2. the predefined varbinds in a log event

For example, the following are the passed-in (dynamic) variables for an event with N varbinds:

1. appid
2. eventid
3. severity
4. subject
5. gentime
6. event_varbind_1
7. event_varbind_2
   ...
   ...
8. event_varbind_N

Note: For more information about showing event parameters, see the show command in the 7450 ESS, 7750 SR, 7950 XRS, and VSR Clear, Show, and Tools Command Reference Guide. Refer to the 7450 ESS, 7750 SR, 7950 XRS, and VSR Log Events Guide for any event's predefined varbinds The passed-in event's gentime is always UTC The event's sequence number is not passed in to the script

When using the classic CLI, an EHS script has the ability to define local (static) variables and uses basic .if or .set syntax inside the script. The use of variables with .if or .set commands within an EHS script adds more logic to the EHS scripting and allows the reuse of a single EHS script for more than one trigger or action.

Both passed-in and local variables can be used within an EHS script either as part of the CLI commands or as part of the .if or .set commands.

The following applies to both CLI commands and .if or .set commands (where X represents a variable).

1. Using $X, without using single or double quotes, replaces the variable X with its string or integer value.

1. Using “X”, with double quotes, means the actual string X.

1. Using “$X”, with double quotes, replaces the variable X with its string or integer value.

1. Using ‘X’, with single quotes does not replace the variable X with its value but means the actual string $X.

This means the following interpretation of single and double quotes applies.

1. All characters within single quotes are interpreted as string characters.

1. All characters within double quotes are interpreted as string characters except for $, which replaces the variable with its value (for example, shell expansion inside a string).

6.3.8.2. Examples of EHS Syntax Supported in Classic CLI

This section describes the supported EHS syntax for the classic CLI.

Note: These scenarios use pseudo syntax.

1. .if $string_variable==string_value_or_string_variable {
             CLI_commands_set1
   .} else {
             CLI_commands_set2
   .} endif
2. .if ($string_variable==string_value_or_string_variable) {
             CLI_commands_set1
   .} else {
             CLI_commands_set2
   .} endif
3. .if $integer_variable==integer_value_or_integer_variable {
             CLI_commands_set1
   .} else {
             CLI_commands_set2
   .} endif
4. .if ($integer_variable==integer_value_or_integer_variable) {
             CLI_commands_set1
   .} else {
             CLI_commands_set2
   .} endif
5. .if $string_variable!=string_value_or_string_variable {
             CLI_commands_set1
   .} else {
             CLI_commands_set2
   .} endif
6. .if ($string_variable!=string_value_or_string_variable) {
             CLI_commands_set1
   .} else {
             CLI_commands_set2
   .} endif
7. .if $integer_variable!=integer_value_or_integer_variable {
             CLI_commands_set1
   .} else {
             CLI_commands_set2
   .} endif
8. .if ($integer_variable!=integer_value_or_integer_variable) {
             CLI_commands_set1
   .} else {
             CLI_commands_set2
   .} endif
9. .set $string_variable = string_value_or_string_variable
10. .set ($string_variable = string_value_or_string_variable)
11. .set $integer_variable = integer_value_or_integer_variable
12. .set ($integer_variable = integer_value_or_integer_variable)

where:

1. CLI_commands_set1 is a set of one or more CLI commands
2. CLI_commands_set2 is a set of one or more CLI commands
3. string_variable is a local (static) string variable
4. string_value_or_string_variable is a string value/variable
5. integer_variable is a local (static) integer variable
6. integer_value_or_integer_variable is an integer value/variable

Note: A limit of 100 local (static) variables per EHS script is imposed. Exceeding this limit may result in an error and partial execution of the script. When a set statement is used to set a string_variable to a string_value, the string_value can be any non-integer value not surrounded by single or double quotes or it can be surrounded by single or double quotes. A "." preceding a directive (for example, if, set, and so on) is always expected to start a new line. An end of line is always expected after {. A CLI command is always expected to start a new line. Passed-in (dynamic) variables are always read-only inside an EHS script and cannot be overwritten using a set statement. .if commands support == and != operators only. .if and .set commands support addition, subtraction, multiplication, and division of integers. .if and .set commands support addition, which means concatenation, of strings.

6.3.8.5. EHS Debounce

EHS debounce (also called dampening) is the ability to trigger an action (for example an EHS script), if an event happens (N) times within a specific time window (S).

N = [2..15]

S = [1..604800]

Note: Triggering occurs with the Nth event not at the end of S. There is no sliding window (for example a trigger at Nth event, N+1 event, and so on), as N is reset after a trigger and count is restarted. When EHS debouncing or dampening is used, the varbinds passed in to an EHS script at script triggering time are from the Nth event occurrence (the Nth triggering event). If S is not specified then the SR OS will continue to trigger every Nth event.

For example:

When linkDown occurs N times in S seconds, an EHS script is triggered to shut down the port.

6.4.1. Python Engine for Syslog

This section discusses syslog-specific aspects of Python processing. Refer to the “Python Script Support for ESM” section of the 7450 ESS, 7750 SR, and VSR Triple Play Service Delivery Architecture Guide for an introduction to Python.

When an event is dispatched to the log manager in SR OS, the log manager asynchronously passes the event context data and varbinds to the Python engine, that is, the logger task is not waiting for feedback from Python. Varbinds are variable bindings that represent the variable number of values that are included in the event. Each varbind consists of a triplet (OID, type, value). Along with other system-level variables, the Python engine constructs a syslog message and sends it to the syslog destination. During this process, the operator can modify the format of the syslog message or leave it intact, as if it was generated by the syslog process within the log manager.

The tasks of the Python engine in a syslog context are as follows:

6.4.2. Python Processing Efficiency

Python retrieves event-related variables from the log manager, as opposed to retrieving pre-assembled syslog messages. This eliminates the need for string parsing of the syslog message to manipulate it constituent parts increasing the speed of Python processing.

To further improve processing performance, Nokia recommends performing string manipulation via the Python native string method, when possible.

6.4.3. Python Backpressure

A Python task assembles syslog messages based on the context information received from the logger and sends them to the syslog server independent of the logger. If the Python task is congested due to a high volume of received data, the backpressure should be sent to the ISA so that the ISA stops allocating NAT resources. This behavior matches the current behavior in which NAT resources allocation is blocked if that logger is congested.

6.4.4. Event Selection for Python Processing

Events destined for Python processing are configured through a log ID that references a Python policy. The selection of the events are performed via a filter associated with this log ID. The remainder of the events destined to the same syslog server can bypass Python processing by redirecting them to a different log ID. The following example clarifies this point:

In the example above, the configuration-only event 2012 from application "nat" will be sent to log-id 33. All other events are forwarded to the same syslog destination via log-id 34, without any modification. As a result, all events (modified via log-id 33 and unmodified via log-id 34) are sent to the syslog 1 destination.

This configuration may cause reordering of syslog messages at the syslog 1 destination due to slight delay of messages processed by Python.

6.4.5. Modifying a Log File

The following displays the current log configuration:

The following displays an example to modify log file parameters:

The following displays the modified log file configuration:

6.4.6. Deleting a Log File

The log ID must be shutdown first before it can be deleted. In a previous example, file 1 is associated with log-id 2.

The following displays an example to delete a log file:

6.4.7. Modifying a File ID

The following displays the current log configuration:

The following displays an example to modify log file parameters:

The following displays the file modifications:

The following displays an example to modify log file parameters:

The following displays the file modifications:

6.4.8. Modifying a Syslog ID

The following displays an example of the syslog ID modifications:

The following displays the syslog configuration:

6.4.9. Modifying an SNMP Trap Group

The following displays the current SNMP trap group configuration:

The following displays an example of the command usage to modify an SNMP trap group:

The following displays the SNMP trap group configuration:

6.4.10. Deleting an SNMP Trap Group

The following displays the SNMP trap group configuration:

The following displays an example to delete a trap target and an SNMP trap group.

6.4.11. Modifying a Log Filter

The following output displays the current log filter configuration:

The following displays an example of the log filter modifications:

The following displays the log filter configuration:

6.4.12. Modifying Event Control Parameters

The following displays the current event control configuration:

The following displays an example of an event control modification:

The following displays the log filter configuration:

The following displays the current event control configuration:

The following displays an example of an event control modification:

The following displays the log filter configuration:

6.4.13. Returning to the Default Event Control Configuration

The no form of the event-control command returns modified values back to the default values.

Use the following CLI syntax to modify event control parameters:

The following displays an example of the command usage to return to the default values:

6.5.1. Accounting Records

An accounting policy must define a record name and collection interval. Only one record name can be configured per accounting policy. Also, a record name can only be used in one accounting policy.

The record name, sub-record types, and default collection period for service and network accounting policies are shown in Table 35. Table 37 (fields per policer stat-mode are given in the stat-mode command descriptions in the 7450 ESS, 7750 SR, 7950 XRS, and VSR Quality of Service Guide), Table 38, and Table 39 provide field descriptions.

When creating accounting policies, one service accounting policy and one network accounting policy can be defined as default. If statistics collection is enabled on a SAP or network port and no accounting policy is applied, then the respective default policy is used. If no default policy is defined, then no statistics are collected unless a specifically defined accounting policy is applied.

Each accounting record name is composed of one or more sub-records which is in turn composed of multiple fields.

Refer to the Application Assurance Statistics Fields Generated per Record table in the 7450 ESS, 7750 SR, and VSR Multiservice Integrated Service Adapter and Extended Services Appliance Guide for fields names for Application Assurance records.

The availability of the records listed in Table 36 depends on the specific platform functionality and user configuration.

Table 37, Table 38, and Table 39 provide field descriptions.

* Enhanced Subscriber Management (ESM) only.

** Enhanced Subscriber Management (ESM) connection bonding only.

6.5.2. Accounting Files

When a policy has been created and applied to a service or network port, the accounting file is stored on the compact flash in a compressed XML file format. The router creates two directories on the compact flash to store the files. The following output displays a directory named act-collect that holds accounting files that are open and actively collecting statistics. The directory named act stores the files that have been closed and are awaiting retrieval.

Accounting files always have the prefix act followed by the accounting policy ID, log ID and timestamp. The accounting log file naming and log file destination properties like rollover and retention are discussed in more detail in Log Files.

6.5.3. Design Considerations for Accounting Policies

The router has ample resources to support large scale accounting policy deployments. When preparing for an accounting policy deployment, verify that data collection, file rollover, and file retention intervals are properly tuned for the amount of statistics to be collected.

If the accounting policy collection interval is too brief there may be insufficient time to store the data from all the services within the specified interval. If that is the case, some records may be lost or incomplete. Interval time, record types, and number of services using an accounting policy are all factors that should be considered when implementing accounting policies.

The rollover and retention intervals on the log files and the frequency of file retrieval must also be considered when designing accounting policy deployments. The amount of data stored depends on the type of record collected, the number of services that are collecting statistics, and the collection interval that is used. For example, with a 1Gb CF and using the default collection interval, the system is expected to hold 48 hours’ worth of billing information.

6.5.4. Reporting and Time-Based Accounting

SR OS on the 7750 SR platform has support for volume accounting and time-based accounting concepts, and provides an extra level of intelligence at the network element level in order to provide service models such as “prepaid access” in a scalable manner. This means that the network element gathers and stores per-subscriber accounting information and compares it with “pre-defined” quotas. Once a quota is exceeded, the pre-defined action (such as re-direction to a web portal or disconnect) is applied.

6.5.5. Overhead Reduction in Accounting: Custom Record

Custom records can be used to decrease accounting messaging overhead as follows:

6.5.6. Immediate Completion of Records

6.5.7. AA Accounting per Forwarding Class

This feature allows the operator to report on protocol/application/app-group volume usage per forwarding class by adding a bitmap information representing the observed FC in the XML accounting files. In case the accounted object is deleted or changed, the latest information will be written in the XML file with a “final” tag indication in the record header.