Redundant Controllers and Multiple Switch Instances

The operator can configure one or more instances of an H-OFS (using SNMP/CLI interfaces) with each instance controlled by an OF-controller over a unique OF channel (using openflow protocol). One OF controller can control multiple H-OFS instances (using dedicated channels), or a dedicated OF controller per switch can be deployed. For each switch, up to two OF controllers can be deployed for redundancy. If two controllers are programmed, they can operate in either OFPCR_ROLE_EQUAL roles or in OFPCR_ROLE_MASTER and OFPCR_ROLE_SLAVE roles. Figure 27 depicts this architecture:

Figure 27:  SR OS Router/Switch OF Controller/Switch Architecture Overview 

GRT-only and Multi-Service H-OFS Mode of Operations

SR OS supports two modes of operations for an H-OFS instance: GRT-only and multi-service. The mode of operations is operator-controlled per H-OFS instance by enabling or disabling switch-defined-cookie option (configure>open-flow>of-switch>flowtable 0). For backward compatibility, GRT-only mode of operation is default but, since multi-service mode is a functional superset, it is recommended to operate in multi-service mode whenever possible. The operator can change the mode in which an H-OFS instance operates but a shutdown is required first. This will purge all the rules forcing the OF controller to reprogram the switch instance once re-enabled in a new mode. An SR OS router supports both H-OFS operational modes concurrently for different switch instances.

Multi-service operational mode uses part of the FlowTable cookie field (higher order 32 bits) to provide the enhanced functionality; the lower order FlowTable cookie bits are fully controlled by the OF controller. Table 65 depicts higher order bit Flow Table cookie encoding used when operating in the multi-service mode.

To enable multi-service mode of operation, an operator must embed the OF switch in an ACL filter policy, and, since multi-service H-OFS supports a mix of VPRN/VPLS/GRT/System rules, an additional scope of embedding must be selected (embed open-flow service, embed open-flow system - grt scope used by default). Since after embedding H-OFS instance, an ACL policy contains rules specific to a particular VPRN or VPLS service instance or to a GRT or to a System Filter Policy, the ACL filter policy can only be used in the scope defined by H-OFS embedding.

Rules programmed by an OF controller with grt, system, and service cookies specified are accepted even if the H-OFS instance is not embedded by a filter activated in a given context. Rules programmed by an OF controller with a service cookie specified, when the service ID is not one of the supported service types, or when the service with the specified id does not exist, are rejected with an error returned back to the controller. If an H-OFS is embedded into a line card policy with a specific service context, the embedding must be removed before that service is deleted.

Table 66 summarizes the main differences between the two modes of operation.

Caveats:

Hybrid OpenFlow Switch Steering using Filter Policies

A router H-OFS instance is embedded into line card IPv4 and IPv6 filter policies to achieve OF-controlled Policy Based Routing (PBR). When H-OFS instance is created, embedded filters (IP and IPv6) required for that instance are automatically created. The filters are created with names, as follows:

“_tmnx_ofs_<ofs_name>”, with the same name for IPv4 and IPv6 filters used.

If embedded filters cannot be allocated due to the lack of filter policy instances, the creation of an H-OFS instance will fail. When the H-OFS instance is deleted, the corresponding embedded filters are freed.

The H-OFS can be embedded only in ingress filter policies on line cards/platforms supporting embedded filters (FP2-based or newer) and for services supporting H-OFS. Embedding of an H-OFS in filter policies on unsupported services is blocked, embedding of an H-OFS in filter policies in unsupported direction or on unsupported hardware follows the general filter policy misconfiguration behavior and is not recommended. Unsupported match fields are ignored. Other match criteria may cause a packet to match an entry.

As soon as an H-OFS instance is created, the controller can program OF rules for that instance. For instance, the rules can be created prior to the H-OFS instance embedding into a filter policy or prior to a filter policy with H-OFS instance embedded being assign to an interface. This allows operator to either preprogram H-OFS steering rules, or to disable the rules without removing them from a flow table by removing the embedding. An error is returned to controller if it attempts to program rules not supported by the system. The following lists examples of the errors returned:

As the OF controller updates traffic steering rules, the Hybrid OpenFlow Switch updates the flow table rules. This automatically triggers programming of the embedded filter, which consequently causes instantiation of the rules for all services/interfaces that have a filter policy embedding this H-OFS instance. Embedding filter policy configuration/operational rules apply also to embedded filters auto-created for an H-OFS instance (see Embedded Filter Support for ACL Filter Policies section of this guide). MPLS cannot be deleted if OFS rules are created that redirect to an LSP.

The auto-created embedded filters can be viewed through CLI but cannot be modified and/or deleted through filter policy CLI/SNMP. Operator can see the above embedded filters under show filter context, including the details on the filters themselves, entries programmed, interface association, statistics, etc.

The following picture depicts the H-OFS to service operator-configurable mapping example.

For an H-OFS with switch-defined-cookie enabled, embedded filters are created for each unique context in the H-OFS instead.

Figure 28:  OF Flow Table Mapping to Router/Switch Service Infrastructure Example — switch-defined-cookie Disabled 

The router allows mixing H-OFS rules from one or more H-OFS instances in a single filter policy. Co-existence of H-OFS rules in a single policy with CLI/SNMP programmed rules and/or BGP flowspec programmed rules in a single line card filter policy is also supported. When a management interface and an OF controller flow entry have the same filter policy entry, the management interface-created entry overrides the OF controller-created entry; see the embedded filter functional description. For mixing of the rules from multiple management entities, the controller should not program an entry in its Flow Table that would match all traffic, as this would stop evaluation of the filter policy.

The router supports HA for the OF Flow Table content and statistics. On an activity switch the channel goes down and is re-established by the newly active CPM. “Fail secure mode” operation takes place during channel re-establishment (OpenFlow rules continue to be applied to the arriving traffic). OF controller is expected to re-synchronize the OF table when the channel is re-established.On a router reboot, H-OFS Flow Table rules and statistics are purged. The same takes place when H-OFS instance is shutdown. The H-OFS instance cannot be deleted unless the H-OFS instance is removed first from all embedding filter policies.

Hybrid OpenFlow Switch Statistics

SR OS Hybrid OpenFlow switch supports statistics retrieval using the OpenFlow protocol. There are two types of statistics that can be collected:

Operational Notes:

OpenFlow Switch Auxiliary Channels

The H-OFS supports auxiliary channels, as defined in OpenFlow version 1.3.1. The packet-in and statistics functions are supported on the auxiliary channels as well as on the main channel. When the auxiliary channel is enabled on a switch (using the aux-channel-enable command), the switch will set up a dedicated auxiliary channel for statistics (Auxiliary ID 1) and a dedicated auxiliary channel for packet-in (Auxiliary ID 2) if a packet-in action is configured, to every controller for a given H-OFS switch instance. Auxiliary connections use the same transport as the main connection. The switch will handle any requests over any established channel and respond on the same channel even if a specific requested auxiliary channel is available. The H-OFS instance uses the packet-in connection for packet-in functionality by default and expects (but does not require) the controller to use the statistics channel for statistics processing by default. The switch uses the auxiliary channels (packet-in for packet-in-specific requests and statistics for statistics-specific requests) as long as they are available. If they are not available, the switch will use the next available auxiliary channel. If none of the auxiliary channels are available, the main channel will be used.Auxiliary connections can be enabled or disabled without shutting down the switch.

Hybrid OpenFlow Switch Traffic Steering Details

As described in the previous section, an update to an OpenFlow Switch’s flow table, results in the embedded filter update(s), which triggers update to all filter policies embedding those filters. The router automatically downloads the new set of rules to the line cards as defined through service configuration. The rules became part of ingress line card pipeline as depicted in Figure 29.

Figure 29:  OpenFlow Switch Embedding in Ingress Pipeline Processing