Executive summary

Software Defined Networking (SDN) delivers more robust and configurable networks by providing a standardized programmable interface between applications and network resources. As SDN deployments mature, pragmatic issues are emerging in relation to: performance, security, variability of resources to be programmed, and many peripheral field issues ranging from dealing with link to power supply failures. These issues highlight the need for a well rounded architectural approach to SDN. Many solutions and frameworks have been proposed and standardized. Most, including the well-known OpenFlow, are relegated to niche environments due to inherent design limitations. The IETF’s standardized “Forwarding and Control Element Separation (ForCES)” framework, however, provides a holistic approach which has proven versatile and powerful when addressing SDN’s emerging issues. Having been developed and improved through the rigorous IETF standards process, ForCES is currently the only fully-featured single architectural solution for SDN that is known to meet the resource abstraction and control and management requirements necessary for implementing a robust SDN solution that can meet the needs for carrier and enterprise deployments.

Mojatatu’s Network Operating System (MojaNET^TM) builds upon the IETF ForCES framework to provide scalability, high availability and high performance, accompanied by a software development toolkit that provides new value added services and functionalities for developers and operators, allowing them to control and manage all network resources in a homogeneous approach.

Software Defined Networking Architecture

An SDN architecture provides applications with interfaces to programmatically control and manage network resources. Exposing network resources programmatically changes the dynamics of the canned ‘cookie-cutter’ solutions that vendors have traditionally provided. Applications can be written to cater to network specific needs to meet specific network loads. An SDN solution must therefore not only have the flexibility to add or enhance features but should also provide a toolset for building a more robust network where redundancy, availability and timely responses are crucial.

A network typically constitutes a heterogeneous environment comprised of devices containing various data path and operational resources. A solid SDN architecture requires a versatile modeling language to abstract resources and export them as data models to a protocol for control and management. MojaNET^TM focuses on the following SDN architectural elements [RFC7426], as can be seen in Figure 1.

1.The Network Operating System (NOS) glues resources, applications and services together. The NOS provides the southbound interface abstractions towards the resources, both for control and management, and northbound APIs to services and applications.

2.The Southbound protocol connects the control and management planes and resources. A single protocol interface such as provided ForCES and, therefore MojaNET, simplifies the programmability and operational interfaces.

3.Four programmatic interfaces provide for different semantic needs. Applications can be written to interface to one or more of these interfaces as illustrated in Figure 1.

·Resource Abstraction Layer (RAL) is responsible for abstracting access to resources.

·Control Abstraction Layer (CAL) abstracts the interface between resources and the NOS.

·Management Abstraction Layer (MAL) abstracts management functionality

·Service Abstraction Layer (SAL) exposes APIs from the NOS to user applications and services.

The most critical SDN elements are the resource abstractions and the southbound protocol. Upon these two elements hinge the efficiency and range of APIs that the NOS can offer to services and applications. Each element has a different set of requirements which we illustrate in this document.

Resource Modeling Requirements

The RAL provides an abstracted view of network device resources, irrespective of actual implementation, software or hardware, whether the resource is physical or virtual, or whether dealing with data path versus supporting operational resources. A versatile RAL is implemented around a modeling language capable of describing resources. In this context it is therefore important for the RAL to satisfy the following requirements:

·Extensibility. The RAL needs to provide the necessary toolset to model resources in an extensible fashion which is both forward and backwards compatible. Such a requirement can only be fulfilled by a modeling language that provides a basic set of data types which can be extended and reused as needed to create new data type definitions. For example, the model should be able to support the description of a forwarding table in the same semantics as a power supply unit simply by adding new data type definitions. As we rapidly move to the Internet of Things (IoT), the extensibility to incorporate varied and disparate network elements – from pollution sensors spread across a city to personal biometric devices to smart cars - will be a critical design consideration.

·Inheritance. Resource types are usually reused, versioned, changed and upgraded. Besides being extensible, in order to support the evolving nature of resources the modeling language should follow an object-oriented approach, with similar features such as inheritance, with resource types as classes and actual resources as instances. For example, a virtual port data model could inherit from a physical port data model.

​

·Capabilities expression. Each resource varies in its characteristics, functionalities and limitations. Similar resource types can differ in terms of their constraints, e.g. an ethernet port implementation from one ASIC vendor could support a subset of encapsulation schemes that a different ASIC vendor supports. An application has to be able to discover the capabilities of all resources, without knowing them a priori, in order to effectively utilize them. Therefore, it is very important that the modeling language support the definition and descriptions of the capabilities that resources support.

·Events definition. Resources have the ability to produce a report when an event occurs. An extensible modeling language has to support formal event definition, and the definition has to include the resource components that must be monitored and the trigger that activates these events, along with what to report when the event occurs. The model must be able to define a variety of event trigger filters such as time-based, count-based, change-based, and threshold crossings.

​

·Flexibility. Resource instances are usually grouped and connected to each other in a static or dynamic and programmable graph to create a packet service. As examples, an ASIC has a fixed resource graph, while software, network processors, or FPGAs can provide programmable graphs. In this context, the modeling language should be able to describe the service graph and the possible interconnection of resources. Additionally, the model should provide an application the ability to programmatically manipulate these interconnections and change the resource graph. Figure 3 is an example of a resource graph.

· Granularity. Sometimes groups of resources must be managed as a single resource. The modeling language has to provide the ability to define resources with different levels of granularity. A resource definition can be as fine-grained as an L3 Forwarding Information table, or as coarse-grained as grouping all components of an L3 router.

·Virtualization support. SDN plays an important role in Network Function Virtualization (NFV). A versatile model must be able to support physical and virtual resources. An expressive model should also support modeling of resources pertaining to resource management such as resource instantiation and release.

·Implementation independent. A resource could be implemented in hardware or in software, or it could exist as a physical resource or within a virtual environment. In addition, it is common for various vendors to implement the same resource using different models. An implementation independent model abstracts these details away and completely decouples controllers from resources. Such a model empowers applications to control resources irrespective of their actual implementation.

SDN Southbound Interface Requirements

There are two types of southbound interfaces in an SDN solution: control and management. While it is possible to have different protocols - one for each interface - MojaNET provides and requires only one, ForCES. Using ForCES for both control and management simplifies and reduces the effort required to implement services and applications. Typically, the control southbound interface is stricter in terms of performance [RFC7426] whereas the management interface tends to be less constrained. The ForCES architecture provides formalization of the data on the wire, via its data model, which make it suitable for both control and management interfaces. In this context we discuss the requirements for the control interface. These are:

·Transport Independence. Untangle transport from the control protocol for southbound interfaces. It should not matter whether the underlying transport is on IP, Ethernet or Optical, the control protocol should stay consistent. In effect, there is a need to separate the transport and data protocol via transport abstractions.

·Inherent security. SDN introduces additional points of attack for networks [1]; the southbound protocol being one of them. Therefore it is imperative that the transport protocol inherently support security mechanisms, e.g. IPSec, TLS, MACSec, etc.

·High Availability (HA). Any resource or control node can fail. It becomes crucial, therefore, that the interface is engineered to accommodate different HA needs. An optimal architecture provides HA as an option with different kinds of high availability, such as cold and hot standby.

·Publish/Subscribe mechanisms. The interface should provide the ability to subscribe to events and get event notifications. Resource level sensors advertise events synchronously or asynchronously based on programmed triggers. This ability is an improvement over traditional polling of management interfaces.

​

·Request/Response mechanisms. Applications need to be able to issue a command request to a resource and receive an asynchronous response. This mechanism should be simple and have a small number of command verbs such as get, set, and delete. The request/response interface complements the publish/subscribe mechanism. For simplicity, in network management and control programmability it is important for one protocol to be capable of supporting both mechanisms.

·Transactions. Applications need to control multiple resources which may have dependencies. When a change needs to take effect on resources in various network locations in an atomic manner, transaction mechanisms are required. Mechanisms like two phase commits and rollbacks need to be supported. In addition, a variety of execution modes should be supported to serve various utility needs for transactions which constitute multiple commands; these include: atomic execute-all-or-none, execute-until-error, and execute-all-despite-error (to allow for a set of batched commands and record, but ignore, failures).

·Traffic sensitive keep alive mechanisms. Keep alive messages could be sent bi-directionally between control and resources. For reasons of efficiency, these heartbeats should be optionally available and sent only during idle periods. It is important to note that keep-alive mechanisms should be available for both the transport and data protocol layers.

·Message priorities. Certain messages, such as configuration messages and their responses, require immediate action. It is important for the interface to distinguish higher priority from lower priority messages.

·Efficiency. The interface should be simple and able to efficiently transfer data across the wire. When possible, a binary protocol would be preferred over text based, to save both compute and wire resources. Additionally, the interface has to include mechanisms that can support message batching to amortize commands or parallelization to reduce latency, as both are highly desired.

There are a number of solutions in the market that offer a range of features that cover parts of the requirements for both the RAL and the southbound interfaces; however, all known solutions other than ForCES provide their offerings in the form of a menu of interfaces and a set of protocols. While such a menu of heterogeneous sets of protocols will work, in effect it reduces usability or ends up requiring extra layers of abstraction to improve programmability. This adds either complexity or maintenance overhead to the NOS implementation. A more elegant solution, like ForCES, utilizes a single framework capable of supporting all of SDN’s requirements outlined above for the model and protocol.

Currently IETF’s ForCES is the only fully-featured single architectural solution for SDN that is known to meet the RAL and southbound requirements.

Forwarding and Control Element Separation

ForCES [2] is an IETF standardized framework developed to model, control and manage network device resources. The ForCES architecture meets all the requirements mentioned above and includes: an expressive and extensible model [3], a transport layer module (TML) [4] abstraction, and a clean control protocol [5].

The ForCES protocol is a wire encoding protocol, efficient at transporting messages with little bandwidth waste. The ForCES data model represents resources as libraries of Logical Functional Blocks (LFBs) classes, each with its own set of parameters. Actual resources are instances of these LFB classes, interconnected in a directed graph to create functionalities. Developers can extend the current model by using the base set of datatypes, scalar or compound, to create their own custom datatypes to represent resource parameters. This extensibility has allowed ForCES use to extend beyond traditional packet processing resources. As an example, MojaNET ships with LFB models of device sensors such as temperature, voltage and fan control. The ForCES protocol and model have been engineered with several features listed below:

MojaNET^TM

The Mojatatu NOS, MojaNET^TM, utilizes the ForCES architecture to provide scalability, high availability, and high performance. MojaNET exposes ForCES LFB abstractions while hiding their detailed intricacies from the programming interface, making it easy to create applications and services. MojaNET is built upon, but goes well beyond, ForCES. MojaNET’s extensible architecture has evolved over time through real-world carrier and enterprise grade deployment experiences.

MojaNET is agnostic to whether underlying resources are virtual/overlay or physical/underlay, and has been deployed on ASICs ranging from fixed function switching chips to more flexible network processors. In addition, it has been deployed for virtual networking solutions on bare-bone machines, VMs and containers; all with the same homogenous interfaces.

References:

[1] Kreutz, Diego, Fernando Ramos, and Paulo Verissimo. "Towards secure and dependable software-defined networks." Proceedings of the second ACM SIGCOMM workshop on Hot topics in software defined networking. ACM, 2013.

[2] Evangelos Haleplidis, Jamal Hadi Salim, Joel M Halpern, Susan Hares, Kostas Pentikousis, Kentaro Ogawa, Wang Weiming, Spyros Denazis, Odysseas Koufopavlou, "Network Programmability with ForCES", IEEE Communications Surveys & Tutorials Issue Date, third quarter 2015, Volume: 17 Issue: 3, On page(s): 1-18. DOI: http://dx.doi.org/10.1109/COMST.2015.2439033

[3] Halpern, Joel and Jamal Hadi Salim, "Forwarding and Control Element Separation (ForCES) Forwarding Element Model", RFC 5812, IETF, March 2010.

[4] Hadi Salim, Jamal, Kentaro Ogawa. "SCTP-Based Transport Mapping Layer (TML) for the Forwarding and Control Element Separation (ForCES) Protocol", RFC 5811, IETF, March 2010.

[5] Doria, Avri, Jamal Hadi Salim, Robert Haas, Horzmud Khosravi, Weiming Wang, Ligang Dong, Ram Gopal and Joel Halpern, "Forwarding and Control Element Separation (ForCES) Protocol Specification", RFC 5810, IETF, March 2010.

[6] Ogawa, Kentaro, Weiming Wang, Evangelos Haleplidis, Jamal Hadi Salim. "ForCES Intra-NE High Availability", RFC 7121. IETF, February 2014.​