High performance software-defined core network

Abstract

A system of nodes configured to form a network including a plurality of virtual links in an overlay network provisioned over an underlay network. The system includes at least one virtual machine (VM) at each node. The VM is coupled to the network and configured as a tenant VM to correspond to a tenant of the network, and a tenant network includes the tenant VM at each node. The system includes numerous control planes, and each control plane is created by the VM of a corresponding tenant network and configured as a tenant control plane for routing of traffic flows of a corresponding tenant. The system includes numerous data planes, and each data plane is created by the VM and configured as a tenant data plane for traffic flows of a corresponding tenant.

What is claimed is: 
     
         1 . A system comprising:
 a plurality of nodes configured to form a network including a plurality of virtual links in an overlay network provisioned over an underlay network;   at least one virtual machine (VM) at each node, wherein the at least one VM is coupled to the network and configured as a tenant VM to correspond to at least one tenant of a plurality of tenants of the network, wherein a tenant network includes the tenant VM at each node;   a plurality of control planes, wherein each control plane is created by the at least one VM of a corresponding tenant network and configured as a tenant control plane for routing of traffic flows of a corresponding tenant; and   a plurality of data planes, wherein each data plane is created by the at least one VM and configured as a tenant data plane for traffic flows of a corresponding tenant.   
     
     
         2 . The system of  claim 1 , wherein each tenant control plane is dedicated to the tenant and isolated from others of the plurality of control planes, and each tenant data plane is isolated from the corresponding tenant control plane. 
     
     
         3 . The system of  claim 1 , wherein the underlay network includes servers of a public network. 
     
     
         4 . The system of  claim 1 , wherein each tenant data plane is isolated from others of the plurality of data planes. 
     
     
         5 . The system of  claim 1 , wherein the plurality of virtual links is a component of the overlay network and utilizes the underlay network for delivery of the tenant traffic flow. 
     
     
         6 . The system of  claim 5 , wherein the plurality of virtual links comprises a plurality of single-hop virtual links coupled between each node of the plurality of nodes. 
     
     
         7 . The system of  claim 1 , wherein the tenant network corresponds to the tenant, and includes a set of virtual links of the plurality of virtual links. 
     
     
         8 . The system of  claim 7 , wherein the network includes a plurality of tenant networks corresponding to the plurality of tenants, wherein each tenant network is isolated from each other tenant network of the plurality of tenant networks. 
     
     
         9 . The system of  claim 8 , wherein the plurality of tenant networks is configured to maintain separation of multi-tenant traffic flows throughout the network. 
     
     
         10 . The system of  claim 7 , wherein the tenant network is configured with a tenant configuration of a corresponding tenant to control routing of tenant traffic flows of the tenant. 
     
     
         11 . The system of  claim 10 , wherein the tenant VM includes a virtual router (VR), wherein the VR is coupled to the network and to the corresponding tenant of the VM. 
     
     
         12 . The system of  claim 11 , wherein at least one VR of the tenant network is configured to receive feedback data from the at least one VM, and to characterize the network using the feedback data. 
     
     
         13 . The system of  claim 12 , wherein the VR includes at least one objective function. 
     
     
         14 . The system of  claim 13 , wherein the VR is configured to characterize the network by applying the at least one objective function to the feedback data. 
     
     
         15 . The system of  claim 14 , wherein the feedback data includes link state data of the plurality of virtual links. 
     
     
         16 . The system of  claim 15 , wherein the link state data of each link represents at least one link metric of the link, wherein the at least one link metric includes at least one of latency, jitter, packet loss, throughput, utilization, link state, and link status. 
     
     
         17 . The system of  claim 15 , wherein the VR is configured to determine a best route of the network using the network characterization, and control routing of a tenant traffic flow of the corresponding tenant according to the best route. 
     
     
         18 . The system of  claim 17 , wherein the control of the routing of the tenant traffic comprises the VR separately controlling routing of each tenant traffic flow to at least one next node of the best route. 
     
     
         19 . The system of  claim 17 , wherein the VR comprises a plurality of routing control algorithms representing a plurality of routing behaviors, wherein each routing control algorithm is configured to determine at least one best route through the network for routing the tenant traffic flow from an ingress node of the plurality of nodes to an egress node of the plurality of nodes. 
     
     
         20 . The system of  claim 19 , wherein each routing behavior corresponds to a traffic classification of a corresponding tenant traffic flow. 
     
     
         21 . The system of  claim 20 , wherein each routing behavior is defined by an objective function of the plurality of objective functions, wherein the plurality of objective functions include the at least one objective function. 
     
     
         22 . The system of  claim 21 , wherein the VR is configured to apply a corresponding objective function to the link state data and generate a link weight for each link of the set of links. 
     
     
         23 . The system of  claim 22 , wherein the VR is configured to determine the at least one best route of the tenant traffic flow according to link weights of the set of links. 
     
     
         24 . The system of  claim 23 , wherein the control of the routing of the tenant traffic flow comprises continually adapting the at least one best route in response to changes in the link state data as determined with the corresponding objective function. 
     
     
         25 . The system of  claim 24 , wherein the VR is configured to periodically receive link state updates that include updated link state data of the set of virtual links. 
     
     
         26 . The system of  claim 25 , wherein the continually adapting of the at least one best route comprises applying the corresponding objective function to the updated link state data. 
     
     
         27 . The system of  claim 26 , wherein the VR is configured to apply the corresponding objective function to the updated link state data and generate an updated link weight for each link of the set of links. 
     
     
         28 . The system of  claim 27 , wherein the VR is configured to determine an updated best route of the tenant traffic flow according to updated link weights of the set of links. 
     
     
         29 . The system of  claim 28 , wherein each VR of a set of VRs corresponding to the tenant includes the plurality of objective functions that characterize the network. 
     
     
         30 . The system of  claim 28 , wherein the at least one VM includes a plurality of VMs, wherein each VM includes a VR, wherein each VR of the plurality of VRs includes the plurality of objective functions that characterize the network. 
     
     
         31 . The system of  claim 19 , wherein the plurality of routing behaviors includes at least one routing behavior configured to route the tenant traffic flow via multiple paths of the network. 
     
     
         32 . The system of  claim 19 , wherein the plurality of routing behaviors includes at least one routing behavior configured to route the tenant traffic flow directly via a shortest path of the network. 
     
     
         33 . The system of  claim 19 , wherein the plurality of routing behaviors includes at least one routing behavior configured to route the tenant traffic flow on a path and maintain the tenant traffic flow on the path until detection of a network event. 
     
     
         34 . The system of  claim 33 , wherein the network event includes at least one of a network topology change and a variation in the link state data. 
     
     
         35 . The system of  claim 34 , wherein the variation in the link state data comprises a variation exceeding a pre-specified threshold. 
     
     
         36 . The system of  claim 17 , wherein the VR is configured to maintain configuration data of the tenant configuration, and to use the configuration data in the control of the routing of the tenant traffic flow. 
     
     
         37 . The system of  claim 36 , wherein the configuration data includes traffic class configuration data, wherein the traffic class configuration data identifies traffic classes. 
     
     
         38 . The system of  claim 36 , wherein the configuration data includes route configuration data, wherein the route configuration data includes data of a service that is a recipient of the tenant traffic flow. 
     
     
         39 . The system of  claim 36 , wherein the VR is configured to maintain topology data including a logical view of the tenant network for the corresponding tenant, and to use the topology data in the control of the routing of the tenant traffic flow. 
     
     
         40 . The system of  claim 17 , wherein the VM includes a monitoring agent coupled to the VR, wherein the monitoring agent is configured to collect the feedback data of the set of virtual links. 
     
     
         41 . The system of  claim 40 , wherein each monitoring agent is configured to collect the feedback data from at least one of a plurality of monitoring agents and at least one other VR of at least one other VM. 
     
     
         42 . The system of  claim 41 , wherein each monitoring agent is configured to collect the feedback data using probe signals exchanged with others of the at least one VM. 
     
     
         43 . The system of  claim 42 , wherein the VM is configured to send the feedback data to the monitoring agent transmitting the probe signals in response to receipt of the probe signals. 
     
     
         44 . The system of  claim 42 , wherein the monitoring agent is configured to generate the link state data of the set of virtual links by processing the feedback data. 
     
     
         45 . The system of  claim 44 , wherein the VR is configured to receive from the monitoring agent the link state data of the set of virtual links. 
     
     
         46 . The system of  claim 40 , wherein the at least one VM includes a plurality of VMs, wherein each VM includes a VR, wherein each VR is configured to receive the link state data of others of the plurality of links from others of a plurality of VRs. 
     
     
         47 . The system of  claim 40 , wherein each VM includes a virtual gateway coupled to the corresponding tenant and the corresponding VR, wherein the virtual gateway is configured to control tenant traffic flows incoming to the VM from the corresponding tenant. 
     
     
         48 . The system of  claim 47 , wherein the virtual gateway is coupled to the monitoring agent. 
     
     
         49 . The system of  claim 47 , wherein the virtual gateway is configured to attract tenant traffic flows of the corresponding tenant, and to reject traffic flows arriving from sources other than the corresponding tenant. 
     
     
         50 . The system of  claim 49 , wherein the at least one VM includes a set of public IP addresses, wherein the set of public IP addresses is dedicated to the corresponding tenant, wherein the corresponding tenant accesses the virtual gateway of the VM using the set of public IP addresses. 
     
     
         51 . The system of  claim 47 , wherein the VR is configured to generate at least one set of flow rules configured to control the routing of the tenant traffic flow through the overlay network. 
     
     
         52 . The system of  claim 51 , wherein the at least one set of flow rules corresponds to the at least one objective function. 
     
     
         53 . The system of  claim 51 , comprising at least one virtual switch coupled to the VR and the virtual gateway of each VM. 
     
     
         54 . The system of  claim 53 , wherein the at least one virtual switch includes a set of routing tables representing the at least one set of flow rules, wherein the set of routing tables is configured to manage the control of the routing of the tenant traffic flow through the network. 
     
     
         55 . The system of  claim 53 , wherein the at least one virtual switch is configured to transfer the tenant traffic flow between the virtual gateway and the VR. 
     
     
         56 . The system of  claim 53 , wherein each node includes at least one aggregator coupled to the at least one virtual switch and the network. 
     
     
         57 . The system of  claim 56 , wherein the aggregator is configured to route via the network the tenant traffic flows of the plurality of tenants corresponding to the node. 
     
     
         58 . The system of  claim 53 , comprising an aggregator coupled to the at least one virtual switch. 
     
     
         59 . The system of  claim 58 , wherein the aggregator is configured to route via the network the tenant traffic flow received at the virtual gateway from the corresponding tenant. 
     
     
         60 . The system of  claim 58 , wherein the aggregator is configured to route to the corresponding tenant the tenant traffic flow received at the node via the network. 
     
     
         61 . The system of  claim 60 , wherein the tenant traffic flow arriving at the aggregator via the network is routed to the corresponding tenant via at least one of the corresponding VR and the virtual gateway. 
     
     
         62 . The system of  claim 60 , wherein the virtual gateway routes the tenant traffic flow arriving at the aggregator via the network to the tenant via a coupling over a public network. 
     
     
         63 . The system of  claim 58 , wherein each node includes a hypervisor, wherein the hypervisor is configured as an operating system of the at least one VM of the node. 
     
     
         64 . The system of  claim 63 , wherein the hypervisor is configured to include at least one of the aggregator and the at least one virtual switch. 
     
     
         65 . The system of  claim 47 , comprising a provisioner coupled to the at least one VM of the plurality of nodes, wherein the provisioner is configured to control provisioning of at least one of the overlay network and the underlay network. 
     
     
         66 . The system of  claim 65 , wherein the provisioner is configured to control configuration of the at least one VM of the plurality of nodes. 
     
     
         67 . The system of  claim 66 , wherein the provisioner is configured to control configuration of components of the at least one VM using a tenant configuration of the corresponding tenant, wherein the components include at least one of the VR and the virtual gateway. 
     
     
         68 . The system of  claim 65 , wherein the provisioner is configured to maintain network data of at least one of the overlay network and the underlay network, wherein the network data includes data representing the overlay network, the underlay network, route configurations, topology data of the network including the plurality of virtual links, and tenant configurations of the plurality of tenants. 
     
     
         69 . The system of  claim 1 , wherein the at least one VM at each node includes a plurality of VMs at each node, wherein each VM at each node is configured as the tenant VM to correspond to a tenant of the plurality of tenants. 
     
     
         70 . The system of  claim 69 , wherein a tenant set of VMs includes the tenant VM of the corresponding tenant at each node, and the tenant network comprises the tenant set of VMs. 
     
     
         71 . The system of  claim 69 , wherein at least one VM of the plurality of VMs at each node includes the VR. 
     
     
         72 . The system of  claim 69 , wherein each VM of the plurality of VMs includes a VR.