High performance software-defined core network

Abstract

A method comprises instantiating at least one virtual machine (VM) at each node of a network including virtual links in an overlay network provisioned over an underlay network. The method includes configuring the VM as a tenant VM to correspond to at least one tenant of the network, such that a tenant network includes the tenant VM at each node. The method comprises configuring the VM of a corresponding tenant network to generate a control plane configured as a tenant control plane to route traffic flows of the corresponding tenant, and a data plane configured as a tenant data plane for the traffic flows of the corresponding tenant. The network includes control planes corresponding to the tenants, and data planes corresponding to the tenants.

What is claimed is: 
     
         1 . A method comprising:
 configuring a plurality of nodes to form a network including a plurality of virtual links in an overlay network provisioned over an underlay network;   instantiating at least one virtual machine (VM) at each node, and configuring the at least one VM as a tenant VM to correspond to at least one tenant of a plurality of tenants of the network, wherein a tenant network includes the tenant VM at each node;   configuring the at least one VM of a corresponding tenant network to generate a control plane configured as a tenant control plane to route traffic flows of the corresponding tenant, wherein the network includes a plurality of control planes corresponding to the plurality of tenants; and   configuring the at least one VM of the corresponding tenant network to generate a data plane configured as a tenant data plane for the traffic flows of the corresponding tenant, wherein the network includes a plurality of data planes corresponding to the plurality of tenants.   
     
     
         2 . The method of  claim 1 , comprising configuring each tenant control plane to be dedicated to the tenant and isolated from others of the plurality of control planes, and configuring each tenant data plane to be isolated from the corresponding tenant control plane. 
     
     
         3 . The method of  claim 1 , wherein the underlay network includes servers of a public network. 
     
     
         4 . The method of  claim 1 , comprising configuring each tenant data plane to be isolated from others of the plurality of data planes. 
     
     
         5 . The method of  claim 1 , comprising configuring the plurality of virtual links as a component of the overlay network to use the underlay network for delivery of the tenant traffic flow. 
     
     
         6 . The method of  claim 5 , comprising configuring the plurality of virtual links to include a plurality of single-hop virtual links coupled between each node of the plurality of nodes. 
     
     
         7 . The method of  claim 1 , comprising configuring the tenant network to correspond to the tenant, and to include a set of virtual links of the plurality of virtual links. 
     
     
         8 . The method of  claim 7 , comprising configuring the network to include a plurality of tenant networks corresponding to the plurality of tenants, and configuring each tenant network to be isolated from each other tenant network of the plurality of tenant networks. 
     
     
         9 . The method of  claim 8 , comprising configuring the plurality of tenant networks to maintain separation of multi-tenant traffic flows throughout the network. 
     
     
         10 . The method of  claim 7 , comprising configuring the tenant network with a tenant configuration of a corresponding tenant to control routing of tenant traffic flows of the tenant. 
     
     
         11 . The method of  claim 10 , comprising configuring the tenant VM to include a virtual router (VR), wherein the VR is coupled to the network and to the corresponding tenant of the VM. 
     
     
         12 . The method of  claim 11 , comprising configuring at least one VR of the tenant network to receive feedback data from the at least one VM, and to characterize the network using the feedback data. 
     
     
         13 . The method of  claim 12 , comprising configuring the VR to include at least one objective function. 
     
     
         14 . The method of  claim 13 , comprising configuring the VR to characterize the network by applying the at least one objective function to the feedback data. 
     
     
         15 . The method of  claim 14 , comprising configuring the feedback data to include link state data of the plurality of virtual links. 
     
     
         16 . The method of  claim 15 , comprising configuring the link state data of each link to represent at least one link metric of the link, wherein the at least one link metric includes at least one of latency, jitter, packet loss, throughput, utilization, link state, and link status. 
     
     
         17 . The method of  claim 15 , comprising configuring the VR to determine a best route of the network using the network characterization, and control routing of a tenant traffic flow of the corresponding tenant according to the best route. 
     
     
         18 . The method of  claim 17 , comprising configuring the VR to separately control the routing of each tenant traffic flow to at least one next node of the best route. 
     
     
         19 . The method of  claim 17 , comprising configuring the VR to include a plurality of routing control algorithms representing a plurality of routing behaviors, and configuring each routing control algorithm to determine at least one best route through the network for routing the tenant traffic flow from an ingress node of the plurality of nodes to an egress node of the plurality of nodes. 
     
     
         20 . The method of  claim 19 , comprising configuring each routing behavior to correspond to a traffic classification of a corresponding tenant traffic flow. 
     
     
         21 . The method of  claim 20 , comprising configuring each routing behavior to be defined by an objective function of the plurality of objective functions, wherein the plurality of objective functions to include the at least one objective function. 
     
     
         22 . The method of  claim 21 , comprising configuring the VR to apply a corresponding objective function to the link state data and generate a link weight for each link of the set of links. 
     
     
         23 . The method of  claim 22 , comprising configuring the VR to determine the at least one best route of the tenant traffic flow according to link weights of the set of links. 
     
     
         24 . The method of  claim 23 , comprising configuring the control of the routing of the tenant traffic flow to continually adapt the at least one best route in response to changes in the link state data as determined with the corresponding objective function. 
     
     
         25 . The method of  claim 24 , comprising configuring the VR to periodically receive link state updates that include updated link state data of the set of virtual links. 
     
     
         26 . The method of  claim 25 , comprising configuring the continual adaptation of the at least one best route to include applying the corresponding objective function to the updated link state data. 
     
     
         27 . The method of  claim 26 , comprising configuring the VR to apply the corresponding objective function to the updated link state data and generate an updated link weight for each link of the set of links. 
     
     
         28 . The method of  claim 27 , comprising configuring the VR to determine an updated best route of the tenant traffic flow according to updated link weights of the set of links. 
     
     
         29 . The method of  claim 28 , comprising configuring each VR of a set of VRs corresponding to the tenant to include the plurality of objective functions that characterize the network. 
     
     
         30 . The method of  claim 28 , wherein instantiating the at least one VM comprises instantiating a plurality of VMs, and configuring each VM to include a VR, and configuring each VR of the plurality of VRs to include the plurality of objective functions that characterize the network. 
     
     
         31 . The method of  claim 19 , comprising configuring the plurality of routing behaviors to include at least one routing behavior configured to route the tenant traffic flow via multiple paths of the network. 
     
     
         32 . The method of  claim 19 , comprising configuring the plurality of routing behaviors to include at least one routing behavior configured to route the tenant traffic flow directly via a shortest path of the network. 
     
     
         33 . The method of  claim 19 , comprising configuring the plurality of routing behaviors to include at least one routing behavior configured to route the tenant traffic flow on a path and maintain the tenant traffic flow on the path until detection of a network event. 
     
     
         34 . The method of  claim 33 , comprising configuring the network event to include at least one of a network topology change and a variation in the link state data. 
     
     
         35 . The method of  claim 34 , comprising configuring the variation in the link state data to include a variation exceeding a pre-specified threshold. 
     
     
         36 . The method of  claim 17 , comprising configuring the VR to maintain configuration data of the tenant configuration, and to use the configuration data in the control of the routing of the tenant traffic flow. 
     
     
         37 . The method of  claim 36 , comprising configuring the configuration data to include traffic class configuration data, wherein the traffic class configuration data identifies traffic classes. 
     
     
         38 . The method of  claim 36 , comprising configuring the configuration data to include route configuration data comprising data of a service that is a recipient of the tenant traffic flow. 
     
     
         39 . The method of  claim 36 , comprising configuring the VR to maintain topology data including a logical view of the tenant network for the corresponding tenant, and to use the topology data in the control of the routing of the tenant traffic flow. 
     
     
         40 . The method of  claim 17 , comprising configuring the VM to include a monitoring agent coupled to the VR, and configuring the monitoring agent to collect the feedback data of the set of virtual links. 
     
     
         41 . The method of  claim 40 , comprising configuring each monitoring agent to collect the feedback data from at least one of a plurality of monitoring agents and at least one other VR of at least one other VM. 
     
     
         42 . The method of  claim 41 , comprising configuring each monitoring agent to collect the feedback data using probe signals exchanged with others of the at least one VM. 
     
     
         43 . The method of  claim 42 , comprising configuring the VM to send the feedback data to the monitoring agent transmitting the probe signals in response to receipt of the probe signals. 
     
     
         44 . The method of  claim 42 , comprising configuring the monitoring agent to generate the link state data of the set of virtual links by processing the feedback data. 
     
     
         45 . The method of  claim 44 , comprising configuring the VR to receive from the monitoring agent the link state data of the set of virtual links. 
     
     
         46 . The method of  claim 40 , wherein instantiating the at least one VM comprises instantiating a plurality of VMs, and configuring each VM to include a VR, and configuring each VR of the plurality of VRs to receive the link state data of others of the plurality of links from others of a plurality of VRs. 
     
     
         47 . The method of  claim 40 , comprising configuring each VM to include a virtual gateway coupled to the corresponding tenant and the corresponding VR, and configuring the virtual gateway to control tenant traffic flows incoming to the VM from the corresponding tenant. 
     
     
         48 . The method of  claim 47 , comprising configuring the virtual gateway to couple to the monitoring agent. 
     
     
         49 . The method of  claim 47 , comprising configuring the virtual gateway to attract tenant traffic flows of the corresponding tenant, and to reject traffic flows arriving from sources other than the corresponding tenant. 
     
     
         50 . The method of  claim 49 , comprising configuring the at least one VM to include a set of public IP addresses dedicated to the corresponding tenant, wherein the corresponding tenant accesses the virtual gateway of the VM using the set of public IP addresses. 
     
     
         51 . The method of  claim 47 , comprising configuring the VR to generate at least one set of flow rules configured to control the routing of the tenant traffic flow through the overlay network. 
     
     
         52 . The method of  claim 51 , comprising configuring the at least one set of flow rules to correspond to the at least one objective function. 
     
     
         53 . The method of  claim 51 , comprising configuring the VR and the virtual gateway of each VM to communicate with at least one virtual switch. 
     
     
         54 . The method of  claim 53 , comprising configuring the at least one virtual switch to include a set of routing tables representing the at least one set of flow rules, and configuring the set of routing tables to manage the control of the routing of the tenant traffic flow through the network. 
     
     
         55 . The method of  claim 53 , comprising configuring the at least one virtual switch to transfer the tenant traffic flow between the virtual gateway and the VR. 
     
     
         56 . The method of  claim 53 , comprising configuring each node to include at least one aggregator coupled to the at least one virtual switch and the network. 
     
     
         57 . The method of  claim 56 , comprising configuring the aggregator to route via the network the tenant traffic flows of the plurality of tenants corresponding to the node. 
     
     
         58 . The method of  claim 53 , comprising configuring the at least one virtual switch to communicate with an aggregator. 
     
     
         59 . The method of  claim 58 , comprising configuring the aggregator to route via the network the tenant traffic flow received at the virtual gateway from the corresponding tenant. 
     
     
         60 . The method of  claim 58 , comprising configuring the aggregator to route to the corresponding tenant the tenant traffic flow received at the node via the network. 
     
     
         61 . The method of  claim 60 , comprising configuring at least one of the corresponding VR and the virtual gateway to route to the tenant the tenant traffic flow arriving at the aggregator via the network. 
     
     
         62 . The method of  claim 60 , comprising configuring the virtual gateway to route to the tenant via a coupling over a public network, the tenant traffic flow arriving at the aggregator via the network. 
     
     
         63 . The method of  claim 58 , comprising configuring each node to include a hypervisor, and configuring the hypervisor as an operating system of the at least one VM of the node. 
     
     
         64 . The method of  claim 63 , comprising configuring the hypervisor to include at least one of the aggregator and the at least one virtual switch. 
     
     
         65 . The method of  claim 47 , comprising configuring a provisioner to couple to the at least one VM of the plurality of nodes, and to control provisioning of at least one of the overlay network and the underlay network. 
     
     
         66 . The method of  claim 65 , comprising configuring the provisioner to control configuration of the at least one VM of the plurality of nodes. 
     
     
         67 . The method of  claim 66 , comprising configuring the provisioner to control configuration of components of the at least one VM using a tenant configuration of the corresponding tenant, wherein the components include at least one of the VR and the virtual gateway. 
     
     
         68 . The method of  claim 65 , comprising configuring the provisioner to maintain network data of at least one of the overlay network and the underlay network, wherein the network data includes data representing the overlay network, the underlay network, route configurations, topology data of the network including the plurality of virtual links, and tenant configurations of the plurality of tenants. 
     
     
         69 . The method of  claim 1 , wherein instantiating the at least one VM at each node comprises instantiating a plurality of VMs at each node, and configuring each VM at each node as the tenant VM to correspond to a tenant of the plurality of tenants. 
     
     
         70 . The method of  claim 69 , comprising configuring a tenant set of VMs to include the tenant VM of the corresponding tenant at each node, and configuring the tenant network to include the tenant set of VMs. 
     
     
         71 . The method of  claim 69 , comprising configuring at least one VM of the plurality of VMs at each node to include the VR. 
     
     
         72 . The method of  claim 69 , comprising configuring each VM of the plurality of VMs to include a VR.