Measuring the Impact of Network Deployments

Abstract

In one embodiment, a computing system may receive a request for a measurement project of a communication network for a geographic region of interest. The computing system may identify network metrics for control regions. The control regions may comprise geographic regions connected by the communication network. The network metrics may comprise features that measure a change in a value of a corresponding network metric during a particular time period. The computing system may determine, based on a comparison between a first control region and a second control region, whether a first network metric has a change greater than a pre-determined threshold after a first network deployment. The computing system may send, responsive to the request for the measurement project, instructions for presenting a measurement result summary for a second network deployment based on the determined change after the first network deployment is greater than the pre-determined threshold.

What is claimed is: 
     
         1 . A method comprising, by a computing system:
 receiving, from a client system, a request for a measurement project of a communication network for a geographic region of interest;   identifying network metrics for control regions, wherein the control regions comprise geographic regions connected by the communication network, wherein the network metrics comprises features that measure a change in a value of a corresponding network metric during a particular time period;   determining, based on a comparison between a first control region and a second control region, whether a first network metric has a change greater than a pre-determined threshold after a first network deployment; and   sending, to the client system, responsive to the request for the measurement project, instructions for presenting a measurement result summary for a second network deployment based on the determined change after the first network deployment is greater than the pre-determined threshold.   
     
     
         2 . The method of  claim 1 , wherein the network metrics are determined based on network performance data collected during a first time period of the particular time period before the first network deployment and a second time period of the particular time period after the first network deployment. 
     
     
         3 . The method of  claim 2 , wherein the features shared by a first group of geographic regions is determined based on the network performance data collected during the first time period before the first network deployment. 
     
     
         4 . The method of  claim 3 , wherein a second group of geographic regions and the geographic region of interest have a similarity being above a similarity threshold, and wherein the similarity is determined based on the features associated with the geographic region of interest and that geographic region before the first network deployment. 
     
     
         5 . The method of  claim 2 , wherein a first network metric baseline is determined based on a weighted combination of the control regions and is determined based on a plurality of control regions. 
     
     
         6 . The method of  claim 5 , wherein a second group of geographic regions are identified by a machine-learning model, and wherein the weighted combination of the control regions are used by a synthetic controls method. 
     
     
         7 . The method of  claim 5 , wherein the first network baseline is determined based on the network performance data collected during the second time period after the first network deployment. 
     
     
         8 . The method of  claim 1 , further comprising:
 identifying a first group of geographic regions based on the features of the network metrics shared by two or more geographic regions,   identifying a second group of geographic regions from the first group of geographic regions based on a comparison between the region of interest and each of the first group of geographic regions,   wherein the second group of geographic regions are hold-out regions excluded from being impacted by the first network deployment, wherein the control regions are randomly selected from the second group of geographic regions, and wherein a first network metric baseline is determined based on the selected one or more controls regions.   
     
     
         9 . The method of  claim 1 , further comprising:
 comparing a second network metric of the geographic region of interest after the network deployment to a network metric baseline; and   determining whether the second network metric of the geographic region of interest has a change greater than a second pre-determined threshold after the first network deployment.   
     
     
         10 . One or more computer-readable non-transitory storage media embodying software that is operable when executed to:
 receive, from a client system, a request for a measurement project of a communication network for a geographic region of interest;   identify network metrics for control regions, wherein the control regions comprise geographic regions connected by the communication network, wherein the network metrics comprises features that measure a change in a value of a corresponding network metric during a particular time period;   determine, based on a comparison between a first control region and a second control region, whether a first network metric has a change greater than a pre-determined threshold after a first network deployment; and   send, to the client system, responsive to the request for the measurement project, instructions for presenting a measurement result summary for a second network deployment based on the determined change after the first network deployment is greater than the pre-determined threshold.   
     
     
         11 . The media of  claim 10 , wherein the network metrics are determined based on network performance data collected during a first time period of the particular time period before the first network deployment and a second time period of the particular time period after the first network deployment. 
     
     
         12 . The media of  claim 11 , wherein the features shared by a first group of geographic regions is determined based on the network performance data collected during the first time period before the first network deployment. 
     
     
         13 . The media of  claim 12 , wherein a second group of geographic regions and the geographic region of interest have a similarity being above a similarity threshold, and wherein the similarity is determined based on the features associated with the geographic region of interest and that geographic region before the first network deployment. 
     
     
         14 . The media of  claim 11 , wherein a first network metric baseline is determined based on a weighted combination of the control regions and is determined based on a plurality of control regions. 
     
     
         15 . The media of  claim 14 , wherein a second group of geographic regions are identified by a machine-learning model, and wherein the weighted combination of the control regions are used by a synthetic controls method. 
     
     
         16 . A system comprising: one or more non-transitory computer-readable storage media embodying instructions; and one or more processors coupled to the storage media and operable to execute the instructions to:
 receive, from a client system, a request for a measurement project of a communication network for a geographic region of interest;   identify network metrics for control regions, wherein the control regions comprise geographic regions connected by the communication network, wherein the network metrics comprises features that measure a change in a value of a corresponding network metric during a particular time period;   determine, based on a comparison between a first control region and a second control region, whether a first network metric has a change greater than a pre-determined threshold after a first network deployment; and   send, to the client system, responsive to the request for the measurement project, instructions for presenting a measurement result summary for a second network deployment based on the determined change after the first network deployment is greater than the pre-determined threshold.   
     
     
         17 . The system of  claim 16 , wherein the network metrics are determined based on network performance data collected during a first time period of the particular time period before the first network deployment and a second time period of the particular time period after the first network deployment. 
     
     
         18 . The system of  claim 17 , wherein the features shared by a first group of geographic regions is determined based on the network performance data collected during the first time period before the first network deployment. 
     
     
         19 . The system of  claim 18 , wherein a second group of geographic regions and the geographic region of interest have a similarity being above a similarity threshold, and wherein the similarity is determined based on the features associated with the geographic region of interest and that geographic region before the first network deployment. 
     
     
         20 . The system of  claim 17 , wherein a first network metric baseline is determined based on a weighted combination of the control regions and is determined based on a plurality of control regions.