Resource management in a big data environment

Abstract

A method and system to automatically analyze, diagnose and repair the state of a network utilizing big data and machine learning techniques. Data from disparate sources related to a first network element is received by a processing layer. Contextual information from a measurements megastore related to the first network element and other network elements is retrieved. The data from the disparate sources and the contextual information is analyzed by an intelligence layer comprising big data and machine learning techniques. Upon determining, by the intelligence layer, that a predetermined condition is met or a predetermined threshold is exceeded, a notification is provided to the first network element. Over time, the intelligence layer adapts to learn based on growing amounts of historical data.

What is claimed is: 
     
         1 . A computer system comprising:
 a processor;   a network interface coupled to the processor configured to enable communications via a communication network;   a storage device for content and programming; and   a program stored in the storage device having a data processing layer and an intelligence layer, wherein execution of the program by the processor configures the computer system to perform acts comprising:
 receiving data from disparate sources related to a first network element by the processing layer; 
 retrieving a contextual information from a measurements megastore related to the first network element and other network elements; 
 analyzing the data from disparate sources and the contextual information by the intelligence layer; and 
 upon determining, by the intelligence layer, that a predetermined condition is met or a predetermined threshold is exceeded, providing a notification to the first network element. 
   
     
     
         2 . The computer system of  claim 1 , wherein the storage device further includes an automation layer and the notification is provided by the automation layer. 
     
     
         3 . The computer system of  claim 1 , wherein the notification is provided to one or more other network elements. 
     
     
         4 . The computer system of  claim 1 , wherein execution of the program further configures the computer system to perform acts comprising:
 upon determining that a group of other network elements in an area of the first network element meet the predetermined condition or exceed the predetermined threshold, automatically sending the notification to the network elements to the group of other network elements.   
     
     
         5 . The computer system of  claim 1 , wherein the contextual information includes at least one of: (i) a type of a data plan of a subscriber of the network, and (ii) a type of first network element. 
     
     
         6 . The computer system of  claim 1 , wherein the data from the disparate sources includes at least one of: (i) a subscriber radio trace, (ii) an operations subsystem (OSS), (iii) a customer care record, (iv) a billing information, and (iv) an application performance monitoring report. 
     
     
         7 . The computer system of  claim 1 , wherein at least some of the disparate sources provide data to the processing layer, which is based on the local knowledge of the network of the at least some disparate sources, respectively. 
     
     
         8 . The computer system of  claim 1 , wherein the analyzing by the intelligence layer includes determining patters or trends across the disparate data from the disparate sources and the contextual information. 
     
     
         9 . The computer system of  claim 1 , wherein the disparate sources comprise network elements from one or more radio access networks. 
     
     
         10 . The computer system of  claim 1 :
 wherein the analyzing by the intelligence layer includes considering a network element subscriber's predetermined preferences, limitations, plan, and type of network element, and   wherein the network element is a mobile device that is subscribed to be used on the network.   
     
     
         11 . The computer system of  claim 1 , wherein execution of the program further configures the computer system to perform acts comprising:
 making the contextual information available to the first network element and the other network elements before analyzing the information.   
     
     
         12 . The computer system of  claim 10 , wherein the contextual information includes general information on subscribers and network nodes. 
     
     
         13 . The computer system of  claim 1 , wherein the contextual information is provided to the first network element via a query performed by the first network element to the measurements megastore via an open application program interface (API). 
     
     
         14 . The computer system of  claim 1 , wherein the notification includes information that is operative to adjust a performance of the first network element. 
     
     
         15 . The computer system of  claim 1 , wherein:
 the network element is a resource management module; and   the notification is operative to adjust at least one of: (i) a scheduler, (ii) a link adaptation, and (iii) a flow control of the resource management module.   
     
     
         16 . The computer system of  claim 1 , wherein the notification is operative to display on a user interface of the first network element that the first network element needs service. 
     
     
         17 . The computer system of  claim 1 , wherein the contextual information is used to adjust at least one of: (i) an admission control, (ii) a scheduler, (iii) a link adaptation, and (iv) a power control procedure of the first network element. 
     
     
         18 . The computer system of  claim 1 , wherein the notification is provided to a network administrator in real time. 
     
     
         19 . The computer system of  claim 1 , wherein the data processing of the data processing layer is performed in real time. 
     
     
         20 . The computer system of  claim 1 , wherein:
 the notification is operative to place a restriction on one or more network elements; and   the restriction is lifted after a threshold time has elapsed or upon a new notification operative to remove the restriction is sent to the one or more network elements.   
     
     
         21 . The computer system of  claim 1 , wherein the measurements megastore uses massive parallel processing technology of at least one of: (i) Hadoop, (ii) Storm, and (iii) Spark. 
     
     
         22 . The computer system of  claim 21 , wherein execution of the program further configures the computer system to perform acts comprising:
 machine learning via one or more clustering models performed on a pre-determined training set by the intelligence layer and operative to identify patterns and trends in the data from the disparate sources and the contextual information.   
     
     
         23 . A network element comprising:
 a processor;   a network interface coupled to the processor configured to enable communications via a communication network;   a storage device for content and programming; and   a program stored in the storage device having a data processing layer and an intelligence layer, wherein execution of the program by the processor configures the network element to perform acts comprising:
 determining local radio resources on the network; 
 receiving contextual information from a measurements megastore, wherein the contextual information is related to radio resources of other network elements on the network; and 
 adjusting a resource allocation based on the local radio resources and the received contextual information. 
   
     
     
         24 . The network element of  claim 23 , wherein the measurements megastore uses massive parallel processing technology of at least one of: (i) Hadoop, (ii) Storm, and (iii) Spark. 
     
     
         25 . The computer system of  claim 23 , wherein execution of the program further configures the network element to perform acts comprising:
 receiving one or more notifications from a data framework associated with the measure measurements megastore, wherein the one or more notifications are based on an analysis of the contextual information for patterns and trends; and   assigning one or more resources of the network element based on the notification.   
     
     
         26 . The computer system of  claim 23 , wherein execution of the program further configures the network element to perform acts comprising:
 providing the determined local radio resources to a measurements megastore via the network.   
     
     
         27 . A monitoring server configured to determine a root cause of a communication network error, the server comprising:
 a processor;   a network interface coupled to the processor configured to enable communications via the communication network;   a storage device for content and programming; and   a program stored in the storage device having, wherein execution of the program by the processor configures the monitoring server to perform acts comprising:
 collecting disparate data from disparate sources into a common data framework; 
 receiving key metrics to be analyzed from a selection performed on a user interface of a network element; 
 analyzing the disparate data from the disparate sources via one or more clustering algorithms operative to identify at least one of patterns and trends; 
 providing at least one of the identified patterns and trends to the user interface of the network element; 
 receiving ratings on at least one of the identified patterns and trends from the user interface of the network element; 
 automatically sending a notification and key performance indicator (KPI); and 
 saving the KPI in a measurements megastore. 
   
     
     
         28 . A health monitoring server configured to monitor a health of a subscriber, the server comprising:
 a processor;   a network interface coupled to the processor configured to enable communications via a communication network;   a storage device for content and programming; and   a program stored in the storage device having a data processing layer and an intelligence layer, wherein execution of the program by the processor configures the computer system to perform acts comprising:
 receiving disparate data related to the subscriber's health from disparate sources; 
 retrieving a contextual information from a measurements megastore related to the health of the subscriber; 
 analyzing the information from the disparate sources and the measurements megastore for at least one of patterns and trends; and 
 upon determining that one or more thresholds are exceeded or one or more criteria are met, generating a notification. 
   
     
     
         29 . The health monitoring server of  claim 28 , wherein the notification is sent to at least one of the disparate sources of the disparate data. 
     
     
         30 . The health monitoring server of  claim 28 , wherein the disparate data from the disparate sources includes at least one of: a subscriber radio trace, healthcare records from one or more sources, customer care records, billing, and application performance monitoring reports. 
     
     
         31 . The health monitoring server of  claim 28 , wherein the sources of the data include one or more implanted or worn personal health monitors of the subscriber, and records from a server storing personal health information of the subscriber. 
     
     
         32 . The health monitoring server of  claim 28 , wherein the disparate data is first received from one or more application servers and collection systems over one or more networks before being received by the remote health monitoring server. 
     
     
         33 . The health monitoring server of  claim 28 , wherein the disparate data is received at least in part from one or more health monitors configured to measure at least one of: heart rate, blood pressure, motion, oxygen saturation, temperature, and glucose level. 
     
     
         34 . The health monitoring server of  claim 28 , wherein the health monitoring server is configured to provide an open platform wherein the disparate data from the disparate sources are combined and processed in real time. 
     
     
         35 . The computer system of  claim 28 , wherein execution of the program further configures the monitoring server to perform acts comprising:
 determining a location information of the subscriber;   identifying any network issues that may prevent data transmission or connectivity; and   upon determining that there is a network issue, preventing the determination of whether one or more thresholds are exceeded or one or more criteria are met to prevent a false alarm.   
     
     
         36 . The computer system of  claim 28 , wherein the notification is sent to at least one of: the subscriber, a care provided, a contact person previously stored in a database of the measurements megastore, and an emergency service.