Anomaly Analysis For Software Distribution

Abstract

A population of devices provides telemetry data and receives software changes or updates. Event buckets for respective events are found. Event buckets have counts of event instances, where each event instance is an occurrence of a corresponding event reported as telemetry by a device. Records of the software changes are provided, each change record representing a software change on a corresponding device. The event buckets are analyzed to identify which indicate an anomaly. Based on the change records and the identified event buckets, correlations between the software changes and the identified event buckets are found.

1 . A method performed by one or more computing devices, the method comprising:
 accessing a plurality of types of telemetry data sources, each telemetry data source comprising a different type of software performance feedback received from devices;   forming pluralities of data sets from each of the respective telemetry data sources, each data set comprising indicia of counts of software event instances on the devices as a function of time from a corresponding telemetry data source;   analyzing each data set to determine whether the data set comprises an anomaly;   computing correlations of software updates of the devices with the data sets determined to comprise respective anomalies, wherein multiple software updates are applied to the devices during timespans that at least partially overlap with timespans of the data sets; and   automatically controlling distribution, application, or availability of one or more of the software updates based on the one or more of the software updates having been sufficiently correlated with one or more of the data sets determined to comprise anomalies.   
     
     
         2 . A method according to  claim 1 , wherein the controlling comprises sending messages to the devices, the messages identifying a software update. 
     
     
         3 . A method according to  claim 1 , wherein the timing of distribution of a software update to the devices depends on the correlations. 
     
     
         4 . A method according to  claim 1 , further comprising accessing source code data linking a source code file of an update with an element in the pluralities of data sets, and, based thereon, selecting one of the software updates. 
     
     
         5 . A method according to  claim 1 , wherein the automatically controlling comprises sending a message that causes a software update to be sent to at least some of the devices. 
     
     
         6 . A method according to  claim 5 , wherein the software update either installs or uninstalls code corresponding to a source code file determined to link the software update with an anomaly. 
     
     
         7 . A method performed by one or more server computer accessible to user computing devices via a data network, the method comprising:
 receiving from the user computing devices, via the data network, telemetry data reports associated with a software product, each computing device comprising the software product installed therein and a reporting module that builds and transmits the telemetry data reports, and each telemetry data report comprising diagnostic information of the software product;   storing the diagnostic information from the telemetry data reports in a data store;   accessing software update information to identify software updates for the software product that have been applied to the user computing devices;   analyzing the diagnostic information in the data store to identify anomalies associated with the software product;   determining correlations between the anomalies and the identified software updates; and   based on the correlations, communicating with the user computing devices to automatically install and/or uninstall a software update configured to update the software product.   
     
     
         8 . A method according to  claim 7 , wherein the software update comprises one of the identified software updates. 
     
     
         9 . A method according to  claim 7 , wherein the telemetry data reports comprise indicia of events on the user computing devices and respective times thereof, and wherein the correlations are determined according to the times of the events. 
     
     
         10 . A method according to  claim 9 , wherein the correlations are determined further according to frequencies of the events. 
     
     
         11 . A method according to  claim 7 , wherein the telemetry data comprises crash data captured in association with crashes of the software product on the user computing devices 
     
     
         12 . A method according to  claim 7 , further comprising, based on the correlations, inhibiting rollout of a software correlated with an anomaly. 
     
     
         13 . A method according to  claim 7 , further comprising accessing update data identifying which software updates have been applied to which user computing devices and determining the correlations according to the update data. 
     
     
         14 . A method according to  claim 13 , wherein the update data is derived from the telemetry data reports. 
     
     
         15 . A method according to  claim 7 , wherein the software product comprises the reporting module. 
     
     
         16 . A method performed by one or more server devices, the method comprising:
 accessing a data store to obtain telemetry data stored in the data store, the telemetry data associated with a software application, the telemetry data provided by client computing devices via a network, each client computing device comprising the software application installed thereon, the telemetry data comprising diagnostic information generated by execution of the software product on the client computing devices;   based on the diagnostic information, identifying anomalies of the software application and times associated with the anomalies;   based on the times associated with the anomalies, automatically selecting a software update configured to update the software application; and   transmitting a message configured to cause the selected software update to be installed or uninstalled on at least some of the client computing devices.   
     
     
         17 . A method according to  claim 16 , wherein the selected software update comprises a package configured to add, remove, or replace an executable file corresponding to the software application. 
     
     
         18 . A method according to  claim 16 , wherein the telemetry data comprises information derived from stack traces of the software product. 
     
     
         19 . A method according to  claim 16 , further comprising enabling display of a user interface, the user interface comprising a graph representing an identified anomaly. 
     
     
         20 . A method according to  claim 19 , the user interface further comprising a graphic representation of the selected software update.