Application launch support

Abstract

A method of software launch regression testing comprises monitoring a user engagement parameter of an existing application running on a plurality of client devices and determining a probability interval from the user engagement parameter of the existing application. A candidate update application is then launched to a subset of the plurality of client devices. The method then proceeds with monitoring a corresponding user engagement parameter of the candidate update version running on the subset of client devices, determining if the corresponding user engagement parameter of the candidate update version falls within the probability interval, and, based on the probability interval falling within the probability interval, providing a testing pass notification.

What is claimed is: 
     
         1 . A method of software launch regression testing, performed by one or more processors, comprising:
 determining a current value of a user engagement parameter of an existing application from data received from a plurality of client devices running the existing application;   determining a probability interval for the user engagement parameter of the existing application from the data received from the plurality of client devices running the existing application;   launching a candidate update application to client devices comprising a subset of the plurality of client devices;   determining a value of the user engagement parameter using data received from of the subset of the plurality of client devices running the candidate update application;   determining if the value of the user engagement parameter of the candidate update application falls within the probability interval; and   based on the value of the user engagement parameter falling within the probability interval, providing a testing pass notification.   
     
     
         2 . The method of  claim 1 , wherein a size of the probability interval depends on whether or not the user engagement parameter is a critical parameter. 
     
     
         3 . The method of  claim 1 , wherein the probability interval comprises a confidence interval within which correct functioning of the candidate update application is assumed, the method further comprising:
 determining a prediction interval from the data received from the plurality of client devices running the existing application, the prediction interval comprising an interval of values of the user engagement parameter within which the candidate update application may or may not be functioning correctly;   determining if the value of the value of the user engagement parameter of the candidate update application falls within the prediction interval; and   based on the value of the value of the user engagement parameter falling outside the confidence interval but within the prediction interval, providing a testing review notification.   
     
     
         4 . The method of  claim 3 , further comprising:
 based on the value of the user engagement parameter falling outside the prediction interval, providing a testing fail notification.   
     
     
         5 . The method of  claim 1 , wherein launching the candidate update application comprises:
 uploading the candidate update application to an app store,   wherein a number of available downloads of the candidate update application from the app store is limited to a predetermined number.   
     
     
         6 . The method of  claim 1 , wherein launching the candidate update application comprises:
 limiting launch of the candidate update application to client devices having a particular characteristic.   
     
     
         7 . The method of  claim 6 , wherein the particular characteristic is location of the client devices within a particular geographical area or time zone. 
     
     
         8 . The method of  claim 1 , wherein the user engagement parameter is a ratio of transmissions of content from the subset of the plurality of client devices to successful viewing of the content on recipient client devices. 
     
     
         9 . A non-transitory computer-readable storage medium, the computer-readable storage medium including instructions that when executed by a computer, cause the computer to perform operations for software launch regression testing, comprising:
 determining a current value of a user engagement parameter of an existing application from data received from a plurality of client devices running the existing application;   determining a probability interval for the user engagement parameter of the existing application from the data received from the plurality of client devices running the existing application;   launching a candidate update application to client devices comprising a subset of the plurality of client devices;   determining a value of the user engagement parameter using data received from of the subset of the plurality of client devices running the candidate update application;   determining if the value of the user engagement parameter of the candidate update application falls within the probability interval; and   based on the value of the user engagement parameter falling within the probability interval, providing a testing pass notification.   
     
     
         10 . The computer-readable storage medium of  claim 9 , wherein a size of the probability interval depends on whether or not the user engagement parameter is a critical parameter. 
     
     
         11 . The computer-readable storage medium of  claim 9 , wherein the probability interval comprises a confidence interval within which correct functioning of the candidate update application is assumed, wherein the instructions further configure the computer to perform operations further comprising:
 determining a prediction interval from the data received from the plurality of client devices running the existing application, the prediction interval comprising an interval of values of the user engagement parameter within which the candidate update application may or may not be functioning correctly;   determining if the value of the value of the user engagement parameter of the candidate update application falls within the prediction interval; and   based on the value of the value of the user engagement parameter falling outside the confidence interval but within the prediction interval, providing a testing review notification.   
     
     
         12 . The computer-readable storage medium of  claim 11 , wherein the instructions further configure the computer to perform operations further comprising:
 based on the value of the user engagement parameter falling outside the prediction interval, providing a testing fail notification.   
     
     
         13 . The computer-readable storage medium of  claim 9 , wherein launching the candidate update application comprises:
 uploading the candidate update application to an app store,   wherein a number of available downloads of the candidate update application from the app store is limited to a predetermined number.   
     
     
         14 . The computer-readable storage medium of  claim 9 , wherein the user engagement parameter is a ratio of transmissions of content from the subset of the plurality of client devices to successful viewing of the content on recipient client devices. 
     
     
         15 . A computing apparatus comprising:
 a processor; and   a memory storing instructions that, when executed by the processor, configure the apparatus to perform operations comprising:   determining a current value of a user engagement parameter of an existing application from data received from a plurality of client devices running the existing application;   determining a probability interval for the user engagement parameter of the existing application from the data received from the plurality of client devices running the existing application;   launching a candidate update application to client devices comprising a subset of the plurality of client devices;   determining a value of the user engagement parameter using data received from of the subset of the plurality of client devices running the candidate update application;   determining if the value of the user engagement parameter of the candidate update application falls within the probability interval; and   based on the value of the user engagement parameter falling within the probability interval, providing a testing pass notification.   
     
     
         16 . The computing apparatus of  claim 15 , wherein a size of the probability interval depends on whether or not the user engagement parameter is a critical parameter. 
     
     
         17 . The computing apparatus of  claim 15 , wherein the probability interval comprises a confidence interval within which correct functioning of the candidate update application is assumed, wherein the operations further comprise:
 determining a prediction interval from the data received from the plurality of client devices running the existing application, the prediction interval comprising an interval of values of the user engagement parameter within which the candidate update application may or may not be functioning correctly;   determining if the value of the value of the user engagement parameter of the candidate update application falls within the prediction interval; and   based on the value of the value of the user engagement parameter falling outside the confidence interval but within the prediction interval, providing a testing review notification.   
     
     
         18 . The computing apparatus of  claim 17 , wherein the operations further comprise:
 based on the value of the user engagement parameter falling outside the prediction interval, providing a testing fail notification.   
     
     
         19 . The computing apparatus of  claim 15 , wherein launching of the candidate update application comprises:
 uploading the candidate update application to an app store,   wherein a number of available downloads of the candidate update application from the app store is limited to a predetermined number.   
     
     
         20 . The computing apparatus of  claim 15 , wherein the user engagement parameter is a ratio of transmissions of content from the subset of the plurality of client devices to successful viewing of the content on recipient client devices.