Monitoring the quality of software systems

Abstract

A machine may be configured to monitor the quality of software systems based on key performance indicators associated with versions of various units of code. For example, the machine accesses a current version of a unit of code that is not marked as evaluated, in a database. The machine generates a current key performance indicator (KPI) value for the current version of the unit of code based on an execution of the current version of the unit of code. The machine identifies a previous KPI value associated with a previous version of the unit of code that is marked as evaluated, in the database. The machine determines that the current KPI value is less than the previous KPI value and generates an alert for presentation in a user interface of a device. The alert may indicate a decreasing quality level associated with the current version of the unit of code.

What is claimed is: 
     
         1 . A method comprising:
 accessing a current version of a unit of code that is not marked as evaluated, in a database;   generating a current key performance indicator (KPI) value for the current version of the unit of code based on an execution of the current version of the unit of code, the generating of the current KPI being performed by one or more hardware processors;   identifying a previous KPI value associated with a previous version of the unit of code that is marked as evaluated, in the database;   determining that the current KPI value is less than the previous KPI value; and   generating an alert based on the determining that the current KPI value is less than the previous KPI value, for presentation in a user interface of a device, the alert indicating a decreasing quality level associated with the current version of the unit of code.   
     
     
         2 . The method of  claim 1 , wherein the generating of the KPI value for the current version of the unit of code is based on an output resulting from the execution of the current version of the unit of code. 
     
     
         3 . The method of  claim 1 , wherein the current and previous KPI values are statistics metrics of a particular type. 
     
     
         4 . The method of  claim 3 , wherein the current KPI value identifies a current precision value generated based on the output resulting from the execution of the current version of the unit of code and the previous KPI value identifies a previous precision value generated based on the output resulting from the execution of the previous version of the unit of code. 
     
     
         5 . The method of  claim 3 , wherein the current KPI value identifies a current recall value generated based on the output resulting from the execution of the current version of the unit of code and the previous KPI value identifies a previous recall value generated based on the output resulting from the execution of the previous version of the unit of code. 
     
     
         6 . The method of  claim 3 , wherein the current KPI value identifies a current F-measure value generated based on the output resulting from the execution of the current version of the unit of code and the previous KPI value identifies a previous F-measure value generated based on the output resulting from the execution of the previous version of the unit of code. 
     
     
         7 . The method of  claim 1 , further comprising:
 storing the current KPI value in a record of a database, in association with a reference to the current version of the unit of code; and   marking the current version of the unit of code as evaluated, in the database.   
     
     
         8 . The method of  claim 1 , further comprising:
 generating a representation of a plurality of KPI values corresponding to a plurality of versions of the unit of code, the plurality of KPI values including the current and previous KPI values; and   causing a display of the representation of the plurality of KPI values in a user interface of a device.   
     
     
         9 . The method of  claim 8 , wherein the representation comprises a graph including a plurality of nodes, each node of the plurality of nodes representing a KPI value associated with a corresponding version of a plurality of versions of the unit of code. 
     
     
         10 . The method of  claim 1 , further comprising:
 generating a representation of a plurality of version descriptions corresponding to a plurality of versions of the unit of code; and   causing a display of the representation of the plurality of version descriptions in a user interface of a device.   
     
     
         11 . The method of  claim 10 , wherein the representation comprises a graph including a plurality of nodes, one or more nodes of the plurality of nodes including a version identifier that identifies a particular version of the plurality of versions of the unit of code and one or more KPI values generated based on the particular version of the plurality of versions of the unit of code. 
     
     
         12 . The method of  claim 1 , wherein the accessing of the current version of the unit of code that is not marked as evaluated includes:
 checking out a source code of a system monitored for quality, from a source code repository to a local code repository, and   checking out test data used to generate one or more KPIs, including the current KPI, from a data repository to a local data repository; and   the method further comprising:
 identifying one or more commits of versions of the unit of code to the source code repository, that are not marked as evaluated, in the database, the one or more versions of the unit of code including the current version of the unit of code; and 
 performing a build of the one or more commits. 
   
     
     
         13 . The method of  claim 12 , wherein the generating of the current KPI value for the current version of the unit of code includes, upon the performing of the build, executing a unit evaluation program that tests a performance of the system monitored for quality, the executing of the unit evaluation program resulting in the current KPI value. 
     
     
         14 . A system comprising:
 a memory for storing instructions;   a hardware processor, which, when executing instructions, causes the system to:
 access a current version of a unit of code that is not marked as evaluated, in a database; 
 generate a current key performance indicator (KPI) value for the current version of the unit of code based on an execution of the current version of the unit of code; 
 identify a previous KPI value associated with a previous version of the unit of code that is marked as evaluated, in the database; 
 determine that the current KPI value is less than the previous KPI value; and 
 generate an alert based on the determining that the current KPI value is less than the previous KPI value, for presentation in a user interface of a device, the alert indicating a decreasing quality level associated with the current version of the unit of code. 
   
     
     
         15 . The system of  claim 14 , wherein the generating of the KPI value for the current version of the unit of code is based on an output resulting from the execution of the current version of the unit of code. 
     
     
         16 . The system of  claim 14 , wherein the hardware processor further causes the system to:
 generate a representation of a plurality of KPI values, including the current and previous KPI values, each of the plurality of the KPI values being associated with a particular version of the unit of code; and   cause a display of the representation of the plurality of KPIs in a user interface of a device.   
     
     
         17 . The system of  claim 14 , wherein the hardware processor further causes the system to:
 generate a representation of a plurality of versions of the unit of code and associated KPI values, including the current and previous KPI values; and   cause a display of the representation of the plurality of versions of the unit of code and associated KPI values in a user interface of a device.   
     
     
         18 . The system of  claim 14 , wherein the accessing of the current version of the unit of code that is not marked as evaluated includes:
 checking out a source code of a system monitored for quality, from a source code repository to a local code repository, and   checking out test data used to generate one or more KPIs, including the current KPI, from a data repository to a local data repository, and   wherein the hardware processor further causes the system to:
 identify one or more commits of versions of the unit of code to the source code repository, that are not marked as evaluated, the one or more versions of the unit of code including the current version of the unit of code, and 
 perform a build of the one or more commits. 
   
     
     
         19 . The system of  claim 18 , wherein the generating of the current KPI value for the current version of the unit of code includes, upon the performing of the build, executing a unit evaluation program that tests a performance of the system monitored for quality, the executing of the unit evaluation program resulting in the current KPI value. 
     
     
         20 . A non-transitory machine-readable storage medium comprising instructions that, when executed by one or more processors of a machine, cause the machine to perform operations comprising:
 accessing a current version of a unit of code that is not marked as evaluated, in a database;   generating a current key performance indicator (KPI) value for the current version of the unit of code based on an execution of the current version of the unit of code;   identifying a previous KPI value associated with a previous version of the unit of code that is marked as evaluated, in the database;   determining that the current KPI value is less than the previous KPI value; and   generating an alert based on the determining that the current KPI value is less than the previous KPI value, for presentation in a user interface of a device, the alert indicating a decreasing quality level associated with the current version of the unit of code.