Glitch detection system

Abstract

The present disclosure provides a system for automating graphical testing during video game development. The system can use Deep Convolutional Neural Networks (DCNNs) to create a model to detect graphical glitches in video games. The system can use an image, a video game frame, as input to be classified into one of defined number of classifications. The classifications can include a normal image and one of a plurality of different kinds of glitches. In some embodiments, the glitches can include corrupted textures, including low resolution textures and stretched textures, missing textures, and placeholder textures. The system can apply a confidence measure to the analysis to help reduce the number of false positives.

What is claimed is: 
     
       1. A computer-implemented method comprising:
 executing a game application in a test mode; 
 receiving a path for a virtual camera within a virtual environment, the path having a start point and an end point, wherein the virtual environment is rendered within the test mode by a game engine, wherein the virtual camera follows the path from the start point to the end point within the virtual environment; 
 during runtime of the test mode, acquiring frames rendered by the game engine, wherein the frames are rendered based on a viewpoint of the virtual camera within the virtual environment, wherein the virtual environment includes a plurality of virtual objects; 
 analyzing the frames using a machine learning model, wherein the analysis by the machine learning model comprises:
 for individual virtual objects of the plurality of virtual objects, 
 
 identifying a virtual object in a plurality of frames generated from a plurality of viewpoints along the path, wherein the plurality of frames capture the virtual object from a plurality of different viewpoints; 
 analyzing a rendered appearance of the virtual object from the plurality of different viewpoints captured within the plurality of frames; 
 determining a classification of the virtual object based on the analysis, wherein the classification is one of a plurality of classifications of graphical glitches, wherein each classification corresponds to different types of graphical errors of the rendered appearance of virtual objects within the virtual environment; 
 determining a confidence score associated with the classification of the virtual object based on the analysis; 
 determining whether the confidence score satisfies a confidence threshold associated with the classification; 
 in response to a determination that the confidence score associated with the classification of the virtual object satisfies the confidence threshold, outputting the classification of the virtual object; and 
 in response to a determination that the confidence score does not satisfy the confidence threshold, adjusting the path to create an intermediate path prior to the end point, and moving the virtual camera along the intermediate path to acquire frames from additional viewpoints of the virtual object. 
 
     
     
       2. The method of  claim 1 , wherein if the confidence score does not satisfy the confidence threshold, the method further comprises, determining a classification based on an analysis including the additional frames. 
     
     
       3. The method of  claim 1 , wherein if the confidence score does not satisfy the confidence threshold, the method further comprises adjusting rendering parameters of the virtual environment and acquiring additional frames including the virtual object, and determining a classification based on an analysis including the additional frames. 
     
     
       4. The method of  claim 3 , wherein adjusting the rendering parameters includes changing lighting conditions of the virtual environment. 
     
     
       5. The method of  claim 1 , wherein the path through the virtual environment is programmatically defined. 
     
     
       6. The method of  claim 1 , wherein the virtual environment is a three dimensional virtual environment. 
     
     
       7. The method of  claim 1 , wherein the classifications of graphical glitches include corrupted textures, stretched textures, low resolution textures, missing textures, and place holder textures. 
     
     
       8. The method of  claim 1  further comprising generating a bug report based on the identified graphical glitches in the virtual environment. 
     
     
       9. The method of  claim 8  further comprising identifying a frame including the virtual object classified as a graphical glitch and a timestamp of the frame. 
     
     
       10. The method of  claim 9  further comprising identifying a portion of the frame including the virtual object. 
     
     
       11. A system comprising:
 a data store storing a machine learning model; and 
 at least one hardware processor configured with computer executable instructions that configure the at least one hardware processor to:
 execute a game application in a test mode; 
 
 receive a path for a virtual camera within a virtual environment, the path having a start point and an end point, wherein the virtual environment is rendered within the test mode by a game engine, wherein the virtual camera follows the path from the start point to the end point within the virtual environment, wherein the virtual environment includes a plurality of virtual objects; 
 acquire frames rendered by the game engine, wherein the frames are rendered based on a viewpoint of the virtual camera within the virtual environment; 
 analyze the frames using a machine learning model, wherein the analysis by the machine learning model comprises: 
 for individual virtual objects of the plurality of virtual objects, 
 identify a virtual object in a plurality of frames generated from a plurality of viewpoints along the path, wherein the plurality of frames capture the virtual object from a plurality of different viewpoints; 
 analyze a rendered appearance of the virtual object from the plurality of different viewpoints captured within the plurality of frames; 
 determine a classification of the virtual object based on the analysis, wherein the classification is one of a plurality of classifications of graphical glitches, wherein each classification corresponds to different types of graphical errors of the rendered appearance of virtual objects within the virtual environment; 
 determine a confidence score associated with the classification of the virtual object based on the analysis; 
 determine whether the confidence score satisfies a confidence threshold associated with the classification; 
 in response to a determination that the confidence score associated with the classification of the virtual object satisfies a confidence threshold, output the classification of the virtual object; and 
 in response to a determination that the confidence score does not satisfy the confidence threshold, adjust the path to create an intermediate path prior to the end point, and move the virtual camera along the intermediate path to acquire frames from additional viewpoints of the virtual object. 
 
     
     
       12. The system of  claim 9 , wherein if the confidence score does not satisfy the confidence threshold, the computer executable instructions further configure the at least one hardware processor to determine a classification based on an analysis including the additional frames. 
     
     
       13. The system of  claim 11 , wherein if the confidence score does not satisfy the confidence threshold, the computer executable instructions further configure the at least one hardware processor to adjust the path of the virtual camera to adjust rendering parameters of the virtual environment to acquire additional frames including the virtual object, and determine a classification based on an analysis including the additional frames. 
     
     
       14. The system of  claim 13 , wherein the adjustment of the rendering parameters includes changing lighting conditions of the virtual environment. 
     
     
       15. The system of  claim 11 , wherein the classifications of graphical glitches include corrupted textures, stretched textures, low resolution textures, missing textures, and place holder textures. 
     
     
       16. The system of  claim 11 , wherein the computer executable instructions further configure the at least one hardware processor to:
 generate a bug report based on the identified graphical glitches in the virtual environment; 
 identify a frame including the virtual object classified as a graphical glitch and a timestamp of the frame; and 
 identify a portion of the frame including the virtual object.