Interacting with a computer based application

Abstract

A computing system runs an application (e.g., video game) that interacts with one or more actively engaged users. One or more physical properties of a group are sensed. The group may include the one or more actively engaged users and/or one or more entities not actively engaged with the application. The computing system will determine that the group (or the one or more entities not actively engaged with the application) have performed a predetermined action. A runtime condition of the application is changed in response to determining that the group (or the one or more entities not actively engaged with the computer based application) have performed the predetermined action. Examples of changing a runtime condition include moving an object, changing a score or changing an environmental condition of a video game.

1 . A method for interacting with a computer based application, comprising:
 performing the computer based application including interacting with one or more actively engaged users;   automatically sensing one or more physical properties of one or more entities not actively engaged with the computer based application;   determining that the one or more entities not actively engaged with the computer based application have performed a predetermined action;   automatically changing a runtime condition of the computer based application in response to determining that one or more entities not actively engaged with the computer based application have performed the predetermined action; and   automatically reporting the changing of the runtime condition in a user interface of the computer based application.   
     
     
         2 . The method of  claim 1 , wherein:
 the automatically sensing one or more physical properties includes sensing a depth image;   the predetermined action is a gesture; and   the determining that the one or more entities not actively engaged with the computer based application have performed the predetermined action includes using the depth image to identify the gesture.   
     
     
         3 . The method of  claim 1 , wherein:
 the automatically sensing one or more physical properties includes sensing a depth image and sensing a visual image;   the predetermined action is a gesture; and   the determining that the one or more entities not actively engaged with the computer based application have performed the predetermined action includes using the depth image and the visual image to identify the gesture.   
     
     
         4 . The method of  claim 1 , wherein:
 the automatically sensing one or more physical properties includes sensing one or more sounds; and   the determining that the one or more entities not actively engaged with the computer based application have performed the predetermined action includes using the one or more sounds to determine that a predetermined sound event has occurred.   
     
     
         5 . The method of  claim 1 , wherein:
 the computer based application is a video game; and   the changing a runtime condition of the computer based application includes changing an appearance of an item in the video game.   
     
     
         6 . The method of  claim 1 , wherein:
 the computer based application is a video game; and   the changing a runtime condition of the computer based application includes changing the score of the video game.   
     
     
         7 . The method of  claim 1 , further comprising:
 detecting an emotion of the one or more entities not actively engaged with the computer based application using a sensor that also senses information about the one or more actively engaged users, the automatically changing a runtime condition is performed in response to detecting the emotion.   
     
     
         8 . The method of  claim 1 , wherein:
 the determining that the one or more entities not actively engaged with the computer based application have performed a predetermined action comprises identifying an action performed as a group.   
     
     
         9 . The method of  claim 1 , wherein:
 the sensing one or more physical properties of one or more entities not actively engaged with the computer based application includes sensing motion of a group;   the determining that the one or more entities not actively engaged with the computer based application have performed a predetermined action comprises identifying an action of an individual of the group; and   the changing the runtime condition is performed based on the action of the individual.   
     
     
         10 . One or more processor readable storage devices having processor readable code embodied on the one or more processor readable storage devices, the processor readable code for programming one or more processors to perform a method comprising:
 performing a video game including interacting with one or more users who are bound to the video game;   receiving information from a first sensor about moving objects, the moving objects include the one or more bound users and one or more persons who are not bound to the video game;   automatically determining and characterizing movement of the moving objects; and   automatically changing the computer based video game in response to movement of the one or more bound users and one or more persons who are not bound to the video game.   
     
     
         11 . The one or more processor readable storage devices of  claim 10 , wherein:
 the receiving information from a first sensor includes receiving a depth image;   the automatically determining and characterizing movement includes recognizing a gesture using the depth image.   
     
     
         12 . The one or more processor readable storage devices of  claim 10 , wherein the method further comprises:
 receiving sound information from a second sensor, the sound information is from the one or more persons who are not bound to the computer based video game, the changing the video game in performed partially in response to the sound information.   
     
     
         13 . The one or more processor readable storage devices of  claim 10 , wherein:
 the changing the video game includes moving an item in the video game in response to motion of the one or more persons who are not bound to the video game.   
     
     
         14 . The one or more processor readable storage devices of  claim 10 , wherein:
 the determining and characterizing movement of the moving objects comprises identifying an action performed as a group.   
     
     
         15 . The one or more processor readable storage devices of  claim 10 , wherein:
 the receiving information from a first sensor about moving objects includes sensing motion of a group;   the determining and characterizing movement of the moving objects includes identifying an action of an individual of the group;   the changing the video game is performed based on the action of the individual.   
     
     
         16 . A computing system, comprising:
 a camera; and   a computer connected to the camera, the computer includes:
 a tracking engine that receives data from the camera and tracks one or more moving objects, the tracking engine provides output information indicative of tracking of the one or more moving objects, 
 a software application in communication with the tracking engine, the software application interacts with the one or more actively engaged users based on output information from the tracking engine, 
 a plurality of filters, each filter of the plurality receives input data about movement perceptible by the camera, and each filter of the plurality determines and outputs to the software application whether one or more entities not actively engaged with the software application have performed a predetermined action, the software application makes a change to a runtime condition reported in a user interface of the software application in response to the filters indicating that one or more entities not actively engaged with the software application have performed a predetermined action, and 
 a recognizer engine that receives data from the camera and output information from the tracking engine and selectively provides the data from the camera and output information from the tracking engine to one or more of the filters as input data for the respective one or more filters. 
   
     
     
         17 . The computing system of  claim 16 , wherein:
 the camera includes a depth sensor; and   the data from the camera includes a depth image.   
     
     
         18 . The computing system of  claim 17 , wherein:
 the software application is a video game; and   the change to the runtime condition is a movement of an object in response to movement of one or more entities not actively engaged with the video game.   
     
     
         19 . The computing system of  claim 16 , wherein:
 the software application is a video game; and   each filter of the plurality determines and outputs to the video game whether one or more entities not actively engaged with the software application have performed a different gesture.   
     
     
         20 . The computing system of  claim 16 , wherein:
 the software application is a video game; and   one particular filter of the plurality determines and outputs to the video game whether a group has performed a predetermined motion as an aggregate, the group including that entities not actively engaged with the software application and the entities that are actively engaged with the software application.