Data-driven activity prediction

Abstract

A physical environment is equipped with a plurality of sensors (e.g., motion sensors). As individuals perform various activities within the physical environment, sensor readings are received from one or more of the sensors. Based on the sensor readings, activities being performed by the individuals are recognized and the sensor data is labeled based on the recognized activities. Future activity occurrences are predicted based on the labeled sensor data. Activity prompts may be generated and/or facility automation may be performed for one or more future activity occurrences.

What is claimed is: 
     
         1 . A method comprising:
 receiving a set of activity recognition training data, the activity recognition training data including a plurality of sensor events, wherein each sensor event is labeled with a corresponding activity class;   training an activity recognition module based on the activity recognition training data;   receiving sensor events from a smart environment;   using the trained activity recognition module to label each sensor event with an activity class to generate labeled sensor event data;   training an activity occurrence predictor based, at least in part, on the labeled sensor event data; and   using the activity occurrence predictor to predict a future occurrence time of one or more activities within the smart environment.   
     
     
         2 . A method as recited in  claim 1 , wherein the future occurrence time is expressed as a date/time. 
     
     
         3 . A method as recited in  claim 1 , wherein the future occurrence time is expressed as a number of time units until a next occurrence of a particular activity. 
     
     
         4 . A method as recited in  claim 3 , wherein a time unit is a second. 
     
     
         5 . A method as recited in  claim 3 , wherein a time unit is a minute. 
     
     
         6 . A method as recited in  claim 1 , wherein training the activity occurrence predictor includes training a regressor function. 
     
     
         7 . A method as recited in  claim 6 , wherein training the regressor function includes training a respective regressor function for each of a plurality of activities. 
     
     
         8 . A method as recited in  claim 1 , wherein training the activity occurrence predictor includes:
 determining local features of the labeled sensor event data;   determining context features associated with the labeled sensor event data;   determining a loss function associated with the labeled sensor event data;   using a multi-output regression learner to train the activity occurrence predictor based on the local features and the context features to minimize the loss function.   
     
     
         9 . A method as recited in  claim 8 , wherein the local features include sensor event data labeled with recognized activity classes. 
     
     
         10 . A method as recited in  claim 8 , wherein the context features include data based on previous activity occurrence predictions. 
     
     
         11 . A method as recited in  claim 8 , wherein the loss function is expressed as a root mean squared error. 
     
     
         12 . A method as recited in  claim 1 , further comprising, performing facility automation within the smart environment based, at least in part, on a predicted future occurrence time of a particular activity. 
     
     
         13 . A method comprising:
 determining an activity of interest;   querying an activity prediction server for a predicted future time associated with the activity of interest;   receiving, from the server, the predicted future time associated with the activity of interest;   comparing the predicted future time to a current time; and   when the current time is equal to or greater than the predicted future time, presenting an activity prompt associated with the activity of interest.   
     
     
         14 . A method as recited in  claim 13 , wherein querying the activity prediction server includes periodically querying the activity prediction server. 
     
     
         15 . A method as recited in  claim 13 , wherein determining the activity of interest includes one or more of:
 determining an activity specified in association with user-configured settings;   determining an activity for which no activity prompts are currently pending; or   determining an activity for which an activity prompt is currently pending and is schedule to be presented within a threshold time period.   
     
     
         16 . A personal computing device configured to perform the method as recited in  claim 13 . 
     
     
         17 . A mobile phone configured to perform the method as recited in  claim 13 . 
     
     
         18 . A personal computing device comprising:
 a processor;   a memory communicatively coupled to the processor;   an activity prompting application stored in the memory and executed on the processor, to configure the personal computing device to perform the method as recited in  claim 13 .   
     
     
         19 . An activity prediction server system comprising:
 a processor;   a memory communicatively connected to the processor;   a sensor event data store stored in the memory;   an activity recognition module stored in the memory and executed on the processor, the activity recognition module configured to:
 receive sensor event data from sensors within a smart environment; 
 label a received sensor event as being triggered during a recognized activity; and 
 store the labeled sensor event data in the sensor event data store; and 
   an activity prediction module stored in the memory and executed on the processor, the activity prediction module configured to analyze the labeled sensor event data in the sensor event data store to predict a future time of occurrence of a particular activity within the smart environment.