Devices, Systems, and Methods that Observe and Classify Real-World Activity Relating to an Observed Object, and Track and Disseminate State Relating the Observed Object

Abstract

An intelligent monitoring device including a processor and an accelerometer and/or a device that includes at least one processor and at least one image sensor. The intelligent monitoring device is configured to observe at least one machine. The intelligent monitoring device is further configured to utilize its processor, or the processor of a coupled system, to recognize actions carried out on or by the at least one machine and infer the state of the machine. The intelligent monitoring device or the coupled system is further configured to provide alerts or help respond to queries about the status of the at least one machine. For example, the intelligent monitoring camera will infer the state of a washing machine based on its observations and provide an alert (optionally only to the user recognized to have loaded the washer) when the washing machine is ready to be unloaded.

I claim: 
     
         1 . A device comprising:
 a) a housing including a magnetic or adhesive portion adapted to attach the device to an observed machine;   b) a processor;   c) a triple-axis accelerometer; and,   d) a wireless communication interface;   wherein the processor is configured to:
 i) periodically record a series of measurements from at least one axis of the triple-axis accelerometer over a period of time, 
 ii) process those measurements using a Fourier transform, 
 iii) compute the average value of the Fourier transform result, 
 iv) compare the average value to a threshold value, 
 v) determine, based on the comparison, whether the device should cause a transition in a state machine model of the observed machine, and 
 vi) if the determination is yes, cause the state machine transition to occur. 
   
     
     
         2 . A device comprising:
 a) a processor;   b) an imaging sensor;   c) a microphone; and,   d) a wireless communication interface;   wherein the processor is configured to:
 i) periodically record an image from the imaging sensor, 
 ii) periodically record a sound from the microphone; 
 iii) convert the sound into a spectrogram; 
 iv) pass the image and sound through a deep neural network to determine the probability that an observed machine has changed state, 
 v) determine, based on the result, whether the device should cause a transition in a state machine model associated with the observed machine, and 
 vi) if the determination is yes, cause the state machine transition to occur.