Methods and systems for a reminder servicer using visual recognition

Abstract

A downloadable reminder application uses object recognition technology to recognize one or more objects appearing on a display screen of the mobile computing device and matches them to a set of known target objects. The downloadable reminder application process any associated trigger conditions for the target object. The downloadable reminder application then sets the future reminder for the target object to activate. The downloadable reminder application when it detects that a mobile computing device is satisfying the trigger condition for a target object for which it has set a future reminder on, then the downloadable reminder application in the mobile computing device fires the future reminder that was set for that target object and provides the alert to the user along with any triggerable contributions associated with that target object.

1 - 20 . (canceled) 
     
     
         21 . A computing device, comprising:
 a display screen;   a processor; and   a memory on which is stored machine readable instructions that when executed by a processor cause the processor to:
 recognize an object appearing on the display screen; 
 match the recognized object to a known target object; 
 set a future reminder for the target object to be activated in response to a determination that a trigger condition associated with the target object has occurred; 
 transmit a communication to a reminder engine in a backend server system, wherein the communication includes an identification of the target object, and wherein the reminder engine is to identify the associated trigger condition and associated triggerable information for the target object; 
 receive a reminder packet from the reminder engine, wherein the reminder packet includes the associated trigger condition and the triggerable information for the target object; and 
 monitor for occurrence of the associated trigger condition. 
   
     
     
         22 . The computing device of  claim 21 , wherein the machine readable instructions are further to cause the processor to:
 detect that the associated trigger condition has occurred; and   responsive to the detection of the occurrence of the associated trigger condition,
 output the future reminder for the target object; and 
 output the triggerable information for the target object. 
   
     
     
         23 . The computing device of  claim 21 , wherein the associated trigger condition is at least one of a time and date trigger condition. 
     
     
         24 . The computing device of  claim 21 , wherein the triggerable information comprises at least one of information pertaining to the target object, advertisement information, link information, user reviews, links to web pages, price comparisons, coupons, and video trailers. 
     
     
         25 . The computing device of  claim 21 , wherein to recognize the object appearing on the display screen, the machine readable instructions are further to cause the processor to:
 identify an object in a frame of a captured video stream; and   determine whether the target object is contained in the frame of the captured video stream.   
     
     
         26 . The computing device of  claim 25 , wherein to determine whether the target object is contained in the frame of the captured video stream, the machine readable instructions are further to cause the processor to compare the identified object in the frame of the captured video stream to target objects stored in a local target object database of the computing device to determine if there is a match between the identified object and one of the target objects stored in the local target object database, wherein a subset of target objects stored in a backend target object database in the backend server system is stored in the local target object database. 
     
     
         27 . The computing device of  claim 25 , wherein to determine whether the target object is contained in the frame of the captured video stream, the machine readable instructions are further to cause the processor to:
 determine distinguishing features of the identified object using optical character recognition (OCR);   extract the determined distinguishing features of the identified object;   code the distinguishing features of the identified object into a digital fingerprint; and   compare the digital fingerprint with a set of digital fingerprints of target objects stored in a local target object database.   
     
     
         28 . The computing device of  claim 27 , wherein the machine readable instructions are further to cause the processor to:
 request digital fingerprints of target objects that have actionable trigger conditions from the backend server system; and   update target objects stored in a local target object database with the digital fingerprints received from the backend server system.   
     
     
         29 . The computing device of  claim 21 , wherein the machine readable instructions are further to cause the processor to:
 determine that a communication between the computing device and the backend server system is interrupted;   store the recognized object in a local target object database responsive to the determination that the communication has been interrupted;   determine a local list of candidate target objects based on data pertaining to objects stored in the local target object database that are determined to be similar to the recognized object;   display the local list of candidate target objects on the display screen;   receive a user selection of a local target object from the displayed local list of candidate target objects;   set the future reminder on the local target object corresponding to the received user selection in the local target object database; and   transmit the selected local target object and the future reminder to the backend server system in response to the communication between the apparatus and the backend server system being reestablished.   
     
     
         30 . A method comprising:
 identifying an object appearing on a display screen;   matching the recognized object to a known target object;   setting a future reminder for the target object to be activated in response to a determination that a trigger condition associated with the target object has occurred;   transmitting a communication to a reminder engine in a backend server system, wherein the communication includes an identification of the target object, and wherein the reminder engine is to identify the associated trigger condition and associated triggerable information for the target object;   receiving a reminder packet from the reminder engine, wherein the reminder packet includes the associated trigger condition and the triggerable information for the target object; and   monitoring for occurrence of the associated trigger condition.   
     
     
         31 . The method of  claim 30 , further comprising:
 detecting an occurrence of the associated trigger condition; and   responsive to detecting the occurrence of the associated trigger condition,
 outputting the future reminder for the target object; and 
 outputting the triggerable information for the target object. 
   
     
     
         32 . The method of  claim 30 , wherein identifying the object and matching the recognized object to a known target object further comprises:
 identifying an object in a frame of a captured video stream and determining whether the target object is contained in the frame of the captured video stream.   
     
     
         33 . The method of  claim 32 , wherein determining whether the target object is contained in the frame of the captured video stream further comprises comparing the identified object in the frame of the captured video stream to target objects stored in a local target object database of the computing device to determine if there is a match between the identified object and one of the target objects stored in the local target object database, wherein a subset of target objects stored in a backend target object database in the backend server system is stored in the local target object database. 
     
     
         34 . The method of  claim 32 , wherein determining whether the target object is contained in the frame of the captured video stream further comprises:
 determining distinguishing features of the identified object using optical character recognition (OCR);   extracting the determined distinguishing features of the identified object;   coding the distinguishing features of the identified object into a digital fingerprint; and   comparing the digital fingerprint with a set of digital fingerprints of target objects stored in a local target object database.   
     
     
         35 . A server system comprising:
 a processor; and   a memory on which is stored machine readable instructions that when executed by a processor cause the processor to:
 receive a communication from a computing device, wherein the communication includes an identification of a target object; 
 identify a trigger condition associated with the target object; 
 identify triggerable information associated with the target object; 
 generate a reminder packet having the associated trigger condition and the associated triggerable information; and 
 transmit the reminder packet to the computing device, wherein the computing device is to use the reminder packet to set a future reminder for the target object to be activated in response to the associated trigger condition being determined to have occurred. 
   
     
     
         36 . The server system of  claim 35 , wherein the instructions are further to cause the processor to:
 identify digital fingerprints of known target objects that have actionable trigger conditions;   generate a packet that includes the identified digital fingerprints; and   transmit the generated packet to the computing device.   
     
     
         37 . The server system of  claim 36 , wherein the instructions are further to cause the processor to:
 generate a plurality of packets that each includes a different set of the identified digital fingerprints; and   transmit each of the generated plurality of packets to a respective computing device of a plurality of computing devices.   
     
     
         38 . The server system of  claim 35 , wherein the instructions are further to cause the processor to:
 set a reminder corresponding to the target object;   receive an indication from the computing device that the associated trigger condition has been detected; and   responsive to receipt of the indication, transmit the reminder to the computing device.   
     
     
         39 . The server system of  claim 35 , wherein the associated trigger condition is at least one of a time and date trigger condition. 
     
     
         40 . The server system of  claim 35 , wherein the triggerable information comprises at least one of information pertaining to the target object, advertisement information, link information, user reviews, links to web pages, price comparisons, coupons, and video trailers.