Automatic screen navigation for media device configuration and control

Abstract

Methods, systems, and apparatuses are described for automatically navigating a graphical user interface (GUI) of a media device. In embodiments, objects (e.g. selectable items) in device GUI menu screens may be located, identified, and selected by another device (e.g., a switch) using several types of image processing, image recognition, and automated screen navigation techniques. For instance, a device may receive an operation request, for example from a user, receive a video signal from the connected media device, extract a video frame, process the video frame using one or more algorithms, and automatically navigate the GUI of the device by transmitting one more navigation commands for application to the GUI. In this manner, multi-step, time-consuming manual GUI navigation to achieve a desired result may be reduced to an automated technique, allowing users to save time and frustration in navigating a GUI using a remote control to find a particular resource.

1 . (canceled) 
     
     
         2 . A method of automatically identifying one or more resources available on at least one media device, the method comprising:
 receiving a video signal from at least one media device;   extracting one or more video frames from the video signal;   processing the one or more video frames to obtain graphical user interface (GUI) information therefrom;   automatically determining, based on the GUI information obtained, that at least one resource is available on a particular media device;   storing a list of the at least one resource available on the particular media device; and   storing mapping information of the particular media device.   
     
     
         3 . The method of  claim 2 , further comprising:
 determining additional resources available on the particular media device by automatically scrolling a GUI screen.   
     
     
         4 . The method of  claim 2 , further comprising:
 determining additional resources available on the particular media device by automatically selecting an object to navigate to a different page.   
     
     
         5 . The method of  claim 2 , further comprising:
 receiving a first operation request via a voice command.   
     
     
         6 . The method of  claim 5 , wherein the voice command is captured via a microphone of a remote control. 
     
     
         7 . The method of  claim 2 , wherein the at least one resource includes an application. 
     
     
         8 . The method of  claim 2 , wherein the at least one resource includes recorded multimedia content. 
     
     
         9 . The method of  claim 5 , wherein the first operation request comprises a request to execute a specified resource from the stored list, and
 wherein the method further comprises:
 determining, based at least in part on the mapping information, the particular media device on which the specified resource is available on; 
 automatically navigating the GUI of the particular media device on which the specified resource is available on; and 
 executing the specified resource. 
   
     
     
         10 . A system for automatically identifying one or more resources available on a plurality of media devices, the system comprising:
 one or more processing components; and   one or more memory devices that store program code that is executed by the one or more processing components to cause the system to:
 receive a video signal from at least one media device via a video signal receiver; 
 extract one or more video frames from the video signal via a video frame extractor; 
 process the one or more video frames to obtain graphical user interface (GUI) information therefrom via a frame processor; 
 automatically determine, based on the GUI information obtained, that at least one resource is available on a particular media device; 
 generate a list of the at least one resource available on the particular media device; 
 generate mapping information of the particular media device; and 
 store the list of at least one resource available on a particular media device and the mapping information of the particular media device. 
   
     
     
         11 . The system of  claim 10 , wherein the system includes a microphone configured to capture a first operation request via a voice command. 
     
     
         12 . The system of  claim 11 , wherein the microphone is included within a remote control. 
     
     
         13 . The system of  claim 10 , wherein the program code, when executed by the one or more processing components, further causes the system to:
 determine if additional resources are available on a particular media device by automatically scrolling a GUI screen of the particular media device.   
     
     
         14 . The system of  claim 10 , wherein the program code, when executed by the one or more processing components, further causes the system to:
 determine if additional resources are available on a particular media device by automatically selecting an object to navigate to a different page.   
     
     
         15 . The system of  claim 10 , wherein the program code, when executed by the one or more processing components, further causes the system to:
 process the one or more video frames to obtain GUI information therefrom by performing optical character recognition (OCR) on a plurality of the portions of the one or more video frames to capture text associated therewith.   
     
     
         16 . A computer-readable storage medium having program logic recorded thereon that, when executed by at least one processor circuit of a multimedia receiver, perform a method, the method comprising:
 receiving a video signal from at least one media device via a video signal receiver;   extracting one or more video frames from the video signal via a video frame extractor;   processing the one or more video frames to obtain graphical user interface (GUI) information therefrom via a frame processor;   automatically determining, based on the GUI information obtained, that at least one resource is available on a particular media device;   generating a list of the at least one resource available on the particular media device;   generating mapping information of the particular media device; and   storing the list of at least one resource available on a particular media device and the mapping information of the particular media device.   
     
     
         17 . The computer-readable storage medium of  claim 16 , wherein the method further includes:
 capturing, using a microphone, a first operation request via a voice command.   
     
     
         18 . The computer-readable storage medium of  claim 17 , wherein the microphone is included within a remote control. 
     
     
         19 . The computer-readable storage medium of  claim 16 , wherein the method further comprises:
 determining if additional resources are available on a particular media device by automatically scrolling a GUI screen of the particular media device.   
     
     
         20 . The computer-readable storage medium of  claim 16 , wherein the method further comprises:
 determining if additional resources are available on a particular media device by automatically selecting an object to navigate to a different page.   
     
     
         21 . The computer-readable storage medium of  claim 16 , wherein the method further comprises:
 processing the one or more video frames to obtain GUI information therefrom by performing optical character recognition (OCR) on a plurality of the portions of the one or more video frames to capture text associated therewith.