Video game system with wireless modular handheld controller

Abstract

A home entertainment system for video games and other applications includes a main unit and handheld controllers. The handheld controllers sense their own motion by detecting illumination emitted by emitters positioned at either side of a display. The controllers can be plugged into expansion units that customize the overall control interface for particular applications including but not limited to legacy video games.

We claim: 
     
       1. A wireless handheld remote controller configured to be held in one hand, comprising:
 a housing including an upper surface and a lower surface; 
 at least one digit operable detector disposed on the upper surface; 
 at least one depressible trigger disposed on said lower surface; 
 an inertial sensor mounted in the housing; 
 a two dimensional radiation detector; 
 a processor that processes an output of the radiation detector and determines an illumination pattern; 
 a wireless transceiver that transmits information based on signals generated by the inertial sensor and the processor; and 
 an output device operatively coupled to the transceiver. 
 
     
     
       2. The controller of  claim 1  wherein the radiation detector is disposed, at least in part, at a front portion of the housing. 
     
     
       3. The controller of  claim 1 , wherein the radiation detector comprises a two dimensional camera. 
     
     
       4. The controller of  claim 1 , wherein the radiation detector comprises:
 a two dimensional radiation sensor array; and 
 an infrared filter that is mounted on the housing in front of the two dimensional radiation sensor array such that only infrared light passing through the filter is received by the radiation sensor array. 
 
     
     
       5. The controller of  claim 1 , wherein the radiation detector generates frames of two dimensional image data, and wherein the processor determines an illumination pattern for each frame of image data. 
     
     
       6. The controller of  claim 5 , wherein each illumination pattern comprises X and Y coordinates for illuminated objects appearing within a frame of image data. 
     
     
       7. The controller of  claim 5 , wherein each illumination pattern comprises X and Y coordinates for illuminated objects appearing within a frame of image data that have an intensity that rises above a predetermined threshold value. 
     
     
       8. The controller of  claim 5 , wherein each illumination pattern comprises X and Y coordinates for illuminated objects appearing within a frame of image data that emit infrared radiation having an intensity that rises above a predetermined threshold value. 
     
     
       9. The controller of  claim 5 , wherein the wireless transceiver transmits information regarding the illumination patterns for frames of image data. 
     
     
       10. The controller of  claim 9 , wherein the inertial sensor comprises an accelerometer. 
     
     
       11. The controller of  claim 10 , wherein the accelerometer is a three axis accelerometer that senses linear acceleration in each of three mutually perpendicular axes, and wherein the inertial sensor outputs three linear acceleration values corresponding to the three mutually perpendicular axes multiple times every second. 
     
     
       12. The controller of  claim 11 , wherein the wireless transceiver also transmits a set of the three acceleration values multiple times every second. 
     
     
       13. The controller of  claim 1 , wherein the inertial sensor comprises an accelerometer. 
     
     
       14. The controller of  claim 13 , wherein the accelerometer is a three axis accelerometer that senses linear acceleration in each of three mutually perpendicular axes, and wherein the inertial sensor outputs three linear acceleration values corresponding to the three mutually perpendicular axes multiple times every second. 
     
     
       15. The controller of  claim 14 , wherein the wireless transceiver transmits a set of the three acceleration values multiple times every second. 
     
     
       16. The controller of  claim 1 , wherein the output device comprises a speaker, and wherein the speaker outputs sounds based on a signal received by the wireless transceiver. 
     
     
       17. The controller of  claim 1 , wherein the output device comprises a vibration module that causes the housing to vibrate based on a signal received by the wireless transceiver. 
     
     
       18. The controller of  claim 1 , wherein the output device comprises at least one indicator light that is selectively illuminated based on a signal received by the wireless transceiver. 
     
     
       19. The controller of  claim 1 , wherein the output device comprises an array of indicator lights that are selectively illuminated based on a signal received by the wireless transceiver. 
     
     
       20. The controller of  claim 1 , wherein the at least one digit operable detector comprises at least one depressible button disposed on the upper surface of the housing. 
     
     
       21. A handheld electronic device, comprising:
 a housing configured to be held by both hands of a user for providing input to a processor, wherein the housing includes a top surface, a bottom surface, and a further surface extending between the top surface and the bottom surface;   a touch-sensitive input panel arranged at a surface of the housing and configured to receive touch input; and   a first input device and a second input device arranged at the top surface on a first side from the lateral center of the top surface, the first and second input devices being operable with a thumb of the user to provide directional inputs to the processor,   wherein one of the first input device or the second input device includes a directional switch input device and the other includes an inclinable stick input device, and   wherein the top surface comprises a proximal portion closer to the body of the user when the user holds the housing in two hands, the first input device is arranged between the second input device and the proximal portion.    
     
     
       22. The handheld electronic device according to claim 21, further comprising a third input device arranged at the top surface on a second side that is opposite the first side from the lateral center of the top surface.  
     
     
       23. The handheld electronic device according to claim 22, wherein the third input device includes at least four control buttons in a cross-shaped arrangement.  
     
     
       24. The handheld electronic device according to claim 22, wherein the third input device includes an inclinable stick input device.  
     
     
       25. The handheld electronic device according to claim 24, wherein the first input device includes a directional switch input device, and wherein the first input device and the third input device are positioned symmetrically on opposite sides of the lateral center of the top surface.  
     
     
       26. The handheld electronic device according to claim 22, further comprising a fourth input device arranged at the left side of the further surface and a fifth input device arranged at the right side of the further surface, wherein the fourth input device is operable with a finger of the user's left hand and the fifth input device is operable with a finger of the user's right hand when the user holds the housing with both hands to provide input to the processor.  
     
     
       27. The handheld electronic device according to claim 22, further comprising at least one fourth input device on a surface of the housing other than the top surface, the at least one fourth input device configured to generate an analog signal based upon a level of user input.  
     
     
       28. The handheld electronic device according to claim 27, wherein the at least one fourth input device is further configured to vary the analog signal based upon how much pressure is exerted on the at least one fourth input device by a finger of the user.  
     
     
       29. The handheld electronic device according to claim 22, further comprising a fourth input device arranged at a lateral center of the top surface, the fourth input device being a button switch.  
     
     
       30. The handheld electronic device according to claim 29, wherein the button switch is recessed in relation to the top surface.  
     
     
       31. The handheld electronic device according to claim 30, wherein the fourth input device is configured to generate an input signal to cause a MENU or HOME operation.  
     
     
       32. The handheld electronic device according to claim 22, wherein a recessed button switch is arranged at a lateral center of the top surface, and wherein one of the first input device or the second input device and a second inclinable stick input device are arranged symmetrically on opposite sides of the lateral center of the top surface.  
     
     
       33. The handheld electronic device according to claim 21, further comprising at least one wireless antenna, wherein the handheld electronic device is configured to communicate with a console device using a wireless protocol over the wireless antenna.  
     
     
       34. The handheld electronic device according to claim 33, wherein the wireless protocol is based upon the Bluetooth protocol standard.  
     
     
       35. The handheld electronic device according to claim 21, further comprising a vibration generator configured to vibrate the housing in response to a signal received via a wireless communication interface.  
     
     
       36. The handheld electronic device according to claim 21, further comprising at least one image sensor.  
     
     
       37. The handheld electronic device according to claim 36, further comprising a focusing lens associated with the image sensor.  
     
     
       38. The handheld electronic device according to claim 21, further comprising at least one inertial sensor.  
     
     
       39. A handheld electronic device, comprising:
 a housing configured to be held by both hands of a user for providing input to a processor, wherein the housing includes a top surface, a bottom surface, and a further surface extending between the top surface and the bottom surface;   a touch-sensitive input panel arranged at a surface of the housing and configured to receive touch input; and   a first input device and a second input device arranged at the top surface on a first side from the lateral center of the top surface, the first and second input devices being operable with a thumb of the user to provide directional inputs to the processor,   the handheld electronic device further comprising at least one light indication devices arranged on the housing and configured to indicate an identification of the handheld electronic device, wherein the identification uniquely identifies the handheld electronic device among a plurality of controllers communicating with a particular console device.    
     
     
       40. A handheld electronic device, comprising:
 a housing configured to be held by both hands of a user for providing input to a processor, wherein the housing includes a top surface, a bottom surface, and a further surface extending between the top surface and the bottom surface;   a touch-sensitive input panel arranged at a surface of the housing and configured to receive touch input; and   a first input device and a second input device arranged at the top surface on a first side from the lateral center of the top surface, the first and second input devices being operable with a thumb of the user to provide directional inputs to the processor,   the handheld electronic device further comprising additional input devices arranged at locations of the top surface, wherein the additional input devices are configured, respectively, to cause a start operation, to cause a select operation, and to cause processing to return to a predetermined configuration.    
     
     
       41. A handheld electronic device, comprising:
 a housing configured to be held by both hands of a user for providing input to a processor, wherein the housing includes a top surface, a bottom surface, and a further surface extending between the top surface and the bottom surface;   a touch-sensitive input panel arranged at a surface of the housing and configured to receive touch input; and   a first input device and a second input device arranged at the top surface on a first side from the lateral center of the top surface, the first and second input devices being operable with a thumb of the user to provide directional inputs to the processor,   wherein the touch-sensitive input panel includes a touch screen, and wherein   the handheld electronic device further comprises another touch-sensitive input panel.    
     
     
       42. A handheld electronic device, comprising:
 a housing configured to be held by both hands of a user for providing input to a processor, wherein the housing includes a top surface, a bottom surface, and a further surface extending between the top surface and the bottom surface;   a touch-sensitive input panel arranged at a surface of the housing and configured to receive touch input; and   a first input device and a second input device arranged at the top surface on a first side from the lateral center of the top surface, the first and second input devices being operable with a thumb of the user to provide directional inputs to the processor,   wherein one of the first input device or the second input device includes a directional switch input device and the other includes an analog directional input device, and   wherein the top surface comprises a proximal portion closer to the body of the user when the user holds the housing in two hands, the first input device is arranged between the second input device and the proximal portion.    
     
     
       43. A handheld electronic device, comprising:
 a housing configured to be held by both hands of a user for providing input to a processor, wherein the housing includes a top surface, a bottom surface, and a further surface extending between the top surface and the bottom surface;   a touch-sensitive input panel arranged at a surface of the housing and configured to receive touch input; and   a first input device and a second input device arranged at the top surface on a first side from the lateral center of the top surface, the first and second input devices being operable with a thumb of the user to provide directional inputs to the processor,   wherein one of the first input device or the second input device includes a directional switch input device and the other includes an analog directional input device, and   wherein the first input device is disposed at a first position on the housing top surface and the second input device is disposed next to the first input device at a second position on the housing top surface, the second position being closer to the body of the user than the first position when the housing is held by both hands of the user.