System and Method for Exploring 3D Scenes by Pointing at a Reference Object

Abstract

The system and method are described for enhancing location-based services by enabling spatial database systems to respond to or answer spatial queries that use a reference object to identify objects or features of interest in environmental scene before a system user. The system and method present invention enhances pointing technology by permitting system users to use queries to identify objects or features within the system user's field of view by pointing at the reference object or feature, and linking it to the object of interest by using spatial prepositions.

1 . A computer-implemented method for identifying objects of interest in an environmental scene based on a reference object, comprising the steps of:
 (A) pointing a pointing device at an object within a system user's visual field within the environmental scene, with the pointing device being capable of determining its geographic location and identifying the object to which the pointing device is pointing in a system user's visual field in a system user's environment;   (B) generating a spatial scene with a scene generator that coincides with the system user's visual field, with the spatial scene including a spatial configuration of the system user's environment;   (C) designating the object identified in step (A) as a reference object;   (D) determining semantics of the spatial scene, with the semantics geographically associating objects within the system user's visual field with the reference object designated at step (C);   (E) generating queries relating to one or more objects of interest within a system user's visual field based on pointing at the reference object designated at step (C); and   (F) identifying one or more objects of interest within the system user's visual field based on the queries relating to such one or more objects of interest generated at step (E) based on pointing at the reference object designated at step (C).   
     
     
         2 . The method as recited in  claim 1 , wherein the spatial configuration at step (B) includes at least an absolute frame of reference, egocentric frame of reference, and an intrinsic frame of reference. 
     
     
         3 . The method as recited in  claim 2 , wherein the egocentric frame of reference includes an object position within the system user's visual field in the system user's environment described from a system user's geographic location. 
     
     
         4 . The method as recited in  claim 2 , wherein the intrinsic frame of reference includes an object position based on an object structure. 
     
     
         5 . The method as recited in  claim 1 , wherein identifying one or more objects of interest within the system user's visual field based on queries relating to such one or more objects of interest includes identifying objects of interest that are hidden by other objects. 
     
     
         6 . The method as recited in  claim 5 , wherein a query includes statements that combine a system user's geographic location and pointing direction, a first expression defining a spatial relationship between the reference object and an object of interest, and a second expression that indicates a topic of interest or action to be performed. 
     
     
         7 . The method as recited in  claim 6 , wherein query regions associated with a spatial scene include a HERE region, THERE region, and SCENE region. 
     
     
         8 . The method as recited in  claim 7 , wherein the absolute frame of reference indicates a spatial relationship between the system user, reference object, and the object of interest according to a query statement. 
     
     
         9 . The method as recited in  claim 8 , wherein the absolute frame of reference includes from one to eight areas for defining the object of interest with respect to the reference object. 
     
     
         10 . The method as recited in  claim 8 , wherein the absolute frame of reference includes at least North, South, East, and West. 
     
     
         11 . The method as recited in  claim 3 , wherein the egocentric frame of reference includes from one to eight areas for defining the object of interest with respect to the reference object. 
     
     
         12 . The method as recited in  claim 3 , wherein the egocentric frame of reference includes being used for qualitatively describing the spatial configuration of the spatial scene with qualitative statements. 
     
     
         13 . The method as recited in  claim 2 , wherein the intrinsic frame of reference includes from one to eight areas for defining the object of interest with respect to the reference object. 
     
     
         14 . The method as recited in  claim 2 , wherein the spatial scene includes a projective two-dimensional scene generated from a three-dimensional scene. 
     
     
         15 . The method as recited in  claim 1 , wherein the pointing device includes a mobile device. 
     
     
         16 . A system for responding to spatial queries about a real-world object of interest using a reference object to identify the object of interest, comprising:
 a pointing device for pointing at a reference object in a real-world scene and generating a query statement including a query topic and spatial preposition, relating to the object of interest according to a position of the reference object, with the pointing device determining a pointing device geodetic position and pointing direction, and with the pointing device communicating the query statement, and the pointing device geodetic position and pointing direction to a system server; and   a system server further comprising,
 a mapping module for receiving and processing a projective three-dimensional representation of an environment that contains a real-world scene including the reference object and the object of interest, 
 a scene generator that connects to the mapping module and receives an output from the mapping module for generating a two-dimensional digital representation of the real-world scene including the reference object and object of interest, and identifies the reference object according to the pointing device geodetic position and pointing direction applied to the two-dimensional digital representation, 
 a scene annotator for annotating the two-dimensional digital representation for identifying the object of interest according to the position of the reference object, and the query topic and spatial preposition of the query statement by linking the query topic and spatial preposition to an object in the two-dimensional digital representation, and communicating to the pointing device the identification of the object of interest. 
   
     
     
         17 . The system as recited in  claim 16 , wherein the pointing device includes a mobile device. 
     
     
         18 . The system as recited in  claim 16 , wherein the scene annotator includes using a egocentric frame of reference for identifying the object of interest. 
     
     
         19 . The system as recited in  claim 18 , wherein the egocentric frame of reference identifies the object of interest according to the reference object position and pointing device geodetic position. 
     
     
         20 . The system as recited in  claim 15 , wherein the system further includes an identifier module for identifying the object of interest in the two-dimensional digital representation according to the position of the reference object and the query topic and spatial preposition of the query statement by linking the query topic and spatial preposition to an object in the two-dimensional digital representation, and communicating to the pointing device the identification of the object of interest.