Techniques for recommending and presenting products in an augmented reality scene

Abstract

Embodiments of the present invention, as presented herein, relate to an augmented reality application-based service, which facilitates techniques for aiding a first end-user (e.g., a room designer) with the selection and placement of objects (e.g., images of home furnishing and related products) in an augmented reality scene that is being, or has been, generated via a mobile computing device that is remote from the first end-user, such that a second end-user (e.g., a potential consumer) operating the mobile computing device can view objects, in the augmented reality scene, as placed by the first, remote end-user.

What is claimed is: 
     
         1 . A computer-implemented method performed by one or more server computers, the method comprising:
 receiving from a mobile computing device an image of a physical space along with a data representation of a virtual space that corresponds with the physical space;   storing the image of the physical space along with the data representation of the virtual space that corresponds with the physical space;   using the image of the physical space as input to an object recognition analysis to generate output information identifying attributes of objects present in the image of the physical space;   with a query formulated based at least in part on the attributes of the objects present in the image of the physical space, querying a database of products to identify a set of candidate products; and   causing information relating to a subset of the set of candidate products to be presented in a user interface at a client computing device, the user interface to facilitate the placement of a representation of one or more candidate products within the virtual space that corresponds with the physical space.   
     
     
         2 . The computer-implemented method of  claim 1 , further comprising:
 storing information relating to a selection of candidate products with position information identifying their position within the virtual space that corresponds with the physical space; and   responsive to receiving, from the mobile computing device, a request for a saved room design, communicating to the mobile computing device the information relating to the stored selection of candidate products with position information identifying their position within the virtual space that corresponds with the physical space.   
     
     
         3 . The computer-implemented method of  claim 1 , further comprising:
 subsequent to receiving the request for the saved room, establishing a voice or video call session between the mobile computing device and the client computing device, thereby enabling end-users of the respective devices to collaborate via the voice or video call session.   
     
     
         4 . The computer-implemented method of  claim 1 , further comprising:
 prior to receiving and storing the image of the physical space and the data representation of the virtual space that corresponds with the physical space, receiving a design consultation request;   accessing end-user preference information of an end-user who initiated the design consultation request; and   selecting a room designer to whom to allocate the design consultation request by comparing the end-user preference information with profile information of the room designer.   
     
     
         5 . The computer-implemented method of  claim 1 , wherein the data representation of the virtual space includes information indicating a position of a virtual camera, within the virtual space, as determined by the mobile computing device using as input obtained from sensors of the mobile computing device, RGB images, depth images, IMU (inertial measurement unit) sampling data, and inferred scene geometry data. 
     
     
         6 . The computer-implemented method of  claim 5 , wherein, subsequent to causing information relating to the subset of the set of candidate products to be presented in a user interface at the client computing device, causing the image of the physical space to be presented in the user interface at the client computing device, the user interface having controls to facilitate overlaying a rendered 3D model of a candidate product of the set of candidate products on top of the image of the physical space, wherein positioning and orientation of the rendered 3D model of the candidate product overlaid on the image of the physical space generates information indicating the position and orientation of the candidate product relative to the virtual space, and the data representation of the virtual space including information indicating the position of the virtual camera is used to render the 3D model of the candidate product in the user interface at the client computing device. 
     
     
         7 . A system comprising:
 at least one processor for executing instructions;   at least one memory storage device storing instructions, which, when executed by the at least one processor, cause the information retrieval system to perform operations comprising:   receiving from a mobile computing device an image of a physical space along with a data representation of a virtual space that corresponds with the physical space;   storing the image of the physical space along with the data representation of the virtual space that corresponds with the physical space;   using the image of the physical space as input to an object recognition analysis to generate output information identifying attributes of objects present in the image of the physical space;   with a query formulated based at least in part on the attributes of the objects present in the image of the physical space, querying a database of products to identify a set of candidate products; and   causing information relating to a subset of the set of candidate products to be presented in a user interface at a client computing device, the user interface to facilitate the placement of a representation of one or more candidate products within the virtual space that corresponds with the physical space.   
     
     
         8 . The system of  claim 7 , wherein the instructions, when executed, cause the system to perform further operations comprising:
 storing information relating to a selection of candidate products with position information identifying their position within the virtual space that corresponds with the physical space; and   responsive to receiving, from the mobile computing device, a request for a saved room design, communicating to the mobile computing device the information relating to the stored selection of candidate products with position information identifying their position within the virtual space that corresponds with the physical space.   
     
     
         9 . The system of  claim 7 , wherein the instructions, when executed, cause the system to perform further operations comprising:
 subsequent to receiving the request for the saved room, establishing a voice or video call session between the mobile computing device and the client computing device, thereby enabling end-users of the respective devices to collaborate via the voice or video call session.   
     
     
         10 . The system of  claim 7 , wherein the instructions, when executed, cause the system to perform further operations comprising:
 prior to receiving and storing the image of the physical space and the data representation of the virtual space that corresponds with the physical space, receiving a design consultation request;   accessing end-user preference information of an end-user who initiated the design consultation request; and   selecting a room designer to whom to allocate the design consultation request by comparing the end-user preference information with profile information of the room designer.   
     
     
         11 . The system of  claim 7 , wherein the data representation of the virtual space includes information indicating a position of a virtual camera, within the virtual space, as determined by the mobile computing device using as input obtained from sensors of the mobile computing device, RGB images, depth images, IMU (inertial measurement unit) sampling data, and inferred scene geometry data. 
     
     
         12 . The system of  claim 7 , wherein subsequent to causing information relating to the subset of the set of candidate products to be presented in a user interface at the client computing device, causing the image of the physical space to be presented in the user interface at the client computing device, the user interface having controls to facilitate overlaying a rendered 3D model of a candidate product of the set of candidate products on top of the image of the physical space, wherein positioning and orientation of the rendered 3D model of the candidate product overlaid on the image of the physical space generates information indicating the position and orientation of the candidate product relative to the virtual space, and the data representation of the virtual space including information indicating the position of the virtual camera is used to render the 3D model of the candidate product in the user interface at the client computing device. 
     
     
         13 . A system comprising:
 means for receiving from a mobile computing device an image of a physical space along with a data representation of a virtual space that corresponds with the physical space;   means for storing the image of the physical space along with the data representation of the virtual space that corresponds with the physical space;   means for using the image of the physical space as input to an object recognition analysis to generate output information identifying attributes of objects present in the image of the physical space;   means for querying a database of products to identify a set of candidate products with a query formulated based at least in part on the attributes of the objects present in the image of the physical space; and   means for causing information relating to a subset of the set of candidate products to be presented in a user interface at a client computing device, the user interface to facilitate the placement of a representation of one or more candidate products within the virtual space that corresponds with the physical space.   
     
     
         14 . The system of  claim 13 , further comprising:
 means for storing information relating to a selection of candidate products with position information identifying their position within the virtual space that corresponds with the physical space; and   means for communicating to the mobile computing device the information relating to the stored selection of candidate products with position information identifying their position within the virtual space that corresponds with the physical space, responsive to receiving, from the mobile computing device, a request for a saved room design.   
     
     
         15 . The system of  claim 13 , further comprising:
 subsequent to receiving the request for the saved room, means for establishing a voice or video call session between the mobile computing device and the client computing device, thereby enabling end-users of the respective devices to collaborate via the voice or video call session.   
     
     
         16 . The system of  claim 13 , further comprising:
 means for receiving a design consultation request, prior to receiving and storing the image of the physical space and the data representation of the virtual space that corresponds with the physical space;   means for accessing end-user preference information of an end-user who initiated the design consultation request; and   means for selecting a room designer to whom to allocate the design consultation request by comparing the end-user preference information with profile information of the room designer.   
     
     
         17 . The system of  claim 13 , wherein the data representation of the virtual space includes information indicating a position of a virtual camera, within the virtual space, as determined by the mobile computing device using as input obtained from sensors of the mobile computing device, RGB images, depth images, IMU (inertial measurement unit) sampling data, and inferred scene geometry data. 
     
     
         18 . The system of  claim 17 , further comprising:
 means for causing the image of the physical space to be presented in the user interface at the client computing device, the user interface having controls to facilitate overlaying a rendered 3D model of a candidate product of the set of candidate products on top of the image of the physical space, wherein positioning and orientation of the rendered 3D model of the candidate product overlaid on the image of the physical space generates information indicating the position and orientation of the candidate product relative to the virtual space, and the data representation of the virtual space including information indicating the position of the virtual camera is used to render the 3D model of the candidate product in the user interface at the client computing device.