Arbitrary view generation

Abstract

Techniques for generating an arbitrary ensemble view from a plurality of other existing views are disclosed. In some embodiments, arbitrary view generation comprises retrieving a set of views corresponding to each of a plurality of objects, transforming the set of views of each of the plurality of objects to a requested perspective, and generating an ensemble view comprising the plurality of objects that has the requested perspective by combining pixels from the transformed sets of views of the plurality of objects.

What is claimed is: 
     
       1. A system, comprising:
 a processor configured to:
 retrieve, for each of a plurality of objects, a corresponding set of views; 
 transform the set of views of each of the plurality of objects to a requested perspective, wherein each view of the transformed sets of views has at least some missing pixels at positions introduced in the requested perspective that are not present in an original perspective of that view; and 
 generate an ensemble view comprising the plurality of objects that has the requested perspective at least in part by combining pixels from the transformed sets of views of the plurality of objects; and 
 
 a memory coupled to the processor and configured to provide the processor with instructions. 
 
     
     
       2. The system of  claim 1 , wherein the sets of views corresponding to the plurality of objects are independently captured, rendered, or both. 
     
     
       3. The system of  claim 1 , wherein the set of views of each of the plurality of objects comprises photographs, photorealistic renderings, or both. 
     
     
       4. The system of  claim 1 , wherein the set of views of each of the plurality of objects comprises perspectives that are different than the requested perspective. 
     
     
       5. The system of  claim 1 , wherein camera characteristics are stored with and known for each retrieved view. 
     
     
       6. The system of  claim 5 , wherein the camera characteristics include one or more of: position, orientation, rotation, angle, depth, focal length, aperture, zoom level, and lighting information. 
     
     
       7. The system of  claim 1 , wherein to generate the ensemble view comprises to interpolate one or more pixels of the ensemble view. 
     
     
       8. The system of  claim 1 , wherein the ensemble view comprises one or more of a virtual environment, a view of a three-dimensional space, and a frame of an animation or video sequence. 
     
     
       9. The system of  claim 1 , wherein the ensemble view comprises a composite object. 
     
     
       10. The system of  claim 1 , wherein the set of views of each of the plurality of objects at least in part comprises orthographic views. 
     
     
       11. The system of  claim 1 , wherein the set of views of each of the plurality of objects at least in part comprises orthographic views from a plurality of angles. 
     
     
       12. The system of  claim 1 , wherein the processor is further configured to receive a specification of the ensemble view in terms of orthographic views of the plurality of objects. 
     
     
       13. The system of  claim 12 , wherein orthographic views of the plurality of objects facilitate manipulation and accurate placement of the plurality of objects comprising the ensemble view. 
     
     
       14. The system of  claim 1 , wherein each retrieved view comprises metadata including relative object and camera location and orientation. 
     
     
       15. The system of  claim 1 , wherein each retrieved view comprises pixel metadata including relative position coordinates of a point in three-dimensional space that projects at a pixel. 
     
     
       16. The system of  claim 1 , wherein each retrieved view comprises pixel metadata including surface normal vectors relative to coordinate axes. 
     
     
       17. The system of  claim 1 , wherein each retrieved view comprises pixel metadata including texture mapping coordinates. 
     
     
       18. The system of  claim 1 , wherein each retrieved view comprises pixel metadata and wherein RGB values of a pixel encode pixel metadata. 
     
     
       19. A method, comprising:
 for each of a plurality of objects, retrieving a corresponding set of views; 
 transforming the set of views of each of the plurality of objects to a requested perspective, wherein each view of the transformed sets of views has at least some missing pixels at positions introduced in the requested perspective that are not present in an original perspective of that view; and 
 generating an ensemble view comprising the plurality of objects that has the requested perspective at least in part by combining pixels from the transformed sets of views of the plurality of objects. 
 
     
     
       20. A computer program product embodied in a non-transitory computer readable storage medium and comprising computer instructions for:
 for each of a plurality of objects, retrieving a corresponding set of views; 
 transforming the set of views of each of the plurality of objects to a requested perspective, wherein each view of the transformed sets of views has at least some missing pixels at positions introduced in the requested perspective that are not present in an original perspective of that view; and 
 generating an ensemble view comprising the plurality of objects that has the requested perspective at least in part by combining pixels from the transformed sets of views of the plurality of objects. 
 
     
     
       21. The method of  claim 19 , wherein the sets of views corresponding to the plurality of objects are independently captured, rendered, or both. 
     
     
       22. The method of  claim 19 , wherein the set of views of each of the plurality of objects comprises photographs, photorealistic renderings, or both. 
     
     
       23. The method of  claim 19 , wherein the set of views of each of the plurality of objects comprises perspectives that are different than the requested perspective. 
     
     
       24. The method of  claim 19 , wherein camera characteristics are stored with and known for each retrieved view. 
     
     
       25. The method of  claim 24 , wherein the camera characteristics include one or more of: position, orientation, rotation, angle, depth, focal length, aperture, zoom level, and lighting information. 
     
     
       26. The method of  claim 19 , wherein generating the ensemble view comprises interpolating one or more pixels of the ensemble view. 
     
     
       27. The method of  claim 19 , wherein the ensemble view comprises one or more of a virtual environment, a view of a three-dimensional space, and a frame of an animation or video sequence. 
     
     
       28. The method of  claim 19 , wherein the ensemble view comprises a composite object. 
     
     
       29. The method of  claim 19 , wherein the set of views of each of the plurality of objects at least in part comprises orthographic views. 
     
     
       30. The method of  claim 19 , wherein the set of views of each of the plurality of objects at least in part comprises orthographic views from a plurality of angles. 
     
     
       31. The method of  claim 19 , further comprising receiving a specification of the ensemble view in terms of orthographic views of the plurality of objects. 
     
     
       32. The method of  claim 31 , wherein orthographic views of the plurality of objects facilitate manipulation and accurate placement of the plurality of objects comprising the ensemble view. 
     
     
       33. The method of  claim 19 , wherein each retrieved view comprises metadata including relative object and camera location and orientation. 
     
     
       34. The method of  claim 19 , wherein each retrieved view comprises pixel metadata including relative position coordinates of a point in three-dimensional space that projects at a pixel. 
     
     
       35. The method of  claim 19 , wherein each retrieved view comprises pixel metadata including surface normal vectors relative to coordinate axes. 
     
     
       36. The method of  claim 19 , wherein each retrieved view comprises pixel metadata including texture mapping coordinates. 
     
     
       37. The method of  claim 19 , wherein each retrieved view comprises pixel metadata and wherein RGB values of a pixel encode pixel metadata. 
     
     
       38. The computer program product of  claim 20 , wherein the sets of views corresponding to the plurality of objects are independently captured, rendered, or both. 
     
     
       39. The computer program product of  claim 20 , wherein the set of views of each of the plurality of objects comprises photographs, photorealistic renderings, or both. 
     
     
       40. The computer program product of  claim 20 , wherein the set of views of each of the plurality of objects comprises perspectives that are different than the requested perspective. 
     
     
       41. The computer program product of  claim 20 , wherein camera characteristics are stored with and known for each retrieved view. 
     
     
       42. The computer program product of  claim 41 , wherein the camera characteristics include one or more of: position, orientation, rotation, angle, depth, focal length, aperture, zoom level, and lighting information. 
     
     
       43. The computer program product of  claim 20 , wherein generating the ensemble view comprises interpolating one or more pixels of the ensemble view. 
     
     
       44. The computer program product of  claim 20 , wherein the ensemble view comprises one or more of a virtual environment, a view of a three-dimensional space, and a frame of an animation or video sequence. 
     
     
       45. The computer program product of  claim 20 , wherein the ensemble view comprises a composite object. 
     
     
       46. The computer program product of  claim 20 , wherein the set of views of each of the plurality of objects at least in part comprises orthographic views. 
     
     
       47. The computer program product of  claim 20 , wherein the set of views of each of the plurality of objects at least in part comprises orthographic views from a plurality of angles. 
     
     
       48. The computer program product of  claim 20 , further comprising computer instructions for receiving a specification of the ensemble view in terms of orthographic views of the plurality of objects. 
     
     
       49. The computer program product of  claim 48 , wherein orthographic views of the plurality of objects facilitate manipulation and accurate placement of the plurality of objects comprising the ensemble view. 
     
     
       50. The computer program product of  claim 20 , wherein each retrieved view comprises metadata including relative object and camera location and orientation. 
     
     
       51. The computer program product of  claim 20 , wherein each retrieved view comprises pixel metadata including relative position coordinates of a point in three-dimensional space that projects at a pixel. 
     
     
       52. The computer program product of  claim 20 , wherein each retrieved view comprises pixel metadata including surface normal vectors relative to coordinate axes. 
     
     
       53. The computer program product of  claim 20 , wherein each retrieved view comprises pixel metadata including texture mapping coordinates. 
     
     
       54. The computer program product of  claim 20 , wherein each retrieved view comprises pixel metadata and wherein RGB values of a pixel encode pixel metadata.