Manipulator-based image reprojection

Abstract

Examples are disclosed that relate to image reprojection. One example provides a method, comprising receiving a first rendered image comprising content associated with a viewer reference frame, receiving a second rendered image comprising content associated with a manipulator reference frame, and reprojecting the first rendered image based on a head pose of a user to thereby produce a first reprojected image. The method further comprises reprojecting the second rendered image based on the head pose of the user and a pose of the manipulator to thereby produce a second reprojected image, and outputting the first reprojected image and the second reprojected image for display as a composited image.

The invention claimed is: 
     
       1. A method, comprising:
 receiving a first rendered image comprising content associated with a viewer reference frame; 
 receiving a second rendered image comprising content associated with a manipulator reference frame; 
 reprojecting the first rendered image based on a head pose of a user to thereby produce a first reprojected image; 
 reprojecting the second rendered image based on the head pose of the user, a first pose of the manipulator at a render time of the second rendered image, and a second pose of the manipulator determined later than the first pose to thereby produce a second reprojected image; and 
 outputting the first reprojected image and the second reprojected image for display as a composited image. 
 
     
     
       2. The method of  claim 1 , further comprising compositing the first reprojected image and the second reprojected image to thereby produce the composited image, wherein the reprojecting and the compositing is performed at a same device. 
     
     
       3. The method of  claim 1 , wherein a first device reprojects the first rendered image and the second rendered image, and wherein a second device composites the first reprojected image and the second reprojected image and outputs the composited image via a display. 
     
     
       4. The method of  claim 1 , further comprising reprojecting the composited image based on a most recently determined head pose of the user. 
     
     
       5. The method of  claim 1 , wherein the manipulator reference frame corresponds to one or more of a hand of the user and an input device operated by the user. 
     
     
       6. The method of  claim 1 , wherein the second rendered image comprises content associated with a first manipulator reference frame corresponding to a left manipulator and content associated with a second manipulator reference frame corresponding to a right manipulator. 
     
     
       7. The method of  claim 1 , wherein the second rendered image comprises content associated with a first manipulator reference frame corresponding to one of a left manipulator and a right manipulator, the method further comprising
 receiving a third rendered image comprising content associated with a second manipulator reference frame corresponding to the other of the left manipulator and the right manipulator; 
 reprojecting the third rendered image based on the head pose of the user and a pose of the other of the left manipulator and the right manipulator to thereby produce a third reprojected image; and 
 outputting the first reprojected image, the second reprojected image, and the third reprojected image for display as the composited image. 
 
     
     
       8. The method of  claim 1 , further comprising receiving a plurality of motion vectors representing motion in the second rendered image, and updating one or more of the plurality of motion vectors based of the pose of the manipulator. 
     
     
       9. The method of  claim 1 , wherein the first reprojected image and the second reprojected image are output for display as the composited image on an at least partially transparent display. 
     
     
       10. A computing device, comprising:
 a logic subsystem; and 
 a storage subsystem comprising instructions executable by the logic subsystem to
 receive a first rendered image comprising content associated with a viewer reference frame; 
 receive a second rendered image comprising content associated with a manipulator reference frame; 
 reproject the first rendered image based on a head pose of a user to thereby produce a first reprojected image; 
 reproject the second rendered image based on the head pose of the user, a first pose of the manipulator at a render time of the second rendered image, and a second pose of the manipulator determined later than the first pose to thereby produce a second reprojected image; and 
 output the first reprojected image and the second reprojected image for display as a composited image. 
 
 
     
     
       11. The computing device of  claim 10 , further comprising instructions executable by the logic subsystem to composite the first reprojected image and the second reprojected image to thereby produce the composited image. 
     
     
       12. The computing device of  claim 10 , wherein the second rendered image comprises content associated with a first manipulator reference frame corresponding to a left manipulator and content associated with a second manipulator reference frame corresponding to a right manipulator. 
     
     
       13. The computing device of  claim 10 , wherein the second rendered image comprises content associated with a first manipulator reference frame corresponding to one of a left manipulator and a right manipulator, further comprising instructions executable to
 receive a third rendered image comprising content associated with a second manipulator reference frame corresponding to the other of the left manipulator and the right manipulator; 
 reproject the third rendered image based on the head pose of the user and a pose of the other of the left manipulator and the right manipulator to thereby produce a third reprojected image; and 
 output the first reprojected image, the second reprojected image, and the third reprojected image for display as the composited image. 
 
     
     
       14. The computing device of  claim 10 , further comprising instructions executable to receive a plurality of motion vectors representing motion in the second rendered image, and update one or more of the plurality of motion vectors based of the pose of the manipulator. 
     
     
       15. The computing device of  claim 10 , further comprising instructions executable to reproject the composited image based on a most recently determined head pose of the user. 
     
     
       16. A method, comprising:
 receiving a first rendered image comprising mixed reality content associated with a reference frame of a user; 
 receiving a second rendered image comprising mixed reality content associated with a reference frame of a manipulator controlled by the user; 
 reprojecting the first rendered image based on a head pose of the user to thereby produce a first reprojected image; 
 reprojecting the second rendered image based on the head pose of the user, a first pose of the manipulator at a render time of the second rendered image, and a second pose of the manipulator determined later than the first pose to thereby produce a second reprojected image; and 
 outputting the first reprojected image and the second reprojected image for display as a composited image on a mixed reality display. 
 
     
     
       17. The method of  claim 16 , further comprising compositing the first reprojected image and the second reprojected image to thereby produce the composited image, wherein the reprojecting and the compositing is performed at a same device.