Generation apparatus, generation method, reproduction apparatus, and reproduction method

Abstract

There is provided a generation apparatus, a generation method, a reproduction apparatus, and a reproduction method by which a display image generated by use of an omnidirectional image is made approximately uniform in image quality in all visual line directions. A down-conversion part down-converts an omnidirectional image. A perspective projection part generates multiple images by projecting on multiple two-dimensional planes the omnidirectional image mapped onto a 3D model. This disclosure may be applied, for example, to generation apparatuses that generate an omnidirectional image from captured images in six directions so as to generate low- and high-resolution streams of the omnidirectional image.

The invention claimed is: 
     
       1. A generation apparatus comprising:
 a down-conversion part configured to down-convert an omnidirectional image; 
 a projection part configured to generate a plurality of images by projecting on a plurality of two-dimensional planes the omnidirectional image mapped onto a 3D model; 
 a low-resolution encoding part configured to generate a low-resolution stream by encoding the omnidirectional image down-converted by the down-conversion part; 
 a high-resolution encoding part configured to generate high-resolution streams by encoding each of the plurality of images generated by the projection part; and 
 a transmission part configured to transmit the low-resolution stream generated by the low-resolution encoding part and the high-resolution streams generated by the high-resolution encoding part, 
 wherein images on adjacent two of the two-dimensional planes partially overlap with each other. 
 
     
     
       2. The generation apparatus according to  claim 1 , wherein the transmission part transmits two-dimensional plane information indicative of the positions of the plurality of two-dimensional planes. 
     
     
       3. The generation apparatus according to  claim 1 , wherein a normal line passing through the center of each of the plurality of two-dimensional planes is a line that passes through the midpoint of each of the sides of a cube and through the center of the cube, and a line that passes through the center of each of the faces of the cube and through the center of the cube. 
     
     
       4. A generation apparatus comprising:
 a down-conversion part configured to down-convert an omnidirectional image; and 
 a projection part configured to generate a plurality of images by projecting on a plurality of two-dimensional planes the omnidirectional image mapped onto a 3D model, 
 wherein a normal line passing through the center of each of the plurality of two-dimensional planes is a line that passes through the midpoint of each of the sides of a cube and through the center of the cube. 
 
     
     
       5. A generation apparatus comprising:
 a down-conversion part configured to down-convert an omnidirectional image; and 
 a projection part configured to generate a plurality of images by projecting on a plurality of two-dimensional planes the omnidirectional image mapped onto a 3D model, wherein 
 the omnidirectional image is constituted by an omnidirectional image from a left-eye viewpoint and an omnidirectional image from a right-eye viewpoint, 
 the down-conversion part generates a low-resolution packed image by packing the down-converted omnidirectional images from the left-eye and right-eye viewpoints, and 
 the projection part generates a high-resolution packed image by packing, on each of the plurality of two-dimensional planes, the plurality of images from the left-eye and right-eye viewpoints generated by projecting on the plurality of two-dimensional planes each of the omnidirectional images from the left-eye and right-eye viewpoints mapped onto the 3D model. 
 
     
     
       6. A generation method comprising:
 causing a generation apparatus to down-convert an omnidirectional image; 
 causing the generation apparatus to generate a plurality of images by projecting on a plurality of two-dimensional planes the omnidirectional image mapped onto a 3D model; 
 causing the generation apparatus to generate a low-resolution stream by encoding the down-converted omnidirectional image; 
 causing the generation apparatus to generate high-resolution streams by encoding each of the generated plurality of images; and 
 causing the generation apparatus to transmit the generated low-resolution stream and the generated high-resolution streams, 
 wherein images on adjacent two of the two-dimensional planes partially overlap with each other. 
 
     
     
       7. A reproduction apparatus comprising:
 an acceptance part configured to accept at least one of a plurality of images generated by projecting on a plurality of two-dimensional planes an omnidirectional image mapped onto a 3D model, and the omnidirectional image having been down-converted; 
 a rendering part configured to generate a display image on a basis of at least either the image accepted by the acceptance part or the down-converted omnidirectional image; 
 a low-resolution decoding part configured to decode a low-resolution stream generated by encoding the down-converted omnidirectional image so as to generate the down-converted omnidirectional image; and 
 a high-resolution decoding part configured to decode a high-resolution stream generated by encoding at least one of the plurality of images so as to generate the image, 
 wherein the acceptance part accepts the low-resolution stream and the high-resolution stream, and 
 wherein images on adjacent two of the two-dimensional planes partially overlap with each other. 
 
     
     
       8. The reproduction apparatus according to  claim 7 , further comprising:
 a selection part configured to select, from the plurality of images, the image accepted by the acceptance part on a basis of the visual line direction of a viewer. 
 
     
     
       9. The reproduction apparatus according to  claim 8 , wherein the acceptance part accepts two-dimensional plane information indicative of the positions of the plurality of two-dimensional planes, and
 the selection part selects the image on a basis of the two-dimensional plane information and the visual line direction. 
 
     
     
       10. The reproduction apparatus according to  claim 7 , further comprising:
 a mapping processing part configured to generate a 3D model image by mapping the image onto the two-dimensional planes and by mapping the down-converted omnidirectional image onto the 3D model; 
 wherein the rendering part generates the display image by projecting on the visual field range of a viewer the 3D model image generated by the mapping processing part. 
 
     
     
       11. The reproduction apparatus according to  claim 7 , wherein a normal line passing through the center of each of the plurality of two-dimensional planes is a line that passes through the midpoint of each of the sides of a cube and through the center of the cube, and a line that passes through the center of each of the faces of the cube and through the center of the cube. 
     
     
       12. A reproduction apparatus comprising:
 an acceptance part configured to accept at least one of a plurality of images generated by projecting on a plurality of two-dimensional planes an omnidirectional image mapped onto a 3D model, and the omnidirectional image having been down-converted; and 
 a rendering part configured to generate a display image on a basis of at least either the image accepted by the acceptance part or the down-converted omnidirectional image, 
 wherein a normal line passing through the center of each of the plurality of two-dimensional planes is a line that planes passes through the midpoint of each of the sides of a cube and through the center of the cube. 
 
     
     
       13. A reproduction apparatus comprising:
 an acceptance part configured to accept at least one of a plurality of images generated by projecting on a plurality of two-dimensional planes an omnidirectional image mapped onto a 3D model, and the omnidirectional image having been down-converted; and 
 a rendering part configured to generate a display image on a basis of at least either the image accepted by the acceptance part or the down-converted omnidirectional image, wherein 
 the down-converted omnidirectional image is a packed image in which the down-converted omnidirectional image from a left-eye viewpoint and the down-converted omnidirectional image from a right-eye viewpoint are packed, and 
 the plurality of images are packed images generated by projecting on each of the two-dimensional planes each of the omnidirectional images from the left-eye and right-eye viewpoints mapped onto the 3D model. 
 
     
     
       14. The reproduction apparatus according to  claim 13 , wherein the rendering part divides by viewpoint the image accepted by the acceptance part and the down-converted omnidirectional image, the rendering part further generating the display image on a basis of at least either the image or the down-converted omnidirectional image having been divided by viewpoint. 
     
     
       15. A reproduction method comprising:
 causing a reproduction apparatus to accept at least one of a plurality of images generated by projecting on a plurality of two-dimensional planes an omnidirectional image mapped onto a 3D model, and the omnidirectional image having been down-converted; 
 causing the reproduction apparatus to generate a display image on a basis of at least either the accepted at least one of the plurality of images or the down-converted omnidirectional image; 
 causing the reproduction apparatus to decode a low-resolution stream generated by encoding the down-converted omnidirectional image so as to generate the down-converted omnidirectional image; and 
 causing the reproduction apparatus to decode a high-resolution stream generated by encoding at least one of the plurality of images so as to generate the image, 
 wherein the reproduction apparatus is further caused to accept the low-resolution stream and the high-resolution stream, and 
 wherein images on adjacent two of the two-dimensional planes partially overlap with each other.