Coding and decoding of interleaved image data

Abstract

Sampled data is packaged in checkerboard format for encoding and decoding. The sampled data may be quincunx sampled multi-image video data (e.g., 3D video or a multi-program stream), and the data may also be divided into sub-images of each image which are then multiplexed, or interleaved, in frames of a video stream to be encoded and then decoded using a standardized video encoder. A system for viewing may utilize a standard video decoder and a formatting device that de-interleaves the decoded sub-images of each frame reformats the images for a display device. A 3D video may be encoded using a most advantageous interleaving format such that a preferred quality and compression ratio is reached. In one embodiment, the invention includes a display device that accepts data in multiple formats.

What is claimed: 
     
       1. A device to encode images, the device comprising:
 a sub-sampler configured to sub-sample a first image and a second image into a first sampled image data and a second sampled image data respectively via vertical sampling; 
 a formatter configured to separate each of the first sampled image data and the second sampled image data into a plurality of sub-images, wherein each of the sub-images comprises image data of the corresponding sampled image data selected via a pattern, and package at least the plurality of sub-images together into a single image frame of a video stream, wherein the packaging comprises tiling the sub-images together according to an arrangement indicating how the sub-images are tiled together, and wherein the arrangement indicates that the plurality of sub-images corresponding to the first sampled image data are tiled into a first rectangular block of the image frame, and the plurality of sub-images corresponding to the second sampled image data are tiled into a second rectangular block of the image frame, and wherein the first rectangular block is positioned in top-bottom relation with the second rectangular block in the image frame; and 
 an encoder configured to encode the single image frame and encode an identifier of the arrangement. 
 
     
     
       2. The device according to  claim 1 , wherein the identifier is a code that is placed in side information of an encoded patterned block and/or that is placed in the image frame. 
     
     
       3. The device according to  claim 1 , wherein the first image and the second image include any one or more of multiple images from multiple program streams, multiple angles of a same scene, or multiple sets of 3D views of a same or different scene, video game, or program. 
     
     
       4. A device to decode an encoded video signal with a processor, the device comprising:
 a decoder configured to decode the encoded video signal comprising more than one image per frame in the video signal and generate a decoded video signal; 
 format converter configured to de-interleave groups of video data interleaved in a frame format in the decoded video signal, wherein the groups of video data comprise one or more groups of video data from a first image and one or more groups of video data from a second image, wherein de-interleaving the groups of video data comprises de-interleaving the groups of video data from multiple interleaving formats according to an identifier of an arrangement indicating how the groups of video data are tiled together, and wherein the arrangement indicates that the one or more groups of video data from the first image are tiled into a first rectangular block of an image frame, and the one or more groups of video data from the second image are tiled into a second rectangular block of the frame, and wherein the first rectangular block is positioned in top-bottom relation with the second rectangular block in the image frame; and 
 an up-converter configured to up-convert the one or more groups of video data from the first image and the one or more groups of video data from the second image into the first image and the second image respectively according to vertical sampling. 
 
     
     
       5. The device according to  claim 4 , wherein the identifier is a code that is placed in side information of an encoded patterned block and/or that is placed in the frame. 
     
     
       6. The device according to  claim 4 , wherein the first image and the second image include any one or more of multiple images from multiple program streams, multiple angles of a same scene, or multiple sets of 3D views of a same or different scene, video game, or program.