Compositing Pairs Of Image Frames From Different Cameras Of A Mobile Device To Generate A Video Stream

Abstract

Some embodiments provide a novel method for in-conference adjustment of encoded video pictures captured by a mobile device having at least first and second cameras. The method may involve real-time modifications of composite video displays that are generated by the mobile devices involved in such a conference. Specifically, in some embodiments, the mobile devices generate composite displays that simultaneously display multiple videos captured by multiple cameras of one or more devices. In some cases, the composite displays place the videos in adjacent display areas (e.g., in adjacent windows). In other cases, the composite display is a picture-in-picture (PIP) display that includes at least two display areas that show two different videos where one of the display areas is a background main display area and the other is a foreground inset display area that overlaps the background main display area.

What is claimed is: 
     
         1 . A method, comprising:
 capturing a plurality of first image frames using a first camera of a device;   capturing a plurality of second image frames using a second camera of the device; and   compositing pairs of image frames from the plurality of first image frames and the plurality of second image frames to generate a video stream,   wherein each frame of the video stream includes image data from a first image frame and image data from a second image frame that are captured substantially contemporaneously.   
     
     
         2 . The method of  claim 1 , further comprising processing a first selected image frame from the plurality of first image frames before compositing the first selected image frame with a second selected image frame from the plurality of second image frames. 
     
     
         3 . The method of  claim 2 , wherein the processing performed on the first selected image comprises: adding elements to the first selected image, replacing elements of the first selected image, or altering elements of the first selected image. 
     
     
         4 . The method of  claim 1 , wherein the act of compositing pairs of image frames from the plurality of first image frames and the plurality of second image frames comprises generating a video frame that includes image data from a first image frame adjacent to image data from a second image frame. 
     
     
         5 . The method of  claim 1 , wherein the act of compositing pairs of image frames from the plurality of first image frames and the plurality of second image frames comprises generating a video frame that includes image data from a first image frame in a background and image data from a second image frame inset in a foreground. 
     
     
         6 . The method of  claim 5 , wherein the foreground of the generated video frame at least partially overlaps the background of the generated video frame. 
     
     
         7 . The method of  claim 1 , wherein the act of compositing pairs of image frames from the plurality of first image frames and the plurality of second image frames comprises generating a video frame that comprises a picture-in-picture (PIP) display. 
     
     
         8 . A non-transitory computer readable medium storing a computer program thereon for encoding video pictures captured by first and second cameras of a mobile device, the computer program comprising sets of instructions to cause the mobile device to:
 capture a plurality of first image frames using the first camera of the mobile device;   capture a plurality of second image frames using the second camera of the mobile device;   composite pairs of image frames from the plurality of first image frames and the plurality of second image frames to generate a video stream,   wherein each frame of the video stream includes image data from a first image frame and image data from a second image frame that are captured substantially contemporaneously.   
     
     
         9 . The non-transitory computer readable medium of  claim 8 , wherein the computer program further comprises instructions to process a first selected image frame from the plurality of first image frames before causing the mobile device to composite the first selected image frame with a second selected image frame from the plurality of second image frames. 
     
     
         10 . The non-transitory computer readable medium of  claim 9 , wherein the instructions to process the first selected image further comprise instructions to: add elements to the first selected image, replace elements of the first selected image, or alter elements of the first selected image. 
     
     
         11 . The non-transitory computer readable medium of  claim 8 , wherein the instructions to composite pairs of image frames from the plurality of first image frames and the plurality of second image frames further comprise instructions to generate a video frame that includes image data from a first image frame adjacent to image data from a second image frame. 
     
     
         12 . The non-transitory computer readable medium of  claim 8 , wherein the instructions to composite pairs of image frames from the plurality of first image frames and the plurality of second image frames further comprise instructions to generate a video frame that includes image data from a first image frame in a background and image data from a second image frame inset in a foreground. 
     
     
         13 . The non-transitory computer readable medium of  claim 12 , wherein the foreground of the generated video frame at least partially overlaps the background of the generated video frame. 
     
     
         14 . The non-transitory computer readable medium of  claim 8 , wherein the instructions to composite pairs of image frames from the plurality of first image frames and the plurality of second image frames further comprise instructions to generate a video frame that comprises a picture-in-picture (PIP) display. 
     
     
         15 . A mobile device comprising:
 first and second cameras for capturing video pictures;   a module for allowing a simultaneous capture of video pictures from the first camera to the second camera during use;   a storage for storing a computer program, the computer program comprising sets of instructions for:   capturing a plurality of first image frames using the first camera of the mobile device;   capturing a plurality of second image frames using the second camera of the mobile device;   compositing pairs of image frames from the plurality of first image frames and the plurality of second image frames to generate a video stream,   wherein each frame of the video stream includes image data from a first image frame and image data from a second image frame that are captured substantially contemporaneously.   
     
     
         16 . The mobile device of  claim 15 , wherein the computer program further comprises instructions for processing a first selected image frame from the plurality of first image frames before compositing the first selected image frame with a second selected image frame from the plurality of second image frames. 
     
     
         17 . The mobile device of  claim 16 , wherein the instructions for processing the first selected image further comprise instructions to: add elements to the first selected image, replace elements of the first selected image, or alter elements of the first selected image. 
     
     
         18 . The mobile device of  claim 15 , wherein the instructions for compositing pairs of image frames from the plurality of first image frames and the plurality of second image frames further comprise instructions for generating a video frame that includes image data from a first image frame adjacent to image data from a second image frame. 
     
     
         19 . The mobile device of  claim 15 , wherein the instructions for compositing pairs of image frames from the plurality of first image frames and the plurality of second image frames further comprise instructions for generating a video frame that includes image data from a first image frame in a background and image data from a second image frame inset in a foreground. 
     
     
         20 . The mobile device of  claim 15 , wherein the instructions for compositing pairs of image frames from the plurality of first image frames and the plurality of second image frames further comprise instructions for generating a video frame that comprises a picture-in-picture (PIP) display.