US10412361B1ActiveUtility

Generated stereoscopic video using zenith and nadir view perspectives

84
Assignee: NVIDIA CORPPriority: Jul 16, 2018Filed: Jul 16, 2018Granted: Sep 10, 2019
Est. expiryJul 16, 2038(~12 yrs left)· nominal 20-yr term from priority
Inventors:David Cook
H04N 13/243H04N 13/194H04N 13/128H04N 13/106H04N 13/117H04N 13/344H04N 13/156H04N 13/361H04N 2013/0088
84
PatentIndex Score
3
Cited by
17
References
21
Claims

Abstract

The disclosure is directed to a method to generate a generated stereoscopic video image stream of a zenith or nadir view perspective of a scene utilizing a blending of stereoscopic and monoscopic view perspectives. In another aspect, a system is disclosed for generating a zenith or nadir view perspective utilizing a relative user view orientation. In another aspect, an apparatus is disclosed capable to generate a zenith or nadir view perspective utilizing a detected user view orientation, and display a generated stereoscopic video image stream of the view perspective.

Claims

exact text as granted — not AI-modified
What is claimed is: 
     
       1. A method generating user view perspectives of scene images captured by multiple cameras, comprising:
 transforming a received user view orientation to a relative user view orientation, relative to said multiple cameras; 
 selecting a first video image stream and a proximate second video image stream, from said scene images, that proximately represent said relative user view orientation, wherein at least one of said video image streams is a zenith or nadir user view perspective of said scene images; and 
 stitching together, to generate a generated stereoscopic video image stream for said user view perspective, said first video image stream and said second video image stream, utilizing a proportionate blending of a monoscopic view perspective and a stereoscopic view perspective, wherein said proportionate blending is proportionate to said relative user view orientation and said first video image stream orientation and said second video image stream orientation. 
 
     
     
       2. The method as recited in  claim 1 , further comprising:
 identifying a set of video image streams of said scene images from which said first video image stream and said second video image stream are selected; and 
 communicating said generated stereoscopic video image stream to a user display device. 
 
     
     
       3. The method as recited in  claim 1 , wherein said first video image stream and said second video image stream are captured at substantially a same time. 
     
     
       4. The method as recited in  claim 1 , wherein said received user view orientation can be measured directionally as one or more of up, down, left, right, rotate, tilt, shift, and depth. 
     
     
       5. The method as recited in  claim 1 , wherein said first video image stream's pixels and said second video image stream's pixels are reversed in relative position relative to said relative user view orientation, and said first video image stream and said second video image stream assignment to a left and right eye perspective are reversed, to generate said generated stereoscopic video image stream 180° relative to a user view perspective generated from received first video image stream and second video image stream. 
     
     
       6. The method as recited in  claim 1 , wherein said stitching includes reducing disparity between said first video image stream and said second video image stream utilizing said relative user view orientation, where said relative user view orientation approaches one of said video image streams' in-line view direction. 
     
     
       7. The method as recited in  claim 1 , wherein said stitching includes morphing said first video image stream and said second video image stream to one of said video image streams, when said relative user view orientation is closer than 35% of a distance between said first video image stream and second video image stream view direction to one of said video image streams' in-line view direction. 
     
     
       8. The method as recited in  claim 1 , wherein said stitching includes increasing disparity between said first video image stream and said second video image stream utilizing said user view orientation, whereby said user view orientation approaches a mid-point between said first video image stream's in-line view direction and said second video image stream's in-line view direction. 
     
     
       9. The method as recited in  claim 1 , wherein said stitching includes increasing a stereoscopic view perspective of said first video image stream and said second video image stream, when said user view orientation is equal to or greater than 35%, and equal to or less than 65%, of a distance between said first video image stream's view direction and said second video image stream's view direction. 
     
     
       10. The method as recited in  claim 1 , wherein one of said first video image stream and second video image stream is adjusted to substantially align said one of video image stream with other of said video image stream, where said adjusted includes at least one of rotation, tilt, scale, and shift. 
     
     
       11. The method as recited in  claim 1 , wherein said first video image stream and said second video image stream are adjusted to substantially align said first video image stream and second video image stream with each other, where said adjusted includes at least one of rotation, tilt, scale, and shift. 
     
     
       12. A video processing system operable to generate user view perspectives of scene images captured by multiple cameras, comprising:
 an orientation device, operable to detect a user view orientation; 
 a receiver, operable to receive a set of video image streams, and corresponding video image orientation parameters, of said scene images; 
 a processor, operable to determine a relative user view orientation utilizing said user view orientation and said video image orientation parameters, operable to select a first video image stream and a second video image stream from said received set of video image streams, wherein at least one of said first video image stream and second video image stream is a zenith or nadir view direction of said scene images and said first video image stream and second video image stream are captured proximate to each other, and operable to adjust at least one of said first video image stream and second video image stream utilizing said relative user view orientation, wherein said adjust includes proportionately blending monoscopic and stereoscopic user view perspectives; 
 a stitcher, operable to combine said first video image stream and said second video image stream to generate a generated stereoscopic video image stream; and 
 a communicator, operable to transmit said generated stereoscopic video image stream, as said user view perspective, to at least one of a network, storage device, and display device. 
 
     
     
       13. The video processing system as recited in  claim 12 , wherein said generated stereoscopic video image stream is a monoscopic view when said relative user view orientation is substantially aligned with one of said first video image stream's in-line view direction or second video image stream's in-line view direction. 
     
     
       14. The video processing system as recited in  claim 12 , wherein said generated stereoscopic video image stream is a stereoscopic view when said relative user view orientation is substantially aligned at a mid-point between said first video image stream's in-line view direction and second video image stream's in-line view direction. 
     
     
       15. The video processing system as recited in  claim 12 , wherein said generated stereoscopic video image stream is a blended view when said relative user view orientation is between one of said first video image stream's in-line view direction or second video image stream's in-line view direction, and a mid-point between said first video image stream's in-line view direction and second video image stream's in-line view direction. 
     
     
       16. The video processing system as recited in  claim 12 , wherein said processor is operable to reverse order of pixels of said first video image stream and second video image stream, and change said first video image stream's and second video image stream's assignments as to a left eye and right eye view perspective, to generate a generated stereoscopic video image stream user view perspective 180° relative to a user view perspective generated from received first video image stream and second video image stream. 
     
     
       17. The video processing system as recited in  claim 12 , wherein said processor is operable to substantially generate an alignment of said first video image stream and second video image stream, where said alignment includes at least one of rotation, tilt, scale, and shift. 
     
     
       18. A video processing computing apparatus to generate user view perspectives of scene images captured by a multi-camera apparatus, comprising:
 a receiver capable of receiving at least a first video image stream and a proximate second video image stream, wherein at least one of said first video image stream and said second video image stream is a zenith or nadir view perspective of said scene images, and receiving a user view orientation; 
 a storage device, communicatively coupled to said receiver, capable of storing at least said first video image stream and said second video image stream, storing an adjusted first video image stream and an adjusted second video image stream, storing a generated stereoscopic video image stream, and storing said user view orientation; 
 a processor, communicatively coupled to said storage device and said receiver, capable of generating said adjusted first video image stream and said adjusted second video image stream utilizing said user view orientation, wherein said adjusted includes proportionally blending a monoscopic and stereoscopic view perspective; and 
 a stitcher, communicatively coupled to said processor and said storage device, capable of generating a generated stereoscopic video image stream utilizing said adjusted first video image stream and said adjusted second video image stream. 
 
     
     
       19. The video processing computing apparatus recited in  claim 18 , wherein said processor is capable of rotating, tilting, and scaling said first video image stream and said second video image stream. 
     
     
       20. The video processing computing apparatus recited in  claim 18 , further comprising a communicator, communicatively coupled to said processor, storage device, and stitcher, capable of sending said generated stereoscopic video image stream to a network, a display device, said storage device, and a separate storage device. 
     
     
       21. The video processing computing apparatus recited in  claim 20 , further comprising:
 the display device, communicatively coupled to said communicator, capable of displaying said generated stereoscopic video image stream; and 
 a user orientation device, communicatively coupled to said communicator, capable of detecting and communicating said user view orientation.

Cited by (0)

No later patents cite this yet.

References (0)

No backward citations on record.