Ultra low latency video communication

Abstract

A video communication system comprising at least one video data transmission unit for sending or receiving video data over a data network is disclosed, as well as a corresponding video data transmission method. The transmission unit comprising an image acquiring circuitry or an image reconstruction circuitry for acquiring or reconstruction an image frame or image field, a video processing unit for processing at least part of the video data and a communication unit for sending or receiving at least part of the data. At least two of the image acquiring circuitry or image reconstruction circuitry, the video processing unit and the communication unit are arranged for simultaneously handling different parts of a same image frame or image same field of the video data.

The invention claimed is: 
     
       1. A video data communication system for ultra-low latency (ULL) video communication, the system comprising:
 at least one video data transmission unit for sending or receiving video data over a data network, the transmission unit comprising:
 an image acquiring circuitry or an image reconstruction circuitry for acquiring or reconstructing an image frame or image field, 
 a video processing unit for processing at least part of the video data, and 
 a communication unit for sending or receiving at least part of the data, 
 wherein at least two of
 (a) the image acquiring circuitry or the image reconstruction circuitry, 
 (b) the video processing unit, and 
 (c) the communication unit 
 are arranged for simultaneously handling configured to simultaneously handle a first part of an image frame and a second part of the image frame, the first part of the image frame and the second part of the image frame being different parts of a the same image frame, each of the parts first part of the image frame and the second part of the image frame being an incomplete image field, or different parts of a same image field of the video data; and 
 
 
 wherein 
 the image acquiring circuitry comprises an image divider configured for dividing the image frame of the video data into image blocks as the first and second parts of the image frame, each image block being one or more entire lines of pixels of the image frame or field, or 
 the image reconstruction circuitry comprises an image merger configured for merging image blocks as the first and second parts of the image frame, each image block being one or more entire lines of pixels of the image frame or field. 
 
     
     
       2. A video data communication system according to  claim 1 , wherein all of the image acquiring circuitry or image reconstruction circuitry, the video processing unit and the communication unit are arranged for simultaneously handling different parts of a same image frame or same image field of the video data. 
     
     
       3. A video data communication system according to  claim 1 , wherein
 the image acquiring circuitry comprises an image divider configured for dividing an image frame or image field into image blocks comprising a part of the image frame or field being one or more entire horizontal lines of pixels of the image frame or field, or 
 the image reconstruction circuitry comprises an image merger configured for merging image blocks comprising only a part of an image frame or image field being one or more lines of pixels of the image frame or field in a full image frame or image field. 
 
     
     
       4. A video data communication system according to claim  3  1, wherein the video communication system comprises an image block header generating means configured for generating an image block header for the image block for identifying an image frame or field wherefrom the image block is derived. 
     
     
       5. A video data communication system according to  claim 1 , wherein the video processing unit comprises an a video encoder or a video decoder configured for encoding or decoding image blocks comprising only a part of an image frame or image field in a sequence of image frames of the video data. 
     
     
       6. A video data communication system according to  claim 1 , wherein the communication unit is configured for sending or receiving data representative of an image block comprising only a part of an image frame or image field. 
     
     
       7. A video data communication system according to  claim 1 , wherein the at least one video data transmission unit is a video sender or a video receiver. 
     
     
       8. A video data communication system according to  claim 1 , wherein the video communication system comprises both
 a video sender comprising an image acquiring circuitry, a video processing unit and a communication unit arranged for simultaneously handling different parts of a same frame or same field of the video data, and 
 a video receiver comprising a communication unit, a video processing unit and an image reconstruction circuitry arranged for simultaneously handling different parts of a same frame or same field of the video data, 
 the system furthermore being adapted for the video sender and the video receiver simultaneously handling different parts of the same frame or same field of the video data. 
 
     
     
       9. A video data communication system according to claim  4  1, wherein the reconstruction circuitry is configured for reconstructing frames or fields taking into account image block headers of the image blocks. 
     
     
       10. A video data communication system according to  claim 1 , the video communication system being a software-implemented controller. 
     
     
       11. A video data communication system according to  claim 1 ,
 wherein the video processing unit comprises a compressing or decompressing unit configured for compressing or decompressing image blocks comprising only a part of an image frame or image field, and/or 
 wherein the video processing unit comprises an encrypting or decrypting unit configured for encrypting or decrypting image blocks comprising only a part of an image frame or image field. 
 
     
     
       12. Use of a video data communication system according to  claim 1 , in transmission of endoscopic video data and/or for combining multiple video data inputs and for providing multiple video data outputs or providing a merged data output and/or for obtaining visual delay free transport of video data. 
     
     
       13. A method for transferring video data of ultra-low latency (ULL) video communication, the video data comprising image frames or image fields of video data, the method comprising:
 (a) acquiring or reconstructing an image frame or image field, 
 (b) processing at least part of the video data, and 
 (c) sending or receiving at least part of the data,
 wherein at least two of
 (a) said acquiring or reconstructing, 
 (b) said processing, and 
 (c) said sending or receiving 
 are performed simultaneously by simultaneously handling a first part of an image frame and a second part of the image frame, the first part of the image frame and the second part of the image frame being different parts of the same image frame, the parts each of the first part of the image frame and the second part of the image frame being an incomplete image field, or different parts of the same image field of the video data; and 
   
 the method further comprising 
 (d) obtaining at least a first image block and a second image block as the first and second parts of the same image frame, each of the first image block and the second image block being one or more entire lines of pixels of the image frame or field. 
 
     
     
       14. A method for transferring video data according to  claim 13 , wherein all of said acquiring or reconstructing, processing and sending or receiving are performed simultaneously by simultaneously handling different parts of the same image frame, the parts not being a complete image field, or different parts of the same image field of the video data. 
     
     
       15. A method for transferring video data according to  claim 13 , the method comprising:
 obtaining at least a first image block and a second image block, each image block being part of a same image frame or a same image field, after obtaining the at least a first image block and during obtaining of the second image block, processing the first image block, and thereafter, before or during processing of the second image block, sending the first image block over a data network. 
 
     
     
       16. A method for transferring video data according to claim  15  13, the method further comprising, upon receiving at least a first image block and a second image block, generating an image block header for the image block for identifying an image frame or field wherefrom the image block is derived. 
     
     
       17. A method for transferring video data according to  claim 13 , the method comprising:
 receiving at least a first image block and a second image, each image block being part of a same image frame or a same image field, over a data network, 
 after receiving the first image block and before or during receiving of the second image block, processing the first image block, and thereafter 
 merging the at least first image block and the second image block, after these image blocks have been processed, into a single image frame or image field. 
 
     
     
       18. A method for transferring video data according to  claim 17 , wherein at least said merging is performed taking into account an image block header for the image block for identifying an image frame or field wherefrom the image blocks are derived. 
     
     
       19. A computer program product programmed for, when executing on a processing unit, transferring video data according to the method of  claim 13 . 
     
     
       20. A data carrier medium storing a computer program product according to  claim 19 . 
     
     
       21. A video data communication system according to claim 1, the video data communication system being a surgical video data communication system and the video data transmission unit being a surgical video data transmission unit for sending or receiving surgical video data.  
     
     
       22. Use according to claim 12, wherein the video data communication system is a surgical video data communication system.  
     
     
       23. A method for transferring surgical video data according to claim 13, wherein the method is a method for transferring surgical video data.  
     
     
       24. A surgical video data communication system for ultra-low latency (ULL) video communication, the system comprising:
 at least one surgical video data transmission unit for sending or receiving surgical video data over a data network, the transmission unit comprising:
 an image acquiring circuitry or an image reconstruction circuitry for acquiring or reconstructing an image frame, 
 a video processing unit for processing at least part of the surgical video data, and 
 a communication unit for sending or receiving at least part of the data; 
 wherein at least two of
 (a) the image acquiring circuitry or the image reconstruction circuitry, 
 (b) the video processing unit, and 
 (c) the communication unit 
 are configured to simultaneously handle a first part of an image frame and a second part of the image frame, the first part of the image frame and the second part of the image frame being different parts of the same image frame, each of the first part of the image frame and the second part of the image frame being an incomplete image field or different parts of a same image field of the video data; and 
 
   wherein   the image acquiring circuitry comprises an image divider configured for dividing the image frame of the video data into image blocks as the first and second parts of the image frame, each image block being one or more entire lines of pixels of the image frame or field, or   the image reconstruction circuitry comprises an image merger configured for merging image blocks as the first and second parts of the image frame, each image block being one or more entire lines of pixels of the image frame or field.    
     
     
       25. Use of a surgical video data communication system according to claim 24, in transmission of endoscopic video data and/or for combining multiple surgical video data inputs and for providing multiple video data outputs or providing a merged data output and/or for obtaining visual delay free transport of surgical video data.  
     
     
       26. A method for transferring surgical video data of ultra-low latency (ULL) video communication, the surgical video data comprising image frames or image fields of surgical video data, the method comprising:
 (a) acquiring or reconstructing an image frame or image field,   (b) processing at least part of the video data, and   (c) sending or receiving at least part of the data,
 wherein at least two of
 (a) said acquiring or reconstructing, 
 (b) said processing, and 
 (c) said sending or receiving 
 are performed simultaneously by simultaneously handling a first part of an image frame and a second part of the image frame, the first part of the image frame and the second part of the image frame being different parts of the same image frame, each of the first part of the image frame and the second part of the image frame being an incomplete image field or different parts of the same image field of the video data; and 
 
   the method further comprising   (d) obtaining at least a first image block and a second image block as first and second parts of the same image frame, each of the first image block and the second image block being one or more entire lines of pixels of the image frame or field.    
     
     
       27. A computer-readable medium having stored thereon executable instructions that, when executed by one or more processors, configure a computer system to perform the method of claim 26.   
     
     
       28. A method for transferring video data according to claim 13, wherein the method further comprises:
 (d) obtaining at least a first image block and a second image block as first and second parts of the same image frame, each of the first image block and the second image block being one or more entire lines of pixels of the image frame or field, and   after obtaining the first image block, and before or during obtaining of the second image block, processing the first image block;   or   (e) processing at least a first image block and a second image block as first and second parts of the same image frame, each of the first image block and the second image block being one or more entire lines of pixels of the image frame or field, and   after processing the first image block, and before or during processing of the second image block, sending the first image block over a data network;   or   (f) receiving at least a first image block and a second image block as first and second parts of the same image frame, each of the first image block and the second image block being one or more entire lines of pixels of the image frame or field, and   after receiving the first image block, and before or during receiving of the second image block, processing the first image block.    
     
     
       29. A surgical video data communication system for ultra-low latency (ULL) video communication according to claim 24, the surgical video data being surgical endoscopic video data.  
     
     
       30. A surgical video data communication system for ultra-low latency (ULL) video communication according to claim 24, wherein the video data transmission unit is configured for sending the received video data over an internet protocol network.  
     
     
       31. A surgical video data communication system for ultra-low latency (ULL) video communication according to claim 24, wherein the surgical video data is raw surgical video data.  
     
     
       32. A surgical video data communication system for ultra-low latency (ULL) video communication according to claim 24, wherein the system has a compression quality without visual loss and without visual delay.  
     
     
       33. A surgical video data communication system for ultra-low latency (ULL) video communication according to claim 24, wherein the latency of the system is less than one twentieth of the frame rate of the surgical video.  
     
     
       34. A video data communication system according to claim 4, wherein the image block header generated by the image block header generating means comprises metadata for signaling to the video processing unit that no image blocks of future image frames are to be used when processing the image block identified by the image block header.  
     
     
       35. A method for transferring video data according to claim 13, wherein (b) said processing comprises processing metadata included in an image block header to either process an image block identified by the image block header without taking into account image blocks obtained with respect to future image frames or process an image block identified by the image block header taking into account image blocks obtained with respect to future image frames.  
     
     
       36. A video data communication system according to claim 1, wherein the image reconstruction circuitry comprises an image merger configured for merging image blocks comprising only a part of an image frame or image field being one or more entire lines of pixels of the image frame or field in a full image frame or image field.  
     
     
       37. A method for transferring surgical video data of ultra-low latency (ULL) video communication, the surgical video data comprising image frames or image fields of surgical video data, the method comprising:
 (a) acquiring or reconstructing an image frame or image field,   (b) processing at least part of the video data, and   (c) sending or receiving at least part of the data,
 wherein at least two of
 (a) said acquiring or reconstructing, 
 (b) said processing, and 
 (c) said sending or receiving 
 are performed simultaneously by simultaneously handling a first part of an image frame and a second part of the image frame, the first part of the image frame and the second part of the image frame being different parts of the same image frame, each of the first part of the image frame and the second part of the image frame being an incomplete image field or different parts of the same image field of the video data; and 
 
   the method further comprising   (d) obtaining at least a first image block and a second image block as first and second parts of the same image frame, each of the first image block and the second image block being one or more entire lines of pixels of the image frame or field, and   after obtaining the first image block, and before or during obtaining of the second image block, processing the first image block;   or   (e) processing at least a first image block and a second image block as first and second parts of the same image frame, each of the first image block and the second image block being one or more entire lines of pixels of the image frame or field, and   after processing the first image block, and before or during processing of the second image block, sending the first image block over a data network;   or   (f) receiving at least a first image block and a second image block as first and second parts of the same image frame, each of the first image block and the second image block being one or more entire lines of pixels of the image frame or field, and   after receiving the first image block, and before or during receiving of the second image block, processing the first image block.    
     
     
       38. The method for transferring surgical video data of ultra-low latency (ULL) video communication according to claim 37, wherein
 the method further comprises   (e) processing at least a first image block and a second image block as first and second parts of the same image frame, each of the first image block and the second image block being one or more entire lines of pixels of the image frame or field, and   after processing the first image block, and before processing of the second image block, starting to send the first image block over a data network.    
     
     
       39. The method for transferring surgical video data of ultra-low latency (ULL) video communication according to claim 37, wherein
 the method further comprises   (e) processing at least a first image block and a second image block as first and second parts of the same image frame, each of the first image block and the second image block being one or more entire lines of pixels of the image frame or field, and   after processing the first image block, and before processing of the second image block, sending the first image block over a data network.