Adaptive media streaming method and apparatus according to decoding performance

Abstract

A method and apparatus for streaming media without installing a plug-in in a web browser are provided. When a JavaScript decoder is used to stream media without installing a plug-in, it is difficult to support decoding of high framerate and high resolution video due to the limitations of JavaScript. When streaming media using a video element of HTML5 without installing a plug-in, a compatibility issue with a conventional video capturing apparatus using RTSP/RTP and an initial delay problem due to a container of MPEG-DASH are inherent. The present disclosure presents an adaptive media streaming method and apparatus capable of performing streaming of high framerate and high resolution video without an initial delay and compatibility issues by addressing these drawbacks.

What is claimed is: 
     
         1 . An adaptive media streaming apparatus comprising:
 a receiver configured to receive media data generated by a media service apparatus using a communication protocol which supports web services;   a video web worker configured to determine whether a video codec of video data included in the media data is supported by a first video decoder embedded in a web browser of the adaptive media streaming apparatus;   a first video player configured to, in response to the video codec of the video data being unsupported by the first video decoder embedded in the web browser, decode the video data transmitted from the video web worker using a second video decoder written in a script which is supported by the web browser; and   a second video player configured to, in response to the video codec of the video data being supported by the first video decoder embedded in the web browser, decode the video data transmitted from the video web worker using the first video decoder embedded in the web browser.   
     
     
         2 . The adaptive media streaming apparatus of  claim 1 , further comprising:
 an audio web worker configured to determine whether an audio codec of audio data included in the media data is supported by a first audio decoder embedded in the web browser;   a first audio player configured to, in response to the audio codec of the audio data being unsupported by the first audio decoder embedded in the web browser, decode the audio data transmitted from the audio web worker using a second audio decoder written in the script; and   a second audio player configured to, in response to the audio codec of the audio data being supported by the first audio decoder embedded in the web browser, decode the audio data transmitted from the audio web worker using the first audio decoder embedded in the web browser.   
     
     
         3 . The adaptive media streaming apparatus of  claim 1 , wherein the script is JavaScript. 
     
     
         4 . The adaptive media streaming apparatus of  claim 2 , wherein each of the second video decoder and the second audio decoder is configured to convert native code written in C/C++ into JavaScript using a code converter. 
     
     
         5 . The adaptive media streaming apparatus of  claim 2 , wherein the first video decoder and the first audio decoder are a video tag and an audio tag, respectively, according to a HTML5 standard. 
     
     
         6 . The adaptive media streaming apparatus of  claim 1 , wherein the receiver comprises:
 a WebSocket client configured to receive the media data from the media service apparatus; and   a Real-Time Streaming Protocol (RTSP) client configured to classify the media data received from the WebSocket client into the video data or audio data.   
     
     
         7 . The adaptive media streaming apparatus of  claim 2 , wherein the video web worker is configured to transmit the video data to the first video player or the second video player according to the video codec of the video data, and
 wherein the audio web worker is configured to transmit the audio data to the first audio player or the second audio player according to the audio codec of the audio data.   
     
     
         8 . The adaptive media streaming apparatus of  claim 1 , wherein the second video player is configured to decode the video data transmitted containers, each container of the containers comprising a plurality of frames. 
     
     
         9 . The adaptive media streaming apparatus of  claim 8 , wherein the video data is packaged in the containers before the decoding by the second video player. 
     
     
         10 . The adaptive media streaming apparatus of  claim 9 , wherein the containers are packaged by a multiplexer (MUX) configured to create Moving Picture Experts Group-Dynamic Adaptive Streaming over Hypertext Transport Protocol (MPEG-DASH)-based containers. 
     
     
         11 . The adaptive media streaming apparatus of  claim 1 , further comprising a decoding time measurer configured to measure a decoding time of the first video player using a frames per second (FPS) meter. 
     
     
         12 . The adaptive media streaming apparatus of  claim 11 , wherein the video web worker is configured to adjust, through machine learning, a threshold value used for determining the decoding time received from the decoding time measurer is acceptable. 
     
     
         13 . The adaptive media streaming apparatus of  claim 12 , wherein the video web worker is further configured to compare the decoding time received from the decoding time measurer with the threshold value, and transmit the video data to the second video player in response to the decoding time exceeding the threshold value and the video codec of the video data being supported by the web browser. 
     
     
         14 . An adaptive media streaming apparatus comprising:
 a receiver configured to receive media data generated by a media service apparatus using a communication protocol which supports web services;   a video web worker configured to determine whether a video codec of video data included in the media data is supported by a first video decoder embedded in a web browser of the adaptive media streaming apparatus;   a first video player configured to decode the video data transmitted from the video web worker using a second video decoder written in a script which is supported by the web browser; and   a second video player configured to, in response to the video codec of the video data being supported by the first video decoder embedded in the web browser and in response to a determination that a decoding delay over a delay threshold has occurred in the first video player, decode the video data using the first video decoder embedded in the web browser.   
     
     
         15 . The adaptive media streaming apparatus of  claim 14 , further comprising a decoding time measurer configured to measure a frame rate of the video data being decoded by the first video player using a frames per second (FPS) meter to determine whether the decoding delay of the first video player is over the delay threshold. 
     
     
         16 . The adaptive media streaming apparatus of  claim 14 , further comprising:
 an audio web worker configured to determine whether an audio codec of audio data included in the media data is supported by a first audio decoder embedded in the web browser;   a first audio player configured to, in response to the audio codec of the audio data being unsupported by the first audio decoder embedded in the web browser, decode the audio data transmitted from the audio web worker using a second audio decoder written in the script; and   a second audio player configured to, in response to the audio codec of the audio data being supported by the first audio decoder embedded in the web browser, decode the audio data transmitted from the audio web worker using the first audio decoder embedded in the web browser.   
     
     
         17 . The adaptive media streaming apparatus of  claim 14 , wherein the script is JavaScript. 
     
     
         18 . The adaptive media streaming apparatus of  claim 16 , wherein the first video decoder and the first audio decoder are a video tag and an audio tag, respectively, according to a HTML5 standard. 
     
     
         19 . The adaptive media streaming apparatus of  claim 16 , wherein the video web worker is further configured to transmit the video data to the first video player or the second video player according to the video codec of the video data and whether the decoding delay over the delay threshold has occurred, and
 wherein the audio web worker is further configured to transmit the audio data to the first audio player or the second audio player according to the audio codec of the audio data.   
     
     
         20 . A media service apparatus for transmitting real-time live video or stored video to a media streaming apparatus, the media service apparatus comprising:
 a module storage configured to store a script module which is supported by a web browser of the media streaming apparatus and is required for playing the real-time live video or the stored video on the web browser;   a module transmitter configured to transmit the script module to the media streaming apparatus in response to establishing a connection with the media streaming apparatus;   a packetizer configured to packetize the real-time live video or the stored video to generate a transmission packet; and   a web server configured to establish a communication session with the media streaming apparatus and transmit the transmission packet to the media streaming apparatus in response to receiving a streaming request from the media streaming apparatus,   wherein the script module comprises a process of determining which video decoder of a plurality of video decoders being executed in the media streaming apparatus decodes the transmission packet.   
     
     
         21 . The media service apparatus of  claim 20 , wherein the script module is code written in JavaScript. 
     
     
         22 . The media service apparatus of  claim 20 , wherein the plurality of video decoders include a first video decoder written in a script which is supported by the web browser, and a second video decoder embedded in the web browser. 
     
     
         23 . The media service apparatus of  claim 22 , wherein the process further comprises measuring a decoding time of the first video decoder using a frames per second (FPS) meter. 
     
     
         24 . The media service apparatus of  claim 23 , wherein the process further comprises adjusting, through machine learning, a threshold value for determining whether the decoding time is acceptable. 
     
     
         25 . The media service apparatus of  claim 24 , wherein the process further comprises comparing the decoding time with the threshold value and switching to the second video decoder for decoding the transmission packet in response to the decoding time exceeding the threshold value and a video codec format of the transmission packet being supported by the web browser.