Method and apparatus for generating immersive-media, mobile terminal using the same

Abstract

A method and apparatus for generating immersive media and a mobile terminal using the method and apparatus is disclosed. An apparatus for generating immersive media comprises an image generation unit generating image data based on image signals, a sensory effect data generation unit generating sensory effect data by obtaining information related to a sensory effect for providing a sensory effect in conjunction with the image data, and an immersive media generation unit generating immersive media based on the sensory effect data.

What is claimed is: 
     
         1 . An apparatus for generating immersive media, comprising:
 an image generation unit generating image data based on image signals;   a sensory effect data generation unit generating sensory effect data by obtaining information related to a sensory effect for providing a sensory effect in conjunction with the image data; and   an immersive media generation unit generating immersive media based on the sensory effect data,   where the sensory effect data generation unit generates the sensory effect data including at least one of sensory effect data related to an image obtained from the image signals, sensory effect data based on annotation information obtained from a user, sensor-related sensory effect data obtained from sensing signals detected by a sensor module, sensory effect data based on environment information obtained from the web, and sensory effect data related to sound obtained from sound signals.   
     
     
         2 . The apparatus of  claim 1 , wherein the sensory effect data generation unit comprises an image sensory effect extraction unit generating sensory effect data related to the image compliant with the image context based on the image signals;
 a sound sensory effect extraction unit extracting a sound characteristic pattern based on the sound signals and generating sensory effect data related to the sound corresponding to the detected sound characteristic pattern;   an annotation generation unit generating sensory effect data based on the annotation information based on the user's voice command;   a sensor sensory effect extraction unit obtaining sensing signals detected by the sensor module and based on the obtained sensing signals, generating the sensor-related sensory effect data;   an environment information generation unit obtaining environment information from the web through wireless communication and based on the obtained environment information, generating the environment information based sensory effect data; and   a metadata conversion unit converting the sensory effect data including at least one of the image-related sensory effect data, sound-related sensory effect data, annotation-related sensory effect data, sensor-related sensory effect data, and environment information-related sensory effect data into metadata.   
     
     
         3 . The apparatus of  claim 2 , wherein the annotation generation unit comprises a voice recognition unit obtaining the voice command from the user and recognizing the voice command; and an annotation information extraction unit determining whether the voice command is an annotation command existing in a predefined annotation table and if the voice command corresponds to the annotation command, generating the annotation information-based sensory effect data based on the voice command. 
     
     
         4 . The apparatus of  claim 2 , wherein the environment information generation unit obtains the environment information including at least one piece of weather information from among temperature, humidity, wind, and the amount of rainfall corresponding to a predetermined time and location from the web. 
     
     
         5 . The apparatus of  claim 2 , wherein the sensor module includes at least one of an orientation detection sensor, gravity sensor, accelerometer, illumination sensor, GPS sensor, ultraviolet sensor, and temperature sensor. 
     
     
         6 . The apparatus of  claim 1 , wherein the image generation unit generates the image data by encoding the image signals obtained by a camera module during image capture and the sound signals obtained by a microphone module. 
     
     
         7 . A mobile terminal, comprising:
 a camera module obtaining image signals by capturing images;   an image generation unit generating image data based on image signals;   a sensory effect data generation unit generating sensory effect data by obtaining information related to a sensory effect for providing a sensory effect in conjunction with the image data; and   an immersive media generation unit generating immersive media based on the sensory effect data,   where the sensory effect data generation unit generates the sensory effect data including at least one of sensory effect data related to an image obtained from the image signals, sensory effect data based on annotation information obtained from a user, sensor-related sensory effect data obtained from sensing signals detected by a sensor module, sensory effect data based on environment information obtained from the web, and sensory effect data related to sound obtained from sound signals.   
     
     
         8 . A method for generating immersive media, comprising:
 generating image data based on image signals;   generating sensory effect data by obtaining information related to a sensory effect for providing a sensory effect in conjunction with the image data; and   generating immersive media based on the sensory effect data,   where the sensory effect data includes at least one of sensory effect data related to an image obtained from the image signals, sensory effect data based on annotation information obtained from a user, sensor-related sensory effect data obtained from sensing signals detected by a sensor module, sensory effect data based on environment information obtained from the web, and sensory effect data related to sound obtained from sound signals.   
     
     
         9 . The method of  claim 8 , wherein the annotation information-based sensory effect data is generated based on the voice command if the voice command is obtained from the user and recognized; the recognized voice command is determined to correspond to an annotation command existing in a predefined annotation table; and the voice command is the annotation command. 
     
     
         10 . The method of  claim 8 , wherein the environment information-based sensory effect data is generated based on environment information obtained from the web through wireless communication. 
     
     
         11 . The method of  claim 10 , wherein the environment information includes at least one piece of weather information from among temperature, humidity, wind, and the amount of rainfall corresponding to a predetermined time and location. 
     
     
         12 . The method of  claim 8 , wherein the sensor module includes at least one of an orientation detection sensor, gravity sensor, accelerometer, illumination sensor, GPS sensor, ultraviolet sensor, and temperature sensor. 
     
     
         13 . The method of  claim 8 , wherein the image-related sensory effect data includes a sensory effect compliant with image context based on the image signals. 
     
     
         14 . The method of  claim 8 , wherein the sound-related sensory effect data includes a sound sensory effect data corresponding to a sound characteristic pattern detected from the sound signals. 
     
     
         15 . The method of  claim 8 , wherein the image data is generated by encoding the image signals obtained by a camera module during image capture and the sound signals obtained by a microphone module. 
     
     
         16 . A method for generating immersive media in a mobile terminal, comprising:
 capturing images by using a camera module;   generating image data based on image signals obtained by the camera module;   generating sensory effect data by obtaining sensor effect-related information for providing a sensory effect in conjunction with the image data; and   generating immersive media based on the image data and the sensor effect data,   where the sensor effect data includes at least one of sensory effect data related to images obtained from the image signals, sensory effect data based on annotation information obtained from a user, sensor-related sensory effect data obtained from sensing signals detected by a sensor module, sensory effect data based on environment information obtained from the web, and sensory effect data related to sound obtained from sound signals.