Video playback device and control method thereof

Abstract

Provided are an artificial intelligence (AI) system that mimics cognitive functions, such as cognition and judgment, of the human brain using a machine learning algorithm such as deep learning and applications thereof. More particularly, provided is a device including a memory storing least one program and a first video, a display, and at least one processor configured to display the first video on at least one portion of the display by executing the at least one program, wherein the at least one program includes instructions for: comparing an aspect ratio of the first video with an aspect ratio of an area in which the first video is to be displayed, generating a second video corresponding to the aspect ratio of the area by using the first video when the aspect ratio of the first video is different from the aspect ratio of the area, and displaying the second video in the area, wherein the generating of the second video is performed by inputting at least one frame of the first video to an AI neural network.

1 . A device comprising:
 a display;   a memory storing one or more instructions; and   at least one processor configured to execute the one or more instructions stored in the memory to cause the device to:
 obtain a first video; 
 compare an aspect ratio of the first video with an aspect ratio of a display area of the display; 
 based on the aspect ratio of the first video being different from the aspect ratio of the display area, obtain an expanded video corresponding to all frames of the first video by applying a reference frame comprised in the first video to a trained artificial intelligence (AI) model, the expanded video having an aspect ratio corresponding to the aspect ratio of the display area, wherein the trained AI model is generated by using the reference frame and at least one related frame associated with the reference frame as training data; and 
 display the expanded video in the display area. 
   
     
     
         2 . The device of  claim 1 , wherein the at least one processor is further configured to execute the one or more instructions to cause the device to identify a letterbox to be displayed in the display area based on the aspect ratio of the first video being different from the aspect ratio of the display area. 
     
     
         3 . The device of  claim 1 , wherein the at least one processor is further configured to execute the one or more instructions to cause the device to:
 extract frames comprised in the first video;   generate the training data to be used to generate the trained AI model based on the extracted frames; and   obtain the expanded video by updating the trained AI model by inputting the training data to the trained AI model.   
     
     
         4 . The device of  claim 1 , wherein the at least one processor is further configured to execute the one or more instructions to cause the device to generate the trained AI model by using the reference frame and a resized frame obtained by resizing the reference frame as the training data. 
     
     
         5 . The device of  claim 1 , wherein the at least one processor is further configured to execute the one or more instructions to cause the device to:
 extract the reference frame, at least one previous frame, and at least one next frame comprised in the first video; and   generate the trained AI model by using the reference frame, the at least one previous frame, and the at least one next frame as the training data.   
     
     
         6 . The device of  claim 1 , wherein the device further comprises a communicate interface configured to transmit the first video to a server and receive, from the server, the trained AI model generated by the server using the first video, and
 wherein the at least one processor is further configured to execute the one or more instructions to cause the device to obtain the expanded video by inputting at least one frame of the first video to the trained AI model received from the server.   
     
     
         7 . The device of  claim 1 , wherein the at least one processor is further configured to execute the one or more instructions to cause the device to generate frames of the expanded video corresponding to the all frames of the first video by training the trained AI model by inputting the reference frame, at least one previous frame, and at least one next frame, to the trained AI model. 
     
     
         8 . The device of  claim 1 , wherein the device further comprises a communicate interface, and
 wherein the at least one processor is further configured to execute the one or more instructions to cause the device to:
 detect at least one of a pattern and a color constituting the reference frame comprised in the first video; 
 search for an image related to the detected at least one of the pattern and the color by using the communicate interface; and 
 generate the trained AI model by using the reference frame and the searched image as the training data. 
   
     
     
         9 . The device of  claim 1 , wherein the at least one processor is further configured to execute the one or more instructions to cause the device to:
 identify an object comprised in the reference frame of the first video;   determine a category of the first video according to the object; and   select the trained AI model related to the category of the first video.   
     
     
         10 . The device of  claim 9 , wherein the category comprises at least one from among a scientific fiction (SF) movie, a documentary, a live performance, a 2D animation, a 3D animation, an augmented reality (AR) video and a hologram video. 
     
     
         11 . A method of displaying a video, the method comprising:
 obtaining a first video;   comparing an aspect ratio of the first video with an aspect ratio of a display area in which the first video is to be displayed;   based on the aspect ratio of the first video being different from the aspect ratio of the display area, obtaining an expanded video corresponding to all frames of the first video by applying a reference frame comprised in the first video to a trained artificial intelligence (AI) model, the expanded video having an aspect ratio corresponding to the aspect ratio of the display area, wherein the trained AI model is generated by using the reference frame and at least one related frame associated with the reference frame as training data; and   displaying the expanded video in the display area.   
     
     
         12 . The method of  claim 11 , wherein the comparing the aspect ratio of the first video with the aspect ratio of the display area comprises identifying a letterbox to be displayed in the display area based on the aspect ratio of the first video being different from the aspect ratio of the display area. 
     
     
         13 . The method of  claim 11 , wherein the obtaining of the expanded video comprises:
 extracting frames comprised in the first video;   generating the training data to be used to generate the trained AI model based on the extracted frames; and   obtaining the expanded video by updating the trained AI model by inputting the training data to the trained AI model.   
     
     
         14 . The method of  claim 11 , wherein the obtaining the expanded video comprises generating the trained AI model by using the reference frame and a resized frame obtained by resizing the reference frame as the training data. 
     
     
         15 . The method of  claim 11 , wherein the obtaining of the expanded video comprises:
 extracting the reference frame, at least one previous frame, and at least one next frame comprised in the first video; and   generating the trained AI model by using the reference frame, the at least one previous frame, and the at least one next frame as the training data.   
     
     
         16 . The method of  claim 11 , wherein the obtaining of the expanded video comprises:
 transmitting the first video to a server and receive, from the server, the trained AI model generated by the server using the first video; and   obtaining the expanded video by inputting at least one frame of the first video to the trained AI model received from the server.   
     
     
         17 . The method of  claim 11 , wherein the obtaining of the expanded video comprises generating frames of the expanded video corresponding to the all frames of the first video by training the trained AI model by inputting the reference frame, at least one previous frame, and at least one next frame, to the trained AI model. 
     
     
         18 . The method of  claim 11 , wherein the obtaining of the expanded video comprises:
 detecting at least one of a pattern and a color constituting the reference frame comprised in the first video;   searching for an image related to the detected at least one of the pattern and the color by using a communicate interface; and   generating the trained AI model by using the reference frame and the searched image as the training data.   
     
     
         19 . The method of  claim 11 , further comprising:
 identifying an object comprised in the reference frame of the first video;   determining a category of the first video according to the object; and   selecting the trained AI model related to the category of the first video.   
     
     
         20 . The method of  claim 19 , wherein the category comprises at least one from among a scientific fiction (SF) movie, a documentary, a live performance, a 2D animation, a 3D animation, an augmented reality (AR) video and a hologram video.