Electronic device and controlling method of electronic device

Abstract

An electronic device and a controlling method of the electronic device is disclosed. Specifically, the electronic device according to the disclosure may identify, based on receiving a user input for expanding an outer area of a first image, a plurality of objects included in the first image, and obtain a first segmentation image including segmentation information on areas corresponding to the respective objects, obtain a second segmentation image in which an outer area of the first segmentation image is expanded based on the segmentation information, obtain a second image in which segmentation information included in the second segmentation image is converted to RGB information, obtain a third image by reflecting a feature of the first image to the second image based on the segmentation information, and provide the obtained third image.

What is claimed is: 
     
       1. An electronic device, comprising:
 a memory configured to store at least one instruction; and 
 a processor configured to execute the at least one instruction; 
 wherein the processor, based on executing the at least one instruction, is configured to:
 based on receiving a user input for expanding an outer area of a first image, identify a plurality of objects comprised in the first image and obtain a first segmentation image comprising first segmentation information on areas corresponding to respective objects; 
 obtain a second segmentation image in which an outer area of the first segmentation image is expanded based on the first segmentation information; 
 obtain a second image in which second segmentation information comprised in the second segmentation image is converted to RGB information; 
 obtain a third image by reflecting a feature of the first image to the second image based on the second segmentation information; and 
 provide the obtained third image. 
 
 
     
     
       2. The electronic device of  claim 1 , wherein the processor is configured to:
 obtain information on an outline comprised in the first image; 
 identify areas of the respective objects comprised in the first image based on information on the outline; and 
 obtain the first segmentation image by allocating classes of the respective objects according to the first segmentation information to the identified areas of the respective objects. 
 
     
     
       3. The electronic device of  claim 1 , wherein the processor is configured to:
 identify an expanding property corresponding to types of the respective objects based on information on types of the respective objects comprised in the first segmentation information; and 
 obtain the second segmentation image by expanding the outer area of the first segmentation image based on the identified expanding property for the respective objects. 
 
     
     
       4. The electronic device of  claim 1 , wherein the processor is configured to:
 divide the first segmentation image into a plurality of patches having a pre-set size; 
 expand a portion of the patches from among the plurality of patches and merge to the outer area of the first segmentation image; and 
 based on a separation line appearing at a joining surface between the expanded portion of the patches, obtain the second segmentation image by moving a location of an object on both sides of the separation line to a center point of the separation line. 
 
     
     
       5. The electronic device of  claim 1 , wherein the processor is configured to:
 identify a parameter affecting a style of the first image; 
 obtain a fourth image based on changing a parameter of the second image to the identified parameter affecting the style of the first image; 
 identify a blending property corresponding to types of the respective objects based on information on types of the respective objects comprised in the second segmentation information; and 
 obtain the third image by blending the first image and the fourth image based on blending properties of the respective objects. 
 
     
     
       6. The electronic device of  claim 1 , wherein:
 the processor is configured to:
 compare filter information on a plurality of images, stored in the memory, respectively with filter information of images in which an expansion of an outer area is assessed as natural, and 
 provide assessment information on a naturalness of expanding an outer area expansion of the plurality of images, and 
 
 the filter information comprises information on a distribution of RGB values for respective pixels of the respective images and information on a distribution of labels for respective pixels according to segmentation information on the respective images. 
 
     
     
       7. The electronic device of  claim 6 , wherein the processor is configured to provide filter information on the third image and the assessment information based on the filter information of images in which the expansion of the outer area is assessed as natural based on receiving a positive feedback from a user on the third image. 
     
     
       8. A controlling method of an electronic device, comprising:
 based on receiving a user input for expanding an outer area of a first image, identifying a plurality of objects comprised in the first image and obtaining a first segmentation image comprising first segmentation information on areas corresponding to respective objects; 
 obtaining a second segmentation image in which the outer area of the first segmentation image is expanded based on the first segmentation information; 
 obtaining a second image by converting second segmentation information comprised in the second segmentation image to RGB information; 
 obtaining a third image by reflecting a feature of the first image to the second image based on the second segmentation information; and 
 providing the obtained third image. 
 
     
     
       9. The method of  claim 8 , wherein the obtaining the first segmentation image comprises:
 obtaining information on an outline comprised in the first image; 
 identifying areas of the respective objects comprised in the first image based on information on the outline; and 
 obtaining the first segmentation image by allocating classes to the respective objects according to the first segmentation information in the identified areas of the respective objects. 
 
     
     
       10. The method of  claim 8 , wherein the obtaining the second segmentation image comprises:
 identifying an expanding property corresponding to types of the respective objects based on information on types of the respective objects comprised in the first segmentation information; and 
 obtaining the second segmentation image by expanding the outer area of the first segmentation image based on the identified expanding property for the respective objects. 
 
     
     
       11. The method of  claim 8 , wherein the obtaining the second segmentation image comprises:
 dividing the first segmentation image into a plurality of patches having a pre-set size; 
 expanding a portion of the patches from among the plurality of patches and merging to the outer area of the first segmentation image; and 
 based on a separation line appearing at a joining surface between the expanded portion of the patches, obtaining the second segmentation image by moving a location of an object on both sides of a separation line to a center point of the separation line. 
 
     
     
       12. The method of  claim 8 , wherein the obtaining the third image comprises:
 identify a parameter affecting a style of the first image; 
 obtain a fourth image based on changing a parameter of the second image to the identified parameter affecting the style of the first image; 
 identifying a blending property corresponding to types of the respective objects based on information on types of the respective objects comprised in the second segmentation information; and 
 obtaining the third image by blending the first image and the fourth image based on blending properties of the respective objects. 
 
     
     
       13. The method of  claim 8 , further comprising:
 comparing filter information on a plurality of images respectively stored in a memory of the electronic device with filter information of images in which an expansion of an outer area is assessed as natural, and providing assessment information on a naturalness of expanding an outer area expansion of the plurality of images, 
 wherein the filter information comprises information on a distribution of RGB values for respective pixels of the respective images and information on a distribution of labels for respective pixels according to segmentation information on the respective images. 
 
     
     
       14. The method of  claim 13 , wherein the providing the assessment information comprises providing, filter information on the third image and the assessment information based on the filter information of images in which the expansion of the outer area is assessed as natural, based on receiving an input of a positive feedback from a user on the third image. 
     
     
       15. A non-transitory computer readable medium comprising a program executing a controlling method of an electronic device, the method comprising:
 based on receiving a user input for expanding an outer area of a first image, identifying a plurality of objects comprised in the first image and obtaining a first segmentation image comprising first segmentation information on areas corresponding to respective objects; 
 obtaining a second segmentation image in which an outer area of the first segmentation image is expanded based on the first segmentation information; 
 obtaining a second image by converting second segmentation information comprised in the second segmentation image to RGB information; 
 obtaining a third image by reflecting a feature of the first image to the second image based on the second segmentation information; and 
 providing the obtained third image. 
 
     
     
       16. The non-transitory computer readable medium of  claim 15 , wherein the obtaining the first segmentation image comprises:
 obtaining information on an outline comprised in the first image; 
 identifying areas of the respective objects comprised in the first image based on information on the outline; and 
 obtaining the first segmentation image by allocating classes to the respective objects according to the first segmentation information in the identified areas of the respective objects. 
 
     
     
       17. The non-transitory computer readable medium of  claim 15 , wherein the obtaining the second segmentation image comprises:
 identifying an expanding property corresponding to types of the respective objects based on information on types of the respective objects comprised in the first segmentation information; and 
 obtaining the second segmentation image by expanding an outer area of the first segmentation image based on the identified expanding property for the respective objects. 
 
     
     
       18. The non-transitory computer readable medium of  claim 15 , wherein the obtaining the second segmentation image comprises:
 dividing the first segmentation image into a plurality of patches having a pre-set size; 
 expanding a portion of the patches from among the plurality of patches and merging to an outer area of the first segmentation image; and 
 based on a separation line appearing at a joining surface between the expanded portion of the patches, obtaining the second segmentation image by moving a location of an object on both sides of a separation line to a center point of the separation line.