Electronic device for colorizing black and white image using GAN based model comprising transformer block and method for operation thereof
Abstract
In accordance with various embodiments, an electronic device for colorizing a black and white image using a Generative Adversarial Network (GAN)-based model comprising a transformer block includes a processor, wherein the processor is set to: obtain a black and white image including only first information about a luminance channel; and generate a pseudo color image including only second information about a chrominance channel by applying the black and white image to the GAN-based model, the GAN-based model includes a generator network including a plurality of transformer blocks for color conversion, a plurality of convolution layers, and a plurality of transpose convolution layers, the plurality of transformer blocks each include a Depth Wise Convolution (DWC) layer, a first Layer Normalization (LN) layer, a Window-based Multi-head Self Attention (W-MSA) layer, a second LN layer, and a Colorization Feed Forward (CFF) block. Other various embodiments are possible.
Claims
exact text as granted — not AI-modifiedWhat is claimed is:
1 . An electronic device for colorizing a black and white image using a Generative Adversarial Network (GAN)-based model comprising a transformer block, the electronic device comprising a processor,
wherein the processor is set to: obtain a black and white image including only first information about a luminance channel; and generate a pseudo color image including only second information about a chrominance channel by applying the black and white image to the GAN-based model, the GAN-based model includes a generator network including a plurality of transformer blocks for color conversion, a plurality of convolution layers, and a plurality of transpose convolution layers, the plurality of transformer blocks each include a Depth Wise Convolution (DWC) layer, a first Layer Normalization (LN) layer, a Window-based Multi-head Self Attention (W-MSA) layer, a second LN layer, and a Colorization Feed Forward (CFF) block, the W-MSA layer includes a first-group MSA and a second-group MSA, and the first-group MSA is provided with a feature map divided into first-type windows and the second-group MSA is provided with a feature map divided into second-type windows obtained by shifting the first-type windows.
2 . The electronic device of claim 1 , wherein the first-group MSA and the second-group MSA are each composed of four heads.
3 . The electronic device of claim 2 , wherein the W-MSA layer transmits a value concatenating a result value of the first-group MSA and a result value of the second-group MSA to a next layer.
4 . The electronic device of claim 3 , wherein the generator network has an encoder-decoder-based architecture that uses the plurality of transformer blocks.
5 . The electronic device of claim 4 , wherein the GAN-based model is trained using a total loss L total considering all of a pixel wise (L1) loss function L L1 , a VGG loss function L VGG , and a WGAN loss function L wgan , and
the pixel wise (L1) loss function L L1 , the VGG loss function L VGG , the WGAN loss function L wgan , and the total loss L total are calculated from the following [Equation 1],
[
Equation
1
]
L
wgan
{
L
G
=
-
E
y
~
[
D
(
y
~
,
x
)
]
L
D
=
E
y
[
D
(
y
,
x
)
]
-
E
y
~
[
D
(
y
~
,
x
)
]
+
λ
×
GP
1
L
L
1
=
J
-
G
(
I
)
1
2
L
VGG
=
φ
(
J
)
-
φ
(
G
(
I
)
)
2
2
3
L
total
=
L
wgan
+
λ
1
L
L
1
+
λ
2
L
VGG
4
where J is a color image corresponding to a ground truth, G(I) is a pseudo color image, and λ, λ 1 , an dλ 2 are weights.
6 . The electronic device of claim 5 , wherein the processor is set to generate a final color image by combining the black and white image and the pseudo color image through the GAN-based model.Cited by (0)
No later patents cite this yet.
References (0)
No backward citations on record.