US10019642B1ActiveUtilityA1
Image upsampling system, training method thereof and image upsampling method
Est. expirySep 17, 2035(~9.2 yrs left)· nominal 20-yr term from priority
G06T 3/4053G06F 18/251G06N 3/045G06T 3/40G06N 3/0464G06N 3/09G06K 9/4628G06K 9/6289G06N 3/04G06N 3/08
95
PatentIndex Score
25
Cited by
21
References
20
Claims
Abstract
An image upsampling system, a training method thereof and an image upsampling method are provided, the feature images of an image are obtained by using the convolutional network, upsampling processing is performed on the images with the muxer layer to synthesize every n×n feature images in the input signal into a feature image with the resolution amplified by n×n times, in the upsampling procedure with the muxer layer, information of respective feature images in the input signal is recorded in the generated feature image(s) without loss; and thus, every time when the image passes through a muxer layer with an upsampling multiple of n, the image resolution can be increased by n×n times.
Claims
exact text as granted — not AI-modifiedWhat is claimed is:
1. An image upsampling system, comprising:
at least one first convolutional network and at least one muxer layer that are cascaded;
wherein an signal input end of the image upsampling system is connected with a signal input end of a first convolutional network in the at least one first convolutional network, and a signal output end of the image upsampling system is connected with a signal output end of a last muxer layer in the at least one muxer layer;
a signal input end of every muxer layer in the at least one muxer layer is connected with a signal output end of a first convolutional network located in a stage prior to the muxer layer in the at least one first convolutional network, or connected with a signal output end of another muxer layer located in a stage prior to the muxer layer in the at least one muxer layer;
the first convolutional network is configured for converting an image input to its signal input end into a plurality of feature images and outputting the feature images to the signal input end of the muxer layer connected therewith;
the muxer layer is configured for synthesizing every n×n feature images in the feature images input to its signal input end into a feature image whose resolution is n×n times that of the input feature image and outputting the same; and a number of feature images input to the muxer layer is a multiple of n×n, n being an integer greater than one.
2. The image upsampling system according to claim 1 , wherein a number of muxer layers is two or three.
3. The image upsampling system according to claim 2 , further comprising:
a second convolutional network, whose signal input end is connected with a signal output end of the last muxer layer in the at least one muxer layer, and whose signal output end is connected with the signal output end of the image upsampling system;
wherein the second convolutional network is configured for optimizing picture quality of the feature images output by the muxer layer.
4. The image upsampling system according to claim 3 , wherein, the first convolutional network and the second convolutional network include at least one convolution layer composed of a plurality of filter units.
5. The image upsampling system according to claim 1 , wherein, a signal input end of each muxer layer is respectively connected with a signal output end of one corresponding first convolutional network in the at least one first convolutional network.
6. The image upsampling system according to claim 5 , further comprising:
a second convolutional network, whose signal input end is connected with a signal output end of the last muxer layer in the at least one muxer layer, and whose signal output end is connected with the signal output end of the image upsampling system;
wherein the second convolutional network is configured for optimizing picture quality of the feature images output by the muxer layer.
7. The image upsampling system according to claim 6 , wherein, the first convolutional network and the second convolutional network include at least one convolution layer composed of a plurality of filter units.
8. The image upsampling system according to claim 1 , wherein, in the case where there are provided a plurality of muxer layers, the muxer layers have a same upsampling multiple.
9. The image upsampling system according to claim 8 , further comprising:
a second convolutional network, whose signal input end is connected with a signal output end of the last muxer layer in the at least one muxer layer, and whose signal output end is connected with the signal output end of the image upsampling system;
wherein the second convolutional network is configured for optimizing picture quality of the feature images output by the muxer layer.
10. The image upsampling system according to claim 9 , wherein, the first convolutional network and the second convolutional network include at least one convolution layer composed of a plurality of filter units.
11. The image upsampling system according to claim 1 , wherein, the muxer layer has an upsampling multiple which is a prime number.
12. The image upsampling system according to claim 11 , wherein, the muxer layer has an upsampling multiple which is 2.
13. The image upsampling system according to claim 1 , wherein, the muxer layer is a self-adaptive interpolation filter.
14. The image upsampling system according to claim 1 , further comprising:
a second convolutional network, whose signal input end is connected with a signal output end of the last muxer layer in the at least one muxer layer, and whose signal output end is connected with the signal output end of the image upsampling system;
wherein the second convolutional network is configured for optimizing picture quality of the feature images output by the muxer layer.
15. The image upsampling system according to claim 14 , wherein, the first convolutional network and the second convolutional network include at least one convolution layer composed of a plurality of filter units.
16. A display device, comprising the image upsampling system according to claim 1 .
17. A training method of the image upsampling system according to claim 1 , comprising:
initializing respective parameters in the image upsampling system;
by using an original image signal as an output signal of the image upsampling system and using an image signal obtained by down-sampling the original image signal as an input signal of the image upsampling system, adjusting the respective parameters in the image upsampling system to allow the down-sampled image signal subjected to upsampling processing with the adjusted respective parameters to be the same as the original image signal.
18. The training method according to claim 17 , wherein, initializing of the respective parameters in the image upsampling system includes:
initializing weights W ij of respective filter units of respective convolution layers of the first convolutional network and the second convolutional network in the image upsampling system according to a formula below:
W
ij
=
{
1
/
(
m
)
(
i
,
j
)
are
preset
anchor
pixel
0
other
pixel
}
where m represents the number of feature images input to the filter unit; and
initializing the biases of respective filter units to 0.
19. The training method according to claim 17 , wherein, initializing of respective parameters in the image upsampling system includes:
initializing the weights W ij of respective filter units of respective convolution layers of the first convolutional network and the second convolutional network in the image upsampling system according to a formula below:
W
ij
=
W
ij
′
+
uniform
(
-
1
,
1
)
m
;
W
ij
′
=
{
1
/
(
m
)
(
i
,
j
)
are
preset
anchor
pixel
0
other
pixel
}
where m represents the number of feature images input to the filter unit, and uniform (−1, 1) represents a random number selected between (−1, 1); and
initializing the biases of respective filter units to 0.
20. A method for performing image upsampling with the image upsampling system according to claim 1 , comprising:
converting, by a first convolutional network, an input image input to the first convolutional network into a plurality of feature images and outputting the feature images;
synthesizing, by a muxer layer, every n×n feature images in the feature images input to the muxer layer into a feature image with a resolution amplified by n×n times as larger as the input feature images, and outputting the same; the number of feature images input to the muxer layer being a multiple of n×n, and n being an integer greater than 1.Cited by (0)
No later patents cite this yet.
References (0)
No backward citations on record.