Regional precipitation nowcasting system and method based on cycle-gan extension

Abstract

A regional precipitation nowcasting system based on cycle-generative adversarial network (GAN) extension includes an input unit configured to receive an input composite hybrid surface rainfall (HSR) image including precipitation information of a region of interest corresponding to a first time, a cycle-GAN configured to generate a resultant composite HSR image including precipitation information of the region of interest corresponding to a second time which comes later than the first time on the basis of the input composite HSR image using a first cycle-GAN and a second cycle-GAN which is complementary to the first cycle-GAN, and an output unit configured to output the resultant composite HSR image as a nowcasting image of the region of interest. The regional precipitation nowcasting system and method based on cycle-GAN extension can ensure robust temporal causality by applying pixel losses to a cycle-GAN.

The invention claimed is: 
     
       1. A regional precipitation nowcasting system based on cycle-generative adversarial network (GAN) extension, the system comprising:
 an input processor configured to receive an input composite hybrid surface rainfall (HSR) image including precipitation information of a region of interest corresponding to a first time; 
 a cycle-GAN configured to generate a resultant composite HSR image including precipitation information of the region of interest corresponding to a second time which is later than the first time, based on the input composite HSR image,
 wherein the cycle-GAN comprises a first cycle-GAN and a second cycle-GAN which is complementary to the first cycle-GAN, and 
 wherein the first cycle-GAN and the second cycle-GAN are configured to perform forward image mapping, which temporally goes forward, and backward image mapping, which temporally goes backward, respectively; and 
 an output processor configured to output the resultant composite HSR image as a nowcasting image of the region of interest. 
 
 
     
     
       2. The regional precipitation nowcasting system of  claim 1 , wherein the first cycle-GAN comprises:
 a forward generator configured to learn predictive mapping of a first predictive composite HSR image of the second time based on the input composite HSR image of the first time; 
 a backward generator configured to learn predictive mapping of a first cycle predictive composite HSR image of the first time based on the first predictive composite HSR image of the second time; 
 a forward discriminator configured to evaluate an accuracy of the predictive mapping of the forward generator and discriminate between the input composite HSR image and the first predictive composite HSR image; and 
 a backward discriminator configured to evaluate an accuracy of the predictive mapping of the backward generator and discriminate between the first predictive composite HSR image and the first cycle predictive composite HSR image. 
 
     
     
       3. The regional precipitation nowcasting system of  claim 2 , wherein the second cycle-GAN is configured to:
 learn predictive mapping of a complementary input composite HSR image corresponding to the input composite HSR image of the first time based on a complementary first predictive composite HSR image obtained by applying a first pixel loss function to the first predictive composite HSR image of the second time using the backward generator, wherein the complementary input composite HSR image obtained by applying a second pixel loss function to the input composite HSR image has a complementary relationship with the input composite HSR image; 
 learn predictive mapping of a second cycle predictive composite HSR image of the second time based on the complementary input composite HSR image of the first time using the forward generator; 
 evaluate the accuracy of the predictive mapping of the forward generator and discriminate between the complementary input composite HSR image and the second cycle predictive composite HSR image using the forward discriminator; and 
 evaluate the accuracy of the predictive mapping of the backward generator and discriminate between the complementary first predictive composite HSR image and the complementary input composite HSR image using the backward discriminator. 
 
     
     
       4. The regional precipitation nowcasting system of  claim 3 , wherein the first cycle-GAN is trained using a first cycle-consistency loss function and the first cycle predictive composite HSR image matches the input composite HSR image,
 wherein the forward discriminator is trained using a forward adversarial loss function to increase a probability of wrongly determining the first predictive composite HSR image as the input composite HSR image, and 
 wherein the backward discriminator is trained using a backward adversarial loss function to increase a probability of wrongly determining the first cycle predictive composite HSR image as the first predictive composite HSR image. 
 
     
     
       5. The regional precipitation nowcasting system of  claim 4 , wherein the second cycle-GAN is trained using a second cycle-consistency loss function and the second cycle predictive composite HSR image matches the complementary first predictive composite HSR image,
 wherein the forward discriminator is trained using the forward adversarial loss function to increase a probability of wrongly determining the second cycle predictive composite HSR image as the complementary input composite HSR image, and 
 wherein the backward discriminator is trained using the backward adversarial loss function to increase a probability of wrongly determining the complementary first predictive composite HSR image. 
 
     
     
       6. The regional precipitation nowcasting system of  claim 5 , wherein the cycle-GAN generates, as the resultant composite HSR image, the first predictive composite HSR image, the complementary first predictive composite HSR image, or the second cycle predictive composite HSR image when the first pixel loss function and the second pixel loss function converge on a predetermined reference pixel loss function value and the first cycle-consistency loss function and the second cycle-consistency loss function converge on a predetermined cycle-consistency loss function value. 
     
     
       7. The regional precipitation nowcasting system of  claim 2 , wherein each of the forward generator and the backward generator comprises:
 an encoder configured to reduce a size of the input composite HSR image and extract a plurality of feature maps from the input composite HSR image; 
 a plurality of squeeze-and-excitation (SE)-residual blocks configured to recalibrate the plurality of feature maps; and 
 a decoder configured to restore the size of the input composite HSR image reduced by the encoder. 
 
     
     
       8. The regional precipitation nowcasting system of  claim 7 , wherein the forward discriminator and the backward discriminator use a patch-GAN architecture that discriminates in units of patches. 
     
     
       9. A regional precipitation nowcasting method based on cycle-generative adversarial network (GAN) extension, the method comprising:
 receiving an input composite hybrid surface rainfall (HSR) image including precipitation information of a region of interest corresponding to a first time; 
 generating, by a cycle-GAN, a resultant composite HSR image including precipitation information of the region of interest corresponding to a second time which is later than the first time, based on the input composite HSR image,
 wherein the cycle-GAN comprises a first cycle-GAN and a second cycle-GAN which is complementary to the first cycle-GAN, and 
 wherein the first cycle-GAN and the second cycle-GAN are configured to perform forward image mapping, which temporally goes forward, and backward image mapping, which temporally goes backward, respectively; and 
 
 outputting the resultant composite HSR image as a nowcasting image of the region of interest. 
 
     
     
       10. The regional precipitation nowcasting method of  claim 9 , wherein the first cycle-GAN is configured to perform:
 learning predictive mapping of a first predictive composite HSR image of the second time based on the input composite HSR image of the first time using a forward generator; 
 learning predictive mapping of a first cycle predictive composite HSR image of the first time based on the first predictive composite HSR image of the first time based on the first predictive composite HSR image of the second time using a backward generator; 
 evaluating an accuracy of the predictive mapping of the forward generator and discriminating between the input composite HSR image and the first predictive composite HSR image using a forward discriminator; and 
 evaluating an accuracy of the predictive mapping of the backward generator and discriminating between the first predictive composite HSR image and the first cycle predictive composite HSR image using a backward discriminator. 
 
     
     
       11. The regional precipitation nowcasting method of  claim 10 , wherein the second cycle-GAN is configured to:
 learn predictive mapping of a complementary input composite HSR image corresponding to the input composite HSR image of the first time based on a complementary first predictive composite HSR image obtained by applying a first pixel loss function to the first predictive composite HSR image of the second time using the backward generator, wherein the complementary input composite HSR image obtained by applying a second pixel loss function to the input composite HSR image has a complementary relationship with the input composite HSR image; 
 learn predictive mapping of a second cycle predictive composite HSR image of the second time based on the complementary input composite HSR image of the first time using the forward generator; 
 evaluate an accuracy of predictive mapping of the forward generator and discriminate between the complementary input composite HSR image and the second cycle predictive composite HSR image using the forward discriminator; and 
 evaluate an accuracy of predictive mapping of the backward generator and discriminate between the complementary first predictive composite HSR image and the complementary input composite HSR image using the backward discriminator. 
 
     
     
       12. The regional precipitation nowcasting method of  claim 11 , wherein the first cycle-GAN is trained using a first cycle-consistency loss function and the first cycle predictive composite HSR image matches the input composite HSR image,
 wherein the forward discriminator is trained using a forward adversarial loss function to increase a probability of wrongly determining the first predictive composite HSR image as the input composite HSR image, and 
 wherein the backward discriminator is trained using a backward adversarial loss function to increase a probability of wrongly determining the first cycle predictive composite HSR image as the first predictive composite HSR image. 
 
     
     
       13. The regional precipitation nowcasting method of  claim 12 ,
 wherein the second cycle-GAN is trained using a second cycle-consistency loss function and the second cycle predictive composite HSR image matches the complementary first predictive composite HSR image, 
 wherein the forward discriminator is trained using the forward adversarial loss function to increase a probability of wrongly determining the second cycle predictive composite HSR image as the complementary input composite HSR image, and 
 wherein the backward discriminator is trained using the backward adversarial loss function to increase a probability of wrongly determining the complementary input composite HSR image as the complementary first predictive composite HSR image. 
 
     
     
       14. The regional precipitation nowcasting method of  claim 13 ,
 wherein the cycle-GAN generates, as the resultant composite HSR image, the first predictive composite HSR image, the complementary first predictive composite HSR image, or the second cycle predictive composite HSR image when the first pixel loss function and the second pixel loss function converge on a predetermined reference pixel loss function value and the first cycle-consistency loss functions and the second cycle-consistency loss function converge on a predetermined cycle-consistency loss function value. 
 
     
     
       15. The regional precipitation nowcasting method of  claim 10 , wherein each of the forward generator and the backward generator is configured to perform:
 reducing a size of the input composite HSR image and extracting a plurality of feature maps from the input composite HSR image using an encoder; 
 recalibrating the plurality of feature maps using a plurality of SE-residual blocks; and 
 restoring the size of the input composite HSR image reduced by the encoder using a decoder. 
 
     
     
       16. The regional precipitation nowcasting method of  claim 15 , wherein the forward discriminator and the backward discriminator discriminates in units of patches using a patch-GAN architecture.