Method and computing device for determining optimal parameter

Abstract

Provided are a method and computing device for determining an optimal parameter set. The method includes receiving an inference model, a dataset, and a constraint, configuring a set of compression methods and a set of parameters, applying a first compression method and a first parameter related to the first compression method to the inference model through a compression pipeline, determining whether a compressed inference model is generated from the inference model through the compression pipeline, when it is determined that the compressed inference model is not generated, applying a second compression method, and a second parameter to the inference model, following the first compression method, when it is determined that the compressed inference model is generated, transmitting the compressed inference model to the target device, and determining an optimal set of parameters on the basis of the performance of the compressed inference model received from the target device.

What is claimed is: 
     
         1 . A method of determining an optimal parameter set that is performed by a computing device including at least one processor, the method comprising:
 receiving an inference model, a dataset, and a constraint;   configuring a set of compression methods to be applied to the inference model and a set of parameters for the set of compression methods on the basis of the constraint;   applying a first compression method included in the set of compression methods and a first parameter related to the first compression method to the inference model, through a compression pipeline;   determining whether a compressed inference model is generated from the inference model through the compression pipeline;   when it is determined that the compressed inference model is not generated, applying a second compression method included in the set of compression methods, and a second parameter among the set of parameters to the inference model, following the first compression method, through the compression pipeline;   when it is determined that the compressed inference model is generated, transmitting the compressed inference model to the target device; and   determining an optimal set of parameters on the basis of the performance of the compressed inference model received from the target device,   wherein the performance of the compressed inference model is measured by the target device using the dataset.   
     
     
         2 . The method of  claim 1 , wherein the configuring the set of compression methods and the set of parameters includes:
 selecting the first compression method and the first parameter for the first compression method;   determining whether to further select a compression method and a parameter on the basis of the constraint and the first compression method; and   when it is determined to further select a compression method and a parameter, selecting the second compression method and the second parameter for the second compression method on the basis of the first compression method.   
     
     
         3 . The method of  claim 1 , wherein when the performance of the compressed inference model satisfies the constraint, the selected set of parameters is determined as the optimal parameter set, and
 wherein when the performance of the compressed inference model does not satisfy the constraint, the configuring the set of compression methods and the set of parameters, the applying the first compression method and the first parameter to the inference model, the determining whether the compressed inference model is generated, the applying the second compression method and the second parameter to the inference model, the transmitting the compressed inference model to the target device, and the determining the optimal parameter set are repeatedly performed.   
     
     
         4 . The method of  claim 1 , wherein the constraint includes a value of at least one item among device, accuracy, model size, latency, compression time, and energy consumption. 
     
     
         5 . The method of  claim 1 , wherein a priority is assigned to each of a plurality of items included in the constraint, and
 wherein the determining the optimal set of parameters includes:   determining whether the compressed inference model satisfies the constraint based on the priority at least a predetermined criterion on the basis of the performance of the compressed inference model.   
     
     
         6 . The method of  claim 1 , wherein the set of compression methods are selected from a compression method pool, and
 wherein the compression method pool includes pruning, quantization, resolution change, and filter decomposition.   
     
     
         7 . The method of  claim 1 , wherein the performance of the compressed inference model includes a value of at least one item among latency, accuracy, and energy consumption. 
     
     
         8 . The method of  claim 1 , wherein the set of compression methods is configured on the basis of a predetermined rule, and
 the predetermined rule includes at least one of a first rule that a quantization-based compression method included in the optimal parameter set is to be positioned last in the compression pipeline or a second rule that an activation change-based compression method is to be positioned before the quantization-based compression method.   
     
     
         9 . A computing device for determining an optimal parameter set, the computing device comprising:
 a memory configured to store at least one instruction; and   at least one processor executing the at least one instruction, wherein the processor is configured to:   receive an inference model, a dataset, and a constraint,   configure a set of compression methods to be applied to the inference model and a set of parameters for the set of compression methods on the basis of the constraint,   apply a first compression method included in the set of compression methods and a first parameter related to the first compression method to the inference model, through a compression pipeline,   determine whether a compressed inference model is generated from the inference model through the compression pipeline,   when it is determined that the compressed inference model is not generated, apply a second compression method included in the set of compression methods, and a second parameter among the set of parameters to the inference model, following the first compression method, through the compression pipeline,   when it is determined that the compressed inference model is generated, transmit the compressed inference model to the target device, and   determine an optimal set of parameters on the basis of the performance of the compressed inference model received from the target device,   wherein the performance of the compressed inference model is measured by the target device using the dataset.   
     
     
         10 . The computing device of  claim 9 , wherein the processor is further configured to:
 select the first compression method and the first parameter for the first compression method,   determine whether to further select a compression method and a parameter on the basis of the constraint and the first compression method, and   when it is determined to further select a compression method and a parameter, select the second compression method and the second parameter for the second compression method on the basis of the first compression method.   
     
     
         11 . The computing device of  claim 9 , wherein when the performance of the compressed inference model satisfies the constraint, the selected set of parameters is determined as the optimal parameter set, and
 wherein when the performance of the compressed inference model does not satisfy the constraint, the processor is further configured to: repeatedly configure the set of compression methods and the set of parameters, apply the first compression method and the first parameter to the inference model, determine whether the compressed inference model is generated, apply the second compression method and the second parameter to the inference model, transmit the compressed inference model to the target device, and determine the optimal parameter set.   
     
     
         12 . The computing device of  claim 9 , wherein the constraint includes a value of at least one item among device, accuracy, model size, latency, compression time, and energy consumption. 
     
     
         13 . The computing device of  claim 9 , wherein a priority is assigned to each of a plurality of items included in the constraint, and
 wherein the processor is further configured to determine whether the compressed inference model satisfies the constraint based on the priority at least a predetermined criterion on the basis of the performance of the compressed inference model.   
     
     
         14 . The computing device of  claim 9 , wherein the processor is further configured to select the set of compression methods from a compression method pool, and
 wherein the compression method pool includes pruning, quantization, resolution change, and filter decomposition.   
     
     
         15 . The computing device of  claim 9 , wherein the performance of the compressed inference model includes a value of at least one item among latency, accuracy, and energy consumption. 
     
     
         16 . The computing device of  claim 9 , wherein the processor is further configured to configure the set of compression methods on the basis of a predetermined rule, and
 the predetermined rule includes at least one of a first rule that a quantization-based compression method included in the optimal parameter set is to be positioned last in the compression pipeline or a second rule that an activation change-based compression method is to be positioned before the quantization-based compression method.