Gate node providing moe service in hybrid peer-to-peer network and method for operating the same

Abstract

A gate node providing MoE service in a hybrid peer-to-peer network and its operating method are disclosed. A method of operation of the disclosed MoE gate node includes receiving a query from a user; determining at least one primary model to generate a response to the query from among a plurality of expert models connected to the MoE gate node; broadcasting the query to the at least one primary model; receiving a response broadcast from each of the at least one primary model; evaluating the response; and providing a final response generated based on the evaluation of the response to the user.

1 . A method of operation of a mixture of experts (MoE) gate node, the method comprising:
 receiving a query from a user;   determining at least one primary model to generate a response to the query from among a plurality of expert models connected to the MoE gate node;   broadcasting the query to the at least one primary model;   receiving a response broadcast from each of the at least one primary model;   evaluating the response; and   providing a final response generated based on the evaluation of the response to the user.   
     
     
         2 . The method of  claim 1 , wherein:
 the determining at least one primary model comprises:   determining the at least one primary model among the plurality of expert models based on the query and metadata of the plurality of expert models.   
     
     
         3 . The method of  claim 2 , wherein:
 the determining at least one primary model comprises:   evaluating an output suitability of the plurality of expert models for an input data corresponding to the query through a gate model included in the MoE gate node;   evaluating a relevance of the plurality of expert models for the query based on the metadata of the plurality of expert models through a meta model included in the MoE gate node; and   determining the at least one primary model according to the output suitability and the relevance.   
     
     
         4 . The method of  claim 1 , wherein:
 expert nodes including each of the multiple expert models are connected to the MoE gate node through an overlay network.   
     
     
         5 . The method of  claim 1 , wherein:
 the determining at least one primary model comprises:   determining a single primary model or a pluarlity of primary models based on at least one of a volume, complexity and uncertainty of input data corresponding to the query.   
     
     
         6 . The method of  claim 1 , further comprising:
 receiving an evaluation of the response of the at least one primary model from a candidate model selected from among the plurality of expert models, and   wherein the evaluating the response comprises:   evaluating the response considering the evaluation received from the candidate model.   
     
     
         7 . The method of  claim 6 , wherein the evaluating the response comprises:
 evaluating the response of at least one primary model using a gate model included in the MoE gate node, and   evaluating a reliability and quality of the response of the at least one primary model using a large language model (LLM) included in the MoE gate node.   
     
     
         8 . The method of  claim 1 , wherein:
 the final response is generated by a LLM included in the MoE gate node based on the responses of at least one primary model and an evaluation of a candidate model.   
     
     
         9 . The method of  claim 1 , wherein:
 the final response is based on an evaluation scores of a candidate model for the response of the at least one primary model, and is determined based on the responses of the at least one primary model and a modification suggestion of the candidate model.   
     
     
         10 . A non-transitory computer-readable recording medium storing instructions that, when executed by one or more processors, configure the one or more processors to perform the method of  claim 1 . 
     
     
         11 . A mixture of experts (MoE) gate node comprising:
 a processor; and   a memory storing instructions,   wherein the instructions, when executed by the processor, cause the MoE gate node to: receive a query from a user;   determine at least one primary model to generate a response to the query among a plurality of expert models connected to the MoE gate node;   broadcast the query to the at least one primary model;   receive a response broadcast from each of the at least one primary model;   evaluate the response; and   provide a final response generated based on the evaluation of the response to the user.   
     
     
         12 . The Moe gate node of  claim 11 , wherein:
 the instructions, when executed by the processor, cause the MoE gate node to determine a primary model among the plurality of expert models based on the query and metadata of the plurality of expert models.   
     
     
         13 . The Moe gate node of  claim 12 , wherein:
 the instructions, when executed by the processor, cause the MoE gate node to:   evaluate an output suitability of the plurality of expert models for an input data corresponding to the query through a gate model included in the MoE gate node;   evaluate a relevance of the plurality of expert models for the query based on the metadata of the plurality of expert models through a meta model included in the MoE gate node; and   
       determine the primary model according to the output suitability and the relevance. 
     
     
         14 . The Moe gate node of  claim 11 , wherein:
 expert nodes including each of the multiple expert models are connected to the MoE gate node through an overlay network.   
     
     
         15 . The Moe gate node of  claim 11 , wherein:
 the instructions, when executed by the processor, cause the MoE gate node to determine a single primary model or a plurality of primary models based on at least one of a volume, complexity and uncertainty of input data corresponding to the query.   
     
     
         16 . The Moe gate node of  claim 11 , wherein:
 the instructions, when executed by the processor, cause the MoE gate node to:   receive an evaluations of the responses of the at least one primary model from a candidate model selected from the plurality of expert models; and   evaluate the responses by considering the evaluation received from the candidate model.   
     
     
         17 . The Moe gate node of  claim 16 , wherein:
 the instructions, when executed by the processor, cause the MoE gate node to:   evaluate the response of the at least one primary model using a gate model included in the MoE gate node; and   evaluate a reliability and quality of the response of the at least one primary model using a large language model (LLM) included in the MoE gate node.   
     
     
         18 . The Moe gate node of  claim 11 , wherein:
 the final response is generated by a LLM included in the MoE gate node based on the responses of at least one primary model and an evaluation of a candidate model.   
     
     
         19 . The Moe gate node of  claim 11 , wherein:
 the final response is based on an evaluation scores of a candidate model for the response of the at least one primary model, and is determined based on the responses of the at least one primary model and a modification suggestion of the candidate model.