Utilizing Small Sized Large Language Models (LLMs) for Performing Domain Classification

Abstract

Systems and methods for utilizing small sized Large Language Models (LLMs) for performing domain classification include responsive to training one or more machine learning models for performing classification of domains, the training including performing one or more optimizations to the one or more machine learning models, receiving a domain; obtaining data associated with the domain including log data from a cloud-based system that performs monitoring of a plurality of users; and analyzing the domain via the one or more trained machine learning models for classifying the domain.

What is claimed is: 
     
         1 . A method comprising steps of:
 responsive to training one or more machine learning models for performing classification of domains, the training including performing one or more optimizations to the one or more machine learning models, receiving a domain;   obtaining data associated with the domain including log data from a cloud-based system that performs monitoring of a plurality of users; and   analyzing the domain via the one or more trained machine learning models for classifying the domain.   
     
     
         2 . The method of  claim 1 , wherein the one or more optimizations comprise any of 4-bit quantization of trainable parameters, utilization of bfloat16 memory for non-trainable parameters, utilization of flash attention mechanisms, and utilization of Rotation Positional Encoding (RoPE). 
     
     
         3 . The method of  claim 1 , wherein the steps further comprise:
 performing an action based on the classifying, the action comprising any of blocking the domain, allowing the domain, and isolating the domain.   
     
     
         4 . The method of  claim 3 , wherein the steps further comprise:
 providing an explanation of a classification of the domain via an interactive User Interface (UI) responsive to performing the action.   
     
     
         5 . The method of  claim 1 , wherein the classifying comprises predicting a likelihood the domain is malicious or benign. 
     
     
         6 . The method of  claim 1 , wherein the classifying comprises categorizing the domain as phishing. 
     
     
         7 . The method of  claim 1 , wherein the classifying comprises predicting a likelihood the domain is a command and control site. 
     
     
         8 . The method of  claim 1 , wherein the classifying comprises determining a category for the domain. 
     
     
         9 . The method of  claim 1 , wherein the one or more machine learning models are Large Language Models (LLMs). 
     
     
         10 . The method of  claim 9 , wherein the one or more LLMs are small sized LLMs. 
     
     
         11 . A non-transitory computer-readable storage medium having computer readable code stored thereon for programming at least one processor to perform steps of:
 responsive to training one or more machine learning models for performing classification of domains, the training including performing one or more optimizations to the one or more machine learning models, receiving a domain;   obtaining data associated with the domain including log data from a cloud-based system that performs monitoring of a plurality of users; and   analyzing the domain via the one or more trained machine learning models for classifying the domain.   
     
     
         12 . The non-transitory computer-readable storage medium of  claim 11 , wherein the one or more optimizations comprise any of 4-bit quantization of trainable parameters, utilization of bfloat16 memory for non-trainable parameters, utilization of flash attention mechanisms, and utilization of Rotation Positional Encoding (RoPE). 
     
     
         13 . The non-transitory computer-readable storage medium of  claim 11 , wherein the steps further comprise:
 performing an action based on the classifying, the action comprising any of blocking the domain, allowing the domain, and isolating the domain.   
     
     
         14 . The non-transitory computer-readable storage medium of  claim 13 , wherein the steps further comprise:
 providing an explanation of a classification of the domain via an interactive User Interface (UI) responsive to performing the action.   
     
     
         15 . The non-transitory computer-readable storage medium of  claim 11 , wherein the classifying comprises predicting a likelihood the domain is malicious or benign. 
     
     
         16 . The non-transitory computer-readable storage medium of  claim 11 , wherein the classifying comprises categorizing the domain as phishing. 
     
     
         17 . The non-transitory computer-readable storage medium of  claim 11 , wherein the classifying comprises predicting a likelihood the domain is a command and control site. 
     
     
         18 . The non-transitory computer-readable storage medium of  claim 11 , wherein the classifying comprises determining a category for the domain. 
     
     
         19 . The non-transitory computer-readable storage medium of  claim 11 , wherein the one or more machine learning models are Large Language Models (LLMs). 
     
     
         20 . The non-transitory computer-readable storage medium of  claim 19 , wherein the one or more LLMs are small sized LLMs.