Machine learning-based query processing of documents including tabular data structures

Abstract

An apparatus comprises at least one processing device configured to obtain a query comprising search text and a context identifying documents including tabular data structures to be searched using the search text, and to generate document chunks by parsing the documents, the tabular data structures being replaced in the document chunks with tabular data structure representations that maintain tabular formatting of the tabular data structures. The at least one processing device is further configured to select a subset of the document chunks based at least in part on determining a similarity between the document chunks and the search text, to generate a prompt for input to a machine learning system comprising the selected document chunks, and to provide an answer to the query that comprises content from at least one of the tabular data structures based at least in part on an output of the machine learning system.

What is claimed is: 
     
         1 . An apparatus comprising:
 at least one processing device comprising a processor coupled to a memory;   the at least one processing device being configured:
 to obtain a query, the query comprising search text and a context, the context identifying one or more documents to be searched using the search text, at least one of the one or more documents comprising one or more tabular data structures; 
 to generate a plurality of document chunks by parsing the one or more documents, each of the plurality of document chunks comprising a portion of content of one of the one or more documents, wherein the one or more tabular data structures are replaced in the plurality of document chunks by one or more tabular data structure representations that maintain a tabular formatting of the one or more tabular data structures; 
 to select a subset of the plurality of document chunks based at least in part on determining a similarity between content of the plurality of document chunks and the search text, the selected subset of the plurality of document chunks comprising at least one document chunk comprising at least one of the one or more tabular data structure representations; 
 to generate, based at least in part on the query, a prompt for input to a machine learning system, the prompt comprising the selected subset of the plurality of document chunks; 
 to apply the prompt to the machine learning system to generate an output; and 
 to provide an answer to the query based at least in part on the output of the machine learning system, the answer comprising at least a portion of content from at least one of the one or more tabular data structures. 
   
     
     
         2 . The apparatus of  claim 1  wherein, for a given one of the plurality of document chunks generated for a given one of the one or more documents including a given one of the one or more tabular data structures, a given tabular data structure representation for the given tabular data structure comprises a plain-text representation of the tabular formatting of the given tabular data structure. 
     
     
         3 . The apparatus of  claim 2  wherein generating the given document chunk comprises:
 determining coordinates of the given tabular data structure relative to surrounding text in the given document chunk; 
 extracting the given tabular data structure from the given document chunk; 
 extracting textual content and layout of the given tabular data structure; and 
 inserting the extracted textual content of the given tabular data structure into the given document chunk at the determined coordinates in accordance with the extracted layout. 
 
     
     
         4 . The apparatus of  claim 3  wherein the determined coordinates comprise a horizontal alignment of the given tabular data structure in the given document chunk. 
     
     
         5 . The apparatus of  claim 1  wherein, for a given one of the plurality of document chunks generated for a given one of the one or more documents including a given one of the one or more tabular data structures, a given tabular data structure representation for the given tabular data structure comprises a placeholder for the given tabular data structure. 
     
     
         6 . The apparatus of  claim 5  wherein generating the given document chunk comprises:
 determining a tabular data structure identifier for the given tabular data structure; 
 determining coordinates of the given tabular data structure relative to surrounding text in a given one of the plurality of document chunks; and 
 inserting, into the given document chunk, the placeholder for the given tabular data structure at the determined coordinates, the placeholder including a reference to the determined tabular data structure identifier. 
 
     
     
         7 . The apparatus of  claim 5  wherein generating the answer to the query comprises augmenting the output of the machine learning system with an original version of the given tabular data structure extracted from the given document responsive to determining that textual content of the output of the machine learning system is sourced from the given tabular data structure. 
     
     
         8 . The apparatus of  claim 7  wherein generating the given document chunk further comprises determining a textual description for the given tabular data structure, and wherein augmenting the output of the machine learning system comprises selecting the given tabular data structure from among multiple tabular data structures in the given document chunk responsive to determining that a similarity between the search text of the query and the textual description for the given tabular data structure exceeds a designated similarity threshold. 
     
     
         9 . The apparatus of  claim 8  wherein determining the textual description comprises extracting a table caption for the given tabular data structure. 
     
     
         10 . The apparatus of  claim 8  wherein determining the textual description comprises applying at least one of natural language processing summarization and natural language processing topic extraction to textual content of the given tabular data structure. 
     
     
         11 . The apparatus of  claim 1  wherein the machine learning system comprises a large language model, and wherein generating the prompt for input to the machine learning system comprises utilizing a prompt template that instructs the large language model to recognize the tabular data structure representations and include content of relevant ones of the one or more tabular data structures in the output of the machine learning system. 
     
     
         12 . The apparatus of  claim 1  wherein, for a given one of the plurality of document chunks generated for a given one of the one or more documents including a given one of the one or more tabular data structures, a given tabular data structure representation for the given tabular data structure comprises one or more image placeholders for one or more images included in the given tabular data structure, and wherein generating the answer to the query comprises augmenting the output of the machine learning system with an original version of at least a given one of the one or more images responsive to determining that textual content of the output of the machine learning system is sourced from a portion of the given tabular data structure associated with the given image. 
     
     
         13 . The apparatus of  claim 1  wherein the query is directed to performing configuration of an information technology asset, and wherein the one or more documents comprise one or more technical guides for the information technology asset. 
     
     
         14 . The apparatus of  claim 1  wherein the query is directed to performing at least one of troubleshooting and remediation of one or more issues encountered on an information technology asset, and wherein the one or more documents comprise one or more support tickets associated with the one or more issues encountered on the information technology asset. 
     
     
         15 . A computer program product comprising a non-transitory processor-readable storage medium having stored therein program code of one or more software programs, wherein the program code when executed by at least one processing device causes the at least one processing device:
 to obtain a query, the query comprising search text and a context, the context identifying one or more documents to be searched using the search text, at least one of the one or more documents comprising one or more tabular data structures;   to generate a plurality of document chunks by parsing the one or more documents, each of the plurality of document chunks comprising a portion of content of one of the one or more documents, wherein the one or more tabular data structures are replaced in the plurality of document chunks by one or more tabular data structure representations that maintain a tabular formatting of the one or more tabular data structures;   to select a subset of the plurality of document chunks based at least in part on determining a similarity between content of the plurality of document chunks and the search text, the selected subset of the plurality of document chunks comprising at least one document chunk comprising at least one of the one or more tabular data structure representations;   to generate, based at least in part on the query, a prompt for input to a machine learning system, the prompt comprising the selected subset of the plurality of document chunks;   to apply the prompt to the machine learning system to generate an output; and   to provide an answer to the query based at least in part on the output of the machine learning system, the answer comprising at least a portion of content from at least one of the one or more tabular data structures.   
     
     
         16 . The computer program product of  claim 15  wherein, for a given one of the plurality of document chunks generated for a given one of the one or more documents including a given one of the one or more tabular data structures, a given tabular data structure representation for the given tabular data structure comprises a plain-text representation of the tabular formatting of the given tabular data structure. 
     
     
         17 . The computer program product of  claim 15  wherein, for a given one of the plurality of document chunks generated for a given one of the one or more documents including a given one of the one or more tabular data structures, a given tabular data structure representation for the given tabular data structure comprises a placeholder for the given tabular data structure. 
     
     
         18 . A method comprising:
 obtaining a query, the query comprising search text and a context, the context identifying one or more documents to be searched using the search text, at least one of the one or more documents comprising one or more tabular data structures;   generating a plurality of document chunks by parsing the one or more documents, each of the plurality of document chunks comprising a portion of content of one of the one or more documents, wherein the one or more tabular data structures are replaced in the plurality of document chunks by one or more tabular data structure representations that maintain a tabular formatting of the one or more tabular data structures;   selecting a subset of the plurality of document chunks based at least in part on determining a similarity between content of the plurality of document chunks and the search text, the selected subset of the plurality of document chunks comprising at least one document chunk comprising at least one of the one or more tabular data structure representations;   generating, based at least in part on the query, a prompt for input to a machine learning system, the prompt comprising the selected subset of the plurality of document chunks;   applying the prompt to the machine learning system to generate an output; and   providing an answer to the query based at least in part on the output of the machine learning system, the answer comprising at least a portion of content from at least one of the one or more tabular data structures;   wherein the method is performed by at least one processing device comprising a processor coupled to a memory.   
     
     
         19 . The method of  claim 18  wherein, for a given one of the plurality of document chunks generated for a given one of the one or more documents including a given one of the one or more tabular data structures, a given tabular data structure representation for the given tabular data structure comprises a plain-text representation of the tabular formatting of the given tabular data structure. 
     
     
         20 . The method of  claim 18  wherein, for a given one of the plurality of document chunks generated for a given one of the one or more documents including a given one of the one or more tabular data structures, a given tabular data structure representation for the given tabular data structure comprises a placeholder for the given tabular data structure.