System and method of providing an interactive development platform in a distributed computing environment

Abstract

A system and method of providing an interactive development environment may include providing a proxy server module, adapted to interface at least one cloud-based platform and one or more client modules, operatively connected to the proxy server, where each client module may be associated with a respective user development platform. At least one client module may receive, from the respective user development platform, one or more interactive computing documents, commonly referred to as notebooks, each representing one or more scripting code elements, commonly referred to as cells. The proxy server may scan the one or more cells, according to a set of predetermined scripting rules, and encapsulate the one or more notebooks in one or more data containers, based on the scan. The proxy server may subsequently transmit the one or more data containers to a cloud-based platform, to execute at least one cell of the one or more notebooks.

1 . A method of providing an interactive development environment, said method comprising:
 providing a proxy server configured to:   operatively connect to one or more client modules, each associated with a respective user development platform;   receive, from at least one client module of the one or more client modules, one or more interactive computing documents, each comprising one or more scripting code elements;   scan the one or more scripting code elements according to a set of predetermined scripting rules;   encapsulate the one or more interactive computing documents in one or more data containers, based on said scan; and   transmit the one or more data containers to at least one cloud-based platform, wherein the at least one cloud-based platform is configured to execute at least one scripting code element of the one or more interactive computing documents.   
     
     
         2 . The method of  claim 1 , wherein the proxy server is further configured to:
 monitor execution of the one or more scripting code elements of the one or more interactive computing documents by the at least one cloud-based platform; and   perform at least one mitigation action based on said monitoring, wherein said mitigation action is selected from a list consisting of: blocking a user of a specific client module from accessing the at least one cloud-based platform, changing an access permission of a role of the user of the specific client module to a specific cloud-based platform, deleting data footprints of the at least one scripting code element, and disabling a specific user of the specific client module from executing the one or more scripting code elements in the user development platform.   
     
     
         3 . The method of  claim 1 , further comprising:
 producing, by at least one client module of the one or more client modules, a workflow data element, defining a scheme for executing the one or more interactive computing documents;   transmitting, by at least one client module, the workflow data element to the to the proxy server; and   transmitting, by the proxy server, the one or more data containers and the workflow data element to the at least one cloud-based platform, to execute the one or more interactive computing documents according to the scheme, as defined by the workflow data element.   
     
     
         4 . A proxy server for providing an interactive development environment, said proxy server configured to:
 receive, from at least one client module associated with a respective user development platform, one or more interactive computing documents, each comprising one or more scripting code elements;   accumulate a subset of scripting code elements, each representing one or more data actions;   classify the subset of scripting code elements, in relation to the at least one client module, to one or more classifications;   determine at least one scripting rule based on the one or more classifications; and   scan at least one scripting code element according to the at least one scripting rule.   
     
     
         5 . The proxy server of  claim 4 , further configured to:
 receive, from the at least one client module, a workflow data element, defining a scheme for executing the one or more interactive computing documents;   encapsulate the workflow data element with the one or more interactive computing documents in the one or more data containers; and   transmit the one or more data containers to at least one cloud-based platform, to execute the at least one scripting code element of the one or more interactive computing documents according to the scheme, as defined by the workflow data element.   
     
     
         6 . The proxy server of  claim 5 , wherein the scheme comprises at least one of: (a) scheduling of execution of the one or more interactive computing documents; (b) an order of execution of the one or more interactive computing documents; (c) a condition for executing the one or more interactive computing documents; and (d) concurrence of execution of two or more interactive computing documents. 
     
     
         7 . The proxy server of  claim 5 , wherein the one or more interactive computing documents further comprise document metadata, and wherein the proxy server is further configured to:
 scan the document metadata according to a set of predetermined metadata rules; and   encapsulate the workflow data element and the one or more interactive computing documents in the one or more data containers, based on said scan of the document metadata.   
     
     
         8 . The proxy server of  claim 5 , further configured to:
 receive, from the at least one client module, at least one execution parameter value, associated with a specific interactive computing document;   inject the received execution parameter value as document metadata into the specific interactive computing document; and   further encapsulate document metadata in the one or more data containers, wherein the at least one cloud-based platform is further configured to execute the at least one scripting code element of the one or more interactive computing documents according to the document metadata.   
     
     
         9 . The proxy server of  claim 8  further comprising:
 a feature extraction module, configured to extract one or more data action features from at least one specific scripting code element of the subset of scripting code elements; and 
 an ML based model configured to classify the specific scripting code element according to the one or more classifications, based on the one or more data action features. 
 
     
     
         10 . The proxy server of  claim 5 , further configured to
 encapsulate the workflow data element and the one or more interactive computing documents by (i) encapsulating the workflow data element in a first data container, and (ii) spawning one or more second data containers, each encapsulating one or more interactive computing documents of the one or more interactive computing documents based on the workflow data element; and   transmit the one or more second data containers to the at least one cloud-based platform according to the scheme, as defined by the workflow data element.   
     
     
         11 . The proxy server of  claim 5  further comprising:
 a monitoring module, adapted to monitor execution of the one or more scripting code elements of the one or more interactive computing documents by the at least one cloud-based platform; and 
 a rule engine module, adapted to identify a problem in the execution according to a set of predefined runtime rules, based on said monitoring. 
 
     
     
         12 . The proxy server of  claim 10 , wherein the rule engine module is further configured to communicate a notification message to a relevant client module, wherein the notification message comprises a representation of the identified execution problem, to be displayed on a user interface (UI) of the user development platform. 
     
     
         13 . The proxy server of  claim 10 , wherein the rule engine module is further configured to perform at least one mitigation action, selected from a list consisting of: blocking a user of a specific client module from accessing data stored on the at least one cloud-based platform, changing an access permission of a role of the user of the specific client module to a specific cloud-based platform, deleting data footprints of the at least one scripting code element, and disabling a specific user of the specific client module from executing the one or more scripting code elements in the user development platform. 
     
     
         14 . The proxy server of  claim 10 , wherein monitoring execution of the one or more scripting code elements comprises at least one of: run-time monitoring of logs generated by the at least one cloud-based platform; run-time monitoring of an order of the execution of the one or more scripting code elements; run-time monitoring of output of the execution of the one or more scripting code elements; monitoring duration of execution of the at least one scripting code element of the one or more interactive computing documents; and performing a comparison between output statistics of a current execution of an interactive computing document and output statistics of a previous execution of the interactive computing document. 
     
     
         15 . The proxy server of  claim 10 , wherein monitoring execution of the one or more scripting code elements comprises at least one of:
 identifying at least one difference between an output of a specific scripting code element at a first execution and an output of the specific scripting code element at a second execution; and   identifying at least one difference between a first metadata element, associated with the specific scripting code element at the first execution and a second metadata element, associated with the specific scripting code element at the second execution.   
     
     
         16 . The proxy server of  claim 5  further comprising a monitoring module, adapted to:
 monitor execution of the one or more scripting code elements of the one or more interactive computing documents by the at least one cloud-based platform; 
 produce a log data element, representing said monitored execution of the one or more scripting code elements; and 
 communicate the log data element to a relevant client module, wherein said relevant client module is configured to display said log data element on a UI of the user development platform. 
 
     
     
         17 . The proxy server of  claim 4  wherein the proxy server is further configured to scan the at least one scripting code element according to the set of predetermined scripting rules, to verify at least one of (a) permission-based legitimacy of data retrieval by the at least one scripting code element, and (b) lack of malicious activity in the at least one scripting code element 
     
     
         18 . A proxy server for providing an interactive development environment, said proxy server configured to:
 receive, from a client module, one or more interactive computing documents, each comprising one or more scripting code elements;   apply a machine learning (ML) based model on the one or more scripting code elements to determine at least one scripting rule;   scan the one or more scripting code elements according to the at least one scripting rule;   encapsulate the one or more interactive computing documents in one or more data containers, based on said scan; and   transmit the one or more data containers to at least one cloud-based platform, to execute at least one scripting code element of the one or more interactive computing documents.   
     
     
         19 . The proxy server of  claim 18 , further configured to:
 monitor execution of the one or more scripting code elements of the one or more interactive computing documents by the at least one cloud-based platform; and   perform at least one mitigation action based on said monitoring, wherein said mitigation action is selected from a list consisting of: blocking a user of a specific client module from accessing the at least one cloud-based platform, changing an access permission of a role of the user of the specific client module to a specific cloud-based platform, deleting data footprints of the at least one scripting code element, and disabling a specific user of the specific client module from executing the one or more scripting code elements in the user development platform.   
     
     
         20 . The proxy server of  claim 18 , further configured to:
 receive, from at least one client module of the one or more client modules, a workflow data element, defining a scheme for executing the one or more interactive computing documents; and   transmit the one or more data containers and the workflow data element to the at least one cloud-based platform, to execute the one or more interactive computing documents according to the scheme, as defined by the workflow data element.