Systems and methods for scientific evaluation of program code outputs

Abstract

Systems and methods for distributed scientific evaluation of software product comprising a digital health intervention or software as a medical device product. In certain embodiments, a scientific evaluation system is operably configured to analyze one or more safety, efficacy, performance, and/or quality assurance aspects for program code outputs of the software product. In further embodiments, a scientific evaluation system is operably configured to configure and administer a distributed scientific evaluation, research study and/or clinical trial for the software product. In certain embodiments, a scientific evaluation system is configured to enable scientific evaluation of one or more program code outputs of a software product for one or more indication or intended use.

What is claimed is: 
     
         1 . A computer-implemented system comprising:
 at least one server communicably engaged with an application server, the at least one server comprising at least one processor; and   a non-transitory computer-readable medium communicably engaged with the at least one processor of the at least one server, the non-transitory computer-readable medium comprising one or more processor-executable instructions stored thereon that, when executed, command the at least one processor to perform one or more operations comprising:   configuring a parallel authentication framework between the application server and the at least one server, wherein the parallel authentication framework is configured to authenticate, in parallel, one or more users associated with one or more user devices at the application server and the at least one server,   wherein the application server comprises at least one software product hosted thereon;   configuring one or more scientific evaluation parameters for the at least one software product, wherein the one or more scientific evaluation parameters comprise one or more usage or session parameters for the one or more users;   providing one or more instance of the at least one software product to the one or more user devices;   receiving user-generated data for the one or more users;   processing the user-generated data according to an analytical framework to determine user outcome data comprising one or more user statistics and one or more stimulus-input patterns for the one or more users; and   analyzing the user outcome data for the one or more users according to at least one statistical framework to determine one or more scientific validation metrics for a targeted output of the at least one software product.   
     
     
         2 . The system of  claim 1  wherein the at least one statistical framework comprises a computer-implemented machine learning framework configured to analyze one or more dependent variables associated with one or more safety, efficacy and performance endpoints for the at least one software product. 
     
     
         3 . The system of  claim 1  wherein the one or more operations further comprise operations for monitoring real-time user activity data to determine a measure of user adherence to the one or more scientific evaluation parameters. 
     
     
         4 . The system of  claim 1  wherein the one or more operations further comprise operations for segmenting mobile endpoint data and the user-generated data into at least two data segments comprising performance data and non-performance data. 
     
     
         5 . The system of  claim 1  wherein the one or more operations further comprise operations for revoking access to the at least one software product according to the one or more scientific evaluation parameters. 
     
     
         6 . The system of  claim 1  wherein the one or more operations further comprise operations for providing one or more notifications to the one or more user devices according to the one or more scientific evaluation parameters, the one or more notifications comprising user instructions or user prompts for executing an instance of the at least one software product. 
     
     
         7 . The system of  claim 6  wherein the one or more operations further comprise operations for providing the one or more notifications to the one or more user devices according to one or more user adherence rules. 
     
     
         8 . The system of  claim 1  wherein the one or more operations further comprise operations for analyzing the user-generated data for the one or more users to determine a measure of protocol compliance or deviation for at least one user in the one or more users. 
     
     
         9 . The system of  claim 1  wherein the one or more operations further comprise operations for dynamically modifying one or more computerized stimuli or interactions for the at least one software product according to the one or more scientific evaluation parameters. 
     
     
         10 . A computer-implemented method comprising:
 providing, with an application server, an instance of a scientific evaluation application to at least one client device;   configuring a parallel authentication framework between the application server and at least one server, wherein the parallel authentication framework is configured to authenticate, in parallel, one or more users associated with one or more user devices at the application server and the at least one server,   wherein the one or more users comprise a user population for at least one scientific study group associated with scientific evaluation of a software product;   configuring one or more scientific evaluation parameters for the software product, wherein the one or more scientific evaluation parameters comprise one or more usage or session parameters for the one or more users;   providing, with the application server, one or more instance of the software product to the one or more user devices;   receiving, with the at least one server, user-generated data for the one or more users in the user population;   processing, with the at least one server, the user-generated data according to an analytical framework to determine user outcome data comprising one or more user statistics and one or more stimulus-input patterns for the one or more users in the user population; and   analyzing, with the at least one server, the user outcome data for the user population according to at least one statistical framework to determine one or more scientific validation metrics for a targeted output of the software product.   
     
     
         11 . The method of  claim 10  further comprising monitoring in real-time, with the application server and the at least one server, user activity data comprising a measure of user adherence to the one or more scientific evaluation parameters. 
     
     
         12 . The method of  claim 10  wherein the at least one statistical framework comprises a computer-implemented machine learning framework configured to analyze one or more dependent variables associated with one or more safety, efficacy and performance endpoints for the software product. 
     
     
         13 . The method of  claim 10  further comprising segmenting, with the at least one server, the user-generated data into at least two data segments comprising performance data and non-performance data. 
     
     
         14 . The method of  claim 10  further comprising dynamically modifying, with the application server, one or more computerized stimuli or interactions or revoking access to the software product according to the one or more scientific evaluation parameters. 
     
     
         15 . The method of  claim 10  further comprising providing, with the application server, one or more notifications to the one or more user devices according to the one or more scientific evaluation parameters, the one or more notifications comprising user instructions or user prompts for executing the one or more instance of the software product. 
     
     
         16 . The method of  claim 15 , further comprising providing, with the application server, the one or more notifications to the one or more user devices according to one or more user adherence rules. 
     
     
         17 . A non-transitory computer-readable medium with one or more processor-executable instructions stored thereon that, when executed, command one or more processor to perform one or more operations, the one or more operations comprising:
 configuring a parallel authentication framework between an application server and at least one server, wherein the parallel authentication framework is configured to authenticate, in parallel, one or more users associated with one or more user devices at the application server and the at least one server,   wherein the application server comprises at least one software product hosted thereon;   configuring one or more scientific evaluation parameters for the at least one software product, wherein the one or more scientific evaluation parameters comprise one or more usage or session parameters for the one or more users;   providing one or more instance of the at least one software product to the one or more user devices;   receiving user-generated data for the one or more users;   processing the user-generated data according to an analytical framework to determine user outcome data comprising one or more user statistics and one or more stimulus-input patterns for the one or more users; and   analyzing the user outcome data for the one or more users according to at least one statistical framework to determine one or more scientific validation metrics for a targeted output of the at least one software product.   
     
     
         18 . The non-transitory computer-readable medium of  claim 17  wherein the at least one statistical framework comprises a computer-implemented machine learning framework configured to analyze one or more dependent variables associated with one or more safety, efficacy and performance endpoints for the at least one software product. 
     
     
         19 . The non-transitory computer-readable medium of  claim 17  wherein the one or more operations further comprise operations for monitoring real-time user activity data to determine a measure of user adherence to the one or more scientific evaluation parameters. 
     
     
         20 . The non-transitory computer-readable medium of  claim 17  wherein the one or more operations further comprise operations for revoking access to the at least one software product according to the one or more scientific evaluation parameters.