Apparatus for Adaptive Problem Determination in Distributed Service-Based Applications

Abstract

A technique for problem determination in a distributed application is provided. Testing results of the application are first obtained through execution of test cases of a test group in the application. The testing of the application is then adaptively refined when the testing results have one or more failures, to expose problems that caused the one or more failures.

1 . Apparatus for problem determination in a distributed service-based application, the apparatus comprising: 
 a memory; and    at least one processor, coupled to the memory, operative to: (i) obtain, via one or more probes, at least one testing result of the application through execution of at least one test group, wherein the test group comprises one or more test cases; and (ii) adaptively refine the at least one executed test group when the at least one testing result comprises at least one failure, to expose at least one problem that caused the at least one failure,    wherein the operation of adaptively refining the at least one executed test group comprises at least one of generating one or more new test cases and refining the one or more probes,    wherein the operation of obtaining testing results comprises the operations of: (a) generating a test group having at least one test case, from a general model of the application; (b) executing the at least one test case of the test group in the application; (c) passing the at least one result of the test group to an outcome analyzer; and (d) verifying the at least one result against expected output at the outcome analyzer, wherein the operation of verifying the at least one result comprises the operation of marking each test case of the test group with a failure or a success.    
     
     
         2 . The apparatus of  claim 1 , wherein the at least one processor is further operative to fix at least one problem that caused at least one failure, when the cause of the at least one problem has been localized.  
     
     
         3 . The apparatus of  claim 1 , wherein the processor is further operative to enable one or more probes that return one or more testing results to the outcome analyzer.  
     
     
         4 . The apparatus of  claim 1 , wherein the operation of adaptively refining the at least one executed test group comprises the operations of: 
 enabling one or more additional probes that return one or more results relating to the at least one problem; and    repeating the method of problem determination in a distributed service-based application.    
     
     
         5 . The apparatus of  claim 1 , wherein the one or more probes collect one or more intermediary results from the application.  
     
     
         6 . The apparatus of  claim 1 , wherein the one or more probes return one or more results relating to the functioning of the application.  
     
     
         7 . The apparatus of  claim 1 , wherein the operation of adaptively refining the at least one executed test group comprises the operations of: 
 adapting the test group to comprise at least one test case focused on the at least one problem; and    repeating the method of problem determination in a distributed service-based application.    
     
     
         8 . The apparatus of  claim 7 , wherein the operation of adapting the test group comprises the operation of representing at least one action that correlates to the at least one failure in a model for generating the test group.  
     
     
         9 . The apparatus of  claim 7 , wherein the operation of adapting the test group comprises the operation of increasing coverage requirements for at least one state that correlates to the at least one failure in a model for generating the test group.  
     
     
         10 . The apparatus of  claim 1 , wherein the test group provides substantially complete coverage across the application.  
     
     
         11 . Apparatus for problem determination in a distributed service-based application, the apparatus comprising: 
 a memory; and    at least one processor, coupled to the memory, operative to: (i) obtain, via one or more probes, at least one testing result of the application through execution of at least one test group, wherein the test group comprises one or more test cases; and (ii) adaptively refine the at least one executed test group when the at least one testing result comprises at least one failure, to expose at least one problem that caused the at least one failure,    wherein the operation of adaptively refining the at least one executed test group comprises at least one of generating one or more new test cases and refining the one or more probes, and    wherein the operation of adaptively refining the at least one executed test group comprises the operations of disabling one or more probes that returned one or more results that did not relate to the at least one problem and repeating the method of problem determination in a distributed service-based application.    
     
     
         12 . Apparatus for problem determination in a distributed service-based application, the apparatus comprising: 
 a memory; and    at least one processor, coupled to the memory, operative to: (i) obtain, via one or more probes, at least one testing result of the application through execution of at least one test group, wherein the test group comprises one or more test cases; and (ii) adaptively refine the at least one executed test group when the at least one testing result comprises at least one failure, to expose at least one problem that caused the at least one failure,    wherein the operation of adaptively refining the at least one executed test group comprises at least one of generating one or more new test cases and refining the one or more probes, and    wherein the operation of adaptively refining the at least one executed test group comprises the operations of: (a) enabling one or more additional probes that return one or more results relating to the at least one problem; (b) disabling one or more probes that returned one or more results that did not relate to the at least one problem; (c) adapting the test group to comprise at least one test case focused on the at least one problem; and (d) repeating the method of problem determination in a distributed service-based application.    
     
     
         13 . An article of manufacture for problem determination in a distributed service-based application, comprising a machine readable medium containing one or more programs which when executed implement the steps of: 
 obtaining, via one or more probes, at least one testing result of the application through execution of at least one test group, wherein the test group comprises one or more test cases; and    adaptively refining the at least one executed test group when the at least one testing result comprises at least one failure, to expose at least one problem that caused the at least one failure,    wherein the step of adaptively refining the at least one executed test group comprises at least one of generating one or more new test cases and refining the one or more probes, and    wherein the step of obtaining testing results comprises the steps of: (a) generating a test group having at least one test case, from a general model of the application; (b) executing the at least one test case of the test group in the application; (c) passing the at least one result of the test group to an outcome analyzer; and (d) verifying the at least one result against expected output at the outcome analyzer, wherein the step of verifying the at least one result comprises the step of marking each test case of the test group with a failure or a success.