System, method and computer program product to maximize server throughput while avoiding server overload by controlling the rate of establishing server-side net work connections

Abstract

A method for maximizing server throughput while avoiding overload of a server is presented. The method involves intercepting, via an interface unit, a client request for information from the server. Next, the interface unit determines the current server performance, where the server performance is based on the number of connections opened to the server, the response time of the server and the rate at which the response time is changing. Finally, the interface unit forwards the client request to the server if the current server performance is close to an optimal performance, whereby avoiding overload of the server.

1. A method for managing throughput while avoiding overload of one or more servers in an environment including an interface unit intercepting requests from clients to a server, transmitting the intercepted requests to the server, and intercepting responses to the requests transmitted by the server to the clients, the method comprising the steps of:
 transmitting, by an interface unit, client requests to a server to maintain performance of server throughput within a predetermined threshold range of a server throughput function, the predetermined threshold range corresponding to values of the server throughput function based on a number of client requests to the server via the interface unit per second of time over a total number of clients communicating with the server via the interface unit; 
 monitoring, by the interface unit, responses to client requests intercepted by the interface unit, changes in response times from the server and changes of a rate in which the response times change; 
 intercepting, by the interface unit, a request from a client to open a transport layer connection with the server; 
 determining, by the interface unit, from the monitoring that the performance of the server throughput exceeds the predetermined threshold range; 
 buffering, by the interface unit in response to the determination, the intercepted request in a queue; and 
 transmitting, by the interface unit, the buffered request to the server upon the interface unit determining that the performance of server throughput is within the predetermined threshold range of the server throughput function and that a total number of clients currently communicating with the server per second of time is less than a predetermined number of clients per second of time. 
 
   
   
     2. The method of  claim 1 , wherein the predetermined threshold range comprises one of a maximum threshold range or an optimal threshold range for server throughput. 
   
   
     3. The method of  claim 2 , wherein the predetermined threshold range comprises a first threshold at a lower point in the predetermined threshold range and a second threshold at a higher point in the predetermined threshold range, the first threshold represents one of a faster response time, a lesser number of users, or a greater number of connections than the second threshold. 
   
   
     4. The method of  claim 3 , comprising transmitting, by the interface unit, client requests to the server to maintain performance of server throughput one of at or near the first threshold. 
   
   
     5. The method of  claim 1 , comprising determining, by the interface unit, the performance of the server throughput based on monitoring one or more of: the number of active connections opened to the server, the response time of the server, the rate at which the response time is changing, and the intercepted request. 
   
   
     6. The method of  claim 1 , determining, by the interface unit, the performance of the server throughput based on a first portion of server resources available to service existing clients and a second portion of server resources available to accept new clients. 
   
   
     7. The method of  claim 1 , comprising identifying a preferred client value for the request of the client, and determining the position of the client request in the queue based on the preferred client value. 
   
   
     8. The method of  claim 7 , comprising determining, by the interface unit, the preferred client value from one or more of the internet address of the client request, the port number of the client request, by a header related to the client request, by previous requests from the client of the client request, and by a cookie related to the client request. 
   
   
     9. The method of  claim 1 , comprising pooling, by the interface unit, a plurality of transport layer connections to the server. 
   
   
     10. The method of  claim 9 , comprising multiplexing, by the interface unit, client requests via the pooled plurality of transport layer connections. 
   
   
     11. The method of  claim 1 , comprising closing, by the interface unit, transport layer connections to the server to bring performance of server throughput within the predetermined threshold range. 
   
   
     12. The method of  claim 1 , comprising determining, by the interface unit, the performance of server throughput by one of a number of requests pending at the server or server error/overload messages from the server. 
   
   
     13. The method of  claim 1 , comprising establishing, by the interface unit, the transport layer connection with the client in response to the request from the client. 
   
   
     14. The method of  claim 1 , comprising opening, by the interface unit, a second transport layer connection to the server if there is not a free transport layer connection to the server. 
   
   
     15. The method of  claim 1 , wherein monitoring further comprises the interface unit monitoring changes in times between the interface unit forwarding intercepted client requests to the server and the interface unit receiving the responses to the forwarded intercepted client requests from the server and changes of a rate in which differences between the changes in times change. 
   
   
     16. A system having an interface unit for managing throughput while avoiding overload of one or more servers, the interface unit intercepting requests from clients to a server, transmitting the intercepted requests to the server, and intercepting responses to the requests transmitted by the server to the clients, the system comprising;
 a device comprising a processor and configured as an interface unit transmitting client requests to a server to maintain performance of server throughput within a predetermined threshold range of a server throughput function, the predetermined threshold range corresponding to a value of the server throughput function based on a number of client requests to the server via the interface unit per second of time over a total number of clients communicating with the server via the interface unit, and intercepting a request from a client to open a transport layer connection with the server; and 
 a queue for storing intercepted client requests; 
 wherein the interface unit determines from monitoring responses to client requests intercepted by the interface unit, changes in response times of the server and changes of a rate in which the response times change, that the performance of the server throughput exceeds the predetermined threshold range, and buffers in the queue the intercepted request in response to the determination; and 
 wherein the interface unit transmits the buffered request upon determining that the performance of server throughput is within the predetermined threshold range of the server throughput function and that a total number of clients currently communicating with the server per second of time is less than a predetermined number of clients per second of time. 
 
   
   
     17. The system of  claim 16 , wherein the predetermined threshold range comprises one of a maximum threshold range or an optimal threshold range for server throughput. 
   
   
     18. The system of  claim 17 , wherein the predetermined threshold range comprises a first threshold at a lower point in the predetermined threshold range and a second threshold at a higher point in the predetermined threshold range, the first threshold represents one of a faster response time, a lesser number of users, or a greater number of connections than the second threshold. 
   
   
     19. The system of  claim 18 , wherein the interface unit transmits client requests to the server to maintain performance of server throughput one of at or near the first threshold. 
   
   
     20. The system of  claim 16 , wherein the interface unit determines the performance of the server throughput based on monitoring one or more of: the number of active connections opened to the server, the response time of the server, the rate at which the response time is changing, and the buffered request. 
   
   
     21. The system of  claim 16 , wherein the interface unit determines the performance of the server throughput based on a first portion of server resources available to service existing clients and a second portion of server resources available to accept new clients. 
   
   
     22. The system of  claim 16 , wherein the interface unit identifies a preferred client value for the request of the client, and determines the position of the client request in the queue based on the preferred client value. 
   
   
     23. The system of  claim 16 , wherein the interface unit determines the preferred client value from one or more of the internet address of the client request, the port number of the client request, by a header related to the client request, by previous requests from the client of the client request, and by a cookie related to the client request. 
   
   
     24. The system of  claim 16 , wherein the interface unit pools a plurality of transport layer connections to the server. 
   
   
     25. The system of  claim 24 , wherein the interface unit multiplexes client requests via the pooled plurality of transport layer connections. 
   
   
     26. The system of  claim 16 , wherein the interface unit closes the transport layer connection to the server to bring performance of server throughput within the predetermined threshold range. 
   
   
     27. The system of  claim 16 , wherein the interface unit determines the performance of server throughput by one of a number of requests pending at the server or server error/overload messages from the server. 
   
   
     28. The system of  claim 16 , wherein the interface unit establishes the transport layer connection with the client in response to the request from the client. 
   
   
     29. The system of  claim 16 , wherein the interface unit opens a second transport layer connection to the server if there is not a free transport layer connection to the server. 
   
   
     30. The system of  claim 16 , wherein the interface unit opens a second transport layer connection to the server if the queue comprises one or more requests from a second client. 
   
   
     31. A method for managing throughput while avoiding overload of one or more servers in an environment including an interface unit intercepting requests from clients to a server, transmitting the intercepted requests to the server, and intercepting responses to the requests transmitted by the server to the clients, the method comprising the steps of:
 transmitting, by a device configured as an interface unit, client requests to a server to maintain performance of server throughput within a predetermined threshold of a server throughput function, the predetermined threshold corresponding to a value of the server throughput function that is based on a number of client requests to the server via the interface unit per second of time over a total number of clients communicating with the server via the interface unit; 
 monitoring, by the interface unit, changes in times between the interface unit forwarding intercepted client requests to the server and the interface unit receiving the responses to the forwarded intercepted client requests from the server and changes of a rate in which differences between the changes in times change; 
 intercepting, by the interface unit, a request from a client to open a connection with the server; 
 determining, by the interface unit, from the monitoring that the performance of the server throughput exceeds the predetermined threshold; 
 buffering, by the interface unit in response to the determination, the intercepted request in a queue; and 
 transmitting, by the interface unit, the buffered request to the server upon the interface unit determining that the performance of server throughput is within the predetermined threshold of the server throughput function and that a total number of clients currently communicating with the server per second of time is less than a predetermined number of clients per second of time.