Highly deterministic latency in a distributed system

Abstract

A distributed computing system, such as may be used to implement an electronic trading system, supports a notion of fairness in latency. The system does not favor any particular client. Thus, being connected to a particular access point into the system (such as via a gateway) does not give any particular device an unfair advantage or disadvantage over another. That end is accomplished by precisely controlling latency, that is, the time between when request messages arrive at the system and a time at which corresponding response messages are permitted to leave. The precisely controlled, deterministic latency can be fixed over time, or it can vary according to some predetermined pattern, or vary randomly within a pre-determined range of values.

1 . A method to process messages with a deterministic latency at an electronic sequencer device, the method comprising:
 receiving an incoming message;   sending a forwarded message to one or more compute engines, the forwarded message depending on the incoming message, and the forwarded message including (a) a time based value and (b) a sequence identifier that determines a relative processing order of the forwarded message;   receiving a compute-response message from the one or more compute engines, the compute-response message including information derivable from the time based value;   
       determining a deterministic egress time that depends on both a deterministic latency value and the information derivable from the time based value as received in the compute-response message; and
 causing an outbound message to be sent at the deterministic egress time, the outbound message based on the compute-response message. 
 
     
     
         2 . The method of  claim 1  additionally comprising:
 in response to receiving a second incoming message,
 causing a second outbound message to be sent at a second deterministic egress time, without either (a) sending a second forwarded message that depends on the second incoming message to the one or more compute engines or (b) receiving a second compute-response message related to the second outbound message. 
 
 
     
     
         3 . The method of  claim 1  wherein the compute-response message comprises one or more compute response messages, each associated with a trade match event between two match parties, and wherein causing the outbound message to be sent further comprises:
 causing a first outbound message to be sent based on one of the one or more compute response messages to a first one of the two match parties, and 
 causing a second outbound message to be sent based on one of the one or more compute response messages to a second one of the two match parties, 
 wherein the first outbound message and second outbound message are caused to be sent simultaneously at the deterministic egress time. 
 
     
     
         4 . The method of  claim 1  additionally comprising:
 receiving an incoming message further comprises receiving a first incoming message from a first one of two participant devices; 
 receiving a second incoming message from a second one of the two participant devices; and 
 wherein the time based value depends on an arrival time associated with either the first incoming message or the second incoming message, whichever has a later arrival time. 
 
     
     
         5 . The method of  claim 1  further comprising:
 causing a market data message to be sent at the deterministic egress time, the market data message based on the compute-response message, and the market data message sent to a device associated with a subscriber of a market data stream. 
 
     
     
         6 . The method of  claim 1  additionally comprising:
 receiving an asynchronous message from the one or more compute engines; and 
 causing the asynchronous message to be sent at the deterministic egress time to two or more participant devices. 
 
     
     
         7 . The method of  claim 1  further comprising:
 assigning the deterministic latency from a set of deterministic latencies such that a particular deterministic latency is not subsequently assigned until other deterministic latencies in the set of deterministic latencies have been assigned. 
 
     
     
         8 . The method of  claim 7  wherein assigning the deterministic latency further comprises:
 assigning the deterministic latency such that a number of consecutively assigned, lower valued deterministic latencies is limited to a predetermined number. 
 
     
     
         9 . The method of  claim 1  wherein:
 receiving the incoming message further comprises receiving a first incoming message and receiving a second incoming message, and 
 causing the outbound message to be sent further comprises:
 causing a first outbound message to be sent to a first one of two participant devices according to the deterministic latency; and 
 causing a second outbound message to be sent to a second one of the two participant devices according to another deterministic latency that is different than the deterministic latency associated with the first outbound message. 
 
 
     
     
         10 . The method of  claim 1  wherein:
 causing the outbound message to be sent further comprises:
 causing a first outbound message to be sent to a participant device according to the deterministic latency; and 
 causing a second outbound message to be sent to a subscriber of a market data stream according to another deterministic latency that is different than the deterministic latency associated with the first outbound message. 
 
 
     
     
         11 . The method of  claim 1  wherein the time based value either
 (a) depends on an ingress time of the incoming message; or 
 (b) depends on a desired egress time determined by adding the ingress time of the incoming message to the deterministic latency before the forwarded message is sent to the one or more compute engines, or 
 (c) is an index value that depends on the ingress time of the incoming message and the deterministic latency. 
 
     
     
         12 . The method of  claim 1  additionally comprising:
 detecting an event related to system performance; and 
 modifying the deterministic latency based on the event. 
 
     
     
         13 . The method of  claim 1  additionally comprising:
 receiving the incoming message from a participant device via a gateway; and 
 sending the outbound message to the participant device via the gateway. 
 
     
     
         14 . The method of  claim 13  wherein sending the outbound message further comprises:
 delaying the outbound message at the electronic sequencer device until the deterministic egress time before sending the outbound message to the gateway. 
 
     
     
         15 . The method of  claim 1  additionally comprising:
 determining the time based value at the electronic sequencer device. 
 
     
     
         16 . The method of  claim 1  wherein receiving an incoming message further comprises receiving a first incoming message;
 and the method additionally comprises: 
 receiving a second incoming message; and 
 wherein the first incoming message comprises a first trading protocol and the second incoming message comprises a second trading protocol different from the first trading protocol. 
 
     
     
         17 . The method of  claim 16  additionally comprising:
 causing the outbound message to be sent further comprises causing a first outbound message to be sent using the first trading protocol; 
 
       causing a second outbound message to be sent using the second trading protocol; and 
       wherein the deterministic latency is the same for the first trading protocol and the second trading protocol. 
     
     
         18 . The method of  claim 16  additionally comprising:
 causing the outbound message to be sent further comprises causing a first outbound message to be sent using the first trading protocol; 
 causing a second outbound message to be sent using the second trading protocol; and 
 wherein the deterministic latency is different for the first trading protocol and the second trading protocol. 
 
     
     
         19 . An electronic sequencer device configured to:
 receive an incoming message;   send a forwarded message to one or more compute engines, the forwarded message depending on the incoming message, and the forwarded message including (a) a time based value and (b) a sequence identifier that determines a relative processing order of the forwarded message;   receive a compute-response message from the one or more compute engines, the compute-response message including information derivable from the time based value;   
       determine a deterministic egress time that depends on both a deterministic latency value and the information derivable from the time based value as received in the compute-response message; and
 cause an outbound message to be sent at the deterministic egress time, the outbound message based on the compute-response message. 
 
     
     
         20 . A method for performing electronic trading, the method comprising:
 receiving, at a sequencer node, an incoming message representing an electronic trade request with an offer to buy or sell a financial instrument;   determining, at the sequencer node, a sequence-marked message being a sequenced version of the incoming message that includes a time based value;   forwarding, from the sequencer node, the sequence-marked message with the time based value to a core compute node;   receiving, at the sequencer node, a compute response message that includes a result of an electronic trading matching function in accordance with a relative ordering of the sequence-marked message, the compute response message also including information derivable from the time based value;   determining, at the sequencer node, a deterministic egress time that depends on both a deterministic latency value and the information derivable from the time based value as received in the compute response message; and   sending an outbound message at the deterministic egress time, the outbound message based on the compute response message.   
     
     
         21 . The method of  claim 20  additionally comprising:
 receiving, at the core compute node, the sequence-marked message with the time based value from the sequencer node; 
 determining, at the core compute node, relative ordering of the sequence-marked message among sequence-marked versions of other messages received by the core compute node; 
 completing, at the core compute node, the electronic trading matching function in accordance with the relative ordering determined, the completing causing the offer to be matched with a counteroffer for the financial instrument; 
 generating, at the core compute node, the compute response message based on the completing and based on the information derivable from the time based value; and 
 forwarding, from the core compute node, the compute response message with the information derivable from the time based value, to the sequencer node.