Handling stalling event for multiple thread pipeline, and triggering action based on information access delay

Abstract

A processing pipeline for processing instructions with instructions from multiple threads in flight concurrently may have control circuitry to detect a stalling event associated with a given thread. In response, at least one instruction of the given thread may be flushed from the pipeline, and the control circuitry may trigger fetch circuitry to reduce a fraction of the fetched instructions which are fetched from the given thread. A mechanism is also described to determine when to trigger a predetermined action when a delay in accessing information becomes greater than a delay threshold, and to update the delay threshold based on a difference between a return delay when the information is returned from the storage circuitry and the delay threshold.

We claim: 
     
         1 . An apparatus comprising:
 a processing pipeline to process instructions with instructions from a plurality of threads in flight in the processing pipeline concurrently, comprising fetch circuitry to fetch instructions from the plurality of threads for processing by the processing pipeline; and   control circuitry to detect a stalling event associated with a given thread of said plurality of threads, and in response to detecting the stalling event, to flush from said processing pipeline at least one pending instruction of said given thread, and to trigger said fetch circuitry to reduce a fraction of fetched instructions which are fetched from said given thread.   
     
     
         2 . The apparatus according to  claim 1 , wherein in response to detecting the stalling event, the control circuitry is configured to retain at least one pending instruction of said given thread in said processing pipeline. 
     
     
         3 . The apparatus according to  claim 1 , wherein after elapse of a period of time following detection of the stalling event, the control circuitry is configured to trigger said fetch circuitry to increase the fraction of fetched instructions which are fetched from said given thread. 
     
     
         4 . The apparatus according to  claim 1 , wherein in response to receipt of an indication of resolution of the stalling event, the control circuitry is configured to trigger said fetch circuitry to increase the fraction of fetched instructions which are fetched from said given thread. 
     
     
         5 . The apparatus according to  claim 1 , wherein the processing pipeline comprises at least one shared tracking structure to track pending instructions from the plurality of threads. 
     
     
         6 . The apparatus according to  claim 5 , wherein said at least one shared tracking structure comprises at least one of:
 a decode queue to queue pending instructions to be decoded;   an issue queue to queue pending instructions to be issued for execution;   a load/store queue to queue pending load instructions for loading data from a data store or pending store instructions for storing data to the data store; and   a reorder buffer to store information for tracking completion of execution of instructions by the processing pipeline.   
     
     
         7 . The apparatus according to  claim 1 , wherein the stalling event comprises a delay in accessing information from storage circuitry in response to a pending instruction of the given thread. 
     
     
         8 . The apparatus according to  claim 7 , wherein the control circuitry is configured to detect the stalling event when the delay in accessing the information from the data store becomes greater than a threshold delay. 
     
     
         9 . The apparatus according to  claim 7 , wherein the storage circuitry comprises a plurality of storage levels and the control circuitry is configured to detect the stalling event when a request for the information misses in a predetermined storage level of the storage circuitry. 
     
     
         10 . The apparatus according to  claim 1 , wherein the control circuitry is configured to detect the stalling event when a pending instruction of the given thread requires a hardware unit which is currently in a power saving state. 
     
     
         11 . The apparatus according to  claim 2 , wherein said at least one pending instruction retained in said processing pipeline comprises an oldest pending instruction of said given thread. 
     
     
         12 . The apparatus according to  claim 1 , wherein said processing pipeline supports out-of-order execution of instructions, and comprises an issue queue to track pending instructions to be issued for execution and a reorder buffer to track completion of execution of pending instructions by the processing pipeline; and
 in response to detecting said stalling event, the control circuitry is configured to detect an oldest uncompleted instruction of said given thread among the pending instructions tracked by said issue queue and said reorder buffer, and to flush from said processing pipeline said oldest uncompleted instruction and any younger instruction of said given thread.   
     
     
         13 . The apparatus according to  claim 2 , wherein said at least one pending instruction retained in said processing pipeline comprises a stalled instruction of said given thread which triggered said stalling event. 
     
     
         14 . The apparatus according to  claim 13 , wherein said at least one pending instruction retained in said processing pipeline also comprises N subsequent instructions of said given thread following said stalled instruction, where N≧1. 
     
     
         15 . The apparatus according to  claim 13 , wherein said at least one pending instruction retained in said processing pipeline also comprises at least one subsequent instruction of said given thread whose outcome is independent of an outcome of said stalled instruction. 
     
     
         16 . The apparatus according to  claim 13 , wherein said at least one pending instruction flushed from said processing pipeline comprises a next stallable instruction of said given thread and any subsequent instruction following the next stallable instruction in said given thread;
 said next stallable instruction comprising a next instruction of said given thread following the stalled instruction that is of an instruction type for which said stalling event can occur.   
     
     
         17 . A data processing method comprising:
 fetching instructions from a plurality of threads for processing by a processing pipeline, wherein instructions from the plurality of threads are in flight in the processing pipeline concurrently;   detecting a stalling event associated with a given thread of said plurality of threads; and   in response to detecting the stalling event, flushing from said processing pipeline at least one pending instruction of said given thread, and reducing a fraction of fetched instructions which are fetched from said given thread.   
     
     
         18 . An apparatus comprising:
 control circuitry to trigger a predetermined action when a delay in accessing information from storage circuitry becomes greater than a delay threshold; and   threshold updating circuitry to determine a return delay when the information is returned from the storage circuitry and to adjust the delay threshold in dependence on a difference between the return delay and the delay threshold.   
     
     
         19 . The apparatus according to  claim 18 , wherein the storage circuitry comprises a plurality of storage levels. 
     
     
         20 . The apparatus according to  claim 19 , wherein the control circuitry is configured to start counting the delay in accessing information from the storage circuitry in response to an access request for information from said storage circuitry missing in a predetermined storage level of the storage circuitry. 
     
     
         21 . The apparatus according to  claim 19 , wherein the predetermined action comprises estimating whether an access request missed in a given storage level of the storage circuitry based on whether the delay is greater than the delay threshold. 
     
     
         22 . The apparatus according to  claim 18 , comprising a processing pipeline to process fetched instructions from a plurality of threads;
 wherein the predetermined action comprises flushing from the processing pipeline at least one instruction from a given thread which requested the information from the storage circuitry and reducing a fraction of the fetched instructions which are fetched from said given thread.   
     
     
         23 . The apparatus according to  claim 18 , wherein the predetermined action comprises placing at least a portion of processing circuitry in a power saving state while awaiting the information from the storage circuitry. 
     
     
         24 . The apparatus according to  claim 18 , wherein the predetermined action comprises updating profiling information. 
     
     
         25 . The apparatus according to  claim 18 , wherein the threshold updating circuitry is configured to increase the delay threshold in response to the difference between the return delay and the delay threshold being smaller than a first amount for at least one access request for information from the data store. 
     
     
         26 . The apparatus according to  claim 18 , wherein the threshold updating circuitry is configured to reduce the delay threshold in response to the difference between the final delay and the delay threshold being greater than a second amount for at least one access request for information from the data store. 
     
     
         27 . The apparatus according to  claim 18 , comprising at least one confidence counter to store at least one confidence value;
 wherein the threshold updating circuitry is configured to increment or decrement said at least one confidence value in dependence on the difference between the return delay and the delay threshold, and to determine whether to update said delay threshold based on said at least one confidence value.   
     
     
         28 . The apparatus according to  claim 18 , comprising a first confidence counter to store a first confidence value, and a second confidence counter to store a second confidence value;
 when t i −T is smaller than a first amount, where t i  is the return delay and T is the delay threshold, the threshold updating circuitry is configured to adjust the first confidence value to move towards a first predetermined value, and to increase the delay threshold when the adjustment to the first confidence value causes the first confidence value to reach said first predetermined value;   when t i −T is greater than a second amount, the threshold updating circuitry is configured to adjust the first confidence value to move away from the first predetermined value, to adjust the second confidence value to move towards a second predetermined value, and to reduce the delay threshold when the adjustment to the second confidence value causes the second confidence value to reach said second predetermined value; and   when T−t i  is smaller than a third amount, the threshold updating circuitry is configured to adjust the second confidence counter to move away from the second predetermined value.   
     
     
         29 . The apparatus according to  claim 28 , wherein the third amount equals the second amount. 
     
     
         30 . The apparatus according to  claim 28 , wherein the first confidence counter and the second confidence counter comprise saturating counters, and the second confidence counter comprises a greater number of bits than the first confidence counter. 
     
     
         31 . A data processing method comprising:
 triggering a predetermined action when a delay in accessing information from storage circuitry becomes greater than a delay threshold;   determining a return delay when the information is returned from the storage circuitry; and   adjusting the delay threshold in dependence on a difference between the return delay and the delay threshold.