AI Accelerator Cores with Integrated CPU for External Communication Over Ethernet

Abstract

A multicore processor includes a Network-on-Chip (NoC) that coordinates operations of a set of AI accelerator cores. The multicore processor also includes a central processing unit (CPU) that accesses information regarding the operation of the AI accelerator cores via the NoC. Based on this information the CPU determines that an operation of the AI accelerator cores requires information that is accessible from a system that is accessible over an Ethernet link. Rather than accessing the Ethernet link through a bus and host associated with the multicore processor, the multicore processor includes an Ethernet node that establishes and maintains direct communications over the Ethernet link. The CPU administers the configuration and operation of the Ethernet node based on the requirements of the AI accelerator nodes.

What is claimed is: 
     
         1 . A multicore processor comprising:
 a set of artificial intelligence (“AI”) accelerator cores;   at least one Ethernet node;   at least one central processing unit (“CPU”); and   a network on chip (“NoC”) that networks the set of AI accelerator cores, the at least one Ethernet node, and the CPU, wherein the CPU executes instructions to administrate Ethernet transfers over an Ethernet link for the set of AI accelerator cores using the at least one Ethernet node.   
     
     
         2 . The multicore processor of  claim 1 , wherein the Ethernet link comprises a first Ethernet link, and wherein the Ethernet transfers are performed without utilizing a host that is in communication with the multicore processor or a second Ethernet link connected to the host. 
     
     
         3 . The multicore processor of  claim 1 , wherein the NoC collects information from the AI accelerator cores for use by the CPU to administrate the Ethernet transfers. 
     
     
         4 . The multicore processor of  claim 3 , wherein the information collected from the AI accelerator cores comprises information from one or more of the AI accelerator cores, information from a physical grouping of AI accelerator cores, or information from a logical grouping of AI accelerator cores. 
     
     
         5 . The multicore processor of  claim 3 , wherein the information collected from the AI accelerator cores comprises information accessed from memory of one or more of the AI accelerator cores or information from communications between AI accelerator cores. 
     
     
         6 . The multicore processor of  claim 3 , wherein, based on the information collected from the AI accelerator cores, the CPU determines that one or more of the AI accelerator cores require access to data from an external system via the Ethernet link, and wherein the determination causes the CPU to execute the instructions. 
     
     
         7 . The multicore processor of  claim 3 , wherein, based on the information collected from the AI accelerator cores, the CPU predicts that one or more of the AI accelerator cores will require access to data from an external system via the Ethernet link, and wherein the prediction causes the CPU to execute the instructions. 
     
     
         8 . The multicore processor of  claim 1 , wherein the instructions to administrate Ethernet transfers comprise configuration instructions and transmit instructions, wherein the CPU executes the configuration instructions to configure the at least one Ethernet node to establish Ethernet communications over the Ethernet link with an external system, and wherein the CPU executes the transmit instructions to transmit messages to the external system over the Ethernet link after the Ethernet communications are established. 
     
     
         9 . The multicore processor of  claim 8 , wherein the configuration instructions cause the at least one Ethernet node to establish or close links, set procedures for MAC resolution, establish network protocols, select between transmission control protocol (“TCP”) or user datagram protocol (“UDP”) protocols, or set error handling procedures. 
     
     
         10 . The multicore processor of  claim 1 , wherein the CPU comprises a first CPU, and wherein each Ethernet node of the at least one Ethernet node comprises:
 a router configured to communicate with the NoC;   a network interface unit (“NIU”) configured to administrate operations within the at least one Ethernet node;   a second CPU, wherein the second CPU comprises a lower-power CPU than the first CPU, and wherein the second CPU is configured to execute instructions to configure operations of the Ethernet node;   a memory comprising the instructions to configure operation of the at least one Ethernet node and storage for data to be exchanged between the NoC and the Ethernet link; and   an Ethernet interface configured to prepare messages to be transmitted over the Ethernet link and to process messages received from the Ethernet link.   
     
     
         11 . The multicore processor of  claim 10 , further comprising an overlay stream unit configured to monitor exchanges of data between the NIU and the memory and coordinate transmissions of the messages over the Ethernet link. 
     
     
         12 . The multicore processor of  claim 11 , wherein the overlay stream unit is further configured to monitor exchanges of data between the Ethernet interface and the memory to coordinate processing of data received via the Ethernet link. 
     
     
         13 . The multicore processor of  claim 10 , further comprising a register cross bar, wherein the register cross bar communicates with the second CPU to initiate changes to a configuration of the Ethernet interface based on the CPU executing the instructions to configure operations. 
     
     
         14 . The multicore processor of  claim 10 , wherein the Ethernet interface comprises an Ethernet transmit/receive controller and a MAC/PCS/PHY controller, wherein the Ethernet transmit/receive controller controls a transmission and reception of messages over the Ethernet link and the MAC/PCS/PHY controller performs media access control processing, physical coding sublayer operations, and physical layer communications for the Ethernet link. 
     
     
         15 . A method for a multicore processor to directly communicate via an Ethernet link, comprising:
 monitoring, by a network on chip (“NoC”), a set of artificial intelligence (“AI”) accelerator cores;   determining, by a central processing unit (“CPU”) of the multicore processor based on the monitoring, that the AI accelerator cores require information that is available via the Ethernet link;   generating, by the CPU, a message for transmission via the Ethernet link;   providing, by the CPU via the NoC, the message for transmission to an Ethernet node of the multicore processor;   processing, by the Ethernet node, the message for transmission as packets for transmission over the Ethernet link; and   transmitting, by the Ethernet node, the packets over the Ethernet link.   
     
     
         16 . The method of  claim 15 , further comprising:
 determining, by the CPU based on the monitoring of the AI accelerator cores, that a configuration of the Ethernet link requires modification;   providing, from the CPU to the Ethernet node via the NoC, an instruction to modify the configuration of the Ethernet link; and   modifying, by the Ethernet node based on the instruction, the configuration of the Ethernet link.   
     
     
         17 . The method of  claim 15 , wherein the monitoring comprises monitoring information from one or more of the AI accelerator cores, information from a physical grouping of AI accelerator cores, or information from a logical grouping of AI accelerator cores. 
     
     
         18 . An artificial intelligence processing system, comprising:
 a host processor;   a first Ethernet link connected to the host processor; and   a multicore processor in communication with the host processor, the multicore processor comprising:
 a set of artificial intelligence (“AI”) accelerator cores; 
 at least one Ethernet node; 
 at least one central processing unit (“CPU”); and 
 a network on chip (“NoC”) that networks the set of AI accelerator cores, the at least one Ethernet node, and the CPU, 
   
       wherein the CPU executes instructions to administrate Ethernet transfers over a second Ethernet link directly accessible to the multicore processor for the set of AI accelerator cores using the at least one Ethernet node, and wherein the Ethernet transfers are performed without utilizing the host processor or the first Ethernet link. 
     
     
         19 . The artificial intelligence processing system of  claim 18 , wherein the NoC collects information from the AI accelerator cores for use by the CPU to administrate the Ethernet transfers. 
     
     
         20 . The artificial intelligence processing system of  claim 19 , wherein the information collected from the AI accelerator cores comprises information from one or more of the AI accelerator cores, information from a physical grouping of AI accelerator cores, or information from a logical grouping of AI accelerator cores.