System and method for efficient remote disk I/O

Abstract

When a client computer requests data from a disk or similar device at a server computer, the client exports the memory associated with an allocated read buffer by generating and storing one or more incoming MMU (IMMU) entries that map the read buffer to an assigned global address range. The remote data read request, along with the assigned global address range is communicated to the server node. At the server, the request is serviced by performing a memory import operation, in which one or more outgoing MMU (OMMU) entries are generated and stored for mapping the global address range specified in the read request to a corresponding range of local physical addresses. The mapped local physical addresses in the server are not locations in the server's memory. The server then performs a DMA operation for directly transferring the data specified in the request message from the disk to the mapped local physical addresses. The DMA operation transmits the specified data to the server's network interface, at which the mapped local physical addresses to which the data is transferred are converted into the corresponding global addresses. The specified data with the corresponding global addresses are then transmitted to the client node. The client converts the global addresses in the received specified data into the local physical addresses corresponding to the allocated receive buffer, and stores the received specified data in the allocated receive buffer.

What is claimed is: 
     
       1. A method of performing a remote disk read operation between first and second computers, comprising the steps of: at the first computer: mapping a range of physical local addresses associated with a receive buffer to a corresponding range of global addresses;   sending a request message to the second computer, the request message specifying data to be retrieved from the disk located at the second computer and the range of global addresses associated with the allocated receive buffer;     at the second computer, responding to receipt of the request message by: mapping the range of global addresses specified in the request message to a corresponding range of local physical addresses, wherein the mapped local physical addresses do not denote locations in a memory in the second computer's memory;   performing a DMA operation for directly transferring the data specified in the request message from the disk to the mapped local physical addresses, the DMA operation transmitting the specified data to a network interface in the second computer at which the mapped local physical addresses to which the data is transferred are converted into the corresponding global addresses; and   transmitting the specified data with the corresponding global addresses to the first computer; and     at the first computer, responding to receipt of the specified data by: converting the global addresses transmitted with the received specified data into the local physical addresses corresponding to the allocated receive buffer; and   storing the received specified data in the allocated receive buffer.     
     
     
       2. A method of performing a remote disk read operation between first and second computers, comprising the steps of: at the first computer, responding to an application program request for data from a disk located at the second computer by: allocating a receive buffer in memory in the first computer, the receive buffer having an associated range of local physical addresses;   storing an entry in an input memory mapping unit in the first computer for mapping the local physical address range associated with the allocated receive buffer to a corresponding range of global addresses; and   sending a request message to the second computer, the request message specifying data to be retrieved from the disk located at the second computer and the range of global addresses associated with the allocated receive buffer;     at the second computer, responding to receipt of the request message by: storing an entry in an output memory mapping unit in the second computer for mapping the range of global addresses specified in the request message to a corresponding range of local physical addresses, wherein the mapped local physical addresses do not denote locations in a memory in the second computer's memory; and   performing a DMA operation for directly transferring the data specified in the request message from the disk to the mapped local physical addresses, the DMA operation transmitting the specified data to a network interface in the second computer at which the mapped local physical addresses to which the data is transferred are converted into the corresponding global addresses; and   transmitting the specified data with the corresponding global addresses to the first computer; and     at the first computer, responding to receipt of the specified data by: converting the global addresses transmitted with the received specified data into the local physical addresses corresponding to the allocated receive buffer; and   storing the received specified data in the allocated receive buffer.     
     
     
       3. The method of claim 2, wherein at the second computer the entry in an output memory mapping unit is deactivated after the specified data is successfully transmitted to the first computer; and   at the first computer the entry in the input memory mapping unit is deactivated after the specified data is successfully received from the second computer.   
     
     
       4. In a distributed computer system, apparatus for performing a remote disk read operation between first and second computers, comprising: at the first computer: a CPU;   memory, including memory in which a receive buffer is allocated;   an input memory management unit (IMMU);   a network interface;   a network interface driver procedure, executable by the first computer's CPU, that: stores in the IMMU an entry for mapping a range of physical local addresses associated with the receive buffer to a corresponding range of global addresses; and   a file system procedure, executable by the first computer's CPU, that sends a request message to the second computer via the network interface and a communication channel coupled to the network interface, the request message specifying data to be retrieved from a disk located at the second computer and the range of global addresses associated with the allocated receive buffer;     at the second computer: a CPU;   memory;   the disk that stores the data specified in the request message;   an output memory management unit (OMMU);   a network interface coupled to the communication channel;   a network interface driver, executable by the CPU, that: stores in OMMU an entry that maps the range of global addresses specified in the request message to a corresponding range of local physical addresses, wherein the mapped local physical addresses do not denote locations in a memory in the second computer's memory; and   a disk controller, coupled to the disk, wherein the disk controller is coupled to the network interface by an internal bus; the disk controller including DMA logic for directly transferring the data specified in the request message from the disk to the mapped local physical addresses, the DMA operation transmitting the specified data to the network interface in the second computer at which (A) the mapped local physical addresses to which the data is transferred are converted into the corresponding global addresses in accordance with the entry stored in the OMMU, and (B) the specified data is transmitted with the corresponding global addresses to the first computer via the communication channel;     wherein, at the first computer, the network interface includes logic for responding to receipt of the specified data by (A) converting the global addresses transmitted with the received specified data into the local physical addresses corresponding to the allocated receive buffer, and (B) storing the received specified data in the allocated receive buffer.   
     
     
       5. The apparatus of claim 4, wherein the network interface driver in the second computer deactivates the OMMU entry after the specified data is successfully transmitted to the first computer; and   the network interface driver in the first computer deactivates the IMMU entry after the specified data is successfully received from the second computer.