WO2009033969A1

WO2009033969A1 - Method and apparatus for digital data storage

Info

Publication number: WO2009033969A1
Application number: PCT/EP2008/061459
Authority: WO
Inventors: Dieter Haupt; Johann Maas; Kai Dorau
Original assignee: Thomson Licensing
Priority date: 2007-09-13
Filing date: 2008-09-01
Publication date: 2009-03-19

Abstract

The present invention relates to a system (100) and method (200) for digital data storage. A method (200) according to the invention comprises determining (206) whether a data packet received via a digital serial interface (102) is initial or continuous and issuing (210) an instruction to continue packet processing if the data packet is determined to be initial. A system (100) according to the invention comprises a digital serial interface (102) through which read packets, write packets and other packets are received. The system (100) further comprises an interface module/packet filter (104) that is adapted to deliver the read packets and the write packets to an autonomous payload data module (108) and to deliver the other packets to a processor (106). The autonomous payload data module (108) is adapted to determine whether the read packets and/or the write packets are initial or continuous and to issue an instruction to continue packet processing if the read packets and/or the write packets are determined to be initial.

Description

Method and Apparatus for Digital Data Storage

The present invention relates to the field of storing digital data via a digital interface. In particular, exemplary embodiments of the present invention relate to the autonomous processing of payload data within a digital interface.

Ethernet is one example of a digital interface. To be more particular,10G Ethernet is a further example of a digital interface. Typically, all of the data, received by a digital interface or sent with help thereof, is moved between memory and the digital interface under control of a processor that executes software that manages an appropriate data flow by performing operations such as packet filtering, determination of source and destination addresses and the like.

In many digital data transfers, payload data, especially in the case of using the IP/UDP/TCP/RPC/NFS protocol layers, are filtered and switched to different data paths:

One path for "payload data". This path is intended to be implemented in hardware. Typically, packets containing RPC/NFS WRITE and RPC/NFS READ call are switched to this path.

A second path for "control". This path is intended to be implemented in software running on a processor. Typically, packets containing RPC/NFS commands other than the READ or WRITE call described above are switched to this path.

U.S. Patent Application Publication No. 20050223166 to Kanai, et al. purports to disclose a method for preventing data corruption in a Floppy Diskette Controller. The disclosed method determines the potential for data loss and/or data corruption on a data transfer by determining if, before each data transfer byte is read out or written in the floppy diskette in DMA mode, the maximum delay time for DMA request from the issue to the removal is greater than a specific value, and initializing a specific process by the computer system according to the comparison result. U.S. Patent Application Publication No. 20060098653 to Adams, et al. purports to disclose stateless storage accelerator modules that comprise network interfaces and filter functions that enable the accelerated processing of stateless protocols associated with network storage. Filter functions examine packets received over the network interface according to contexts associated with the destination of the packets. Contexts are found quickly through a fast look-up to ensure high performance of the over all system. The contexts comprise information associated with storage areas including addresses associated with the storage areas.

U.S. Patent Application Publication No. 20040030745 to Boucher, et al. purports to disclose an intelligent network interface card or communication processing device that works with a host computer for data communication. The device provides a fast-path that avoids protocol processing for most messages, greatly accelerating data transfer and offloading time-intensive processing tasks from the host CPU. The host retains a fallback processing capability for messages that do not fit fast-path criteria, with the device providing assistance such as validation even for slow-path messages, and messages being selected for either fast-path or slow-path processing. A context for a connection is defined that allows the device to move data, free of headers, directly to or from a destination or source in the host. The context can be passed back to the host for message processing by the host. The device contains specialized hardware circuits that are much faster at their specific tasks than a general purpose CPU. A preferred embodiment includes a trio of pipelined processors devoted to transmit, receive and utility processing, providing full duplex communication for four Fast Ethernet nodes.

U.S. Patent Application Publication No. 20020152339 to Yamamoto purports to disclose a storage system that includes a storage controller and storage media for reading data from or writing data to the storage media in response to SCSI, NFS, CIFS, or HTTP type read/write requests. The storage controller includes SCSI, NFS, CIFS, and HTTP interface adapters for receiving the read/write requests and effecting the reading of data from or the writing of data to the storage media.

U.S. Patent Application Publication No. 20030046330 to Hayes purports to disclose a method and apparatus for the selective offloading of protocol processing are disclosed. In the disclosed system, computationally intensive and memory bandwidth intensive protocol processing tasks are offloaded from the host processor of a computer to an auxiliary processor. In a preferred embodiment, the auxiliary processor has the ability to return the requested task, thereby allowing complex, non-performance oriented tasks to be performed by the host processor. This enables the auxiliary processor to have necessary resources for the specific tasks for which it has been designed, and does not require that the auxiliary processor has enough resources to accomplish the task of offloading the entire network protocol processing task. In one embodiment, the auxiliary processor may refuse requests to offload additional tasks from the host processor when resources are low. In a preferred embodiment, the auxiliary processor is able to discern between various network applications running over the same network protocol and treat them differently, even though both applications are utilizing the same network and transport protocols. This capability allows the optimization of the protocol processing for each network application.

An improved system and method that facilitates efficient data transfer at high speeds while conserving processor power for other tasks is desired.

A method in accordance with the present invention is set forth in claim 1. The method comprises determining whether a data packet received via a digital serial interface is initial or continuous and issuing an instruction to continue packet processing if the data packet is determined to be initial.

A method according to the invention may further comprise determining whether a set of request identifiers associated with the data packet plus a first offset is unequal to a set of identifiers and a second offset plus a length of a previous data packet. The method may further comprise converting the data packets to a cluster of a first length for storage on a storage system if the data packets are write packets and converting the data packets to a series of interface packets of a second length if the data packets are read packets.

In addition, a method according to the invention may comprise performing a permission check to determine whether a requester has permission to authorize transfer of the data packet. Finally, a method according to the invention may comprise converting data in a payload of the data packet to or from a data cluster depending on whether a transaction that initiated transfer of the data packet is a READ call or a WRITE call.

A system according to the present invention is set forth in claim 6. The system comprises a digital serial interface through which read packets, write packets and other packets are received. The system further comprises an interface module/packet filter that is adapted to deliver the read packets and the write packets to an autonomous payload data module and to deliver the other packets to a processor. The autonomous payload data module is adapted to determine whether the read packets and/or the write packets are initial or continuous and to issue an instruction to continue packet processing if the read packets and/or the write packets are determined to be initial.

In a system according to the present invention, the autonomous payload data module may be adapted to convert the write packets to a cluster of a first length for storage on a storage system and to convert the read packets to a series of interface packets of a second length. The first length may be larger than the second length. In addition, the series of interface packets of the second length may comprise a so-called JumboFrame. Determining whether the read packets and the write packets are initial or continuous may comprise determining whether a set of request identifiers associated with a current packet plus a first offset is unequal to a set of identifiers and a second offset plus a length of a previous data packet. A preferred embodiment of the present invention is described with reference to the accompanying drawings. The preferred embodiment merely exemplifies the invention. Plural possible modifications are apparent to the skilled person. The gist and scope of the present invention is defined in the appended claims of the present application.

Fig. 1 is a block diagram of a data storage system in accordance with an exemplary embodiment of the present invention.

Fig. 2 is a process flow diagram that shows a method in accordance with an exemplary embodiment of the present invention.

The block diagram of a data storage system in accordance with an exemplary embodiment of the present invention is generally referred to by the reference number 100. The data storage system 100 comprises an interface module/packet filter 102 that receives data from a digital serial interface 104.

In a data storage system in accordance with an exemplary embodiment of the present invention, various hardware elements process READ and WRITE calls autonomously after an initial configuration. These hardware elements are also able to generate appropriate REPLY packets autonomously. Moreover, high data rates on digital interfaces attached to a storage system, or processor systems with storage, can be achieved using an exemplary embodiment of the present invention. Processor load in case of high data rates and/or large data volumes is also reduced, in some cases, dramatically.

The data storage system shown in Fig. 1 is generally referred to by the reference number 100. Digital data is delivered to the data storage system 100 via a digital serial interface 102 to an interface module/packet filter 104. The digital serial interface 102 may comprise a 10G Ethernet interface in an exemplary embodiment of the present invention. A microcontroller or processor 106 controls the overall operation of the data storage system 100, which further includes an autonomous payload data module 108. The autonomous payload data module 108 may be implemented in hardware in an exemplary embodiment of the present invention. The processor 106 is adapted to exchange information with a storage system 110 via a processor communication path 112. The autonomous payload data module 108 is adapted to exchange information with the storage system 110 via an autonomous payload data module communication path 114.

In an exemplary embodiment of the present invention, the interface module/packet filter 102 delivers payloads of incoming packets to either the processor 106 or the autonomous payload data module 108. In particular, payloads of packets resulting from READ or WRITE operations are delivered to the autonomous payload data module 108 through a READ/WRITE payload packet data path 116. All other packets, including their payload data, are delivered by the interface module/packet filter 102 to the processor 106 via an other packet data path 118. For example, in an exemplary embodiment of the present invention employing the RPC/NFS protocol, only the READ and WRITE calls are transferred to the autonomous payload data module 108. The interaction between the various components shown in Fig. 1 is described in detail below.

In case of a WRITE call, the following procedures are performed:

1. The interface module/packet filter 104 determines, by checking the IP and port address of the packet, whether the packet is addressed to the correct device.

WRITE call packets not dedicated to the current device will not be forwarded to the autonomous payload data module 108.

2. The interface module/packet filter 104 removes the IP and UDP or TCP header. 3. The autonomous payload data module 108 decides - with help of a set of request identifiers found in the IP, UDP/TCP and RPC/NFS packet headers, such as client IP address, file handle, file permissions, and the Offset and Length fields within the NFS header - whether the incoming data is initial or continuous, with other words, whether the incoming WRITE call constitutes an initial call at the start of a new transfer, or a continuous WRITE call extending or continuing another WRITE call that has preceded it.

As used herein, the term "initial" applies to a WRITE call if the packet is the first WRITE call of data transmission. Criteria for deciding this may include whether the set of request identifiers plus offset is unequal to the set of identifiers and offset plus length information of the previous WRITE call packet or the like.

As used herein, the term "continuous" applies to a WRITE call if the set of request identifiers and the offset of the current WRITE call are equal to the identifiers and offset plus length of the previous WRITE call.

4. If the WRITE call is an initial WRITE call, the autonomous payload data module 108 informs the processor 106 by writing into appropriate configuration registers via a configuration path 120 configuration information such as the client IP address, client port number, file handle, user identifiers such as UID and GIDs and offset and length information. The autonomous payload data module 108 then issues an interrupt to the processor 106 via an interrupt line 122. Using the configuration information obtained from the autonomous payload data module 108, the processor 106 is able to check the file permissions, that is, to test whether the client machine and the user on client machine have the permission to access the file in question in the requested way. If the permission check results in a positive decision, the processor 106 locates the requested file on the storage system 110 and sets up a streaming process that contiguously and consecutively allocates memory for the associated file. The resulting cluster address is provided to the storage system 110. Finally, the processor 106 informs the autonomous payload data module 108 either to go on with packet processing - in other words, to move the packet data to the storage system 110 - or to reject the request with an appropriate error code. If the WRITE call is continuous, no interaction between the autonomous payload data module 108 and the processor 106 takes place. Instead, the payload data will be transferred autonomously to the storage system 110.

5. The autonomous payload data module 108 provides an RPC/NFS reply message without any further interaction with the processor 106.

In an exemplary embodiment of the present invention, a READ call results in the performance of the following actions:

1. The interface module/packet filter 104 determines by checking IP and port address whether the packet is addressed to the correct device. READ call packets not dedicated to the current device will not be forwarded to the autonomous payload data module 108.

2. The interface module/packet filter 104 removes the IP and UDP or TCP header.

3. The autonomous payload data module 108 decides - with help of a set of request identifiers found in IP, UDP/TCP and RPC/NFS packet headers, which may include the client IP address, file handle, file permissions or the like and the Offset and Length fields within the NFS header - whether the requested data is initial or continuous.

As used herein, the term "initial" applies to a READ call if a packet is the first READ call of data transmission. The criteria to check may include whether the set of request identifiers plus offset is unequal to the set of identifiers and offset plus length information of the previous READ call packet or the like.

As used herein, the term "continuous" applies to a READ call if the set of request identifiers and the offset of the READ call are equal to the identifiers and offset plus the length of the previous READ call. 4. If the READ call is initial, the autonomous payload data module 108 informs the processor 106 by writing into appropriate configuration registers via the configuration path 120 configuration information such as the client IP address, client port number, file handle, user identifiers such as UID and GIDs and offset and length information or the like. The autonomous payload data module 108 then issues an interrupt to the processor 106 via an interrupt line 122.

Using the configuration information received from the autonomous payload data module 108, the processor 106 is able to check the file permissions, that is, to test whether the client machine and the user on client machine have the permission to access the file in question in the requested way. If the permission check results in a positive decision, the processor 106 locates the requested file on the storage system 110 and sets up a streaming process that contiguously and continuously reads the associated file from memory starting with the provided offset. The processor 106 then communicates the resulting cluster addresses to storage system 110. Finally, the processor 106 informs the autonomous payload data module 108 either to continue with packet processing, i.e. to read the packet data from buffers on the storage system 110, or to reject the request with an appropriate error code.

If the READ call is continuous, no interaction between the autonomous payload data module 108 and the processor 106 is needed. The payload data will be transferred autonomously from the storage System 110.

5. The autonomous payload data module 108 then provides a reply, such as data or reject, as a RPC/NFS reply message without any further interaction with the processor 106.

The skilled person will appreciate that the exemplary digital serial interface 102 receives interface packets that contain RPC/NFS procedure calls. For WRITE calls, the incoming call contains not only the command header or preamble, but also the data to be written. Whenever, on the application level, data of a total length of more than 9kB are to be written, the data are packetized and arrive as sequences of written data. The answer to a procedure call packet conning in is an acknowledge or response packet going back through the digital serial interface 102.

In the case of a WRITE call, the acknowledge packet, other than the WRITE call packet, does not contain any data proper. The reverse is true for a READ call. That is, the READ call packet contains just the procedure call preamble or header, whereas the associated acknowledge packet or packets contain the data proper that had been requested to be read.

In the autonomous payload data module 108, for the data path direction from the digital serial interface 102 towards the storage system 110, the interface packets or their content are intelligently concatenated into bigger units like clusters. The concatenated data are then buffered in a RAID control unit and a cache that exist somewhere in the symbolically denoted communication path 114 between the autonomous payload data module 108 and the storage system 110.

Correspondingly, for data going from the storage system 110 towards the digital serial interface 102, the autonomous payload data module 108 receives bigger data units like clusters, and apportions or packetizes them into payloads of - typically several consecutive - interface packets.

The skilled person will appreciate that the procedures described above may be useful for performing data transfers where processors are involved, and where the processors are supported by autonomous hardware units in accordance with an exemplary embodiment of the present invention.

Fig. 2 is a process flow diagram that shows a method in accordance with an exemplary embodiment of the present invention. The method is generally referred to by the reference number 200. At step 202, the process begins. At step 204, a data storage system such as the data storage system 100 receives a data packet via the digital serial interface 102. The packet is determined to be initial or continuous at step 206. Further processing as described herein may be performed depending on whether the packet is identified as initial or continuous. In particular, an instruction to continue packet processing is issued if the data packet is determined to be initial, as shown at step 208. At step 210, the process ends.

The skilled person will appreciate that combining any of the above-recited features of the present invention together may be desirable.

Claims

1. Method (200) for digital data storage, comprising:

- determining (206) whether a data packet received via a digital serial interface (102) is initial or continuous; and

- issuing (208) an instruction to continue packet processing if the data packet is determined to be initial.

2. Method (200) for digital data storage according to claim 1 , wherein determining whether the data packet received via the digital serial interface

(102) is initial or continuous comprises determining whether a set of request identifiers associated with the data packet plus a first offset is unequal to a set of identifiers and a second offset plus a length of a previous data packet.

3. Method (200) for digital data storage according to claims 1 or 2, comprising:

- converting the data packets to a cluster of a first length for storage on a storage system (110) if the data packets are write packets; and - converting the data packets to a series of interface packets of a second length if the data packets are read packets.

4. Method (200) for digital data storage according to any preceding claim, comprising performing a permission check to determine whether a requester has permission to authorize transfer of the data packet.

5. Method (200) for digital data storage according to any preceding claim, comprising converting data in a payload of the data packet to or from a data cluster depending on whether a transaction that initiated transfer of the data packet is a READ call or a WRITE call.

6. Apparatus (100) for digital data storage, comprising: - a digital serial interface (102) through which read packets, write packets and other packets are received;

- an interface module/packet filter (104) that is adapted to deliver the read packets and the write packets to an autonomous payload data module (108) and to deliver the other packets to a processor (106); and

- wherein the autonomous payload data module (108) is adapted to determine whether the read packets and/or the write packets are initial or continuous and to issue an instruction to continue packet processing if the read packets and/or the write packets are determined to be initial.

7. Apparatus (100) for digital data storage according to claim 6, wherein the autonomous payload data module (108) is adapted to convert the write packets to a cluster of a first length for storage on a storage system (110) and to convert the read packets to a series of interface packets of a second length.

8. Apparatus (100) for digital data storage according to claim 7, wherein the first length is larger than the second length.

9. Apparatus (100) for digital data storage according to claim 7, wherein the series of interface packets of the second length each comprises a JumboFrame.

10. Apparatus (100) for digital data storage according to any of the preceding claims, wherein determining whether the read packets and the write packets are initial or continuous comprises determining whether a set of request identifiers associated with a current packet plus a first offset is unequal to a set of identifiers and a second offset plus a length of a previous data packet.