WO2004099970A1 - Data transmission processing system and method - Google Patents

Data transmission processing system and method Download PDF

Info

Publication number
WO2004099970A1
WO2004099970A1 PCT/NZ2004/000087 NZ2004000087W WO2004099970A1 WO 2004099970 A1 WO2004099970 A1 WO 2004099970A1 NZ 2004000087 W NZ2004000087 W NZ 2004000087W WO 2004099970 A1 WO2004099970 A1 WO 2004099970A1
Authority
WO
WIPO (PCT)
Prior art keywords
data
comparison
administrative
processing
input digital
Prior art date
Application number
PCT/NZ2004/000087
Other languages
French (fr)
Inventor
Erez Birenzwig
Anthony David Williams
Original Assignee
Endace Technology Limited
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Endace Technology Limited filed Critical Endace Technology Limited
Publication of WO2004099970A1 publication Critical patent/WO2004099970A1/en

Links

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L45/00Routing or path finding of packets in data switching networks
    • H04L45/74Address processing for routing
    • H04L45/742Route cache; Operation thereof
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L43/00Arrangements for monitoring or testing data switching networks
    • H04L43/02Capturing of monitoring data
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L43/00Arrangements for monitoring or testing data switching networks
    • H04L43/18Protocol analysers
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L69/00Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
    • H04L69/12Protocol engines
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L69/00Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
    • H04L69/22Parsing or analysis of headers

Abstract

The present invention relates to a method of processing data transmissions which include the steps of initially receiving input digital administrative data, where this administrative data is associated with a particular data transmission. Next input digital comparison data is received and then a logic comparison operation is completed between the corresponding bits of the administrative data and comparison data. Next data transmission processing instructions are retrieved using the output of the comparison operation completed. A data transmission processing system which implements such a method is also within the ambit of the invention claimed.

Description

DATA TRANSMISSION PROCESSING SYSTEM AND METHOD
TECHNICAL FIELD
This invention relates to an improved method, apparatus and system for processing data transmissions. Preferably the present invention may be adapted to process digital internet protocol based data transmissions used to exchange data over computer networks. The present invention will also be referred to as being used in conjunction with high speed or high bandwidth computer networks, but those skilled in the art should appreciate that other applications for the present invention are also envisioned.
BACKGROUND ART
Data transmissions are used in a wide variety of fields and applications to transfer information between people and to facilitate business activities. Computer based networks are currently the primary mechanism by which the majority of such transmissions are made. For example, internet based computer network facilities allow web pages to be browsed by users, files to be shared and electronic mails to be exchanged between remote users, in addition to many other data sharing and transmitting applications.
Internet based communications protocols (IP protocols) follow a standard set of common rules which computer systems networked together use to make such data transmissions.
In general terms, IP data transmissions divide data to be transmitted into a series of packets, where a packet consists of administrative 'head' portion and a subsequent data based 'payload' portion. The head or header of the packet provides information used in the routing and delivery of the packet as well as other parameters used in the management and implementation of the transmission protocol.
Network connections between remote internet users are generally facilitated or provide by internet service providers and telecommunications (telecoms) organisations. These organisations provide network hubs or major nodes in the network to which a number of users are connected. In general terms, ISP's facilitate the distribution of a high bandwidth network connection into large number of low bandwidth connections to a plurality of users.
In some instances it is also preferable for ISP's and telecommunication companies to monitor the network transmissions or traffic passing through the network hubs they provide. For example, monitoring of IP packets can be used to optimise the hardware and software configurations of an ISP or Telecom, and potentially speed up the routing of packets to between individual users. Furthermore, network monitoring can also allow for the detection of malicious activities such as virus distribution, and also attempts by unauthorised users to gain access rights to other user's computer systems. Network traffic monitoring applications can also be used in law enforcement and national security applications to track the movements and activities of persons of interest.
However, in network traffic monitoring applications, the volume of packets passing through a hub or a node can cause processing difficulties. Existing computer hardware can find it difficult to investigate and potentially capture all packets passing through a node on high speed or high bandwidth network lines.
The header portion of an IP packet in basic terms consists of an array of consecutive data words where each word is made of a predefined number of digital or logic bits. Hardware employed to investigate the values or content present in such headers needs to read in the bits involved and subsequently convert same into recognisable data structures for comparison with alternative threshold values or parameters of interest. This approach involves a degree of operational overhead due to the fact that the bits involved need to be converted into recognisable data on which a comparison operation must be made. Furthermore, this approach must be taken for each and every parameter of interest within a packet's header, which can overall require a significant amount of instructions to be executed by the CPU involved.
In general, this type of traditional algorithmic approach is taken in network traffic monitoring applications, which places a high load on the CPU or CPU's of the monitoring system employed. Each and every packet passing through the node under investigation preferably has its header portion captured and processed in real time to ensure that the monitoring system can keep pace with the packet currently passing through the network node.
One potential improvement to this approach would be to make an initial selection of packets of potential interest based on the content of the header, with the monitoring system subsequently capturing these packets only for further investigation. Again, however, this does not solve the main problem associated with this approach. Existing computer hardware and CPU capacities will still struggle to process in real time the headers associated with all packets at the same rate at which traffic is passing through a network node.
It would be of advantage to have an improved data transmission processing system which addressed any or all of the above problems. A processing system which could quickly select and capture particular transmissions or preferably IP packets with specific characteristics of interest would be of advantage. A system which could also operate at high speeds in conjunction with high speed communication links or networks to investigate and potentially select transmissions or packets from substantially all the traffic passing the current network point or node being monitored would be of advantage.
All references, including any patents or patent applications cited in this specification are hereby incorporated by reference. No admission is made that any reference constitutes prior art. The discussion of the references states what their authors assert, and the applicants reserve the right to challenge the accuracy and pertinency of the cited documents. It will be clearly understood that, although a number of prior art publications are referred to herein, this reference does not constitute an admission that any of these documents form part of the common general knowledge in the art, in New Zealand or in any other country.
It is acknowledged that the term 'comprise' may, under varying jurisdictions, be attributed with either an exclusive or an inclusive meaning. For the purpose of this specification, and unless otherwise noted, the term 'comprise' shall have an inclusive meaning - i.e. that it will be taken to mean an inclusion of not only the listed components it directly references, but also other non-specified components or elements. This rationale will also be used when the term 'comprised' or 'comprising' is used in relation to one or more steps in a method or process.
It is an object of the present invention to address the foregoing problems or at least to provide the public with a useful choice.
Further aspects and advantages of the present invention will become apparent from the ensuing description which is given by way of example only.
DISCLOSURE OF INVENTION
According to one aspect of the present invention there is provided a method of processing data transmissions characterised by the steps of; (i) receiving input digital administrative data where this administrative data is associated with a particular data transmission, and
(ii) receiving input digital comparison data, and
(iii) completing a logic comparison operation between the corresponding bits of the administrative data and comparison data, and
(iv) retrieving data transmission processing instructions using the output of the comparison operation.
According to a further aspect of the present invention there is provided a method of processing transmission data characterised by the steps of;
(i) receiving input digital administrative data where this administrative data is associated with a particular data transmission, and
(ii) receiving input digital comparison data, and
(iii) eliminating from consideration at least a portion of the received administrative data using a mask operation, and
(iv) completing a logic comparison operation between the corresponding bits of the administrative data and comparison data, and
(v) retrieving transmission processing instructions using the output of the comparison operation.
According to a further aspect of the present invention there is provided a method of processing transmission data characterised by the steps of;
(i) receiving input digital administrative data where this administrative data is associated with a particular data transmission, and (ii) receiving input digital comparison data, and
(iii) completing a logic comparison operation between the corresponding bits of the administrative data and comparison data, and
(iv) eliminating from consideration at least a portion of the output of the comparison operation by using a mask operation, and
(v) retrieving transmission processing instructions using the modified output of the comparison operation.
According to a further aspect of the present invention there is provided a method of processing transmission data substantially as described above further characterised in that reception of comparison data is enabled or disabled depending on at least one characteristic of a current portion of the administrative data currently being received.
According to a further aspect of the present invention there is provided a method of processing transmission data substantially as described above wherein the position or location of the portion of the administrative data currently being received within the entire administration data set is used to determine whether the reception of comparison data is disabled.
According to yet another aspect of the present invention there is provided a data transmission processing system which includes,
at least one filtering element, said filtering element including
an input stage adapted to receive input digital administrative data where this administrative data is associated with a particular data transmission, said input stage also being adapted to receive input digital comparison data, and a processing stage adapted to complete a logic comparison operation between corresponding bits of the administrative and comparison data received, and
an output stage adapted to output the result of the logic comparison operation, wherein the result output is adapted to be used to retrieve transmission processing instructions for the particular data transmission associated with the administrative data received.
According to a further aspect of the present invention there is provided a data transmission processing system and apparatus substantially as described above wherein the supply of comparison data to the processing stage is disabled depending on at least one characteristic of the current portion of the administrative data being received.
According to a further aspect of the present invention there is provided a data transmission processing system and apparatus substantially as described above wherein the position of the portion of the administrative data currently being received within the entire administrative data set is used to determine whether the supply of comparison data should be enabled or disabled.
According to a further aspect of the present invention there is provided a transmission processing system substantially as described above which includes a plurality of filter elements, wherein each filter element is adapted to receive the same administrative data with respect to a single data transmission substantially simultaneously with all other filter elements provided, and
each filter element is adapted to output the result of the logic comparison operation completed with respect to the same input administrative data and differing comparison data at substantially the same time.
The present invention is adapted to provide a method, apparatus and system for the processing of at least one data transmission. Preferably such data transmissions may be made over computer networks with the transmissions involved being made using internet protocols. IP protocol packet based data transmissions may preferably be monitored and processed in conjunction with the present invention.
Preferably the present invention may be adapted to consider or analyse at high speed the content of the administrative or header portion of IP packet traffic. These headers may be processed to provide metadata with respect to the data payload of each packet, to in turn allow a packet to be captured and further processed if required.
Such administrative data may be received as a set of consecutive data words from the transmission involved. An entire set of administrative data may be received over time in conjunction with the present invention.
Reference throughout this specification will also be made to the present invention being employed as a filter to selectively identify and capture data transmissions, and preferably IP packets of interest passing through a computer network node at high speeds. The present invention may preferably be employed to investigate or consider all traffic passing through a node to select for further investigation or further processing in the packets of interest based on data contained within the headers of these packets.
However, those skilled in the art should appreciate that the present invention may be used with other types of communications protocols and also in other applications as well. Reference to IP protocol transmissions and data packets in isolation only should not be considered essential to the present invention nor limiting with respect to same, nor should specific references to the present invention being used as a transmission nor packet filtering system. Those skilled in the art should appreciate that other applications are also envisioned and reference to the above throughout the specification should in no way be seen as limiting.
Preferably the present invention may be adapted to initially receive input digital administrative data associated with a particular data transmission or IP packet. Preferably this input administrative data set may consist of the header or administrative portion of the IP packet transmitted. This administrative data may preferably be in a standard format for header data used with internet protocol data transmissions and also in some instances may also include additional protocol layer data which need not necessarily be considered essential to a basic IP packet header.
For example, preferably the content of an IP packet may include the following data:
1. Length of the Header
2. Type of Service (TOS) 3. Total length of the data with the header
4. Identification number, for use with fragmenting packets
5. Flags to tell about fragments
6. Fragment offset for fragment reassembly
7. Time to Live (Helps in misrouted packets) 8. Underlying protocol (such as TCP or UDP or others)
9. Header Checksum for error checking
10. Source IP Addresses
11. Destination IP Address
12. Options
In addition to the above, the header it may also be provided a second layer of protocols such as TCP, UDP, ICMP etc. Each of these further secondary layers may have its own characteristic administrative data which may again be considered in conjunctions with the present invention.
Preferably the input administrative data or packet header may consist of an array of logic bits organised into a series of data words. A consecutive array of data words can become available as a packet is received and passed through the network node being monitored. Each word of the packet header can be consecutively clocked into the components employed in conjunction with the present invention which should provide an array of bits one data word wide with a variable array length depending on the overall size of the packet header.
Preferably the present invention may be adapted to receive input comparison data which has a corresponding or identical organisation, format, or structure to that normally expected from the administrative data involved, preferably being IP packet headers. Such comparison data may be structured as an array of logic bits which is one arbitrary data word wide and which has a length equal to a predefined or standard length for IP packet headers.
Preferably the comparison data received may represent a filtering case or rule used to determine how the data transmitted with a particular packet header is to be processed further. The bits present in the administrative data can represent a particular series of values or settings for the metadata normally embedded in a packet header. This comparison data may then represent a specific case or type of IP packet which is to be captured, filtered or otherwise processed using a unique set of instructions.
Preferably once the required administrative and comparison data have been received, a logic comparison operation may be executed with the corresponding bits of these two sets of data. In such instances corresponding bit pairs of each data set may be supplied to a single logic comparison element. A logic comparison operation may result in an output array of bits of the same size, structure and arrangement as the arrays of administrative and comparison data, where the content of the resulting array will be determined by the standard comparison operator comparing corresponding bits of each data set. With the Boolean comparison operators involved, a voltage high or digital one will be provided within the result array if and only if the corresponding bits of each data set are identical.
In a further preferred embodiment, the logic comparison operation executed may be an exclusive OR Boolean logic operation. This type of operator will provide a true, one or voltage high output when its inputs differ, and will provide a false, zero, or voltage low output when its inputs are identical. This type of logic operator provides the efficient and fast logic comparison operation or function to be employed in conjunction with the present invention.
Reference throughout this specification will also be made to the logic comparison operation executed being an exclusive OR Boolean logic operation. However, those skilled in the art should appreciate that other types of Boolean logic operator which achieve the same aims may also be used in conjunction with the present invention.
In a preferred embodiment, the output of the logic comparison operation can be used to retrieve further transmission processing instructions for the packet associated with a particular header data considered. For example, the output bits obtained may in some instances be used to address a memory element at which a processing instruction for the particular type of packet identified can be contained and subsequently retrieved and executed.
In a preferred embodiment the output of the present invention may consist of a single output bit per set of data words compared or considered. This single output bit may flag whether or not the current data word investigated is of interest or whether further specific processing functions or facilities are to be provided in accordance with the present invention. This approach substantially reduces the amount of data which needs to be handled at the output stage of the present invention.
In a further preferred embodiment, the present invention may also include at least one memory or accumulation element used to receive and store the output bits generated as discussed above. Such an accumulator may store all output bits generated by compare operations on sets of data words preferably until all words of interest within the administrative data set of a transmission or packet have been received and compared with comparison data. At this stage the accumulator or memory component provided may transmit the accumulative output bits recorded for a further memory element to be used as an address at which processing instructions for the particular type of packet identified are contained.
In a preferred embodiment, the processing instructions stored and subsequently retrieved and executed may deal with the filtering, selection and subsequent capture of particular packets passing through a network node which are identified through the content of each packets header. The parameters specified in the packet header may be used to set up or implement a series of filtering rules to identify particular types of packets and subsequently capture these packets. This implementation of the present invention allows a relatively high speed network node monitoring system to be implemented which may investigate all data transmissions passing through the node in real time. A low level comparison bit operation only is required to investigate the content of each packet header, thereby allowing this operation to be completed at high speeds. The comparison data may identify a particular type of packet without necessarily requiring either the received administrative packet data or the comparison data to be converted into a standard data type for relative comparison.
In a preferred embodiment at least a portion of the administrative data received or alternatively at least a portion of the output of the comparison operation completed may be eliminated from consideration and use in retrieving further transmission processing instructions. In some instances, only a subset of all the parameters or data incorporated into a packet header may be used to make filtering decisions. Furthermore in some instance it will also be preferable to search for and select packets which have header data parameters within a specific range of values, as opposed to an exact value attempted by a single comparison operation. In this way a range of administrative data values can trigger the retrieval of a single data transmission processing instruction.
This therefore allows relatively broad filtering rules to be implemented, as opposed for searches for specific, exact bit patterns present in packet headers. For example, selective portions of header parameters of interest may be eliminated from consideration to allow ranges of numeric values based on most significant bits to be implemented. This approach thereby allows a significant degree of flexibility with respect to the number and type of particular packets which can be identified and potentially captured using a single comparison operation result.
In a further preferred embodiment, a mask operation may be completed in some instances on the administrative data prior to the logic comparison operation. In such a situation, the bits not relevant to the header parameters of interest may all be set either high or low depending on the particular filtering and addressing rules employed to retrieve the transmission processing instructions required. This may modify the administrative or header data so that only variations in the bits of the parameters of interest will be used to make a selection decision.
However, in an alternative embodiment, the mask operation may be implemented on the result of the logic comparison operation. This mask operation may again modify or reset selected bits of the array of output result bits to in turn again only provide variation with respect to the bits corresponding to the header parameter on which a selection decision is to be made.
Reference throughout this specification will however be made to the mask operation being completed on the administrative or packet header data after the comparison operation required. However, those skilled in the art should appreciate that other implementations of the present invention are envisioned and reference to the above only throughout this specification should in no way be seen as limiting.
In a preferred embodiment the present invention may be implemented through a specific arrangement of selected hardware components, which preferably are implemented through an integrated circuit design. Such components may be adapted to provide input and output functions in addition to the logic comparison operations required. Furthermore, in some instances the mask operation or operations discussed above may also be implemented using such hardware components.
However, those skilled in the art should also appreciate that a software based algorithm approach may also be used to implement the present invention without necessarily requiring a specific customised hardware implementation.
In a preferred embodiment, a data transmission processing system implemented in accordance with the present invention may include at least one filtering element. Preferably such a filtering element may compose a base unit which may complete a comparison between a single word of header data bits and a single word of comparison data bits.
In a further preferred embodiment, a filtering element may include an input stage which is adapted to receive both the input packet header data in addition to input digital comparison data.
In a preferred embodiment a filtering element may also include a processing stage which is adapted to complete an array of logic comparison operations between the corresponding bits of the administrative and comparison data received. Those skilled in the art should appreciate that simple silicon based logic gate arrangements may be implemented in hardware to provide such a processing stage which can in turn complete the logic comparison operations required rapidly. Those skilled in the art will appreciate such a processing stage need not necessarily be provided through a microprocessor. The implementation of the processing stage with silicon logic gates can substantially speed up the operation of the present invention.
In a further preferred embodiment, a processing stage of a filter element may incorporate an array of logic comparison gates into a single element. This array of logic comparison gates may process the bits of a pair of data words as inputs simultaneously as the bits of these words are clocked onto the inputs of each of the comparison gates provided. The output result from each bit comparison may in turn be clocked from the output stage of each of the comparison gates, thereby providing a substantially synchronised or output result to the comparison of two arbitrary size data words. For example, in a preferred embodiment, the single processing stage may receive a word of header data simultaneously with a word of comparison data, and subsequently provide an output data word on the following clock cycle.
In a preferred embodiment, the processing stage may also include elements adapted to implement a mask operation on the output result word to be provided from the logic comparison operation. However, in alternative embodiments the processing stage may be adapted to implement a mask operation on received input digital administrative data prior to completing a logic comparison operation.
In a further preferred embodiment, the mask operation to be completed may be implemented through use of at least one Boolean logic AND operation. In such embodiments, the comparison operation completed may employ an exclusive OR Boolean operation so that the input to the masking AND gates will be false/low if a match is found and high/true if a difference is found. The mask AND gate will then compare the comparison operation output with a mask data word to provide a final compared and masked output.
In a preferred embodiment the present invention may include a plurality of filter elements which can provide a simultaneous comparison to test a plurality of selection or filtering rules. Each particular filter element may be provided with its own set of comparison data while all elements are in synchronisation provided with the same input packet header data. This array of filter elements will be operated to provide a series of synchronised output results which test series of selection rules simultaneously and preferably at high speeds.
Preferably each filter element may also be provided with its own sequential based memory component which can receive, store and subsequently supply consecutive words of the comparison data set. Preferably the comparison data words may be provided in synchronisation with incoming words of header data. The memory components associated with each filter element may therefore be loaded with varying different comparison data sets to in turn execute a number of differing selection rules or tests.
In a preferred embodiment, the reception of administrative and/or comparison data may be enabled or disabled depending on the characteristics of the current header data word currently being received in conjunction with the present invention. In a further preferred embodiment the operation of each of the filtering elements provided may be enabled or disabled depending on the particular position and current word or portion of the header data currently being received and processed. If for example the current data word of the header does not contain any specific data parameters required to be make a selection decision the operation of the filtering elements and their associated compare functions may not be enabled.
In a preferred embodiment an index count may be maintained on the number of header data words processed in relation to a packet under investigation. The index maintained will therefore represent the position of the word in the packet header currently under investigation and therefore potentially the administrative data currently available on which a selection decision may be made.
In a preferred embodiment, a further memory element may be provided within a processing system implemented in accordance with the present invention. Such a memory element may be referred to as the header index memory, which preferably may be implemented through the provision of a look up table data structure in a standard memory component. The look up table provided may be addressed through the index recorded for the current data word of a packet header. The corresponding value stored in the table may therefore be output to provide an enable or disable signal to be propagated through to the filter elements provided.
In a further preferred embodiment, the present invention may be implemented with a plurality of chains of filter elements. A plurality of filter elements may be linked to and driven by one of a plurality of header index memory elements in such an embodiment. This configuration of the present invention allows configuration and control of subsets or chains of the plurality of filter elements provided. The operational behaviours of the filter elements can be finely tuned or adjusted by the contents of each header index memory element used to drive a specific chain, which will in turn allow a final control of the overall operational behaviour and processing facilities provided in accordance with the present invention.
The present invention may provide many potential advantages over the prior art.
The present invention may be used to monitor specific administrative data associated with data transmissions and to in turn potentially filter, select or otherwise process further data transmissions of interest.
Preferably the present invention may be used to implement a network monitoring system for comparatively high speed or high bandwidth network nodes. Through the use of a low level Boolean bit wise operators and preferably through the provision of customised circuitry or hardware, the present invention may be used to complete a number of filtering checks or rules in real time as data transmissions are received from a network node.
The implementation of the present invention may be used to distribute the processing work required for a single incoming transmission over the transmission time in which the administrative portion of the transmission involved is received. This allows the present invention to use all the time available from the reception of such administrative data to complete the processing work required. This may be contrasted with alternative approaches which would need to complete potentially high speed processing operations after the reception of all administrative data, which could potentially slow down the operation of a processing system.
BRIEF DESCRIPTION OF DRAWINGS
Further aspects of the present invention will become apparent from the following description which is given by way of example only and with reference to the accompanying drawings in which: Figure 1 illustrates a block schematic diagram of hardware element and associated communication lines employed between same used to implement a data transmission processing system in accordance with a preferred embodiment of the present invention; and
Figure 2 shows elements of an operation enable components employed in the system discussed with respect to figure 1 and plots of particular signal values with respect to time used and generated in conjunction with said enable components.
Figure 3 shows a block schematic illustration of logic operators and their associated input data words employed to provide comparison and mask operations in accordance with the embodiment of the invention discussed with respect to figure 1 and 2.
BEST MODES FOR CARRYING OUT THE INVENTION
Figure 1 illustrates a block schematic diagram of hardware element and associated communication lines employed between same used to implement a data transmission processing system in accordance with a preferred embodiment of the present invention.
The system described with respect to figure 1 is implemented with a pair of filter elements, shown as filter (1) and filter (2). Each filter element is linked to a sequential access memory component loaded with comparison data unique to the particular filter involved. Each sequential access memory element also includes mask data to be supplied in addition to the comparison data required.
The system is also linked to a series of input signal and data lines.
The primary line involved is the data line on which incoming data words of the packet header under investigation are transmitted.
The line DVLD represents a signal flag from network interface element which provides the transmission data to the system provided. The DVLD line is held with a voltage high signal when valid data is present in the current word on the data line.
A further SOP line carries a start of packet flag signal, where a high voltage signal on the SOP line will reset the components employed with the arrival of a new packet.
A word clock line is provided to implement a clock signal to drive the sequential operation of each of the logic elements provided. This clock signal is synchronised to pulse with the arrival of data from the network.
The index counter and match look up table component are used to provide and enable/disable signal to each of the sequential access memories provided in conjunction with each filter element. These components, as discussed further with respect to figure 2, control the operation of the present invention and when a particular logic comparison is made by each filter element. The match enable signal line supplied to each memory element increments the state of the memory element and results in the output of a data word from the comparison data and mask data to each filter element.
Each filter element receives its topmost input line a single data word value for each of the mask and comparison data sets, whereas a single input data word from the incoming packet header data line is received on the lower input line of each filter element.
On reception of these data words, a compare operation is completed on the input header data word and received comparison data word. The results of this compare operation are then subsequently supplied to components employed to complete the mask operation. The modified or mast output produced is accumulated or stored within memory based components integrated into the filter element until the SOP line again flags the start of a new packet.
In the embodiment discussed, each filter element generates a single output bit for accumulation or storage on every incrementation of the hardware. This single output bit may indicate simply whether the current data word (when compared and subsequently masked) is of interest to a filtering system employed in conjunction with the embodiment discussed. These accumulative output bits can then be used to address a further memory element which can contain processing instructions for the data associated with the header which has just been investigated.
Figure 2 shows elements of the operation enable components employed in the system discussed with respect to figure 1 , and plots of particular signal values with respect to time used and generated in conjunction with said enable component.
The signal plots shown illustrate the operation of the index counter used in conjunction with the present invention. This counter will only be incremented when the data valid line goes high, with the subsequent count generated being provided as an index to the match look up table. The index counter involved will also be reset when the start of packet (SOP) line goes high, indicating that a new packet header has arrived.
As can be seen from the plot shown with respect to figure 2, the match output line only produces an enabling voltage high output signal on the reception of the header words indexed at position 0 and position 1. No other words of the header are of interest and therefore the contents of the match look up table are configured to only generate a high output result line when the table receives an input index of 0 or 1 only. This facility of the present invention is illustrated in the signal plots shown, where data words of interest are marked with a whereas data words to be ignored are marked with a .
Figure 3 shows a block schematic illustration of logic operators and their associated input data words employed to provide comparison and mask operations in accordance with the embodiment of the invention discussed with respect to figure 1 and 2.
In the embodiment discussed, three input data words provided being operand A, operand B and a mask input data word. Operand A in the embodiment discussed can be provided by a single input data word whereas operand B can provide a comparison data word. Conversely, the mask data word can be used to modify the output of the system provided to consider only bits of interest in the particular operation or scheme employed.
In the embodiment discussed, operands A and B, being the input data word and comparison data are supplied to a Boolean logic exclusive OR gate. This gate completes the comparison operation required to produce a first intermediate result, as shown with respect to figure 3.
The output of the comparison operation provides logic zeros for each bits which are identical and logic ones for any differences between the bits of operand A and operand B.
This intermediate result is then supplied to Boolean logic AND gate in conjunction with the mask data word received. These two data words are then AND'ed together to provide a final output result word. As can be seen from figure 3, the last four bits of the intermediate result are masked by the last four zeros in the mask word as these four bits are not of interest and the resulting processing operation completed.
This final compared and masked result data word can then be reduced further to a single output bit through the final Boolean logic OR gate provided. The output of this final gate will simply flag whether the input data word represented by operand A has passed or failed the filter rule implemented in conjunction with the invention discussed above.
Aspects of the present invention have been described by way of example only and it should be appreciated that modifications and additions may be made thereto without departing from the scope thereof as defined by the appended claims.

Claims

WHAT WE CLAIM IS:
1 . A method of processing data transmissions characterised by the steps of;
(i) receiving input digital administrative data where this administrative data is associated with a particular data transmission, and
(ii) receiving input digital comparison data, and
(iii) completing a logic comparison operation between the corresponding bits of the administrative data and comparison data, and
(iv) retrieving data transmission processing instructions using the output of the comparison operation.
2. A method of processing data transmissions as claimed in claim 1 wherein the data transmissions to be processed are made over computer networks using internet protocols.
3. A method of processing data transmissions as claimed in claim 2 wherein digital administrative data forms the header portion of an internet protocol transmission packet.
4. A method of processing data transmissions as claimed in any previous claim wherein the input digital comparison data has the same format as the input digital administrative data.
5. A method of processing data transmissions as claimed in claim 4 wherein the corresponding bits of the input digital administrative data and input digital comparison data are supplied to a single logic comparison element.
6. A method of processing data transmissions as claimed in any previous claim wherein the logic comparison operation executed is an exclusive OR logic operation.
7. A method of processing data transmissions as claimed in any previous claim wherein the output of the comparison operation is used to address a memory location at which processing instructions are stored.
8. A method of processing data transmissions as claimed in any previous claim wherein an accumulation element is used to store the output of the comparison operation until all bits of interest in the input digital administrative data set have been compared with input digital comparison data.
9. A method of processing data transmissions as claimed in any previous claim wherein the reception of input digital comparison data is disabled depending on at least one characteristic of a portion of the input digital administrative data currently being received.
10. A method of processing data transmissions as claimed in claim 9 wherein the position of the portion of the administrative data currently being received within the entire administrative data set is used to determine whether the reception of digital comparison data is disabled.
11. A method of processing transmission data characterised by the steps of;
(i) receiving input digital administrative data where this administrative data is associated with a particular data transmission, and
(ii) receiving input digital comparison data, and
(iii) eliminating from consideration at least a portion of the received administrative data using a mask operation, and (iv) completing a logic comparison operation between the corresponding bits of the administrative data and a comparison data, and
(v) retrieving transmission processing instructions using the output of the comparison operation.
12. A method of processing transmission data characterised by the steps of;
(i) receiving input digital administrative data where this administrative data is associated with a particular data transmission, and
(ii) receiving input digital comparison data, and
(iii) completing a logic operation between the corresponding bits of the administrative data and comparison data, and
(iv) eliminating from consideration at least a portion of the output of the comparison operation using a mask operation.
(v) retrieving transmission processing instructions using the modified output of the comparison operation.
13. A method of processing data transmissions as claimed in either claim 11 or 12 wherein eliminating from consideration at least a portion of the output of the comparison operation and/or administrative data allows a range of input administrative data values to trigger the retrieval of a single data transmission processing instruction.
14. A method of processing data transmissions as claimed in any one of claims 11 to 13 wherein a mask operation is implemented through the completion of a logic AND operation.
15. A data transmission processing system which includes, at least one filtering element, said filtering element including,
an input stage adapted to receive input digital administrative data where this administrative data is associated with a particular data transmission,
said input stage also being adapted to receive input digital comparison data, and
a processing stage adapted to complete a logic comparison operation between corresponding bits of the administrative and comparison data received, and
an output stage adapted to output the result of the logic comparison operation, wherein the result of output is adapted to be used to retrieve transmission processing instructions for the particular data transmission associated with the administrative data received.
16. A data transmission processing system as claimed in claim 15 wherein each filter element input stage includes a sequential memory component adapted to receive and subsequently supply consecutive data words of an input digital comparison data set.
17. A data transmission processing system as claimed in either claim 15 or claim
16 wherein a filter element processing stage includes a plurality of logic comparison gates adapted to simultaneously compare the bits of a single word of administrative data with the corresponding bits of a single word of comparison data.
18. A data transmission processing system as claimed in any one of claims 15 to
17 wherein a filter element processing stage is adapted to perform a mask operation on received input digital administrative data prior to completing a logic comparison operation.
19. A data transmission processing system as claimed in any one of claims 15 to 17 wherein a filter element processing stage is adapted to perform a mask operation on the resulting output of the logic comparison operation.
20. A data transmission processing system as claimed in any one of claims 15 to 19 wherein the supply of comparison data to the processing stage is disabled depending on at least one characteristic of the current portion of the input digital administrative data being received.
21. A data transmission processing system as claimed in claim 20 wherein the position of the portion of the administrative data currently being received within the entire administrative data set is used to determine whether the supply of comparison data is disabled.
22. A data transmission processing system as claimed in any one of claims 15 to 21 wherein the data transmission processing system includes a plurality of filter elements, wherein
each filter element is adapted to receive the same administrative data with respect to a single data transmission substantially simultaneous with all other filter elements provided, and
each filter element is adapted to output the result of the logic comparison operation completed with respect to the same administrative data and differing comparison data at substantially the same time.
23. A data transmission processing system as claimed in claim 22 wherein each filter element provided is adapted to test one of a plurality of selection rules.
24. A data processing system substantially as herein described with reference to and as illustrated by the accompanying drawings and/or examples.
25. A filtering element substantially as herein described with reference to and as illustrated by the accompanying drawings and/or examples.
26. A method of processing transmission data substantially as herein described with reference to and as illustrated by the accompanying drawings and/or examples.
PCT/NZ2004/000087 2003-05-08 2004-05-07 Data transmission processing system and method WO2004099970A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
NZ525800 2003-05-08
NZ52580003 2003-05-08

Publications (1)

Publication Number Publication Date
WO2004099970A1 true WO2004099970A1 (en) 2004-11-18

Family

ID=33432548

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/NZ2004/000087 WO2004099970A1 (en) 2003-05-08 2004-05-07 Data transmission processing system and method

Country Status (1)

Country Link
WO (1) WO2004099970A1 (en)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5793954A (en) * 1995-12-20 1998-08-11 Nb Networks System and method for general purpose network analysis
US5948080A (en) * 1996-04-26 1999-09-07 Texas Instruments Incorporated System for assigning a received data packet to a data communications channel by comparing portion of data packet to predetermined match set to check correspondence for directing channel select signal
US6041058A (en) * 1997-09-11 2000-03-21 3Com Corporation Hardware filtering method and apparatus
WO2000052897A2 (en) * 1999-03-01 2000-09-08 Sun Microsystems, Inc. Dynamic parsing in a high performance network interface

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5793954A (en) * 1995-12-20 1998-08-11 Nb Networks System and method for general purpose network analysis
US5948080A (en) * 1996-04-26 1999-09-07 Texas Instruments Incorporated System for assigning a received data packet to a data communications channel by comparing portion of data packet to predetermined match set to check correspondence for directing channel select signal
US6041058A (en) * 1997-09-11 2000-03-21 3Com Corporation Hardware filtering method and apparatus
WO2000052897A2 (en) * 1999-03-01 2000-09-08 Sun Microsystems, Inc. Dynamic parsing in a high performance network interface

Similar Documents

Publication Publication Date Title
EP1908219B1 (en) Active packet content analyzer for communications network
US10735221B2 (en) Flexible processor of a port extender device
US7191468B2 (en) System and method for multidimensional data compression
US9866540B2 (en) System and method for rule matching in a processor
EP1103129B1 (en) System and method for filtering data
US7669240B2 (en) Apparatus, method and program to detect and control deleterious code (virus) in computer network
CN101345759B (en) Internet protocol security matching values in an associative memory
US20020198981A1 (en) Method and system for exploiting likelihood in filter rule enforcement
EP1336915B1 (en) Method and apparatus for flexible frame processing and classification engine
US7808897B1 (en) Fast network security utilizing intrusion prevention systems
US6377577B1 (en) Access control list processing in hardware
US8761182B2 (en) Targeted flow sampling
CN110266556A (en) The method and system of service exception in dynamic detection network
US8272056B2 (en) Efficient intrusion detection
US20180367431A1 (en) Heavy network flow detection method and software-defined networking switch
KR20090079945A (en) Flow information restricting apparatus and method
CN101399711A (en) Network monitoring system and network monitoring method
Varghese et al. Detecting evasion attacks at high speeds without reassembly
US7984235B2 (en) Reducing content addressable memory (CAM) power consumption counters
JP6671112B2 (en) Method and apparatus for flexible and efficient analysis in network switch
WO2006008307A1 (en) Method, system and computer program for detecting unauthorised scanning on a network
US10547532B2 (en) Parallelization of inline tool chaining
US7957372B2 (en) Automatically detecting distributed port scans in computer networks
US7266088B1 (en) Method of monitoring and formatting computer network data
WO2004099970A1 (en) Data transmission processing system and method

Legal Events

Date Code Title Description
AK Designated states

Kind code of ref document: A1

Designated state(s): AE AG AL AM AT AU AZ BA BB BG BR BW BY BZ CA CH CN CO CR CU CZ DE DK DM DZ EC EE EG ES FI GB GD GE GH GM HR HU ID IL IN IS JP KE KG KP KR KZ LC LK LR LS LT LU LV MA MD MG MK MN MW MX MZ NA NI NO NZ OM PG PH PL PT RO RU SC SD SE SG SK SL SY TJ TM TN TR TT TZ UA UG US UZ VC VN YU ZA ZM ZW

AL Designated countries for regional patents

Kind code of ref document: A1

Designated state(s): GM KE LS MW MZ NA SD SL SZ TZ UG ZM ZW AM AZ BY KG KZ MD RU TJ TM AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IT LU MC NL PL PT RO SE SI SK TR BF BJ CF CG CI CM GA GN GQ GW ML MR NE SN TD TG

121 Ep: the epo has been informed by wipo that ep was designated in this application
122 Ep: pct application non-entry in european phase