WO1995030294A1 - Atm architecture and switching element - Google Patents

Atm architecture and switching element Download PDF

Info

Publication number
WO1995030294A1
WO1995030294A1 PCT/US1995/004613 US9504613W WO9530294A1 WO 1995030294 A1 WO1995030294 A1 WO 1995030294A1 US 9504613 W US9504613 W US 9504613W WO 9530294 A1 WO9530294 A1 WO 9530294A1
Authority
WO
WIPO (PCT)
Prior art keywords
cell
cells
data
input
priority
Prior art date
Application number
PCT/US1995/004613
Other languages
French (fr)
Inventor
Brian D. Holden
Original Assignee
Integrated Telecom Technology, Inc.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Integrated Telecom Technology, Inc. filed Critical Integrated Telecom Technology, Inc.
Priority to EP95915672A priority Critical patent/EP0761055A4/en
Priority to JP7528264A priority patent/JPH09512683A/en
Publication of WO1995030294A1 publication Critical patent/WO1995030294A1/en
Priority to KR1019960706081A priority patent/KR970703078A/en

Links

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L12/00Data switching networks
    • H04L12/54Store-and-forward switching systems 
    • H04L12/56Packet switching systems
    • H04L12/5601Transfer mode dependent, e.g. ATM
    • CCHEMISTRY; METALLURGY
    • C07ORGANIC CHEMISTRY
    • C07KPEPTIDES
    • C07K14/00Peptides having more than 20 amino acids; Gastrins; Somatostatins; Melanotropins; Derivatives thereof
    • C07K14/195Peptides having more than 20 amino acids; Gastrins; Somatostatins; Melanotropins; Derivatives thereof from bacteria
    • C07K14/32Peptides having more than 20 amino acids; Gastrins; Somatostatins; Melanotropins; Derivatives thereof from bacteria from Bacillus (G)
    • C07K14/325Bacillus thuringiensis crystal protein (delta-endotoxin)
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L49/00Packet switching elements
    • H04L49/10Packet switching elements characterised by the switching fabric construction
    • H04L49/104Asynchronous transfer mode [ATM] switching fabrics
    • H04L49/105ATM switching elements
    • H04L49/108ATM switching elements using shared central buffer
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L49/00Packet switching elements
    • H04L49/20Support for services
    • H04L49/201Multicast operation; Broadcast operation
    • H04L49/203ATM switching fabrics with multicast or broadcast capabilities
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L49/00Packet switching elements
    • H04L49/25Routing or path finding in a switch fabric
    • H04L49/253Routing or path finding in a switch fabric using establishment or release of connections between ports
    • H04L49/255Control mechanisms for ATM switching fabrics
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L49/00Packet switching elements
    • H04L49/30Peripheral units, e.g. input or output ports
    • H04L49/3081ATM peripheral units, e.g. policing, insertion or extraction
    • H04L49/309Header conversion, routing tables or routing tags
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04JMULTIPLEX COMMUNICATION
    • H04J2203/00Aspects of optical multiplex systems other than those covered by H04J14/05 and H04J14/07
    • H04J2203/0001Provisions for broadband connections in integrated services digital network using frames of the Optical Transport Network [OTN] or using synchronous transfer mode [STM], e.g. SONET, SDH
    • H04J2203/0064Admission Control
    • H04J2203/0067Resource management and allocation
    • H04J2203/0071Monitoring
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L12/00Data switching networks
    • H04L12/54Store-and-forward switching systems 
    • H04L12/56Packet switching systems
    • H04L12/5601Transfer mode dependent, e.g. ATM
    • H04L2012/5629Admission control
    • H04L2012/5631Resource management and allocation
    • H04L2012/5632Bandwidth allocation
    • H04L2012/5635Backpressure, e.g. for ABR
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L12/00Data switching networks
    • H04L12/54Store-and-forward switching systems 
    • H04L12/56Packet switching systems
    • H04L12/5601Transfer mode dependent, e.g. ATM
    • H04L2012/5638Services, e.g. multimedia, GOS, QOS
    • H04L2012/5646Cell characteristics, e.g. loss, delay, jitter, sequence integrity
    • H04L2012/5651Priority, marking, classes
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L12/00Data switching networks
    • H04L12/54Store-and-forward switching systems 
    • H04L12/56Packet switching systems
    • H04L12/5601Transfer mode dependent, e.g. ATM
    • H04L2012/5678Traffic aspects, e.g. arbitration, load balancing, smoothing, buffer management
    • H04L2012/5681Buffer or queue management
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L12/00Data switching networks
    • H04L12/54Store-and-forward switching systems 
    • H04L12/56Packet switching systems
    • H04L12/5601Transfer mode dependent, e.g. ATM
    • H04L2012/5678Traffic aspects, e.g. arbitration, load balancing, smoothing, buffer management
    • H04L2012/5681Buffer or queue management
    • H04L2012/5682Threshold; Watermark

Definitions

  • This invention relates to a class of digital communication systems known as asynchronous transfer mode (ATM) switching systems and generally to intercomputer communications architectures. More particularly, this invention relates to switch fabric-type architecture of an ATM switching element. The invention is not to be confused with a bus-type architecture of an ATM switching element. This invention is useful in telecommunications systems which require real-time routing and switching of digitized cells of data. A particular application is in the field of ISDN data switching at telephone central offices.
  • ATM synchronous transfer mode
  • Bus architectures provide multiple-point-to- multiple-point connections.
  • Switch fabric architectures provide single-point-to-single-point connections.
  • a building block in a switch-fabric architecture ATM switch system is a structure known as a switch element.
  • a switch element provides packet signal routing from one of a plurality of input ports to one or more of a plurality of output ports by maintaining an array of crosspoints for connecting any input port to any output port.
  • Switch elements may be aggregated in various patterns to provide an arbitrarily large N by N possible interconnections of input ports to output ports, each via a unique path.
  • a "brute-force" technique for handling the queuing problem is to provide sufficient data storage at each possible crosspoint in the switch element wherein if the amount of data accumulated at the crosspoint exceeds capacity of the data storage, data is discarded, thus forcing the destination port to request that data be resent.
  • Such a solution is offered in the ATM self-routing switch element Model MB86680, an integrated circuit available from Fujitsu Microelectronics, Inc. of San Jose, California. A representation of this prior art switch element 1 is shown in Fig. 1.
  • the element has a number of input terminals II to In (3, 5, 7) connected via crosspoints 31-39 to a number of output terminals 01 to On (9, 11, 13) through buffer memories 2, 4, 6, 8, 10, 12, 14, 16, and 18.
  • the solution employed by prior art switch element 1 is very wasteful of crosspoint memory because each memory element is connected to just one crosspoint.
  • Memory element 6, for example can only store and buffer data traveling from input In to output 01.
  • memory at many of the crosspoints in the switch element 1 will not be needed, while at high traffic crosspoints more memory will be required than is provided in the switch element 1, and data packets will therefore have to be discarded.
  • Discarding data packets is extremely wasteful of network resources because of the time and processing involved in the destination element requesting that the data packet be resent, and the subsequent retransmission of the data packets by the source element. What is needed is an architecture and switching element that optimally uses available memory for queuing and buffering data packets at high-traffic crosspoints without slowing switching operations.
  • an ATM switching system architecture of a switch fabric-type comprises two different types of components, namely, a plurality of ATM switch element circuit means which make up the switch fabric, and a routing table circuit means for each physical connection from the switch fabric to a workstation.
  • these two components are contained in two separate integrated circuit packages and there is a shared pool of memory.
  • the routing table circuit means of the present invention is by way of example a self-contained circuit that is operative to receive standard 53-byte ATM packets or cells from a source on up to 2048 "Virtual Channels" defined by a header of an ATM packet.
  • a standard ATM cell has 48 bytes of data and a 5-byte header which defines the source, destination, and priority of the cell.
  • the routing table circuit means is operative to implement a routing table that uses the address field in the header of the ATM packet to look up the intended address in its RAM. The routing table then adds a six-byte routing tag to the cell and causes the resultant cell to be output to the switch fabric.
  • the routing tag determines the exact path the cell is to take through the switch fabric and also specifies other characteristics about the cell, such as its priority and its type, i.e., whether or not it is a multicast cell being transmitted to more than one output.
  • the routing table is also used to control a queuing function in which queued cells are stored in an accompanying SRAM before being placed on the switch fabric.
  • the routing table maintains a marked interrupt linked list for storing information about which ones of its virtual channels are experiencing congestion. This linked list is available to a processor in the external workstation to alert the processor when a congestion condition exists in one of the virtual channels.
  • a specific switch element circuit according to the present invention is a circuit that has eight 4-bit-wide nibble inputs and eight 4-bit-wide nibble outputs and is capable of connecting cells received at any of its inputs to any of its outputs, based on the information in a routing tag uniquely associated with each cell.
  • a switch fabric-based architecture built according to the present invention includes a plurality of switch element circuit means interconnected together in any number of known network wiring configurations such as the Clos Network, Delta Network, or Reversed Delta Network.
  • a switch element circuit means of the present invention includes a shared cell buffer pool.
  • the shared cell buffer pool is a memory shared by all of the inputs to the switch element and connected to all of the outputs.
  • the shared cell buffer pool memory provides a similar function to the crosspoint memories of prior art circuits but is far more efficient and flexible because of its unique design.
  • the switch element further includes an input controller connected to input crosspoints, an output controller connected to output crosspoints, and a multi-priority buffer pool controller
  • MPBPC All cells passing through the switch element are written into the cell buffer pool during a first cell cycle, which is equivalent to 118 clock cycles. Cells may be sent to a switch element output during the next cell cycle, or if the destination output of the cell is not available, the cell may remain in the buffer pool for several cell cycles before being connected to the switch element output.
  • the MPBPC reads tags from the input controller and directs the storage of cells in the cell buffer pool.
  • the MPBPC also, through the output controller, directs when cells are read out of the cell buffer pool and sent to the output lines through the output I/O crosspoints.
  • the MPBPC includes circuitry for generating and responding to back-pressure signals generated when a receiving channel cannot receive data at a rate fast enough to keep up with the transmitter.
  • the back-pressure signals first notify the routing table, and then the workstation to which it is attached, to halt data transmission to allow the receiver an opportunity to catch up with the transmitted data stream.
  • the back-pressure signal handling circuitry allows the switch element of the present invention to be employed in "closed- loop" ATM systems that can maximize data throughput through the switch without losing cells.
  • the switch element also has a single bit per set of four inputs and per set of four outputs, known as an aggregate bit that, when set, directs the switch element to treat the four inputs as one input for the purposes of routing and FIFO queuing.
  • the MPBPC of the switch element also supports proportional bandwidth queuing, which ensures that virtual channels that are assigned to different bandwidths receive a fair share of the available bandwidth in proportion to other channels.
  • the switch elements and routing tables support multicast transmission.
  • data from one source is distributed to several destinations, i.e., the multicast group, which comprise some, but not necessarily all, of the possible switch fabric outputs.
  • the multicast group which comprise some, but not necessarily all, of the possible switch fabric outputs.
  • An example of such an application is a video conference over a network in which several workstations are connected by a switch fabric, and voice and image data are transmitted from each workstation to each of the other workstations.
  • each switching element conserves available cell memories and thereby avoids dropping cells by issuing back-pressure commands to inputs on a per-input, per-priority basis to halt the flow of cells having a given priority from a given input.
  • Back-pressure is asserted for a given input and given priority whenever the number of currently enqueued cells of the given priority supplied by the given input exceeds a predetermined threshold.
  • Back-pressure for a given priority is asserted for all inputs whenever the number of available cell memories falls below a threshold associated with that priority.
  • Fig. 1 is a block diagram of a prior art ATM switch element of an ATM switch fabric architecture.
  • Fig. 2 is a block diagram of an ATM switch fabric architecture implemented in accordance to the invention.
  • Fig. 3 is a block diagram illustrating an example of a portion of a switch fabric architecture with ATM routing table circuits and switch element circuits configured according to the invention in an ATM switch fabric architecture.
  • Fig. 4 is a block diagram of an ATM switch element circuit with external SRAM according to the invention.
  • Fig. 5 is a block diagram of an ATM switch element circuit with a cell buffer pool according to the invention.
  • Fig. 6 is a block diagram showing an address multiplexer coupled to a linked list controlling a buffer pointer according to the invention.
  • Fig. 7 is a block diagram of a back-pressure controller according to the invention.
  • Fig. 8A and Fig. 8B are block diagrams of configurations for switch elements circuits with back-pressure control.
  • Fig. 9 is a block diagram of an aggregate bit controller.
  • Fig. 10 is a table illustrating service order for one service order period.
  • Fig. 11 is a block diagram showing the source cell duplication multicasting of the prior art.
  • Fig. 12 is a block diagram showing mid-cell duplication multicasting according to the prior art.
  • Fig. 13 is a block diagram showing mid-switch duplication multicasting according to the prior art.
  • Fig. 14 is a block diagram showing tree-based duplication multicasting according to the prior art.
  • Fig. 15 is a block diagram showing tree-based duplication multicasting according to a specific embodiment of the present invention.
  • Fig. 16 is a tabular illustration of per-priority queuing with per VPC cell counts in a routing table according to the invention.
  • Fig. 17 is a tabular illustration of a per VC count of queued cells according to the invention. DESCRIPTION OF SPECIFIC EMBODIMENTS SWITCH FABRIC
  • Fig. 2 is a block diagram of an ATM switch fabric architecture 20 according to the invention. Shown in the figure is a 4 x 5 array of switch element circuit blocks (SE) 40. Each switch element 40 accepts eight 4-bit-wide input lines and has eight 4-bit-wide output lines.
  • the switch fabric 20 thus has a total of thirty-two 4-bit-wide input lines shown at the left side of the switch fabric 20. In an operating switch, each of these thirty-two four-bit-wide input lines is connected from a separate routing table circuit, and each such routing table circuit is connected to a workstation or other digital device.
  • the switch fabric has thirty-two four-bit-wide output lines shown at the right side of the figure. Each of the output lines is connected to individual further routing table circuits, each of which is connected to a workstation.
  • the switch fabric shown in Fig. 2 may provide a physical connection to up to thirty-two workstations and may connect data from any one of those thirty-two workstations to any other one of those or other thirty-two workstations.
  • the switch fabric architecture 20 as shown in Fig. 2 is known as a Reversed Delta Network architecture.
  • the switch element circuits of the present invention may be used in any number of other known network wiring configurations such as the Clos network or the Delta network, and the switch array may be expanded to provide any number of input lines and any number of output lines.
  • Fig. 2 is a configuration bus connected to each one of the switch elements. The configuration bus is used by a configuration processor of the switch system to set up a number of switch parameters in a memory in the switch element.
  • FIG. 3 shows a portion of a switch fabric made up of four switch elements 40.
  • One of the switch elements is shown with interconnections through eight routing tables 30, to a number of workstations 50, and to a server computer 52.
  • each input to a switch fabric is connected to a routing table circuit 30.
  • a routing table circuit 30 is typically connected to some type of digital workstation 50, which may transmit and receive voice, video, and digital data via the switch fabric.
  • FIG. 3 Also shown in Fig. 3 is an aggregate input connection to a switch element in accordance with one specific embodiment of the invention.
  • an aggregate connection four of the input lines of the switch fabric are grouped together and act as one input to receive and transmit data to a high-speed data device such as server computer 52.
  • the same switch element and switch fabric can handle two speeds of packet data, the first speed being the speed of one input line and the second speed being four times faster or the speed of the aggregated input lines.
  • Fig. 4 is a block diagram of a routing table circuit 42 according to the present invention.
  • the routing table circuit 42 is a combination storage and control device that is used with external memory, e.g., SRAM 90, and includes a receive queue controller 80 which sends data to the switch fabric and receives a back-pressure signal from the switch fabric, and a transmission buffer controller 82 which receives data from the switch fabric after that data has been processed by the multicast header translation circuit 84 and asserts back pressure to the switch fabric.
  • the transmission buffer controller 82 also includes a small buffer memory 86 for storing cells received from the switch fabric.
  • a further controller called a connection table controller 88, is for reading header information from the workstation interface and is operative to use that header information to add an appropriate switch tag to the cells before they are transmitted to the switch fabric. Controller 88 stores information about switch tags and buffers data in external SRAM 90. Further included are an interrupt processor 92 and processor interface 94, which are for sending control signals to the workstation. Optionally included is an OAM ⁇ BECN cell transmit circuit 96 for inserting control cells to the outgoing data stream.
  • the routing table circuit 42 in each instance operates by receiving one 8-bit-wide segment of data via connection from a workstation as input and provides one 8-bit-wide workstation output.
  • the routing table includes one 4-bit output to the switch fabric and receives one 4-bit input from the switch fabric.
  • Fig. 5 is a block diagram of the structure of switch element circuit 40 according to the present invention.
  • the switch element circuit 40 includes a very small cell buffer pool memory 100 for storing and queuing cells being transmitted through the switch element, input I/O crosspoint block 110 for connecting any input line to any cell memory in the cell buffer pool, output I/O crosspoint block 120 for connecting any output line to any cell memory in the cell buffer pool, input bus controller 130 for controlling data flow on the input bus to the cell memories, output bus controller 140 for controlling data flow from the cell memories to the output lines, and multipriority buffer pool controller (MPBPC) 150 for controlling assignment of cell memories to connections defined by the cross-point blocks.
  • MPBPC multipriority buffer pool controller
  • the switch element circuit 40 is connected to a configuration bus 41 which supplies configuration data to controller 150.
  • the switch element 40 has eight input interfaces, labeled 10 through 17, and eight output interfaces, labeled OO through 07.
  • Each of the eight inputs and eight outputs is a four-bit or nibble-wide interface capable of operating at for example up to 50 Mhz, i.e., sufficient to support digital communications at the current ATM OC-3 standard.
  • Each of the inputs receives cells from another switch element in the switch fabric or from a routing table, as previously outlined.
  • ATM cells of data are transferred as one hundred and eighteen four-bit nibbles. This allows the standard fifty-three ATM byte cells to be transferred along with six overhead bytes.
  • a cell start signal goes high every one hundred and eighteen clock cycles to indicate the start of a cell.
  • Cell buffer pool 100 is a pool of random access memory.
  • the pool contains thirty-two individual cell memories, each capable of storing an entire cell of one hundred and eighteen nibbles.
  • the thirty-two memories can be connected to any one of the eight inputs by input crosspoint block 110 which is controlled by input bus controller 130.
  • crosspoint block 110 contains a plurality of multiplexers 112 for connecting input buses to any of the cell memories. Multiplexers 112 are controlled by signals from the input bus controller 130 that are transmitted on six-bit wide connection control bus lines 132. Any of the cell memories may be connected to any of the output lines via output crosspoint block 120.
  • Output crosspoint block 120 is controlled by the output bus controller 140 via output connection control bus lines 142.
  • MPBPC 150 contains a link list RAM 152 for storing queue assignment information about the cell buffer pool memories, a service order table 154 for controlling the service order of the proportional bandwidth queues, a memory for multicast group bits 156 for storing information about multicast cell transmission, and a back-pressure control circuit 158 for asserting multipriority back-pressure on the eight back-pressure lines of the switch element.
  • MPBPC 150 uses its linked list RAM 152 to maintain five First-In/First-Out (FIFO) queues by means of lists of pointers to the next entry in the cell memory for each of the five output lines for a total of 40 possible virtual queues.
  • FIFO First-In/First-Out
  • Fig. 6 is a representation of the linked-list RAM 152 and associated head register set 153 and tail register set 155 for the forty queues defined for the 32 cell memories accounted for in the list RAM 152.
  • a buffer pointer is constructed from the head address of the queue and tail address of the queue for one of the forty queues stored in the head register set 153 and the tail register set 155.
  • a head pointer and a tail pointer is kept for each one of the forty queues.
  • the forty queues share a linked list of up to thirty-two entries. Each entry can identify one of the thirty-two cell memories in cell buffer pool 100.
  • the linked-list thereby specifies which cell memories are part of each queue in FIFO order.
  • Cells are enqueued onto the tail of the proper queue and dequeued from the head of the proper queue in accordance with a queue service procedure which generates the Q-dequeue pointer value to the head register set 155 and the Iqueue pointer to the tail register set 155.
  • a mux 157 switches between register sets, depending on whether the procedure calls for enqueuing or dequeuing a cell.
  • An input buffer pointer specifies where the input cell is currently stored, and the output pointer designates where the cell is to be directed.
  • the queues for one output line are assigned five different priorities. Three of the queues are proportional bandwidth queues of equal priority but having an assigned bandwidth of 5/8, 2/8 (1/4), or 1/8. Of the remaining two queues, one is designated a high-priority queue which may be used for very time dependent data such as voice, and the other a multicast queue which is used for data being sent from one transmitting workstation to more than one receiving workstation which might be the case in video conferences.
  • Multipriority buffer pool controller (MPBPC) 150 controls the overall function of the switch element 40 as follows. During each cell cycle, cells having a length of 118 nibbles may be received on any or all of the eight input lines. Prior to the start of a cell cycle, the controller 150 has specified which input line is connected to which cell memory via input crosspoint block 110 by setting bits in input controller 130. The first twelve of the 118 nibbles from each input line are read by input bus controller 130 and transmitted to the multipriority buffer pool controller 150 while the ATM cell is being stored in its designated cell memory. From these tags, MPBPC 150 determines the priority and destination of the cell that has just been stored in each of the eight cell memories connected to one of the input interfaces.
  • the MPBPC 150 then adds the cell memories to their appropriate queues by updating its linked lists.
  • the MPBPC 150 determines which of the output interfaces to which cells are directed can receive the cells during the next clock cycle.
  • An output interface may be unavailable to receive all of the data which is directed towards it during a cycle when more than one input lines are directing cells to a single output interface, or when the output interface has asserted back-pressure to the MPBPC 150.
  • the MPBPC 150 handles the problem of output interfaces being unavailable to receive cells by establishing queues for the connections for which output lines are temporarily unavailable in the cell buffer pool 100. Cells may be stored in these queues in a first-in-first-out FIFO fashion for some number of cell cycles until the output interfaces are available for outputting the cells.
  • the MPBPC 150 Once the MPBPC 150 has made determinations regarding which cells can be transmitted to their destination output interfaces during the next clock cycle and which cells will be stored in queues in the cell buffer pool 100, it directs the output interfaces to receive data from cell memories in the cell buffer pool by sending control signals to output bus controller 140. It also directs the input interfaces to available cell memories by sending input control signals to input bus controller 130.
  • One problem that may arise in a switch element 40 as packets are being routed through cell memories from input lines to output lines is the unavailability of cell memories for queuing prior to a clock cycle when new cells may be received on the input interfaces. If a cell is received at an input interface to the switch element when no cell memory is available to receive the cell, the cell must be dropped and the data resent.
  • each switching element 150 avoids dropping cells by issuing back ⁇ pressure signals to each connection to each of its input interfaces on a per-input, per-priority basis to halt the flow of cells having a given priority to a given input.
  • Back ⁇ pressure is asserted for a given input and given priority whenever the number of currently enqueued cells of the given priority supplied by the given input exceeds a predetermined threshold.
  • Back-pressure for a given priority is also asserted for all inputs whenever the total number of available cell memories falls below a threshold associated with that priority.
  • the switching element of the present invention virtually eliminates the deletion of cells due to exhaustion of available memory.
  • even infrequent cell drops are harmful in that the loss of one cell necessitates the retransmission of many cells, substantially reducing network efficiency.
  • cell drops occur at a routing table rather than at a switching element, since routing tables employ sophisticated congestion management strategies unavailable at a switching element when dropping cells.
  • One such sophisticated congestion management strategy is the standard ATM Adaption Layer 5 (AAL5) early frame discard technique, AAL5 being a technique for segmenting frames into cells.)
  • Fig. 7 is a simplified representation of the elements within back-pressure controller 150 used to implement the back-pressure capability.
  • Back-pressure controller 150 includes a time domain multiplexer 402, a state machine 404, a time domain demultiplexer 406, a queue service controller 408, an index memory 410, a variable delay circuit 412, and a variable delay register 414.
  • Back-pressure signals are generated by state machine 404 based on criteria as discussed below.
  • Back-pressure signals from other switching elements or a routing table are received by a queue service controller 408 which selects cells for output.
  • Back-pressure is asserted for a given input and given priority whenever the number of currently-enqueued cells of the given priority supplied by the given input exceeds a predetermined threshold.
  • a problem is posed in that queues, with the exception of the multicast queue, are organized by output rather than input.
  • index memory 410 is maintained within the back-pressure controller 158 with an entry for each cell memory location which identifies the source of the cell stored there. When a new cell is enqueued, index memory 410 is updated and a counter, internal to state machine 404 and associated with the source and priority of the cell, is incremented.
  • the index entry for that cell is read to identify the source for that cell and the appropriate counter is then decremented.
  • the counter for that input and priority is compared to a predetermined threshold.
  • the predetermined threshold is the same for each input and priority. Thus, back-pressure is allocated among inputs so that no one input blocks incoming traffic from other inputs by occupying a disproportionate share of the cell memory locations.
  • a counter is maintained for each priority for the aggregated inputs as a group rather than for each input.
  • a count of empty cell memories is maintained within state machine 404 and compared to thresholds stored for each priority.
  • back-pressure is asserted for that priority for every input.
  • the higher priorities have lower thresholds set so that high priority traffic is impeded last as the count of available cell memories decreases.
  • the thresholds are normally set so that high-priority traffic has strict priority over lower priority traffic.
  • back ⁇ pressure signals for the various priorities are time-domain multiplexed together by time-domain multiplexer 402 so that each input is provided with a single back-pressure signal.
  • Received back-pressure signals are demultiplexed by time domain demultiplexer 406. Each priority then corresponds to a different time slot within the time-domain multiplexed back ⁇ pressure signal.
  • a switching element and an associated input device may or may not be on the same printed circuit board.
  • Fig. 8A and Fig. 8B are a simplified representation of these two situations.
  • a set of retiming buffers 51-54 compensate for inter-card delays.
  • a switching element 40 according to one embodiment of the present invention is provided with internal means for establishing a variable delay in the back-pressure line.
  • Fig. 7 shows variable delay circuit 412 inserted in one of the back-pressure lines. The variable delay is selected by writing to variable delay register 414 within the switching element.
  • a switch element 40 includes two aggregate input bits, agg_in(0) 151 and agg_in(l) 153 and two aggregate output bits, agg_out(0) 155 and agg_out(l) 157, which may be set by the configuration controller (not shown) to allow for aggregating either the group of inputs 10 to 13, the group of inputs 14 to 17, the group of outputs 00 to 03, or the group of outputs 04 to 07.
  • some types of digital devices particularly other switching systems, such as server computer 52, may need to transmit data through the switch fabric at a higher rate than may be provided by one switch element input, e.g., at the 622 Mbps rate provided by a conventional 0C-12 ATM interface.
  • Fig. 9 shows the groupings of inputs and outputs of a switch element 40 when the signals agg_in(0) and agg_in(l) and agg_out(0) and agg_out(l) are set.
  • Switch element 40 has two bits for input lines and two bits for output lines that signal the switch element when a set of four of its input lines is being configured as a single input which receives four cells at once and retains FIFO order.
  • Input bits agg_in(0) and agg_in(l) are set in the multipriority buffer pool controller 150 via the configuration bus . When bit agg_in(0) is set true, inputs 0 through 3 are treated as if they are a single stream of cells.
  • a switch element 40 can support a mixture of data rates on its inputs.
  • a key problem that must be addressed when aggregating input lines is maintaining FIFO order between the cells that arrive simultaneously.
  • the cells from each input are enqueued separately.
  • the cells are enqueued as if they were from a single input with the cell received on input 0 placed in the single aggregated FIFO queue first, the cell received on input 1 placed in the single FIFO queue second, and so on.
  • FIFO order could be violated as the MPBPC 150 uses a round-robin procedure to enqueue multicast cells to improve fairness. This procedure intentionally chooses cells from the inputs in differing orders from cell time to cell time.
  • a second problem that the invention addresses is that cells bound for an aggregated output can go out any one of the outputs in the aggregated output, depending on availability.
  • the aggregate bit is set, cells bound for the aggregated output are dequeued for any one of its outputs.
  • the MPBPC 150 also uses the aggregate bit to determine how to assert back ⁇ pressure to an output from the previous stage in an aggregated input. Back-pressure is given if cells from a given input are queued excessively.
  • the MPBPC 150 measures the counts of the cells from any of the inputs in the aggregated input, rather than from the individual inputs. The back-pressure is then applied to all of the inputs in the aggregated input, rather than the individual inputs. More specifically, to aggregate, in the first level of the switch fabric (Fig. 2) , the agg_in value is set for all inputs actually connected to a high speed input. Agg_out is set for all possible destinations of the inputs that agg_in is set for. In subsequent levels, agg_in is set for those links which have agg_out set in the previous level.
  • agg_out is set for all possible destinations of an input in which agg_in has been set.
  • agg_in is set for those links which have agg_out set in the previous level.
  • Agg_out is set for those links which are actually connected to a 622 Mbps output port.
  • Switch elements 40 and the routing table circuits 30 can also support proportional bandwidth queues. Proportional bandwidth queues solve a problem that arises when data traffic from sources of drastically different bandwidths coexist on an intermediate link.
  • the switch element 40 and routing table circuit 30 can support for example three queues that are of equal priority but which have bandwidths of 1/8, 1/4 and 5/8 of the available bandwidth.
  • MPBPC 150 maintains a service order table 154 (Fig. 10) which enhances fairness among connections having different bandwidths. The assigned proportions can be adjusted by externally altering the contents of the service order table 154.
  • the proportional bandwidth queues are implemented by the MPBPC 150 by having a service order table 154 for the dequeuing process that specifies the order of queue service for each of the output queues.
  • the schedule in each stage is delayed by one cell period which tends to minimize queuing and thus the cell memories required by trying to dequeue a cell from a given queue just after a cell from that queues is likely to have arrived.
  • MPBPC 150 must make sure that all of the possible competitions between differing bandwidth queues turn out as predicted. For example, if cells only in the l/8th and l/4th queues arrive, then the l/8th queue should get l/3rd of the available bandwidth on the output channel, and the l/4th queue should get 2/3rd of the bandwidth.
  • Fig. 10 shows a service order table 154 stored in MPBPC 150 for determining dequeuing from cell buffer pool 100 when more than one proportional bandwidth queue is queued for a given output.
  • the MPBPC 150 defines a service order interval of eight cell transmission cycles for determining the priority of serving the proportional bandwidth queues. These cycles are represented by the eight columns labelled 0 to 7 in Fig. 10.
  • MPBPC 150 examines which queues for a given output wish to transmit data to that output. It will be seen that during any given cycle there is a queue service order listing the priority with which bandwidth queues will be serviced.
  • any cycle only one queue is serviced and that queue is the queue having the highest priority, 1st through 3rd, as listed in the service order table during that cycle. For example, during cell cycle 4, the priority list is 3, 4 and 2. Should cells from two proportional bandwidth queues both be ready to transmit during cycle 4, the cell from the queue having bandwidth of the higher priority will be transmitted. During the next clock cycle, cycle 5, if both of those queues wish to transmit, the next bandwidth queue will be transmitted because the queue service order table shows that it has a higher priority during that cell cycle.
  • One data transmission application which the present invention may be employed to support is multicast transmission.
  • data from one source is distributed to several destinations, i.e., the multicast group, which comprise some, but not necessarily all, of the possible switch fabric outputs.
  • destinations i.e., the multicast group
  • An example of such an application is a video conference over a network in which several workstations are connected by a switch fabric, and voice and image data are transmitted from each workstation to each of the other workstations.
  • Source cell duplication is a simple, but brute-force solution to the multicast support problem.
  • Fig. 11 is a simplified representation of the source cell duplication solution 60 known in the prior art.
  • the source 62 of the data cells creates copies of each cell for transmission to each destination.
  • This solution suffers from a number of significant disadvantages. Not only does the task of duplication place a severe load on the source 60, it also places limits on the number of destinations 64, 66, 68 in the multicast group connected by a switch element 40.
  • the size of the multicast group for a network which supports multicast may be drastically limited (e.g., to half the size of connection for the network). Additionally, expensive bandwidth is wasted. For example, in cases where more than one destination is at the same distant location, redundant copies of the information are transmitted over the entire distance, thereby unnecessarily contributing to system traffic.
  • Mid-switch cell duplication 70 is an alternate multicast support solution. A simplified representation of a mid-switch duplication 70 solution is shown in Fig. 12.
  • a module 72 is provided at some point in the switching system which duplicates the transmitted cells from a source 74 as necessary for distribution to destinations 64, 66, 68 in the multicast group.
  • this solution does not suffer from all of the disadvantages of the source cell duplication solution 60, bandwidth is still unnecessarily consumed by the transmission of the cell duplicates through the remainder of the system.
  • the optimal solution for supporting multicast is referred to as tree- based cell duplication.
  • a simplified representation of a tree-based cell duplication system 76 according to the invention is provided in Fig. 13.
  • a tree-based cell duplication system 76 With a tree-based cell duplication system 76, the transmitted cells are not duplicated until the last points of divergence 77, 79 to the destinations 64, 66, 68, e.g., by means of cell replication within the switch element 40, represented herein as a serial redirector 78. This avoids the unnecessary consumption of bandwidth encountered with the previously described solutions.
  • One possible complication of this solution is that all destinations 64, 66, 68 of a multicast cell may not be reachable with a cell having the same address.
  • the present invention implements a tree-based cell duplication system 76 using a specific embodiment of the switch element 40 described above. The solution is described with reference to Fig. 14.
  • An eight-bit field in the routing tag of each transmitted cell determines what is called the multicast group for that cell.
  • the routing tag a twelve-nibble field placed on the front of a cell by the routing table circuit based on the content of a multicast group bit register 81, dictates the path of the cell through the switch fabric.
  • the multicast group consists of the group of network destinations to which the cell is to be transmitted.
  • the multicast group field determines which switch element outputs upon which a received cell is to be placed in order to get the information to the desired destinations.
  • the switch element of the present invention stores an array of multicast group bits in its RAM, the array including one eight-bit word for each of the multicast groups. Each bit in each word represents one switch element output.
  • the multicast group field in the cell is used as an index into the multicast group bits array, pointing to a particular word in the array. Any bits which are set in the selected word correspond to the switch element outputs on which the cell in the multicast queue is to be placed.
  • One difficulty encountered in multicast transmissions is that it is often not possible to place a given cell on all of the desired outputs simultaneously. This is referred to as the problem of multicast completion. Such a situation might arise, for example, if a cell from a higher priority queue has already been placed on the selected output. This situation can also occur if a subsequent switch element has exerted back-pressure on the selected output, thereby preventing the transmission of cells from that output.
  • Some open loop switching systems simply allow cells to be dropped if congestion causes cell buffers to overflow. If this occurs with the transmission of video information, for example, the penalty incurred from such a drop is relatively high. An entire video frame might be lost due to the loss of just one cell. Other penalties are incurred if the system protocol requires the retransmission of the entire frame or a series of frames.
  • the switch element 40 of the present invention solves this problem by keeping a record of the switch element outputs upon which the cell in the multicast queue has been successfully placed.
  • multicast queue controller 156 of switch element 40 sets bits in a multicast queue completion register 83 for each output on which the cell is actually placed.
  • the ATM cell in the multicast queue is dequeued only when the bits in the completion register 83 match the bits in the word selected from the multicast group bits array stored in the multicast group bits register 81.
  • cell M 85 in the multicast queue is to be placed on outputs (2) , (3) , and (5) 89, 91, 93.
  • the routing table circuit 30 of the present invention receives a cell from a source, looks up the intended address in its RAM, adds the appropriate routing tag to the cell, and then puts the cell out onto the switch fabric via the switching elements.
  • the routing table circuit 30 also performs a queuing function in which it stores queued cells in an accompanying SRAM before placing them on the switch fabric.
  • the routing table circuit 30 of each source queues the cells on a per priority basis, but also keeps track of how many cells from each connection are in the queue at any given time. Unlike a strict per connection queuing discipline, a transmission scheduler is not required.
  • Fig. 16 is a table which illustrates the manner in which the routing table queues incoming cells.
  • cells having priorities 0, 2, and 5 have been queued by the routing table.
  • the count of queued cells per connection is maintained as shown in Fig. 17, showing the number of queued cells for each VPC.
  • the routing table uses the connection count to perform closed loop functions such as sending a back-pressure signal to a particular data cell source.
  • One method for doing this is to generate an interrupt signal each time a congestion condition is encountered. However, it is not desirable to generate an interrupt every time a data cell is queued for a congested channel, especially if the processor has already been notified of the congestion on that channel. Also, more than one channel may experience congestion before the processor is able to respond to a congestion interrupt. It is therefore necessary to keep track of channels experiencing congestion so that the processor may take appropriate action for all such channels when it is ready to do so.
  • One method for keeping track of congested channels includes assigning a bit for each of the channels, and setting the bits corresponding to channels which experience congestion.
  • the routing table maintains a linked list of currently congested channels. Each channel in the list is also marked (i.e., a "congestion" bit is set) so that the queuing of further data cells for those channels does not generate additional interrupts.
  • a data cell is queued for a particular channel, the current queue depth for that channel is compared with the configured congested queue depth. If the current queue depth is longer and the channel is not marked, the routing table generates an interrupt and the channel is marked and added to the end of the linked list. If the channel is already marked, nothing happens.
  • the processor When the processor is able to respond to an interrupt, it first looks at the interrupt head pointer which points to an address which represents the first virtual channel in the linked list. The processor then reads from that channel the address for the next channel experiencing congestion. The processor continues to read the addresses for the channels experiencing congestion until it reaches the end of the linked list. The processor then takes appropriate action for each of the channels in the list to resolve the congestion. Such action might include, for example, sending an appropriate feedback message, or changing the queue depth which causes interrupts.
  • a congestion-relieved threshold is determined by multiplying the congestion threshold by a fractional constant (e.g., 0.75). And whenever a cell is dequeued, and the current queue depth falls below the congestion-relieved threshold, a second interrupt is generated, and the congestion is cleared.
  • a global "enable" for interrupts allows the system processor to read the linked list of congested channels atomically. If a channel becomes congested while the interrupts are disabled, once the interrupts are re-enabled, the next cell queued for that channel will cause an interrupt if the channel is still congested.

Abstract

An ATM switching system architecture of a switch fabric type (20) is built of a plurality of ATM switch element circuits (40) and routing table circuits (30) for each physical connection to/from the switch fabric (20). A shared pool of memory (100) is employed to eliminate the need to provide memory at every crosspoint. Each routing table (30) maintains a marked interrupt linked list for storing information about which ones of its virtual channels are experiencing congestion. This linked list is available to a processor in an external workstation to alert the processor when a congestion condition exists in one of the virtual channels. The switch element circuit (40) typically has up to eight 4-bit-wide nibble inputs (I0-I7) and eight 4-bit-wide nibble outputs (O0-O7) and is capable of connecting cells received at any of its inputs (I0-I7) to any of its outputs (O0-O7), based on the information in a routing tag uniquely associated with each cell.

Description

ATM ARCHITECTURE AND SWITCHING ELEMENT
BACKGROUND OF THE INVENTION This invention relates to a class of digital communication systems known as asynchronous transfer mode (ATM) switching systems and generally to intercomputer communications architectures. More particularly, this invention relates to switch fabric-type architecture of an ATM switching element. The invention is not to be confused with a bus-type architecture of an ATM switching element. This invention is useful in telecommunications systems which require real-time routing and switching of digitized cells of data. A particular application is in the field of ISDN data switching at telephone central offices.
There is a need for high-speed switching and throughput in a telecommunications switch for digital communication applications wherein the primary usage is switching data between an external source or input resource and an external output or destination resource. Two types of architectures are known: bus architectures and switch-fabric architectures. Bus architectures provide multiple-point-to- multiple-point connections. Switch fabric architectures provide single-point-to-single-point connections.
A building block in a switch-fabric architecture ATM switch system is a structure known as a switch element. A switch element provides packet signal routing from one of a plurality of input ports to one or more of a plurality of output ports by maintaining an array of crosspoints for connecting any input port to any output port. Switch elements may be aggregated in various patterns to provide an arbitrarily large N by N possible interconnections of input ports to output ports, each via a unique path.
Problems arise where the receiving port cannot assimilate information as fast as it is delivered or where the priority of the traffic varies. A "brute-force" technique for handling the queuing problem is to provide sufficient data storage at each possible crosspoint in the switch element wherein if the amount of data accumulated at the crosspoint exceeds capacity of the data storage, data is discarded, thus forcing the destination port to request that data be resent. Such a solution is offered in the ATM self-routing switch element Model MB86680, an integrated circuit available from Fujitsu Microelectronics, Inc. of San Jose, California. A representation of this prior art switch element 1 is shown in Fig. 1. The element has a number of input terminals II to In (3, 5, 7) connected via crosspoints 31-39 to a number of output terminals 01 to On (9, 11, 13) through buffer memories 2, 4, 6, 8, 10, 12, 14, 16, and 18. The solution employed by prior art switch element 1 is very wasteful of crosspoint memory because each memory element is connected to just one crosspoint. Memory element 6, for example, can only store and buffer data traveling from input In to output 01. Depending on packet traffic through the switch element 1, memory at many of the crosspoints in the switch element 1 will not be needed, while at high traffic crosspoints more memory will be required than is provided in the switch element 1, and data packets will therefore have to be discarded. Discarding data packets is extremely wasteful of network resources because of the time and processing involved in the destination element requesting that the data packet be resent, and the subsequent retransmission of the data packets by the source element. What is needed is an architecture and switching element that optimally uses available memory for queuing and buffering data packets at high-traffic crosspoints without slowing switching operations.
SUMMARY OF THE INVENTION According to the invention, an ATM switching system architecture of a switch fabric-type comprises two different types of components, namely, a plurality of ATM switch element circuit means which make up the switch fabric, and a routing table circuit means for each physical connection from the switch fabric to a workstation. In a specific embodiment of the invention, these two components are contained in two separate integrated circuit packages and there is a shared pool of memory.
The routing table circuit means of the present invention is by way of example a self-contained circuit that is operative to receive standard 53-byte ATM packets or cells from a source on up to 2048 "Virtual Channels" defined by a header of an ATM packet. A standard ATM cell has 48 bytes of data and a 5-byte header which defines the source, destination, and priority of the cell. The routing table circuit means is operative to implement a routing table that uses the address field in the header of the ATM packet to look up the intended address in its RAM. The routing table then adds a six-byte routing tag to the cell and causes the resultant cell to be output to the switch fabric. The routing tag determines the exact path the cell is to take through the switch fabric and also specifies other characteristics about the cell, such as its priority and its type, i.e., whether or not it is a multicast cell being transmitted to more than one output. The routing table is also used to control a queuing function in which queued cells are stored in an accompanying SRAM before being placed on the switch fabric. According to the invention, there is one routing table for each physical connection into the switch fabric. The routing table maintains a marked interrupt linked list for storing information about which ones of its virtual channels are experiencing congestion. This linked list is available to a processor in the external workstation to alert the processor when a congestion condition exists in one of the virtual channels.
A specific switch element circuit according to the present invention is a circuit that has eight 4-bit-wide nibble inputs and eight 4-bit-wide nibble outputs and is capable of connecting cells received at any of its inputs to any of its outputs, based on the information in a routing tag uniquely associated with each cell. A switch fabric-based architecture built according to the present invention includes a plurality of switch element circuit means interconnected together in any number of known network wiring configurations such as the Clos Network, Delta Network, or Reversed Delta Network.
A switch element circuit means of the present invention includes a shared cell buffer pool. The shared cell buffer pool is a memory shared by all of the inputs to the switch element and connected to all of the outputs. The shared cell buffer pool memory provides a similar function to the crosspoint memories of prior art circuits but is far more efficient and flexible because of its unique design. The switch element further includes an input controller connected to input crosspoints, an output controller connected to output crosspoints, and a multi-priority buffer pool controller
(MPBPC) . All cells passing through the switch element are written into the cell buffer pool during a first cell cycle, which is equivalent to 118 clock cycles. Cells may be sent to a switch element output during the next cell cycle, or if the destination output of the cell is not available, the cell may remain in the buffer pool for several cell cycles before being connected to the switch element output. The MPBPC reads tags from the input controller and directs the storage of cells in the cell buffer pool. The MPBPC also, through the output controller, directs when cells are read out of the cell buffer pool and sent to the output lines through the output I/O crosspoints. The MPBPC includes circuitry for generating and responding to back-pressure signals generated when a receiving channel cannot receive data at a rate fast enough to keep up with the transmitter. The back-pressure signals first notify the routing table, and then the workstation to which it is attached, to halt data transmission to allow the receiver an opportunity to catch up with the transmitted data stream. The back-pressure signal handling circuitry allows the switch element of the present invention to be employed in "closed- loop" ATM systems that can maximize data throughput through the switch without losing cells. The switch element also has a single bit per set of four inputs and per set of four outputs, known as an aggregate bit that, when set, directs the switch element to treat the four inputs as one input for the purposes of routing and FIFO queuing. The MPBPC of the switch element also supports proportional bandwidth queuing, which ensures that virtual channels that are assigned to different bandwidths receive a fair share of the available bandwidth in proportion to other channels. In accordance with one aspect of the invention, the switch elements and routing tables support multicast transmission. During multicast transmission, data from one source is distributed to several destinations, i.e., the multicast group, which comprise some, but not necessarily all, of the possible switch fabric outputs. An example of such an application is a video conference over a network in which several workstations are connected by a switch fabric, and voice and image data are transmitted from each workstation to each of the other workstations. In accordance with one aspect of the invention, each switching element conserves available cell memories and thereby avoids dropping cells by issuing back-pressure commands to inputs on a per-input, per-priority basis to halt the flow of cells having a given priority from a given input. Back-pressure is asserted for a given input and given priority whenever the number of currently enqueued cells of the given priority supplied by the given input exceeds a predetermined threshold. Back-pressure for a given priority is asserted for all inputs whenever the number of available cell memories falls below a threshold associated with that priority.
The invention will be better understood upon reference to the following detailed description taken in conjunction with the accompanying drawings.
BRIEF DESCRIPTION OF THE DRAWINGS
Fig. 1 is a block diagram of a prior art ATM switch element of an ATM switch fabric architecture. Fig. 2 is a block diagram of an ATM switch fabric architecture implemented in accordance to the invention.
Fig. 3 is a block diagram illustrating an example of a portion of a switch fabric architecture with ATM routing table circuits and switch element circuits configured according to the invention in an ATM switch fabric architecture.
Fig. 4 is a block diagram of an ATM switch element circuit with external SRAM according to the invention.
Fig. 5 is a block diagram of an ATM switch element circuit with a cell buffer pool according to the invention.
Fig. 6 is a block diagram showing an address multiplexer coupled to a linked list controlling a buffer pointer according to the invention.
Fig. 7 is a block diagram of a back-pressure controller according to the invention.
Fig. 8A and Fig. 8B are block diagrams of configurations for switch elements circuits with back-pressure control.
Fig. 9 is a block diagram of an aggregate bit controller.
Fig. 10 is a table illustrating service order for one service order period.
Fig. 11 is a block diagram showing the source cell duplication multicasting of the prior art. Fig. 12 is a block diagram showing mid-cell duplication multicasting according to the prior art.
Fig. 13 is a block diagram showing mid-switch duplication multicasting according to the prior art.
Fig. 14 is a block diagram showing tree-based duplication multicasting according to the prior art.
Fig. 15 is a block diagram showing tree-based duplication multicasting according to a specific embodiment of the present invention.
Fig. 16 is a tabular illustration of per-priority queuing with per VPC cell counts in a routing table according to the invention.
Fig. 17 is a tabular illustration of a per VC count of queued cells according to the invention. DESCRIPTION OF SPECIFIC EMBODIMENTS SWITCH FABRIC
Fig. 2 is a block diagram of an ATM switch fabric architecture 20 according to the invention. Shown in the figure is a 4 x 5 array of switch element circuit blocks (SE) 40. Each switch element 40 accepts eight 4-bit-wide input lines and has eight 4-bit-wide output lines. The switch fabric 20 thus has a total of thirty-two 4-bit-wide input lines shown at the left side of the switch fabric 20. In an operating switch, each of these thirty-two four-bit-wide input lines is connected from a separate routing table circuit, and each such routing table circuit is connected to a workstation or other digital device. The switch fabric has thirty-two four-bit-wide output lines shown at the right side of the figure. Each of the output lines is connected to individual further routing table circuits, each of which is connected to a workstation. Thus the switch fabric shown in Fig. 2 may provide a physical connection to up to thirty-two workstations and may connect data from any one of those thirty-two workstations to any other one of those or other thirty-two workstations.
It will be seen that the interconnections among the switch elements 40 are such that data entering any one of the switch fabric input lines may be routed to any one of the switch fabric output lines after passing through a total of four stages of switch elements. The switch fabric architecture 20 as shown in Fig. 2 is known as a Reversed Delta Network architecture. The switch element circuits of the present invention may be used in any number of other known network wiring configurations such as the Clos network or the Delta network, and the switch array may be expanded to provide any number of input lines and any number of output lines. (Not shown in Fig. 2 is a configuration bus connected to each one of the switch elements. The configuration bus is used by a configuration processor of the switch system to set up a number of switch parameters in a memory in the switch element.) Fig. 3 shows a portion of a switch fabric made up of four switch elements 40. One of the switch elements is shown with interconnections through eight routing tables 30, to a number of workstations 50, and to a server computer 52. As shown in the figure, in a typical application for the present invention, each input to a switch fabric is connected to a routing table circuit 30. A routing table circuit 30 is typically connected to some type of digital workstation 50, which may transmit and receive voice, video, and digital data via the switch fabric.
Also shown in Fig. 3 is an aggregate input connection to a switch element in accordance with one specific embodiment of the invention. In an aggregate connection, four of the input lines of the switch fabric are grouped together and act as one input to receive and transmit data to a high-speed data device such as server computer 52. With the aggregate input feature according to the invention, the same switch element and switch fabric can handle two speeds of packet data, the first speed being the speed of one input line and the second speed being four times faster or the speed of the aggregated input lines.
ROUTING TABLE
Fig. 4 is a block diagram of a routing table circuit 42 according to the present invention. The routing table circuit 42 is a combination storage and control device that is used with external memory, e.g., SRAM 90, and includes a receive queue controller 80 which sends data to the switch fabric and receives a back-pressure signal from the switch fabric, and a transmission buffer controller 82 which receives data from the switch fabric after that data has been processed by the multicast header translation circuit 84 and asserts back pressure to the switch fabric. The transmission buffer controller 82 also includes a small buffer memory 86 for storing cells received from the switch fabric. A further controller, called a connection table controller 88, is for reading header information from the workstation interface and is operative to use that header information to add an appropriate switch tag to the cells before they are transmitted to the switch fabric. Controller 88 stores information about switch tags and buffers data in external SRAM 90. Further included are an interrupt processor 92 and processor interface 94, which are for sending control signals to the workstation. Optionally included is an OAM\BECN cell transmit circuit 96 for inserting control cells to the outgoing data stream.
The routing table circuit 42 in each instance operates by receiving one 8-bit-wide segment of data via connection from a workstation as input and provides one 8-bit-wide workstation output. The routing table includes one 4-bit output to the switch fabric and receives one 4-bit input from the switch fabric.
SWITCH ELEMENT
Fig. 5 is a block diagram of the structure of switch element circuit 40 according to the present invention. The switch element circuit 40 includes a very small cell buffer pool memory 100 for storing and queuing cells being transmitted through the switch element, input I/O crosspoint block 110 for connecting any input line to any cell memory in the cell buffer pool, output I/O crosspoint block 120 for connecting any output line to any cell memory in the cell buffer pool, input bus controller 130 for controlling data flow on the input bus to the cell memories, output bus controller 140 for controlling data flow from the cell memories to the output lines, and multipriority buffer pool controller (MPBPC) 150 for controlling assignment of cell memories to connections defined by the cross-point blocks.
The switch element circuit 40 is connected to a configuration bus 41 which supplies configuration data to controller 150. The switch element 40 has eight input interfaces, labeled 10 through 17, and eight output interfaces, labeled OO through 07. Each of the eight inputs and eight outputs is a four-bit or nibble-wide interface capable of operating at for example up to 50 Mhz, i.e., sufficient to support digital communications at the current ATM OC-3 standard. Each of the inputs receives cells from another switch element in the switch fabric or from a routing table, as previously outlined. ATM cells of data are transferred as one hundred and eighteen four-bit nibbles. This allows the standard fifty-three ATM byte cells to be transferred along with six overhead bytes. A cell start signal goes high every one hundred and eighteen clock cycles to indicate the start of a cell.
Cell buffer pool 100 is a pool of random access memory. The pool contains thirty-two individual cell memories, each capable of storing an entire cell of one hundred and eighteen nibbles. The thirty-two memories can be connected to any one of the eight inputs by input crosspoint block 110 which is controlled by input bus controller 130. crosspoint block 110 contains a plurality of multiplexers 112 for connecting input buses to any of the cell memories. Multiplexers 112 are controlled by signals from the input bus controller 130 that are transmitted on six-bit wide connection control bus lines 132. Any of the cell memories may be connected to any of the output lines via output crosspoint block 120. Output crosspoint block 120 is controlled by the output bus controller 140 via output connection control bus lines 142. MPBPC 150 contains a link list RAM 152 for storing queue assignment information about the cell buffer pool memories, a service order table 154 for controlling the service order of the proportional bandwidth queues, a memory for multicast group bits 156 for storing information about multicast cell transmission, and a back-pressure control circuit 158 for asserting multipriority back-pressure on the eight back-pressure lines of the switch element.
LINKED LISTS
Referring to Fig. 5, MPBPC 150 uses its linked list RAM 152 to maintain five First-In/First-Out (FIFO) queues by means of lists of pointers to the next entry in the cell memory for each of the five output lines for a total of 40 possible virtual queues. Fig. 6 is a representation of the linked-list RAM 152 and associated head register set 153 and tail register set 155 for the forty queues defined for the 32 cell memories accounted for in the list RAM 152. For each of the forty queues, a buffer pointer is constructed from the head address of the queue and tail address of the queue for one of the forty queues stored in the head register set 153 and the tail register set 155. A head pointer and a tail pointer is kept for each one of the forty queues. The forty queues share a linked list of up to thirty-two entries. Each entry can identify one of the thirty-two cell memories in cell buffer pool 100. The linked-list thereby specifies which cell memories are part of each queue in FIFO order. Cells are enqueued onto the tail of the proper queue and dequeued from the head of the proper queue in accordance with a queue service procedure which generates the Q-dequeue pointer value to the head register set 155 and the Iqueue pointer to the tail register set 155. A mux 157 switches between register sets, depending on whether the procedure calls for enqueuing or dequeuing a cell. An input buffer pointer specifies where the input cell is currently stored, and the output pointer designates where the cell is to be directed.
The queues for one output line are assigned five different priorities. Three of the queues are proportional bandwidth queues of equal priority but having an assigned bandwidth of 5/8, 2/8 (1/4), or 1/8. Of the remaining two queues, one is designated a high-priority queue which may be used for very time dependent data such as voice, and the other a multicast queue which is used for data being sent from one transmitting workstation to more than one receiving workstation which might be the case in video conferences.
It will be seen that while there are forty possible virtual queues definable by MPBPC 150, only up to thirty-two queues may be active at any one time because there are only thirty-two available cell buffer pool memories. In practice fewer than thirty-two queues may be active at any one time, because it is likely that there will always be some queues which are using more than one cell memory.
Multipriority buffer pool controller (MPBPC) 150 controls the overall function of the switch element 40 as follows. During each cell cycle, cells having a length of 118 nibbles may be received on any or all of the eight input lines. Prior to the start of a cell cycle, the controller 150 has specified which input line is connected to which cell memory via input crosspoint block 110 by setting bits in input controller 130. The first twelve of the 118 nibbles from each input line are read by input bus controller 130 and transmitted to the multipriority buffer pool controller 150 while the ATM cell is being stored in its designated cell memory. From these tags, MPBPC 150 determines the priority and destination of the cell that has just been stored in each of the eight cell memories connected to one of the input interfaces. The MPBPC 150 then adds the cell memories to their appropriate queues by updating its linked lists. The MPBPC 150 then determines which of the output interfaces to which cells are directed can receive the cells during the next clock cycle. An output interface may be unavailable to receive all of the data which is directed towards it during a cycle when more than one input lines are directing cells to a single output interface, or when the output interface has asserted back-pressure to the MPBPC 150. The MPBPC 150 handles the problem of output interfaces being unavailable to receive cells by establishing queues for the connections for which output lines are temporarily unavailable in the cell buffer pool 100. Cells may be stored in these queues in a first-in-first-out FIFO fashion for some number of cell cycles until the output interfaces are available for outputting the cells. Once the MPBPC 150 has made determinations regarding which cells can be transmitted to their destination output interfaces during the next clock cycle and which cells will be stored in queues in the cell buffer pool 100, it directs the output interfaces to receive data from cell memories in the cell buffer pool by sending control signals to output bus controller 140. It also directs the input interfaces to available cell memories by sending input control signals to input bus controller 130.
BACK-PRESSURE CONTROL
One problem that may arise in a switch element 40 as packets are being routed through cell memories from input lines to output lines is the unavailability of cell memories for queuing prior to a clock cycle when new cells may be received on the input interfaces. If a cell is received at an input interface to the switch element when no cell memory is available to receive the cell, the cell must be dropped and the data resent.
In accordance with one aspect of the invention, each switching element 150 avoids dropping cells by issuing back¬ pressure signals to each connection to each of its input interfaces on a per-input, per-priority basis to halt the flow of cells having a given priority to a given input. Back¬ pressure is asserted for a given input and given priority whenever the number of currently enqueued cells of the given priority supplied by the given input exceeds a predetermined threshold. Back-pressure for a given priority is also asserted for all inputs whenever the total number of available cell memories falls below a threshold associated with that priority.
By employing a shared buffer pool, the switching element of the present invention virtually eliminates the deletion of cells due to exhaustion of available memory. In many ATM applications, even infrequent cell drops are harmful in that the loss of one cell necessitates the retransmission of many cells, substantially reducing network efficiency. Furthermore, in the event of excessive cell traffic through a switching fabric, it is preferable that cell drops occur at a routing table rather than at a switching element, since routing tables employ sophisticated congestion management strategies unavailable at a switching element when dropping cells. (One such sophisticated congestion management strategy is the standard ATM Adaption Layer 5 (AAL5) early frame discard technique, AAL5 being a technique for segmenting frames into cells.)
Fig. 7 is a simplified representation of the elements within back-pressure controller 150 used to implement the back-pressure capability. Back-pressure controller 150 includes a time domain multiplexer 402, a state machine 404, a time domain demultiplexer 406, a queue service controller 408, an index memory 410, a variable delay circuit 412, and a variable delay register 414. Back-pressure signals are generated by state machine 404 based on criteria as discussed below. Back-pressure signals from other switching elements or a routing table are received by a queue service controller 408 which selects cells for output.
Back-pressure is asserted for a given input and given priority whenever the number of currently-enqueued cells of the given priority supplied by the given input exceeds a predetermined threshold. A problem is posed in that queues, with the exception of the multicast queue, are organized by output rather than input. To maintain a count of enqueued cells for each input and priority, index memory 410 is maintained within the back-pressure controller 158 with an entry for each cell memory location which identifies the source of the cell stored there. When a new cell is enqueued, index memory 410 is updated and a counter, internal to state machine 404 and associated with the source and priority of the cell, is incremented. When a cell is dequeued for output, the index entry for that cell is read to identify the source for that cell and the appropriate counter is then decremented. To determine the necessity of back-pressure for a given input and priority, the counter for that input and priority is compared to a predetermined threshold.
The predetermined threshold is the same for each input and priority. Thus, back-pressure is allocated among inputs so that no one input blocks incoming traffic from other inputs by occupying a disproportionate share of the cell memory locations. When inputs are aggregated, a counter is maintained for each priority for the aggregated inputs as a group rather than for each input.
To assure that availability of cell memories is also properly allocated among priorities, a count of empty cell memories is maintained within state machine 404 and compared to thresholds stored for each priority. When the number of empty cell memories falls below the threshold associated with a given priority, back-pressure is asserted for that priority for every input. The higher priorities have lower thresholds set so that high priority traffic is impeded last as the count of available cell memories decreases. In addition, the thresholds are normally set so that high-priority traffic has strict priority over lower priority traffic. In one embodiment of the present invention, back¬ pressure signals for the various priorities are time-domain multiplexed together by time-domain multiplexer 402 so that each input is provided with a single back-pressure signal. Received back-pressure signals are demultiplexed by time domain demultiplexer 406. Each priority then corresponds to a different time slot within the time-domain multiplexed back¬ pressure signal.
A switching element and an associated input device (switching element or routing table) may or may not be on the same printed circuit board. A problem arises in that if the devices are on the same printed circuit board, no delay is required on the interconnecting data or back-pressure lines while if the devices are on separate printed circuit boards, the interconnecting lines may be retimed with D flip-flops. Fig. 8A and Fig. 8B are a simplified representation of these two situations. A set of retiming buffers 51-54 compensate for inter-card delays. To compensate for the resulting delays, a switching element 40 according to one embodiment of the present invention is provided with internal means for establishing a variable delay in the back-pressure line. Fig. 7 shows variable delay circuit 412 inserted in one of the back-pressure lines. The variable delay is selected by writing to variable delay register 414 within the switching element.
AGGREGATE BITS
Referring to Fig. 9, a switch element 40 includes two aggregate input bits, agg_in(0) 151 and agg_in(l) 153 and two aggregate output bits, agg_out(0) 155 and agg_out(l) 157, which may be set by the configuration controller (not shown) to allow for aggregating either the group of inputs 10 to 13, the group of inputs 14 to 17, the group of outputs 00 to 03, or the group of outputs 04 to 07. Referring back to Fig. 3, some types of digital devices, particularly other switching systems, such as server computer 52, may need to transmit data through the switch fabric at a higher rate than may be provided by one switch element input, e.g., at the 622 Mbps rate provided by a conventional 0C-12 ATM interface. Fig. 9 shows the groupings of inputs and outputs of a switch element 40 when the signals agg_in(0) and agg_in(l) and agg_out(0) and agg_out(l) are set. Switch element 40 has two bits for input lines and two bits for output lines that signal the switch element when a set of four of its input lines is being configured as a single input which receives four cells at once and retains FIFO order. Input bits agg_in(0) and agg_in(l) are set in the multipriority buffer pool controller 150 via the configuration bus . When bit agg_in(0) is set true, inputs 0 through 3 are treated as if they are a single stream of cells. This makes the input capable of handling data at an effective rate four times higher than possible on one input line, which in one specific embodiment is 622 Mbps. With this feature, a switch element 40 can support a mixture of data rates on its inputs. A key problem that must be addressed when aggregating input lines is maintaining FIFO order between the cells that arrive simultaneously. When the inputs are not aggregated, the cells from each input are enqueued separately. When the inputs are aggregated, then the cells are enqueued as if they were from a single input with the cell received on input 0 placed in the single aggregated FIFO queue first, the cell received on input 1 placed in the single FIFO queue second, and so on. In the absence of the aggregate bit, FIFO order could be violated as the MPBPC 150 uses a round-robin procedure to enqueue multicast cells to improve fairness. This procedure intentionally chooses cells from the inputs in differing orders from cell time to cell time. A second problem that the invention addresses is that cells bound for an aggregated output can go out any one of the outputs in the aggregated output, depending on availability. When the aggregate bit is set, cells bound for the aggregated output are dequeued for any one of its outputs. The MPBPC 150 also uses the aggregate bit to determine how to assert back¬ pressure to an output from the previous stage in an aggregated input. Back-pressure is given if cells from a given input are queued excessively.
When inputs are aggregated to boost effective speed from 155 Mbps to 622 Mbps, the MPBPC 150 measures the counts of the cells from any of the inputs in the aggregated input, rather than from the individual inputs. The back-pressure is then applied to all of the inputs in the aggregated input, rather than the individual inputs. More specifically, to aggregate, in the first level of the switch fabric (Fig. 2) , the agg_in value is set for all inputs actually connected to a high speed input. Agg_out is set for all possible destinations of the inputs that agg_in is set for. In subsequent levels, agg_in is set for those links which have agg_out set in the previous level. Thus agg_out is set for all possible destinations of an input in which agg_in has been set. In the last level, agg_in is set for those links which have agg_out set in the previous level. Agg_out is set for those links which are actually connected to a 622 Mbps output port.
PROPORTIONAL BANDWIDTH QUEUES
Switch elements 40 and the routing table circuits 30 can also support proportional bandwidth queues. Proportional bandwidth queues solve a problem that arises when data traffic from sources of drastically different bandwidths coexist on an intermediate link. In one specific embodiment of the invention, the switch element 40 and routing table circuit 30 can support for example three queues that are of equal priority but which have bandwidths of 1/8, 1/4 and 5/8 of the available bandwidth. MPBPC 150 maintains a service order table 154 (Fig. 10) which enhances fairness among connections having different bandwidths. The assigned proportions can be adjusted by externally altering the contents of the service order table 154.
The proportional bandwidth queues are implemented by the MPBPC 150 by having a service order table 154 for the dequeuing process that specifies the order of queue service for each of the output queues. The schedule in each stage is delayed by one cell period which tends to minimize queuing and thus the cell memories required by trying to dequeue a cell from a given queue just after a cell from that queues is likely to have arrived. MPBPC 150 must make sure that all of the possible competitions between differing bandwidth queues turn out as predicted. For example, if cells only in the l/8th and l/4th queues arrive, then the l/8th queue should get l/3rd of the available bandwidth on the output channel, and the l/4th queue should get 2/3rd of the bandwidth. Similar results should apply for all of the possible x-way competitions. These issues have been addressed by a careful design of the service order table 154 stored within MPBPC 150. This table 154 provides each of the participants in the possible competitions with approximately the proper bandwidth while reducing overhead processing that must be done by the MPBPC 150 to determine dequeuing order. Additionally, the MPBPC 150 can update the service order table 154 on the fly so that moment-by-moment adjustments may be made by certain types of switching systems that will use these devices to enhance fairness in competition.
As a further detail, Fig. 10 shows a service order table 154 stored in MPBPC 150 for determining dequeuing from cell buffer pool 100 when more than one proportional bandwidth queue is queued for a given output. The MPBPC 150 defines a service order interval of eight cell transmission cycles for determining the priority of serving the proportional bandwidth queues. These cycles are represented by the eight columns labelled 0 to 7 in Fig. 10. During any given cycle, MPBPC 150 examines which queues for a given output wish to transmit data to that output. It will be seen that during any given cycle there is a queue service order listing the priority with which bandwidth queues will be serviced. During any cycle, only one queue is serviced and that queue is the queue having the highest priority, 1st through 3rd, as listed in the service order table during that cycle. For example, during cell cycle 4, the priority list is 3, 4 and 2. Should cells from two proportional bandwidth queues both be ready to transmit during cycle 4, the cell from the queue having bandwidth of the higher priority will be transmitted. During the next clock cycle, cycle 5, if both of those queues wish to transmit, the next bandwidth queue will be transmitted because the queue service order table shows that it has a higher priority during that cell cycle.
MULTICAST
One data transmission application which the present invention may be employed to support is multicast transmission. During multicast transmission, data from one source is distributed to several destinations, i.e., the multicast group, which comprise some, but not necessarily all, of the possible switch fabric outputs. An example of such an application is a video conference over a network in which several workstations are connected by a switch fabric, and voice and image data are transmitted from each workstation to each of the other workstations.
Generally, multicast may be supported in a variety of ways. Source cell duplication is a simple, but brute-force solution to the multicast support problem. Fig. 11 is a simplified representation of the source cell duplication solution 60 known in the prior art. With source cell duplication, the source 62 of the data cells creates copies of each cell for transmission to each destination. This solution suffers from a number of significant disadvantages. Not only does the task of duplication place a severe load on the source 60, it also places limits on the number of destinations 64, 66, 68 in the multicast group connected by a switch element 40. As a result, the size of the multicast group for a network which supports multicast may be drastically limited (e.g., to half the size of connection for the network). Additionally, expensive bandwidth is wasted. For example, in cases where more than one destination is at the same distant location, redundant copies of the information are transmitted over the entire distance, thereby unnecessarily contributing to system traffic.
Mid-switch cell duplication 70 is an alternate multicast support solution. A simplified representation of a mid-switch duplication 70 solution is shown in Fig. 12.
According to a typical mid-switch duplication 70 solution, a module 72 is provided at some point in the switching system which duplicates the transmitted cells from a source 74 as necessary for distribution to destinations 64, 66, 68 in the multicast group. Although this solution does not suffer from all of the disadvantages of the source cell duplication solution 60, bandwidth is still unnecessarily consumed by the transmission of the cell duplicates through the remainder of the system. The optimal solution for supporting multicast, made practicable by the present invention, is referred to as tree- based cell duplication. A simplified representation of a tree-based cell duplication system 76 according to the invention is provided in Fig. 13. With a tree-based cell duplication system 76, the transmitted cells are not duplicated until the last points of divergence 77, 79 to the destinations 64, 66, 68, e.g., by means of cell replication within the switch element 40, represented herein as a serial redirector 78. This avoids the unnecessary consumption of bandwidth encountered with the previously described solutions. One possible complication of this solution, however, is that all destinations 64, 66, 68 of a multicast cell may not be reachable with a cell having the same address. The present invention implements a tree-based cell duplication system 76 using a specific embodiment of the switch element 40 described above. The solution is described with reference to Fig. 14. An eight-bit field in the routing tag of each transmitted cell determines what is called the multicast group for that cell. As described above, the routing tag, a twelve-nibble field placed on the front of a cell by the routing table circuit based on the content of a multicast group bit register 81, dictates the path of the cell through the switch fabric. The multicast group consists of the group of network destinations to which the cell is to be transmitted. For each switch element, the multicast group field determines which switch element outputs upon which a received cell is to be placed in order to get the information to the desired destinations. The switch element of the present invention stores an array of multicast group bits in its RAM, the array including one eight-bit word for each of the multicast groups. Each bit in each word represents one switch element output. When the multicast queue of the switch element is selected and a data cell placed therein (as determined by a one nibble field in the routing tag) , the multicast group field in the cell is used as an index into the multicast group bits array, pointing to a particular word in the array. Any bits which are set in the selected word correspond to the switch element outputs on which the cell in the multicast queue is to be placed.
MULTICAST COMPLETION
One difficulty encountered in multicast transmissions is that it is often not possible to place a given cell on all of the desired outputs simultaneously. This is referred to as the problem of multicast completion. Such a situation might arise, for example, if a cell from a higher priority queue has already been placed on the selected output. This situation can also occur if a subsequent switch element has exerted back-pressure on the selected output, thereby preventing the transmission of cells from that output. Some open loop switching systems simply allow cells to be dropped if congestion causes cell buffers to overflow. If this occurs with the transmission of video information, for example, the penalty incurred from such a drop is relatively high. An entire video frame might be lost due to the loss of just one cell. Other penalties are incurred if the system protocol requires the retransmission of the entire frame or a series of frames.
The switch element 40 of the present invention solves this problem by keeping a record of the switch element outputs upon which the cell in the multicast queue has been successfully placed. Referring to Fig. 15, multicast queue controller 156 of switch element 40 sets bits in a multicast queue completion register 83 for each output on which the cell is actually placed. The ATM cell in the multicast queue is dequeued only when the bits in the completion register 83 match the bits in the word selected from the multicast group bits array stored in the multicast group bits register 81. As indicated by the multicast group bits word, cell M 85 in the multicast queue is to be placed on outputs (2) , (3) , and (5) 89, 91, 93. However, cell H 87 in the higher priority queue has already been placed on output (3) 93, thereby preventing immediate placement of cell M 85 on that output 93. This is reflected by the fact that bit number 3 in the completion register 83 (corresponding to output (3) 93) has not yet been set. When cell M 85 is eventually placed on output (3) 93, this bit 3 is set. The word in the completion register 83 then matches the word from the multicast group bits array 81, allowing cell M 85 to be dequeued.
PER PRIORITY QUEUING WITH PER CONNECTION COUNTS As described above, the routing table circuit 30 of the present invention receives a cell from a source, looks up the intended address in its RAM, adds the appropriate routing tag to the cell, and then puts the cell out onto the switch fabric via the switching elements. The routing table circuit 30 also performs a queuing function in which it stores queued cells in an accompanying SRAM before placing them on the switch fabric. The routing table circuit 30 of each source queues the cells on a per priority basis, but also keeps track of how many cells from each connection are in the queue at any given time. Unlike a strict per connection queuing discipline, a transmission scheduler is not required.
Fig. 16 is a table which illustrates the manner in which the routing table queues incoming cells. In the illustrated example, cells having priorities 0, 2, and 5 have been queued by the routing table. - Within each of the priorities, cells from different connections have been queued. The count of queued cells per connection is maintained as shown in Fig. 17, showing the number of queued cells for each VPC. The routing table uses the connection count to perform closed loop functions such as sending a back-pressure signal to a particular data cell source. Thus, with the present invention, the simplicity of per priority queuing is enjoyed, while at the same time per connection queue depths are kept so that congestion management techniques can be employed.
MARKED INTERRUPT LINKED LIST
It is one of the functions of the routing table to alert the external processor that a virtual channel is experiencing congestion. One method for doing this is to generate an interrupt signal each time a congestion condition is encountered. However, it is not desirable to generate an interrupt every time a data cell is queued for a congested channel, especially if the processor has already been notified of the congestion on that channel. Also, more than one channel may experience congestion before the processor is able to respond to a congestion interrupt. It is therefore necessary to keep track of channels experiencing congestion so that the processor may take appropriate action for all such channels when it is ready to do so. One method for keeping track of congested channels includes assigning a bit for each of the channels, and setting the bits corresponding to channels which experience congestion. The processor then checks the bits for all of the channels to determine which channels are congested. However, because of the number of channels made possible by the present invention, such a solution is undesirably slow, consuming valuable processor time. Therefore, according to a specific embodiment of the invention, the routing table maintains a linked list of currently congested channels. Each channel in the list is also marked (i.e., a "congestion" bit is set) so that the queuing of further data cells for those channels does not generate additional interrupts. When a data cell is queued for a particular channel, the current queue depth for that channel is compared with the configured congested queue depth. If the current queue depth is longer and the channel is not marked, the routing table generates an interrupt and the channel is marked and added to the end of the linked list. If the channel is already marked, nothing happens.
When the processor is able to respond to an interrupt, it first looks at the interrupt head pointer which points to an address which represents the first virtual channel in the linked list. The processor then reads from that channel the address for the next channel experiencing congestion. The processor continues to read the addresses for the channels experiencing congestion until it reaches the end of the linked list. The processor then takes appropriate action for each of the channels in the list to resolve the congestion. Such action might include, for example, sending an appropriate feedback message, or changing the queue depth which causes interrupts. A congestion-relieved threshold is determined by multiplying the congestion threshold by a fractional constant (e.g., 0.75). And whenever a cell is dequeued, and the current queue depth falls below the congestion-relieved threshold, a second interrupt is generated, and the congestion is cleared.
A global "enable" for interrupts allows the system processor to read the linked list of congested channels atomically. If a channel becomes congested while the interrupts are disabled, once the interrupts are re-enabled, the next cell queued for that channel will cause an interrupt if the channel is still congested.
The invention has now been explained with reference to specific embodiments. Other embodiments will be apparent to those of ordinary skill in the art upon review of this description. It is therefore not intended that the invention be limited, except as indicated by the appended claims.

Claims

CLAIMS :
1. An asynchronous transfer mode (ATM) switching system using a switch fabric architecture comprising: a backplane having a plurality of distinguishable locations for supporting interface functions, switching functions and control functions from said distinguishable locations; and a plurality of ATM switch modules for connection to the backplane at said switching function locations, each one of said ATM switch modules including a closed loop switch element.
2. A switch element comprising: a plurality of input interfaces for receiving cells of data; a plurality of output interfaces for transmitting cells of data; a shared pool memory comprised of a plurality of cell memories; an input crosspoint circuit capable of connecting any of said plurality of input interfaces to any one of said plurality of cell memories; an output crosspoint circuit for connecting any one of said plurality of cell memories to any one of said output interfaces and; controller means for controlling which of said inputs is connected to which of said cell memories and for controlling which of said outputs is connected to which of said cell memories.
3. The device according to claim 2 wherein said controller means further comprises: a linked-list random access memory; a set of multipriority queues for each of said output interfaces each one of said queues in said set residing in said random access memory and comprising: a connection address; a queue priority identifier; and a first-in/first-out linked list of identifiers to said cell memories in said shared pool memory.
4. The device according to claim 2 wherein there are eight input lines each capable of receiving four-bit wide nibbles and eight output lines each capable of transmitting four-bit wide nibbles and wherein the shared pool memory is comprised of 32 cell memories.
5. The device according to claim 4 wherein there are five multipriority queues for each of said eight output lines making a total of 40 virtual queues.
6. The device according to claim 3 wherein each of said active queues represents one connection through a switch fabric, said connection defined as connecting one data transmitter to one data receiver at a specified priority over a determined and fixed path in said switch fabric.
7. In a switch element comprising a plurality of input interfaces, a plurality of output interfaces, a shared pool memory having a plurality of cell memories, a controller, and means responsive to the controller for connecting any of said input lines and any of said output interfaces to any one of said cell memories, a method for switching a cell of data from any input interface to any output interface comprising the steps of: receiving the cell of data over one input interface and storing the cell in a cell memory designated by said controller for that input interface; using the controller to examine a tag portion of said cell, said tag portion identifying the output interface to which the cell must be directed; using the controller to enqueue the cell by adding an identifier for the cell memory in which the cell has been stored to a linked list; using the controller to dequeue the cell when its desired output is available by connecting the cell memory in which the cell is stored to the cell's desired output interface; and transmitting the cell over its desired output interface.
8. The method according to claim 7 further capable of sending to one output interface cells of data having different priorities further comprising the steps of: using the controller to examine said tag portion of said cell, said tag portion additionally identifying the priority at which the cell is being transmitted; using the controller to enqueue the cell at its appropriate priority by adding an identifier for the cell memory in which the cell has been stored to a linked list having a specified priority; and using the controller to dequeue the cell when its desired output interface is available to receive a cell of that cell's priority by connecting the cell memory in which the cell is stored to the cell's desired output interface.
9. The device according to claim 8 wherein said multipriority queues further comprise a means for designating a bandwidth assignment associated with each of a subset of said plurality of queues.
10. The device according to claim 9 wherein said controller further comprises a queue service order table for determining, during a cell cycle, the service order for said plurality of bandwidth queues.
11. In a switch element having a plurality of queues for storing cells of data being routed from an input interface to an output interface and having a controller, a method for determining which of the queues will be connected to the output interface during a given cell cycle in accordance with a proportional bandwidth scheme comprising the steps of: designating a specified number of cell cycles as a service order interval; constructing a service order table that, for each of the cell cycles within the service order interval, lists the queues in accordance with the service order priority they will have during that cell cycle; using the controller to determine which of the plurality of queues contain a cell ready to be transmitted to the output interface during a given cell cycle; and using the controller to decide, by reference to the service order table, which of the queues needing service during that cell cycle will be connected to the output interface.
12. The method according to claim 11 wherein the queues are assigned a proportional bandwidth that is a fraction consisting of a numerator and a denominator common to each queue and wherein the service order interval has a number of cell cycles equal to said denominator.
13. The method according to claim 11 wherein there are three queues assigned proportional bandwidths 1/8, 2/8 and 5/8 and there are seven cell cycles in each service order interval.
14. The method according to claim 7 further capable of sending to one output interface cells of data having different priorities and cells having different assigned proportional bandwidths at the same priority further comprising the steps of: using the controller to examine said tag portion of said cell, said tag portion additionally identifying the assigned proportional bandwidth at which the cell is being transmitted; using the controller to enqueue the cell at its appropriate assigned proportional bandwidth by adding an identifier for the cell memory in which the cell has been stored to a linked list defining a queue having a specified proportional bandwidth; using the controller to determine by reference to a service order table the service order for the proportional bandwidth queues; and using the controller to dequeue the cell when its desired output is available to receive a cell of that cells priority and when that cell is in the highest service order during a cell cycle according to the service order table by connecting the cell memory in which the cell is stored to the cell's desired output interface.
15. The switch element of claim 2 further comprising: a plurality of aggregate input control bits; a plurality of aggregate output control bits; means within said controller responsive to an active state of one of said aggregate bits and capable of causing said controller to treat cells received on a subset of said plurality of input interfaces as though the cells were received on one input interface.
16. The switch element of claim 3 further comprising: a plurality of aggregate input control bits; a plurality of aggregate output control bits; means within said controller responsive to an active state of one of said aggregate bits and capable of causing said controller to treat cells received on a subset of said plurality of input interfaces as though the cells were received on one input interface by establishing in said linked-list random access memory a set of aggregated multipriority queues for receiving cells from all of the subset of input interfaces.
17. The switch element of claim 16 wherein said control means is operative to respond to one of said aggregate input signals by preserving FIFO order on cells received over said subset of said input interfaces.
18. In a switch element comprising a plurality of input interfaces, a plurality of output interfaces, a shared pool memory comprised of a plurality of cell memories, a controller, aggregate bits, and means responsive to the controller for connecting any of said input interfaces and any of said output interfaces to any one of said cell memories, a method for switching cells of data from a designated aggregate subset of input interfaces to any available interfaces of a designated aggregate subset of output interfaces comprising the steps of: receiving cells of data over said aggregate subset of input interfaces and storing the cells in cell memories designated by said controller for each of said subset of input interfaces; using the controller to examine a tag portion of said cells, said tag portion identifying the output interface to which the cell must be directed; using the controller to enqueue the cells from the aggregate input interfaces by adding in FIFO order an identifier for each of the cell memories in which aggregate input interface cells have been stored to a single linked list for the aggregate output; using the controller to dequeue the cells when any of the desired aggregate output interfaces are available by connecting the cell memories in which the cells are stored to the available output interfaces; and transmitting the cells over the desired aggregated output interfaces.
19. The method according to claim 18 further capable of receiving, via aggregated input interfaces, cells of data having different priorities, said method further comprising the steps of: using the controller to examine said tag portions of said cella, said tag portions additionally identifying the priority at which the aggregated cells are being transmitted; using the controller to enqueue the cells at their appropriate priority by adding an the identifies for the cell memories in which the cells have been stored to a aggregate linked list having a specified priority; and using the controller to dequeue the cells when any of the desired aggregate output interfaces are available to receive cells at the aggregated queue's priority by connecting the cell memories in which the cells are stored to the available aggregate output interfaces.
20. An ATM switching element comprising: a data input coupled to an ATM cell output of a source; a back-pressure output coupled to said source for issuing a back-pressure signal to block transmission of cells to said data input; and control means coupled to said back-pressure output for asserting back-pressure responsive to congestion within said switching element.
21. The ATM switching element of claim 20 wherein said control means comprise a state machine.
22. The ATM switching element of claim 20 further comprising: variable delay means for delaying said back-pressure signal responsive to a delay value stored in a register within the ATM switching element; and input port means for externally adjusting said delay value.
23. An ATM switching element comprising: a cell output for transmitting cells having one of a plurality of priorities; a first back-pressure input for receiving an indication of back-pressure necessity for cells having a first priority; means, coupled to said cell output and said first bac -pressure input, for blocking transmission of cells having said first priority responsive to said indication of back- pressure necessity received on said first back-pressure input; a second back-pressure input for receiving an indication of back-pressure necessity for cells having a second priority; and means, coupled to said cell output and said second back-pressure input, for blocking transmission of cells having said second priority responsive to said indication of back- pressure necessity received on said second back-pressure input.
24. The ATM switching element of claim 23 wherein said first back-pressure input and said second back-pressure input are derived from a single time-domain multiplexed back- pressure input.
25. In an ATM switching element employing a cell memory pool, a method for providing back-pressure to a plurality of inputs and a plurality of priorities, comprising the steps of: receiving a cell having one of said plurality of priority levels from one of said plurality of inputs; thereafter enqueuing said cell by recording said cell in a first memory location in the cell memory pool; using an address of said first memory location to obtain an address of a second memory location within an index memory; recording a reference specifying said one plurality level and said one input at said second memory location; incrementing a cell count indicator associated with said one priority level and said one input; thereafter comparing said cell count indicator to a predetermined threshold to obtain an indication of back-pressure necessity for said one priority level and said one input; and thereafter upon a positive indication of back-pressure necessity, asserting a back-pressure signal associated with said one input and said one priority.
26. The method of claim 25 further comprising the steps of: dequeuing the cell by transferring the cell from said first memory location to an output of the switching element; using said address of said first memory location to obtain said address of said second memory location; thereafter retrieving said reference specifying said one plurality level and said one input from said second memory location; using said reference to identify said cell count indicator associated with said one input; and thereafter decrementing said cell count indicator associated with said one priority level and said one input.
27. In an ATM switching element employing a cell memory pool, a method for providing back-pressure to a plurality of priorities, comprising the steps of: receiving a cell via an input of the switching element; thereafter enqueuing said cell by recording said cell in a first memory location in the cell memory pool; decrementing an empty cell indicator, said empty cell counter being an enumeration of empty cells within said cell memory pool; comparing said empty cell indicator to a threshold associated with a first priority to obtain an indication of back-pressure necessity for said first priority; upon a positive indication of back-pressure necessity for said first priority, asserting a back-pressure signal associated with said first priority; comparing said empty cell indicator to a threshold associated with a second priority to obtain an indication of back-pressure necessity for said second priority; and upon a positive indication of back-pressure necessity for said second priority, asserting a back-pressure signal associated with said second priority.
28. The method of claim 27 further comprising the step of time-domain multiplexing said back-pressure signal associated with said first priority with said back-pressure signal associated with said second priority.
29. A switching device for transmitting a data cell, the data cell comprising a first data field, the switching device comprising: plurality of outputs; a queue for temporarily storing the data cell; and a memory for storing a plurality of data words, each data word comprising a plurality of first bits, each of said first bits corresponding to one of said outputs; wherein the data cell is placed on selected ones of the outputs, the selected ones of the outputs corresponding to first bits which have been set in a first data word selected from the plurality of data words, the first data word being selected in response to the first data field in the data cell.
30. The switching device of claim 29 wherein there are up to eight outputs and eight first bits in each data word.
31. The switching device of claim 29 wherein the memory comprises random access memory (RAM) .
32. The switching device of claim 29 wherein the data cell comprises up to 118 nibble wide segments.
33. The switching device of claim 29, further comprising a queue completion register, the queue completion register comprising a plurality of second bits, each of the second bits corresponding to one of the outputs, the second bits being set when the data cell is placed on the outputs corresponding to the second bits, the data cell remaining in the queue until the second bits set in the queue completion register match the first bits set in the first data word.
34. The switching device of claim 33 wherein there are up to eight outputs and eight second bits.
35. A method for transmitting a data cell to a plurality of destinations using a switch element having a plurality of outputs, the data cell having a first data field, the method comprising the steps of: storing the data cell in a queue; using the first data field to select a first data word from a memory, the first data word comprising first bits, each of the first bits corresponding to one of the switch element outputs; and placing the data cell on selected ones of the switch element outputs corresponding to first bits in the first data word which have been set.
36. The method of claim 35, further comprising the steps of: setting second bits in a queue completion register, each of the second bits set in the queue completion register corresponding to the outputs upon which the data cell has been placed; comparing the second bits in the queue completion register to the first bits in the first data word; and dequeuing the data cell when the second bits set in the queue completion register match the first bits set in the first data word.
37. A method for queuing incoming data cells in a switching system, the data cells being received from a plurality of connections, each of the data cells having an associated priority, the method comprising the steps of: queuing the incoming data cells with respect to each other in a queue according to the associated priority of each of the incoming data cells; and maintaining a count of the incoming data cells from each of the connections which remain in the queue.
38. The method of claim 37, further comprising the step of generating feedback signals to selected data cell sources when the count for connections corresponding to the selected data cell sources exceeds a first number.
39. A method for monitoring congestion on channels in a switching system, comprising the steps of: storing a first data cell in a queue, the first data cell comprising a routing tag field, the routing tag field corresponding to a first channel; comparing an actual data cell count for the first channel to a predetermined count; generating an interrupt signal if the actual data cell count is greater than the predetermined count and the first channel is not marked; marking the first channel by setting a bit if the first channel is not marked; and placing the first channel on a linked list of congested channels if the first channel is not already on the linked list.
40. The method of claim 39, further comprising the steps of: receiving the interrupt signal; reading the congested channels in the linked list; and taking action to resolve the congestion for the congested channels in the linked list.
41. The method of claim 40 wherein the step of taking action comprises sending a feedback message to selected sources of data cells.
42. The method of claim 40 wherein the step of taking action comprises changing the predetermined count.
PCT/US1995/004613 1994-04-28 1995-04-13 Atm architecture and switching element WO1995030294A1 (en)

Priority Applications (3)

Application Number Priority Date Filing Date Title
EP95915672A EP0761055A4 (en) 1994-04-28 1995-04-13 Atm architecture and switching element
JP7528264A JPH09512683A (en) 1994-04-28 1995-04-13 ATM architecture and switching elements
KR1019960706081A KR970703078A (en) 1994-04-28 1996-10-28 ATM architecture and switching devices

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US08/235,006 US5583861A (en) 1994-04-28 1994-04-28 ATM switching element and method having independently accessible cell memories
US08/235,006 1994-04-28

Publications (1)

Publication Number Publication Date
WO1995030294A1 true WO1995030294A1 (en) 1995-11-09

Family

ID=22883666

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US1995/004613 WO1995030294A1 (en) 1994-04-28 1995-04-13 Atm architecture and switching element

Country Status (7)

Country Link
US (3) US5583861A (en)
EP (1) EP0761055A4 (en)
JP (1) JPH09512683A (en)
KR (1) KR970703078A (en)
CA (1) CA2188882A1 (en)
TW (1) TW281840B (en)
WO (1) WO1995030294A1 (en)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP0785699A2 (en) * 1996-01-16 1997-07-23 AT&T Corp. Multicast routing in multistage networks
EP0785697A3 (en) * 1996-01-16 2000-05-31 AT&T Corp. Multistage network having multicast routing congestion feedback
EP0785698A3 (en) * 1996-01-16 2000-05-31 AT&T Corp. Buffering of multicast cells in switching networks
EP2477366A1 (en) * 2010-11-12 2012-07-18 Huawei Technologies Co., Ltd. Data transmission method, apparatus and system

Families Citing this family (159)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2882545B2 (en) * 1993-10-23 1999-04-12 インターナシヨナル・ビジネス・マシーンズ・コーポレーシヨン Selective congestion control mechanism for information networks
US5901140A (en) * 1993-10-23 1999-05-04 International Business Machines Corporation Selective congestion control mechanism for information networks
US6151301A (en) * 1995-05-11 2000-11-21 Pmc-Sierra, Inc. ATM architecture and switching element
AUPM699394A0 (en) * 1994-07-25 1994-08-18 Curtin University Of Technology Link level controlled access to available asynchronous network service
EP0700229B1 (en) * 1994-08-22 2006-06-28 Fujitsu Limited Connectionless communications system, test method, and intra-station control system
US5548587A (en) * 1994-09-12 1996-08-20 Efficient Networks, Inc. Asynchronous transfer mode adapter for desktop applications
US5528588A (en) * 1994-09-14 1996-06-18 Fore Systems, Inc. Multicast shared memory
EP0706297A1 (en) * 1994-10-07 1996-04-10 International Business Machines Corporation Method for operating traffic congestion control in a data communication network and system for implementing said method
JP3014080B2 (en) * 1994-12-28 2000-02-28 三菱電機株式会社 Exchange adapter and general-purpose computer
US5790539A (en) * 1995-01-26 1998-08-04 Chao; Hung-Hsiang Jonathan ASIC chip for implementing a scaleable multicast ATM switch
US6523060B1 (en) * 1995-04-07 2003-02-18 Cisco Technology, Inc. Method and apparatus for the management of queue pointers by multiple processors in a digital communications network
JPH08288965A (en) * 1995-04-18 1996-11-01 Hitachi Ltd Switching system
JP2856104B2 (en) * 1995-04-18 1999-02-10 日本電気株式会社 ATM switch
US5737314A (en) * 1995-06-16 1998-04-07 Hitachi, Ltd. ATM exchange, ATM multiplexer and network trunk apparatus
US5838915A (en) * 1995-06-21 1998-11-17 Cisco Technology, Inc. System for buffering data in the network having a linked list for each of said plurality of queues
FR2736737B1 (en) * 1995-07-12 1997-08-14 Alcatel Nv DEVICE FOR MANAGING RELATIONSHIPS BETWEEN OBJECTS
US7468977B1 (en) 1995-07-12 2008-12-23 Nortel Networks Limited LAN/ATM switch having local packet switching and an ATM core fabric
US5790770A (en) 1995-07-19 1998-08-04 Fujitsu Network Communications, Inc. Method and apparatus for reducing information loss in a communications network
WO1997004561A1 (en) * 1995-07-19 1997-02-06 Fujitsu Network Communications, Inc. Link scheduling
AU6501796A (en) * 1995-07-19 1997-02-18 Ascom Nexion Inc. Network switch utilizing centralized and partitioned memory for connection topology information storage
DE69637027T2 (en) * 1995-08-02 2007-08-23 Nippon Telegraph And Telephone Corp. CONTROL DEVICE FOR DYNAMIC TRANSMISSION RATES
US5724349A (en) * 1995-08-31 1998-03-03 Lucent Technologies Inc. Terabit per second ATM packet switch having out-of-band control with multi casting
AU6970896A (en) 1995-09-14 1997-04-01 Ascom Nexion Inc. Transmitter controlled flow control for buffer allocation in wide area atm networks
US6445708B1 (en) * 1995-10-03 2002-09-03 Ahead Communications Systems, Inc. ATM switch with VC priority buffers
JP2842522B2 (en) * 1995-12-06 1999-01-06 日本電気株式会社 ATM switch and control method thereof
KR100318956B1 (en) * 1995-12-26 2002-04-22 윤종용 Apparatus and method for multiplexing cells in asynchronous transmission mode
US5774463A (en) * 1995-12-29 1998-06-30 Gte Laboratories Incorporated Switching matrix with contention arbitration
US5813040A (en) * 1995-12-29 1998-09-22 Gte Laboratories Inc Write controller for a CAM-based switch with lineraly searchable memory utilizing hardware-encoded status tags to indicate avaliablity of each memory location for writing
US6452927B1 (en) 1995-12-29 2002-09-17 Cypress Semiconductor Corporation Method and apparatus for providing a serial interface between an asynchronous transfer mode (ATM) layer and a physical (PHY) layer
US5862128A (en) * 1995-12-29 1999-01-19 Gte Laboratories Inc Merged buffer signal switch
WO1997026737A1 (en) * 1996-01-16 1997-07-24 Fujitsu Limited A reliable and flexible multicast mechanism for atm networks
US6430186B1 (en) * 1996-03-15 2002-08-06 Pmc-Sierra, Inc. Asynchronous bit-table calendar for ATM switch
KR100387048B1 (en) * 1996-03-30 2003-08-21 삼성전자주식회사 Medium low speed subscriber multicasting embodying method and apparatus therefor
US5923654A (en) * 1996-04-25 1999-07-13 Compaq Computer Corp. Network switch that includes a plurality of shared packet buffers
US5864539A (en) * 1996-05-06 1999-01-26 Bay Networks, Inc. Method and apparatus for a rate-based congestion control in a shared memory switch
US6034945A (en) 1996-05-15 2000-03-07 Cisco Technology, Inc. Method and apparatus for per traffic flow buffer management
US5872787A (en) * 1996-05-15 1999-02-16 Gte Laboratories Incorporated Distributed switch buffer utilizing cascaded modular switch chips
US5966380A (en) * 1996-08-02 1999-10-12 Fore System, Inc. Processing of TLV based link-state packets
US5748905A (en) 1996-08-30 1998-05-05 Fujitsu Network Communications, Inc. Frame classification using classification keys
KR100318957B1 (en) * 1996-09-02 2002-04-22 윤종용 Congestion notification device and congestion control method in asynchronous transmission mode network
US6031842A (en) 1996-09-11 2000-02-29 Mcdata Corporation Low latency shared memory switch architecture
US5991867A (en) * 1996-09-12 1999-11-23 Efficient Networks, Inc. Transmit scheduler for an asynchronous transfer mode network and method of operation
US5959993A (en) * 1996-09-13 1999-09-28 Lsi Logic Corporation Scheduler design for ATM switches, and its implementation in a distributed shared memory architecture
US6049546A (en) * 1996-10-15 2000-04-11 At&T Corporation System and method for performing switching in multipoint-to-multipoint multicasting
JPH10126419A (en) * 1996-10-23 1998-05-15 Nec Corp Atm exchange system
US6229812B1 (en) 1996-10-28 2001-05-08 Paxonet Communications, Inc. Scheduling techniques for data cells in a data switch
US6226298B1 (en) * 1996-11-08 2001-05-01 Pmc-Sierra (Maryland), Inc. Method and apparatus for detecting disabled physical devices and deleting undeliverable cells
US6188690B1 (en) * 1996-12-12 2001-02-13 Pmc-Sierra, Inc. Method and apparatus for high speed, scalable communication system
US6011779A (en) * 1996-12-30 2000-01-04 Hyundai Electronics America ATM switch queuing system
US6002692A (en) * 1996-12-30 1999-12-14 Hyundai Electronics America Line interface unit for adapting broad bandwidth network to lower bandwidth network fabric
US6201789B1 (en) * 1996-12-30 2001-03-13 Compaq Computer Corporation Network switch with dynamic backpressure per port
US6233242B1 (en) 1996-12-30 2001-05-15 Compaq Computer Corporation Network switch with shared memory system
US6052376A (en) * 1996-12-30 2000-04-18 Hyundai Electronics America Distributed buffering system for ATM switches
GB2321821B (en) * 1997-01-17 1999-03-24 Neill Eugene O Method for distributing and recovering buffer memories in an asynchronous transfer mode edge device
GB2322761B (en) * 1997-01-17 1999-02-10 Donal Casey Method for selecting virtual channels based on address p;riority in an asynchronous transfer mode device
GB2321351B (en) * 1997-01-17 1999-03-10 Paul Flood System and method for data transfer across multiple clock domains
GB2323744B (en) * 1997-01-17 1999-03-24 Connell Anne O Method of supporting unknown addresses in an interface for data transmission in an asynchronous transfer mode
US6115373A (en) * 1997-01-24 2000-09-05 The Hong Kong University Of Science And Technology Information network architecture
US5889776A (en) * 1997-01-31 1999-03-30 Alpnet Corporation Physical layer switch system for ethernet local area network communication system
US6085250A (en) * 1997-03-20 2000-07-04 Efficient Networks, Inc. Method and system for using layered networking application program interfaces (APIs) using a native asynchronous transfer mode (ATM) API
US7490169B1 (en) 1997-03-31 2009-02-10 West Corporation Providing a presentation on a network having a plurality of synchronized media types
US6161137A (en) * 1997-03-31 2000-12-12 Mshow.Com, Inc. Method and system for providing a presentation on a network
US7412533B1 (en) 1997-03-31 2008-08-12 West Corporation Providing a presentation on a network having a plurality of synchronized media types
US7143177B1 (en) 1997-03-31 2006-11-28 West Corporation Providing a presentation on a network having a plurality of synchronized media types
SE520265C2 (en) * 1997-04-01 2003-06-17 Ericsson Telefon Ab L M Method and arrangement for fair distribution of bandwidth in an ATM switch
US5987028A (en) * 1997-05-12 1999-11-16 Industrial Technology Research Insitute Multiple channel ATM switch
US6084881A (en) * 1997-05-22 2000-07-04 Efficient Networks, Inc. Multiple mode xDSL interface
KR100216368B1 (en) * 1997-06-11 1999-08-16 윤종용 The input buffer controller and logical buffer size decision algorithm
KR100247022B1 (en) * 1997-06-11 2000-04-01 윤종용 A single switch element of atm switching system and buffer thresholds value decision method
US6260072B1 (en) * 1997-06-12 2001-07-10 Lucent Technologies Inc Method and apparatus for adaptive routing in packet networks
US6201813B1 (en) 1997-06-30 2001-03-13 Cisco Technology, Inc. Method and apparatus for using ATM queues for segmentation and reassembly of data frames
US6487202B1 (en) 1997-06-30 2002-11-26 Cisco Technology, Inc. Method and apparatus for maximizing memory throughput
US6430191B1 (en) 1997-06-30 2002-08-06 Cisco Technology, Inc. Multi-stage queuing discipline
US6219352B1 (en) * 1997-11-24 2001-04-17 Cabletron Systems, Inc. Queue management with support for multicasts in an asynchronous transfer mode (ATM) switch
FR2771573B1 (en) * 1997-11-27 2001-10-19 Alsthom Cge Alkatel PACKET SWITCHING ELEMENT WITH BUFFER MEMORIES
US6526060B1 (en) 1997-12-05 2003-02-25 Cisco Technology, Inc. Dynamic rate-based, weighted fair scheduler with explicit rate feedback option
US7872969B2 (en) * 1997-12-23 2011-01-18 Ciena Corporation Method and apparatus for auto detection of AAL5 type frames for VCC and VPC switches
US20010055307A1 (en) * 1997-12-23 2001-12-27 Bernard St-Denis Method and apparatus for auto detection of aal5 type frames
US6625120B1 (en) 1997-12-23 2003-09-23 Nortel Networks Limited Method and apparatus for auto detection of AAL5 type frames for VCC and VPC switches
US5963499A (en) * 1998-02-05 1999-10-05 Cypress Semiconductor Corp. Cascadable multi-channel network memory with dynamic allocation
US6178159B1 (en) * 1998-03-02 2001-01-23 Lucent Technologies Inc. Available bit rate flow control algorithms for ATM networks
US6310875B1 (en) * 1998-03-30 2001-10-30 Nortel Networks Limited Method and apparatus for port memory multicast common memory switches
US6570850B1 (en) 1998-04-23 2003-05-27 Giganet, Inc. System and method for regulating message flow in a digital data network
US6721325B1 (en) * 1998-04-23 2004-04-13 Alcatel Canada Inc. Fair share scheduling of multiple service classes with prioritized shaping
US6795442B1 (en) * 1998-04-23 2004-09-21 Emulex Design & Manufacturing Corporation System and method for scheduling message transmission and processing in a digital data network
US6320864B1 (en) * 1998-06-19 2001-11-20 Ascend Communications, Inc. Logical multicasting method and apparatus
US6434115B1 (en) * 1998-07-02 2002-08-13 Pluris, Inc. System and method for switching packets in a network
JP3141850B2 (en) * 1998-07-10 2001-03-07 日本電気株式会社 Time division switching device, time division switching method, and recording medium
US6657961B1 (en) 1998-08-18 2003-12-02 Efficient Networks, Inc. System and method for enhanced end station to end station data flow control
US6633543B1 (en) * 1998-08-27 2003-10-14 Intel Corporation Multicast flow control
US6829218B1 (en) 1998-09-15 2004-12-07 Lucent Technologies Inc. High speed weighted fair queuing system for ATM switches
US6920146B1 (en) * 1998-10-05 2005-07-19 Packet Engines Incorporated Switching device with multistage queuing scheme
US6697362B1 (en) * 1998-11-06 2004-02-24 Level One Communications, Inc. Distributed switch memory architecture
US6661774B1 (en) 1999-02-16 2003-12-09 Efficient Networks, Inc. System and method for traffic shaping packet-based signals
US6590897B1 (en) 1999-03-08 2003-07-08 Efficient Networks, Inc. System and method for bridging universal serial bus and asynchronous transfer mode communication links
US6606326B1 (en) 1999-07-02 2003-08-12 International Business Machines Corporation Packet switch employing dynamic transfer of data packet from central shared queue path to cross-point switching matrix path
US7911960B1 (en) 1999-08-13 2011-03-22 International Business Machines Corporation Delayed-start method for minimizing internal switch congestion
US7016301B1 (en) 1999-09-01 2006-03-21 Cisco Technology, Inc. Fair multiplexing scheme for multiple input port router
US6859435B1 (en) * 1999-10-13 2005-02-22 Lucent Technologies Inc. Prevention of deadlocks and livelocks in lossless, backpressured packet networks
US6775292B1 (en) 2000-01-24 2004-08-10 Cisco Technology, Inc. Method for servicing of multiple queues carrying voice over virtual circuits based on history
US6907041B1 (en) * 2000-03-07 2005-06-14 Cisco Technology, Inc. Communications interconnection network with distributed resequencing
US6757284B1 (en) 2000-03-07 2004-06-29 Cisco Technology, Inc. Method and apparatus for pipeline sorting of ordered streams of data items
US6781998B1 (en) 2000-04-07 2004-08-24 Telefonaktiebolaget Lm Ericsson (Publ) Random reordering system/method for use in ATM switching apparatus
US7142558B1 (en) 2000-04-17 2006-11-28 Cisco Technology, Inc. Dynamic queuing control for variable throughput communication channels
US7236489B1 (en) * 2000-04-27 2007-06-26 Mosaid Technologies, Inc. Port packet queuing
US7079525B1 (en) 2000-04-27 2006-07-18 Cisco Technology, Inc. Network switch having a hybrid switch architecture
EP1301864A4 (en) * 2000-04-28 2005-09-07 Sheer Networks Inc Network management method and system
CA2368619C (en) * 2000-04-28 2008-10-21 Sheer Networks, Inc. Topology discovery in atm networks
US7337209B1 (en) 2000-04-28 2008-02-26 Sheer Networks, Inc. Large-scale network management using distributed autonomous agents
US7106728B1 (en) 2000-05-01 2006-09-12 Industrial Technology Research Institute Switching by multistage interconnection of concentrators
US6591285B1 (en) * 2000-06-16 2003-07-08 Shuo-Yen Robert Li Running-sum adder networks determined by recursive construction of multi-stage networks
US6697332B1 (en) * 2000-07-13 2004-02-24 Pmc-Sierra, Inc. Forward performance monitoring cell generation in ATM OAM processing
US6816492B1 (en) 2000-07-31 2004-11-09 Cisco Technology, Inc. Resequencing packets at output ports without errors using packet timestamps and timestamp floors
US6721313B1 (en) 2000-08-01 2004-04-13 International Business Machines Corporation Switch fabric architecture using integrated serdes transceivers
US6977925B2 (en) * 2000-08-08 2005-12-20 Cesura, Inc. Folded fabric switching architecture
US7463626B2 (en) * 2000-11-21 2008-12-09 Roy Subhash C Phase and frequency drift and jitter compensation in a distributed telecommunications switch
US7254139B2 (en) * 2000-12-28 2007-08-07 International Business Machines Corporation Data transmission system with multi-memory packet switch
US7130301B2 (en) * 2000-12-28 2006-10-31 International Business Machines Corporation Self-route expandable multi-memory packet switch with distributed scheduling means
US7130302B2 (en) * 2000-12-28 2006-10-31 International Business Machines Corporation Self-route expandable multi-memory packet switch
US6819675B2 (en) * 2000-12-28 2004-11-16 International Business Machines Corporation Self-route multi-memory expandable packet switch with overflow processing means
US7092393B1 (en) 2001-02-04 2006-08-15 Cisco Technology, Inc. Method and apparatus for distributed reassembly of subdivided packets using multiple reassembly components
US6832261B1 (en) 2001-02-04 2004-12-14 Cisco Technology, Inc. Method and apparatus for distributed resequencing and reassembly of subdivided packets
US6934760B1 (en) * 2001-02-04 2005-08-23 Cisco Technology, Inc. Method and apparatus for resequencing of packets into an original ordering using multiple resequencing components
US6947418B2 (en) * 2001-02-15 2005-09-20 3Com Corporation Logical multicast packet handling
AU2002234853A1 (en) * 2001-03-07 2002-09-19 Sheer Networks Inc. Method for correlating behavior between two elements of a system to determine the presence of mutual interaction between the elements
US7239641B1 (en) * 2001-04-24 2007-07-03 Brocade Communications Systems, Inc. Quality of service using virtual channel translation
US6807186B2 (en) 2001-04-27 2004-10-19 Lsi Logic Corporation Architectures for a single-stage grooming switch
JP3897994B2 (en) * 2001-05-31 2007-03-28 富士通株式会社 Switch device and data transfer system
US7103059B2 (en) * 2001-06-15 2006-09-05 Industrial Technology Research Institute Scalable 2-stage interconnections
US7609695B2 (en) * 2001-06-15 2009-10-27 Industrial Technology Research Institute Optimizing switching element for minimal latency
US6728857B1 (en) 2001-06-20 2004-04-27 Cisco Technology, Inc. Method and system for storing and retrieving data using linked lists
US6901455B2 (en) * 2001-06-29 2005-05-31 Intel Corporation Peripheral sharing device with unified clipboard memory
US6813676B1 (en) * 2001-07-27 2004-11-02 Lsi Logic Corporation Host interface bypass on a fabric based array controller
US7389359B2 (en) 2001-10-19 2008-06-17 Foundry Networks, Inc. Method and system for intelligently forwarding multicast packets
US7647422B2 (en) 2001-11-06 2010-01-12 Enterasys Networks, Inc. VPN failure recovery
US7505458B2 (en) * 2001-11-27 2009-03-17 Tellabs San Jose, Inc. Apparatus and method for a fault-tolerant scalable switch fabric with quality-of-service (QOS) support
US20030101158A1 (en) * 2001-11-28 2003-05-29 Pinto Oscar P. Mechanism for managing incoming data messages in a cluster
US7111248B2 (en) * 2002-01-15 2006-09-19 Openwave Systems Inc. Alphanumeric information input method
GB2385233B (en) * 2002-02-07 2004-04-21 3Com Corp Network switch with parallel working of look-up engine and network processor
US20030163618A1 (en) * 2002-02-27 2003-08-28 Vishal Anand Shared queue for multiple input-streams
US7606938B2 (en) 2002-03-01 2009-10-20 Enterasys Networks, Inc. Verified device locations in a data network
US7069557B2 (en) * 2002-05-23 2006-06-27 International Business Machines Corporation Network processor which defines virtual paths without using logical path descriptors
WO2005032166A1 (en) * 2003-09-29 2005-04-07 British Telecommunications Public Limited Company Channel assignment process
US7535898B2 (en) * 2003-11-06 2009-05-19 Intel Corporation Distributed switch memory architecture
US7539190B2 (en) * 2004-01-05 2009-05-26 Topside Research, Llc Multicasting in a shared address space
US7580403B2 (en) 2004-02-26 2009-08-25 Enterasys Networks, Inc. Status transmission system and method
US7945945B2 (en) 2004-08-06 2011-05-17 Enterasys Networks, Inc. System and method for address block enhanced dynamic network policy management
US7347628B2 (en) 2004-11-08 2008-03-25 Enterasys Networks, Inc. Optical interface identification system
US7480304B2 (en) * 2004-12-29 2009-01-20 Alcatel Lucent Predictive congestion management in a data communications switch using traffic and system statistics
US8086232B2 (en) 2005-06-28 2011-12-27 Enterasys Networks, Inc. Time synchronized wireless method and operations
JP2009177256A (en) * 2008-01-21 2009-08-06 Fujitsu Ltd Packet switch apparatus and packet switch method
US8248930B2 (en) * 2008-04-29 2012-08-21 Google Inc. Method and apparatus for a network queuing engine and congestion management gateway
US8948193B2 (en) * 2008-08-19 2015-02-03 Check Point Software Technologies, Ltd. Methods for intelligent NIC bonding and load-balancing
ES2359522B1 (en) * 2008-12-18 2012-04-02 Vodafone España, S.A.U. RADIO BASE PROCEDURE AND STATION FOR PLANNING TRAFFIC IN CELL PHONE NETWORKS OF RE? WIDE AREA.
US20130138420A1 (en) * 2009-12-23 2013-05-30 Texas Instruments Incorporated Managing Varying Instrumentation Volumes to Prevent Data Loss
US8644139B2 (en) * 2010-04-26 2014-02-04 International Business Machines Corporation Priority based flow control within a virtual distributed bridge environment
JP2013034133A (en) * 2011-08-02 2013-02-14 Fujitsu Ltd Transmission apparatus, transmitting-receiving system, and control method
GB2495543A (en) * 2011-10-14 2013-04-17 St Microelectronics Res & Dev An arrangement for routing requests
US10108466B2 (en) * 2015-06-29 2018-10-23 International Business Machines Corporation Optimizing the initialization of a queue via a batch operation

Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5119372A (en) * 1989-05-10 1992-06-02 At&T Bell Laboratories Multi-access ATD multiplexer with congestion detection circuitry
US5144619A (en) * 1991-01-11 1992-09-01 Northern Telecom Limited Common memory switch for routing data signals comprising ATM and STM cells
US5231630A (en) * 1990-04-10 1993-07-27 Fujitsu Limited Network termination enabling relief of call congestion in an exchanger
US5233606A (en) * 1991-08-02 1993-08-03 At&T Bell Laboratories Arrangement for controlling shared-buffer-memory overflow in a multi-priority environment
US5271004A (en) * 1990-09-05 1993-12-14 Gpt Limited Asynchronous transfer mode switching arrangement providing broadcast transmission
US5317561A (en) * 1991-05-07 1994-05-31 Siemens Aktiengesellschaft Method for switching ATM message cell streams having a high bit rate via a switching equipment having a lower bit rate
US5325356A (en) * 1992-05-20 1994-06-28 Xerox Corporation Method for aggregating ports on an ATM switch for the purpose of trunk grouping
US5353282A (en) * 1993-03-18 1994-10-04 Northern Telecom Limited Local area network embedded in the communication switch core
US5381410A (en) * 1990-12-20 1995-01-10 Thomson-Csf Transit switch for an asynchronous network, notably an ATM network

Family Cites Families (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2736092B2 (en) * 1989-01-10 1998-04-02 株式会社東芝 Buffer device
CA1320257C (en) * 1989-04-20 1993-07-13 Ernst August Munter Method and apparatus for input-buffered asynchronous transfer mode switching
JP2907886B2 (en) * 1989-09-14 1999-06-21 株式会社日立製作所 Switching system
US5050161A (en) * 1989-12-04 1991-09-17 Bell Communications Research, Inc. Congestion management based on multiple framing strategy
US5285444A (en) * 1990-02-09 1994-02-08 Hitachi, Ltd. Multi-stage link switch
EP0487235B1 (en) * 1990-11-21 1999-02-03 AT&T Corp. Bandwidth and congestion management in accessing broadband ISDN networks
US5130984A (en) * 1990-12-18 1992-07-14 Bell Communications Research, Inc. Large fault tolerant packet switch particularly suited for asynchronous transfer mode (ATM) communication
US5303078A (en) * 1990-12-18 1994-04-12 Bell Communications Research, Inc. Apparatus and method for large scale ATM switching
US5241536A (en) * 1991-10-03 1993-08-31 Northern Telecom Limited Broadband input buffered atm switch
SE470039B (en) * 1992-03-17 1993-10-25 Ellemtel Utvecklings Ab Ways to achieve link grouping in a packet selector
US5313454A (en) * 1992-04-01 1994-05-17 Stratacom, Inc. Congestion control for cell networks
US5278828A (en) * 1992-06-04 1994-01-11 Bell Communications Research, Inc. Method and system for managing queued cells
US5299190A (en) * 1992-12-18 1994-03-29 International Business Machines Corporation Two-dimensional round-robin scheduling mechanism for switches with multiple input queues
US5392280A (en) * 1994-04-07 1995-02-21 Mitsubishi Electric Research Laboratories, Inc. Data transmission system and scheduling protocol for connection-oriented packet or cell switching networks

Patent Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5119372A (en) * 1989-05-10 1992-06-02 At&T Bell Laboratories Multi-access ATD multiplexer with congestion detection circuitry
US5231630A (en) * 1990-04-10 1993-07-27 Fujitsu Limited Network termination enabling relief of call congestion in an exchanger
US5271004A (en) * 1990-09-05 1993-12-14 Gpt Limited Asynchronous transfer mode switching arrangement providing broadcast transmission
US5381410A (en) * 1990-12-20 1995-01-10 Thomson-Csf Transit switch for an asynchronous network, notably an ATM network
US5144619A (en) * 1991-01-11 1992-09-01 Northern Telecom Limited Common memory switch for routing data signals comprising ATM and STM cells
US5317561A (en) * 1991-05-07 1994-05-31 Siemens Aktiengesellschaft Method for switching ATM message cell streams having a high bit rate via a switching equipment having a lower bit rate
US5233606A (en) * 1991-08-02 1993-08-03 At&T Bell Laboratories Arrangement for controlling shared-buffer-memory overflow in a multi-priority environment
US5325356A (en) * 1992-05-20 1994-06-28 Xerox Corporation Method for aggregating ports on an ATM switch for the purpose of trunk grouping
US5353282A (en) * 1993-03-18 1994-10-04 Northern Telecom Limited Local area network embedded in the communication switch core

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
XIV INTERNATIONAL SWITCHING SYMPOSIUM, October 1992, OSHIMA et al., "A New ATM Switch Architecture Based on STS-Type Shared Buffering and its LSI Implementation". *

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP0785699A2 (en) * 1996-01-16 1997-07-23 AT&T Corp. Multicast routing in multistage networks
EP0785697A3 (en) * 1996-01-16 2000-05-31 AT&T Corp. Multistage network having multicast routing congestion feedback
EP0785699A3 (en) * 1996-01-16 2000-05-31 AT&T Corp. Multicast routing in multistage networks
EP0785698A3 (en) * 1996-01-16 2000-05-31 AT&T Corp. Buffering of multicast cells in switching networks
EP2477366A1 (en) * 2010-11-12 2012-07-18 Huawei Technologies Co., Ltd. Data transmission method, apparatus and system
EP2477366A4 (en) * 2010-11-12 2012-07-18 Huawei Tech Co Ltd Data transmission method, apparatus and system
US8549193B2 (en) 2010-11-12 2013-10-01 Huawei Technologies Co., Ltd. Data transmission method, device and system

Also Published As

Publication number Publication date
EP0761055A1 (en) 1997-03-12
US5583861A (en) 1996-12-10
KR970703078A (en) 1997-06-10
EP0761055A4 (en) 1999-12-01
JPH09512683A (en) 1997-12-16
US5570348A (en) 1996-10-29
US5557607A (en) 1996-09-17
CA2188882A1 (en) 1995-11-09
TW281840B (en) 1996-07-21

Similar Documents

Publication Publication Date Title
US5570348A (en) Method and apparatus for enqueueing data cells in an ATM switch fabric architecture
US6151301A (en) ATM architecture and switching element
CA2271883C (en) Many dimensional congestion detection system and method
US7006438B2 (en) Distributed control of data flow in a network switch
US6011779A (en) ATM switch queuing system
US6434155B1 (en) Weighted round robin engine used in scheduling the distribution of ATM cells
US5629928A (en) Dynamic fair queuing to support best effort traffic in an ATM network
EP0864244B1 (en) Apparatus and methods to change thresholds to control congestion in atm switches
EP0533900B1 (en) Low delay or low loss cell switch for atm
EP0763915B1 (en) Packet transfer device and method adaptive to a large number of input ports
US6205118B1 (en) Adaptive time slot scheduling apparatus and method for end-points in an ATM network
EP0785699B1 (en) Multicast routing in multistage networks
KR100329130B1 (en) Data transfer switches, access controlled asynchronous transfer mode (ATM) switches and information cell flow control methods
US20040151197A1 (en) Priority queue architecture for supporting per flow queuing and multiple ports
JP4006205B2 (en) Switching arrangement and method with separate output buffers
US6717912B1 (en) Fair discard system
US6052376A (en) Distributed buffering system for ATM switches
JP3859864B2 (en) Queue management system
US6768717B1 (en) Apparatus and method for traffic shaping in a network switch
WO1997031461A1 (en) High speed packet-switched digital switch and method
EP1111851B1 (en) A scheduler system for scheduling the distribution of ATM cells
EP1198098B1 (en) Switching arrangement and method with separated output buffers
JP4504606B2 (en) Apparatus and method for shaping traffic in a network switch
EP1111853A2 (en) A method of providing a guaranteed frame rate in a hierarchical scheduler
Zervanos et al. Design and implementation of an ABR server in a shared-bus ATM switch

Legal Events

Date Code Title Description
AK Designated states

Kind code of ref document: A1

Designated state(s): CA JP KR

AL Designated countries for regional patents

Kind code of ref document: A1

Designated state(s): AT BE CH DE DK ES FR GB GR IE IT LU MC NL PT SE

121 Ep: the epo has been informed by wipo that ep was designated in this application
DFPE Request for preliminary examination filed prior to expiration of 19th month from priority date (pct application filed before 20040101)
WWE Wipo information: entry into national phase

Ref document number: 2188882

Country of ref document: CA

WWE Wipo information: entry into national phase

Ref document number: 1995915672

Country of ref document: EP

WWP Wipo information: published in national office

Ref document number: 1995915672

Country of ref document: EP

WWW Wipo information: withdrawn in national office

Ref document number: 1995915672

Country of ref document: EP