CA2113647A1 - Crossbar with return net for scalable self-routing non-blocking message switching and routing system - Google Patents
Crossbar with return net for scalable self-routing non-blocking message switching and routing systemInfo
- Publication number
- CA2113647A1 CA2113647A1 CA002113647A CA2113647A CA2113647A1 CA 2113647 A1 CA2113647 A1 CA 2113647A1 CA 002113647 A CA002113647 A CA 002113647A CA 2113647 A CA2113647 A CA 2113647A CA 2113647 A1 CA2113647 A1 CA 2113647A1
- Authority
- CA
- Canada
- Prior art keywords
- crossbar
- switches
- switch
- message
- address
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Abandoned
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04Q—SELECTING
- H04Q3/00—Selecting arrangements
- H04Q3/64—Distributing or queueing
- H04Q3/68—Grouping or interlacing selector groups or stages
Abstract
Described is a crossbar having a return network of switches arranged and connected in a manner so as to return messages which encounter blocked message pathways. It is particularly suited to truly non-blocking network designs, which are also described.
Description
2 PCr/USg2/06653 -` 2 1 ~
-1- ' "'' ~
(~ROSSBAR WlT~ ~TURN N13:T ~R SCALABL~ SELF-ROUTING ~ ~ :
NON-BLOÇKIN(;~ l~ESS~GE SVVll C~ AND ROUI~NG SYSTEM
This invention relates to a self-routing, non-blocl~ng switch system for routing messages f~om any one of a number of inputs to any one of a number of outputs in an efficient and ccst effective manner.
BACKRO~ND QFT~IE~NVENI~ON
As data ~ansfer becomes more and more complex, the numbers of the messages which need to be t~ansferred across a nehvork from one location to ~ ~
another, or one area of a computing device to another, has grown remarkably. . ::
Accordingly, any system or subcomponen~ thereof which allows ~or ease of routing messages from any of a large number of inputs to any palticular~one of a large number of outputs is imponant. The invention: described herein allows for the routing of such messages wi~h very little overhead, at high speed.
The invention herein empioys a reh~rn netw~lc, is capable of using ;
different hnds of switches In the switching network, may operate in a ~ -broadc~ ast mode, may reduce the number of switches necessary by using a Clos network or by reliance on busses and controllers, and uses a self-routing algorithm, elinrinabng ~e need for~p~ocessing overh~.
2 0 ~ Self routing algorithms use information contained in the message itself ;
t~;route the message;at th~e swi~h~l~vel to the desi ed location. ~
Prior self-routing alf,orithms such as ~e ~one dosGribed in the,1~90 IEEE (Vol. 78, No. l),~entitled 'Pastp~çket Switch Archi~for ~
Broa~-Band Inte~rat~d~Sçr~ices Digital Networks", by Tobagi, do not indicate how these algo~ms can actually opeRe~ and do not descnbe how their ~ "
operation can be accomplished when using a Clos-tS pe networlc.
Clos networb ~e prefelTed for larger networks b~ause they reduce the number of switches necessary to accomplish a N x N message switching ;
system to a much smaller number than N2. See, for example, U.S. Patent , :.. :~ ~
..
WO 93/03582 Pcr/US92/~6~53 No. 4,696,000, issued to Payne, which refers to ~e Clos network and -rea~angeable n~tworlcs.
The inven~on described herein simplifies all these ideas.
. . , ,. , , ~ . :
Th,e switches may be arran,ged, in a standard crossbar network for small scale implementations, a Clos network or a va~ation of a Clos network using busses, and the types of switches and routings, dependent on ~}r,,e switch type -:., :, ~.. .
used, may also valy depending on the implementation. As described in the ~;
detailed description, a preferred implementation employs a return network for all blocked, messages that wish to address the same output node and returns lo them to the source or sending node. Due to the nature of the routing system ~-itself and the construction of the switching network, essential,ly no overhead or ` -preprocessing is required to route messages through the sys~m descnbed from the input nodes to the output nodes.
BRIEF DESCRIPIIQN OF l~IE DRA~INGS ;
Fig. 1 is a tw~dimensional layout of the switches in an N x N ;
crossbar. i ~ ~
Flg. 2 is a block diagram of a trap network for removing connection ; ~ `
request conflicts.
, . . . .
, : ~,: :...
Fig. 3 is a layout of a second implementation of an n by m crossbar.
2~ Fig. 4 is a layout~of a~ n by m crossbar wi~ an as~ return : ~ ~ , . .
network.~
. Sa is a~hema~c ~diag~ of 2 x 2 cro~b~ $witch.
Fig. Sb~ is a diagram of a gate-level implementalion of the switch descnbed in ~Fi~. 5a.
~ Fig. 6a (i), (ii),~ and ~l~i) are three possible m~des~ of operation of the 2 x 2 broadcast switch.
Fig. 6a is a diagram of a gate-level impl,ementation of a broadcast switch.
,,, ''~, ;' "
. .. ..
.. . .: : , Fig. 7 is an electroni~-level diagram of an implementation of a crossbar swiuh.
Fig. 8 is a 36 x 36 Clos network diagram.
Fig. 9 is a 16 x 16 Clos network diagram.
Fig. 10 is a layout of a non-blocking nr x nr self-routing Clos network with re~urnnetworks.
Figs. 11a, b and c are diagrams of the inputs and outputs of the crossbars used m.Fig. 10.
Fig. 12 is a schematic diagram of the bus implementation of a truly non-blocking network in accord with the invention.
Fig. 13 is a block diagram of a finite-state machine model of the bus controllerwhich may be used with the invention as described in Fig. 12.
Fig. 14 is a detailed schematic diagram of the databus con~roller blocks .: . . , employed in one of the 2n-1 databus controller blocks used with the invenuon as ;~
described with reference to Fig. 12.
Fig. 15 is a drawing of a complete 4 x 4 crossbar wlth associated return net.
~ ~ Fig. 16 is a block diagram of one message embodiment M for the invention havmg a header H and a data porlion D.
~ ; SUMM~RY OF THE ~lENTIoN ~ - ;
Crossbar With~Return Nçt This invention teaches a crossbar for use in a switching system w~uch switches messages and returns ~messages which do not!get through, on the first ~. I has ; headers in the messages wherein the headers have source and destination address and -25~ ~two~turn~;signal indicators~ which are bc~th set at the hme the message is sent. The crossbar has a number of inputs, n, and a nu~nber of outputs, m, wherein the m inputs are connected by an n X m array of 2 X 2 switches organized into logical colurnns and logical rows labeled 1 to m and 1 to n, respectively, and wherein the swltches are comlected between the n input sources and the m output destinations. The two-by-two -~
switches W0 93/03s82 ~ Pcr/us92~ 3 5 ~ 4 ~ ~ ~ :
may be set into a pass or exchange mode (but are all set into the s~ne mode, called a default mode, at the outset) based on the coincidence of two events, first ~at the message header des~nation address matches the column address of the switch and second that the turn signal has not a Iready been YeSet by a previous switch having a matching column address. Ihe crossbar has in communicative association therewith a ~eturn network ha~ing similar or identical 2 X 2 switches organized into logical diagonals with respect to the logical organization of the crossbar switches. The longest diagonal having the highest number of switches has addresses in its switches matching row 1. `
o Succeedingly smaller diagonals have succeeding larger addresses, and less switches. The ultimate diagonal has only one switch of address "n" (The longest diagonal may be shortened by one switch, as mentioned in the detailed description.) ',, These 2 x 2 switches are also setable into pass or exchange mode based on Ihe coincidence of two events, bu~ all set into the one mode, called default mode, at the outset. The f1rst event is that the message header source address '~ ;
matches the diagonal address of the return net switch. The second event is ' ,' ~ ,~,;' ' that the turn signal has not already been reset by a previous switch with the , ' , same column address.,, ~EA~ DES~I~ON ()F I~IE INVENIION , ~ -, , j~ Several conce~ts a~e, included he,re ,which are all centejIed on,the idea -, ~at ,self-routing algonthms may be employed to provide local control of 2 x 2 , ~ ' "~ switches which comprise various networks. In doing this, the n~ for a 2s ~ , global controller mondtonng all of the inputs and all of the outputs, and ~elr ; ~, ;
connections, is not required. Where plain crossbars are used, a total of NM of ' ~
*e 2 x 2 switches would be required for an N input and M output network. ,~ ,'" ,' ~ ' Because the switches are relatively simple in functionality and layout of the ' ' design would be extremely regular, such networks can be fabricated easily in . ~ , .
WO 93/03582 ~ ~ ~ 3 6 ~ ~ PCr/USs2/06653 electronic as well as optical implementations. Using a Clos variety of truly non-blocking switehing network, even the number of switches may be substantially reduced. Even further reductions in ~e num~er of switches required is possible using the bus design. This kind of savings improves the scalability of the inventive concepts described herein.
For e~sample, in a 36 x 36 netwo~k using a s~ight crossbar design, 1,296 2 x 2 switches would be required. Using a Clos network design, only 1,188 switches would be required. When moving to a network of approximately 128 x 128 design, 16,384 switches would be ~equired, whereas 0 wi~ a Clos design, only 99936 switches would be required.
An additional and important part of this set of inventions is the use of a return network. The return network operates to return messages that attempt to ~oute themselves to outputs which are already occupied. l~is re-routing or routing back is automatic, and takes less time than a bme~ut, especially in larger networks, to inform the sending node tbat ~e message has not gotten ~ ;~
through. (A timeout is a counter or timer that waits some increment longer than the maximum delay, and upon finding the time expired or the counter full, a new action is begun.) In providing this description, some terminology ought first be defined.
20 ~ ~ ~ The~firstdef~initionwouldbenTrulyNon-Blocl~ng~Networks" (lNBs). A
truly non-b ~ing ne~rk is one in which reanangement of existing , ; conditions is not requi~ed in order that ,every message ~get throug~. It ~equires a set of features, including ~e follow~g~
(i) set~ng the appropriate network switches for all requested ;~25~ connections, (ii) ensuring that no conflict occurs (a conflict occurs when ; -multiple inputs are connected to the same output), (iii) in the event of a conflict, only one connection may be ~ -satisfied while the other requests must be lcept walting or informed, ; - -'"~ ' `
WO 93/03582 ~ Pcr/us92/~6653 - 6 ~
(iv) in case of broadcasting networks, the control must also allow a single source to be connected to mul~ple outputs. ~` :
The inventive concep~s desc~ibed in this ap~lication show how TNBs ean provide for practical implementation for high ~peed and high band width ap~lications using a distribut0d local routing control. (l~ey are principally ~;:
oriented toward elec~ical switching but additional benefits can be gained using vptical switching, such as no sepa~ate line being required for a return signal.) ~:
The crossbar switch in general can be thought of as a square or rectangular tw~dimensional array connecting a set of inputs to an equal or, if rectangular, unequal, number of outputs. Thus, for N inputs and M outputs, a crossbar has NM crosspoints and can simultaneously provide any combination of input and output one-t~one connections in a non-blocking manner. In practice, other methods besides using an a~ray of switches, where each switch rel)resents a crosspoint, have been used. These include for example, in ~ ~ -electronics, bus arbitrated architecture used to simpli~y the quadratic complexity of switches and control and, in optics, outerproduct matrix ;
multiplying architectures have been used. The primary approach here is to use switch level design for self-routing. `
A TNB that requires less switches than are required in a crossbar was 20 ~ first proposed by Clos in ~Studv of Non-Blocldn~ Switçking Nrt~o h, Bell ;;
System Technical Journal, 32, ~. 4~24, 1953. A general Clos ne~worlc (CN) bas an odd numb~r, say, 2k+1 (where k-1,2,.~...), of stages ~nd is built in a modular desigh e ploy-ng a ~multiplicity of c~ossbars which are :
substan~ly smaller than the total N x N size desired. Thus, the number of 25 ~ ~ switches which would be r~equired for a Clos network would be on the order of ~N x ~/N, when a comparably sized crossbar would be N x N. (See above paragraph.) Also, networks of more than three stages can be built from -.. ,.. ~ ;. .
three-stage networks by successiYely replacing the center-stage by another three-stage CN.
.
~: :
WO 93/03582 Pcr/US92/o66~3 2 ~ 13 ~
It is hlown ~at in a N x N Clos network, if ~e input stage is built of r crossbars of dimension n x m where rn=N, the second stage is made up of m, r ~ r crossbars, and the third stage is made up of r, m x n crossbars. The size of a crossbar can be designated with the sequence (n,m,r). ~e Clos network will be ~uly non-bloel~ng if m 2 2n-1. The CN is not truly non-blocking but may be rearrangeably non-blocking if m < 2n~
It is not particularly desirable to have a rearrangeably non-blocking network in that messages may be required to be interrupted to rea~ange the network when such rearrangement becomes necessary. Por applications, therefore, requiring long duration or large messages, interruptions are extremely undesirable.
Refer first to Fig. 1 in which a simple c~ssbar network (10) is shown. ~ -It hlas inputs 1 through n and outputs 1 through m as shown, and each one of the 2 x 2 switches may be labeled as points in a mat~c or, 1,1 to n,m. The switchos may be label~ with their column address only since ~is is the one . . which will be compared wi~ the message header as will be explained later.
In Fig. 8, a Clos network is drawn for a 36 x 36 array of inputs to out~uts. The network is comprised of 6, 6 x ll crossbars in column 1, plus 11, 6 x 6 crossbars in column 2, plus another 6, 11 x 6 crossbars in column 3.
2o ~ Thus, an input on line 81 may be directed at switch or c~osspoint 82 to - .. connect ~to: the first top line 83 of crossbar 84 in oolurnn 2, row l, which may .
again ~e di~d by sw~tch 85 to output 86 of the sarne crossbar9 c~nnecting it ~ .
to inpult 87 of crossbar 88. At ~is point it m~y on!y~be rout~d to one of ~e six outputs of c~ossbar 88 by one of the six switches in that column of crossbar~ 88. It will occur~to the reader at this point that three crosspoints are mvolved ;. . ~
in transferring the message from input 81 to one of ~e outputs in cr~ssbar 88. . ~ :
The Clos network, it will also be noted, is generally cons~ucted having square root of N rows ~lus m rows of center crossbars, not labeled~ by three columns. The network may be expanded and the number of switches further - -W0 93/03582 Pcr/uss2/o6653 c~ .t~ - 8 ~
~, ~
reduced by splitting the central portion into three in acc~rdance with the ; .. - ;. ,:
teachings of Clos.
Refer now ~o Fig. 2 in which a trap network scheme is used to remove conneetion ~equest conflicts befor~ they happen, according bo the design 20. ;
s This design is avoided by the invention. In Fig. 25 inputs 1-n are provided to a parallel sorting network 21 and provided, sorted, to a compalator stage 22.
The comparator stage 22 provides an indication of a conflict to the duplicate ;
router stage 23 along with each message being passed. The duplicate router stage 23 returns messages, along lines 24 or 25, to the inputs prior to the ..
s~rting network a, to be handled in accordance with whatever scheme is desired. For instance, either the message header itself may go back to ~e input node where ~e message header and the message may be directed to a buffer, or any number of other things may be desired for messages which may require resending at some other ~ime. The crossbar 27 ~en allows its switches : to be set in accordance with the instruc~ons carried in the header which indicate the output address for each message routed through it. Using the teachings described herein, such an implementation, including units 21, 22, 23 ~ :
and outputs 24 and 25, is not required.
. ;
~ ~ Switches 2 0 ~ Before mo~ing on to a: description of the algoAthm for self-routing, and . : .~ because it would create some cornplicttions to describe ~em later, a ~;
descnption of the switcheslwhich may be used with this device, or at least their functionality, is now descnbed.
While this invention may be~well suited to light switches, only the ?5 :~ electronic versions are shown, however9 it should be clear to one of ordinary skill in the art that light swltches will prwide enhanced flexibility allowing for ~;
return signals over any path once established. Higher speed communications, higher bandwidth and so forth become available when using light switching :' -' WO 93/03582 P~r/Us92/o~6~3 2 ~ ~. 3 ~ ~1 7 devices. Also, by not requiring messages to get a return routing, the return network desc~ib0d herein may even be avoided when using light switches. :
The basic 2 x 2 switch can be seen in Fig. Sa, 50, as hav~ng two inputs, IN 1 and I~ 2, and two outputs, OIJT 1 and O~ 2. It may be ~ :
generally set in either a pass or e~change mode. Under the pass mode, lines :~ ~:
51 and 52 will be open and lines 53 and S~ will be closed. In the exchange mode, thereverse1s~ue.
In Fig. Sb, the switch 50 is shown in logic diagram form. The "change" in input 55 will control whether the switch is in the pass or exchange mode. Thus, when the input to SS is zero, OUT 1 is open to I 1 and closed to I 2, and OUI 2 is open to I 2 and dosed to I 1. When SS is one, the reverse is true. OIJT 1 is open to IN 2 and our 2 is open to IN 1.
Refening now to Fig. 6a, three modes of operation of a brbadcast sw;tch logical connection are shown, i, ii, and iii. In mode i, one input may ;
s be ~connected to both outputs. In mode ii, the inputs::are connected to their ` ` ~
direct outputs. In mode iii, the inputs are connected to their crossed outputs. ~ :
In Fig. 6b, a gate-level implementation of such a switch is shown.
Thus, where input 66 is zero and input 65 is hi (Gr "one"), the output of a' :
will be b and not a, and the output of b' urill be a and not b, thus, a switch 2 o condition. If input 66 is ~æro, or low, and output 65 is also low, there wlll be ~ -a bwadcast pass aondibon in which both outputs will be b. ~ - -Lik~wise, if ~e inpu~ 66 is hi and the output 65 is lqw, the ijnRut b u ill :~ be passed on a', iand the input b will also be pasised on the output b'. In ~e ,-.
case where both 66 and~65 are hi, the output of a' will be b, and the output of : 2s b' will be b, another broadcast condition. -~
The switch may be redesigned to eliminate broadcasit conditions if desir~d. As is also true with the gate-level implementation of the switches described previously, this switch may be designed differently as is well h~own to those of ordiniary skill in the art.
- ,- ~ , Wo 93/03582 Pcr/uss2/o66~3 ~r$~ - 10~
In set~ng the switches,.or not seffing them, or setting them into a broadcast state, this invention re~quires reference to the intended address of the message. ~us, it requires a d~vice similar to that described with reference to Fig. 7, 70, which contains the 2 ~ 2 switch, sw. Input data containing the s address is received by ~e crossbar switch device 70 across datalines 71 and ~ ; .;
72~ Depending on the overall implementation, either a serial or pa~allel input may be desired. Either way, a delay buffer 73a and 73b is provided before the data reaches the switch sw. The address label part of the message is read into the address compa~ator 74 at line 75. The signal output by comparator 74 on .
line 75 is positive if the switch address matches the appropriate part of the .
destination address. Such a positive signal on line 75 latches And-g~e 78. In : ~; :
this way, if the And 78 is not disabled by line 76 or reset by a signal from line 77, the control output will provide input C to switch sw.: The C signal also . ~
:: changes the output on 77a fIom "turn" to "no-turn' by means of latched And . . ;.; . . ;.
~ : 78. ~hus, all succeeding switches would receive ~e changed turn signal,: .. ~,', ' ,'' ,. "'','.!;'' "' ~ allowing the message to continue down the column. :Different schema for thisswitch which perform similar fimctions may be developed easily within the skill of those of : ~:
, ,, .. ::, . . ~
..... .
.:. : : ., : ~ : ' ' ' ~ ~ .
, . .
' ~:
:
WO 93/03s87 Pcr/uss2/Q66s3 ~3~7 ordinary skill in the art. This par~cular design is preferred for the nPtwor~
and algorithms desc~ibed, but ~rariations will ~ apparent to ~ose of ordinary ~ -sl~ll in the art. For example, line 76 could be removed without any loss of function desc~ibed herein. There may be paIallel input paths if desired. May other variations may be constructed without circumventing this invention as ~ ~
daimed. ~; -The Basic Sçlf-routin~ Al~rithm Please refer to Fig. 1. The switches should all have a default setting, either pass or exchange, and all should be set in that mode when beginning to ;
make any connec~ons. For the preferred embodiment, the default setting is exchange. (If desired, the default sefflngs and switches could all be reversed andl appropriate adjustments made to accommodate ~e reversed order. Such adjustments would be principally to reverse the input lines to each switcb.
The switch itself may require redesign.) Whcn a message arnves at any switcb, the heador hav~g a destinabon address will be routed along the row corresponding to the source from which it is sent. Thus, if it enters from input port or node 1, it will be routed along switches 1,1; 1,2; 1,3; .. l,m. The message wiD be accompanied with a ~ -se~ate, active signal which will be denoted for the purposes of this explanation "turnn. The label on the switch will be compared with the - des~nabon address in the header of the message at each switch at which the message arnf ves. If ey are ~e ~same and the turn signal is high, the switch will be set into,~ for ~ample, the pass mode, and the turn signal will be reset by the ~switch (see Fig. 73. If a destina~don label or indicator does no~ match .
~; ~25 ~ the address of ~e switeh, or if the tum signal is low, the switch will remain in .
- ~ the default modo (in the preferred embodiment, the case exchange mode).
Thus, if a message is to be transferred from node 1 to output node 3, at switch 1,1, it will be told to exchange to switch 1,2 across line a'. The turn signal ' ` ' wo s3/o3582 Pcr/us92/o6653 c~ 3~ 12~
will not have been reset since it is ~o continue to t~avel across this row. No ~ -:
matcb has been made so the switch will remain in an e~change condition.
On arriving at switch 1,2, the message header will again be compared with the address of the switch. On again finding no match, and the turn signal :~
again not reset, the message again will be e~changed through to switch 1,3 on line a'2. When the message reaches switch 1,3, the turn signal will be reset.
When the conhol signal is provided to switch sw (see Fig. 7) b~cause the ~ ::
address matches, the switch position will be changed to pass, and the message ;~
header and ensuing message will be ~ansferred from input line a'2 to output line b'3 of switch 1,3. The same control signal C, will reset the tum signal to ; ~ : .
its opposite.
There are seYeral ways to reset turn signals when the message is ;
complete, such as resetting the entire network toge~er at con~rolled intervals determined based on predeterrnined message sizing. However, the inventive concepts~ herein will work with nume~ous global or pass message reset - ~
techniques so these are not explained further here. `
Refe~ now to Fig. 3 in which a network 30 ~s shown having nearly the identical layout as the network of Fig. 1. lhe difference here is that each one of the input nodes broadcasts its message across all of the columns of switches - ~
2 0 ~ so that; each one may compare its address at ~e same time, providing for much ~ ~
. .
faster~connections. These broadcast lines are indicated as lines 31-34.
Return Networks The key concept for elimina~ng the ne~d ~vr a conflict resolution device is the provision of the return network for each input node or line. A
~ ~dmplified implement~tion is described with reference to Fig. 4 in which a crossbar network 40 and an associated retum net 41 are shown. For each node 42 or input line 1, an output as well as the input line is provided. A
modification is also required in the header: both the destination address and the source address are required in order to route back the message which has WO 93/03582 Pcr/US92/o6653 2 ~ ~ 3 ~ d ;~ 3 ~
found a contict. Each should also havbe its own turn si~nal. The ISrst part of the header should contain the des~nation address in the pre~erred embodiment. ;
Thus, for a message from node 1, it is ` 1' of 1 to m outputs. If all goss well,~he header routes itself to ~e proper ou~ut as described previously.
However, if the destination output line is already busy, the header will be ~ -diverted to the return network 41. Consider, for example, a case where input node 42 using input line 1, aind a second input node using input line 2, have connection requests for output nocte 3 (and thus send messages with headers to nocle 3). The header from the second node, for example, reaches the switch 2,3 before the header from nocte 42, if the two headers are clocked out together. Thus, switch 2,3 will be set (in the preferred embodiment) into the pass mocte by the header from the second node and, therefore, the header from nocte 42 (input line 1~ on arriving at nocte 2,3 will be routed through switch ~;
2,n into the return net 41.
The switches in ~e return net wïtl look only at the source a~tdress part of the header and its associated turn signal. Since the source has sent a message request, it should not be busy when it receives the request back, in this case across tine 44. To get there it will have come through switch s from switch t. The routing through the switches in the return net may work exactly ~ ~ i .
the same way as in the forward net, with a separate turn signal for the sending ~: ~ddress, which;is only reset when the switch address matches the origin addIess, and it then stays in reset condition through the ~switches in that ~ ;
(diagonal) return switch line until the header with its message (if any) is -~
I . ~
~; returned across one of ~e long honzontal lines in the crossbar (see Fig. 15~.
2s ~In cases where it is~b~eing routed back to a previous stage crossbar, the turn signal must change again before it is processed in the lower stage.) See Fig. 15 (or Fig. 4) for a complete 4 x 4 array of switches 150 and an associated retum networ~c. If Node 1 sends a message Ml at the same time Node 2 sends a message M2 the routing through both the cross~ars and ~he W0 93/03s~2 Pcr/us92/o6653 return net lSl can be ~ced on this diagram as labelled by following the M~
and M2 labels. (Note that to ~e right of dotted line 153, the switches may be shorted permanently if a different b~rn signal scheme is used than that describ~d in ~e preferred embodiment.) (You can also eliminate the :
connection from the topmost IOW switch in the crossbar to the return net and the lower right corner switch in the reh~rn net beeause no output from row 1 will ever need to be returned unless the crossbar receives messages back from a later stage crossbar by bus.) Th~Clos ~mplemçntation This self-routing algorithm may be applied to any N x N Clos network with r n x m crossbars in the first and third stages, and where m=N,~ m rXr -crossbars in the second stage. Also, because we are interested in TNBs, m 22n-1. -Previously, solving a self-routing crossbar network problem in a dis'aibuted manner, permutation of the rn requests such that the requests (to route a message) routed to addresses in different crossbars in the third stage are ~outed from different crossbars in the input stages. See, for example, Lin et al, T~Dimçn~ional Optical (:~los Interconnection Network and Its Uses, Applied Optics, Volume 27, No. 9, May 1, 1988, pp. 1734-1741. This solution,~ how~ver, requi~es perrnutating or rea~anging all nr requests ~simultaneous]y and cannot be implemented in the self-routing distributed manner used; by this inven~n. ~ j f Ihe self-routing approach described previously works fine for crossbars but not for Clos net vorks. The important inventive aspect for Clos networks .~ ,.
2 s for achieving non-blocking self-routing is in selecting the crossbars to be used in routing the requests through the second stage of the network~ Therefore, when conflicts are detected in the first n shared crossbars of the second stage crossbars, these messages are re-routed using the remaining n-l crossbars in the second stage.
W0 93/03582 2 ~ 1 3 ~ ~ ~ P(~r/US92/~ 3 "G~eedy" Se1f-Ro~ting Al~onthm For Multista~e or Clos Networks ;
Refer now to Fig. 9 which illustrates a crossbar network similar to that illustrated in Fig. 8 but srnaller. Each input line C1 has an address, in this network being a 16 X 16 Clos network there add~esses being (for base lû, 1 ~ ~-through 16 but as shown) for base 2, 0000 through 1111. I~e destinadon addresses in column C3 have the same designation and C3 is likewise comprised of the same number of crossbar switches. C2 has a greater numbPr of smaller crossbar switches in accordance with the requirements of a Clos -network. ~he crossbars in columns C1 and C3 are of size n by 2n-1). For 10 ~ the pulposes of the preferred embodiment if the number of crossbar switchesin C1 and C3 is not a power of 2 (thus r is not a power of 2) the number should be inc~eased until a power of 2 is achieved.
l~us it ean: be seen that, for ~e preferred embodiment, with four digits the: number of bits and the label for each address is I log2r ~
In the preferred embodiment again, no switches are considered latched . :, ` thus they are all in the pass position. ;
When any message arrives at any switch, the header containing the : ~
~: ~ destination addressisrouk~d following theself-routing algorithm described ~: ;
above for the first column. ~ The destination address~will bc nude up of two ~: ~20 ~ ts: ~ the output crossbar:address of llog~l bits and a lo~al destinadon address ¦log2nl~ :bits vvithirl ~e indiYidual crosslbar. This matching is .
~'` '~ i'~l .;``' ~'~ f ~ formed~onlycn thefirst llog2rl bits.
:If ~e messago header finds that the desired address ( l log2r l, say OO -) ~' ' . ' output is busy in the first of the inte~nediary switches in column C2, it will be -:
2 5 ~ ~ : routed to the next intennediary switch and so on until a ~ath to the cr~ssbar switch in column C3 desired is achieved. Thus the~ algorithm is terrned greedy in that any unused column may be used for routing. In the illustrated e~ample ::~ of Fig. 9 the address 00 - was free and a connection made on 9 l to the second - ~
stage crossbar is C2. The other message that wants this crossbar is coming in : j: :
~ ~.
Wo93/~3~8~ Pc~r~us92/o6653 16- ~ ~ -on 92. If they are going to different final crossbars as shown, across lines 93 and 94 there is no problem, otherwise the second message ~hrough will be blocked and have to be rerouted. This is why the greedy algorithm was created. ' .
The key to resolving conflicts in each stage is the use of the crossbar with the return network for conflict detection and resolution. Stage 1 and 3 conflicting requests, due to tme input request conflicts (where two input reques~s specify the same output address) are returned to the source node.
Ho~ever, conflicts in the second stage or second column that arise due to shaling of the crossbars in the second stage are resolved by detecting conflicts ~ ;
in those labelled crossbars and then, on receiving the return header, scanning available connections in the remaining second column crossbars. In the worst case then, a ~equest may scan through n outputs in its stage one or Cl crossbar before its request is met by the (2n-l)th output.
Another way to say this is there are two forms of 10 conflict that can occur, the first occurs in the first or third stages due to direct input conflicts.
This can be where two or more inputs request the same output address in the third stage or subaddresses in the first stage. These are handled by tbe conflict resolu~on as described in the case of the single crossb~. The second Idnd of 2 o c onflict aIises due to ~e ~aiing of crossbars m ~e middle ~tages and requires ; ~ -sc~ning the ~st n~ e crossbars to resolve the conflicts.
Because of ~ese multiple stage conflic~s, crossbars 2û in the second or middle and ~ird stages must provide return paths so that when the header part of the message is sent, the response can be sent back to the source to confinn that a routing path is available. Thus, the multi-stage network acts more like arouter ~an a pure interconnection network. ~ ;
Please refer to Fig. 10 which describes such a system, including the ;
return nets and extra return paths Wo 93/03582 ~ ~ ~ 3 ~ 7 PCr/USs2/~653 Again, if light switches are used, retum paths may be obviated because the light can ~avei back over ~e same open switch that it traveled to get to ~hedestination node. It should also be noted here that electronic switches with -~
. ~:
return paths opened at the same time and location as sending paths would also ~ ~ -obviate the necd for a return net o~ separate return pa~ system.
Crossbars la to ra establish ~e first stage, lb to 2n-lb ~e second and lc to rc the third. Each has a retum net 102, similar to the one described in detail with reference Fig. 15. For eve~r connection between crossbars like line 103 there is a return line like 103r. Please re~er briefly back to Pig. 15 ~o and line 103r thereon.
.~ . .
For ease of explanation, assume the network is populated with 4 x 4 crossbars in the first stage like 150 of Flg. 15. Output 1 goes to the first crossbar in the second stage, output 2 to the second, etc. The line returmng from the first crossbar of the second stage would be line 103r, ~rom the second crossbar of the second stage would be line 104r, from the third crossbar of ~e s~cond stage would be return line 105r, and from the fourth crossbar of the second stage, the return line would be 106r.
Reviewing the two message state of c~ossbar 150 described earlier, two : . , ,.. . ~
- ~ messages, Ml and M2, have ~ied to reach output -3, only message M2 made ~ it. ~ Ml was reùlmed. Let us assume M2 did not make it through the third ~ ;
c rossbar of the second stage. It~would~then be returned on line lO~r. Switch 3 in ~e firs~ row is still held in the pass mode and the network message is, ~ ~ `
~routed through across to the retum net. ~e return net routes ~e message back to node 2. (All switches, once changed from the default state, should be 2 5 held in that state for the maximum delay required for the message to return if ;
blocked at the last stage.) M2 then would be routed through the second row, 4th column switch, into the return net which would forward the header back to node 2, using the return address. As in the explanation which follows, destination address can then be incremental and the header resent.
".' ~.. -.`.,, Wo 93/~3582 PCr/US92/06653 Figs. 11a, b, and c detail the number of lines used in each of the crossbars of Fig. 10.
The remaining detail is how to scan to the ne~t or subsequent second -stage crossbar when the original choice is blocl~d. When the message is returned to its onginating node, the header's destination address is simply incremented and the header reset to ~y the next second s~age switch. The incrementing is only done to the f~st part of the destination address, however.
So, for example, (with refaence to Fig. 9 assuming it has return nets), if output node at receiving address 0000 wants to send a message to a receiving ~:
node at 0000, the incrementing can be done to the first two digits of the 0000 address, maldng ~e next attempt through line 01-of c~ossbar 9S. This would send the header to the second crossbar in C2 (not show~) whose switches would look at the second part of the destination address, thus ~outing~ the ;~
mesxage to the u~rrect stage 3 or C3 crossbar, in this case crossbar 96. If a : :
third~ crossb~ in s~e 3 is tried, because the second part of ~e da~nadon address remains unchanged, the message will still be outed to crossbar 96 if the output address is 0000.
:Use of Busses ~
~-,. ..
Instead of Cloi networks having an intermediary stage or set of st es of ~20 ~ crossbar~s,~they can use~busses~for routing messages. This~embodiment fur~er reduces the number of switches but;adds complexity in that it lequires :bus ontrollers., Refer to Fig. ~12 in which:a networlc 120 is shown~ having again l through r erossb~rs with return ~nets- for both tile first and third stages. Again, 2s ~ ~:: as~in Fig. 10, ~e return lines are shown attached near the sending line even ; ~ though they are located as described with reference to Fig. 15 on ~e side :
opposite the output. The difference in structure between this Fig. and Fig. 10 IS that there are 2n-1 controllers 1211 - 1212n l. Each controller itself has a : set of r data bus controllers 1211d - 121rd. ~
~: ,;~., .:
WO 93/03582 ~ ? Pcr/us9~/o66s3 The algonthm, however, worlcs about the same way on ~is structure as it does on the ldnd illustrated in Fig. 10. Thus, a signal or message sent from node 1 of crossbar 123, passes on output line 1 to Data and Bus Controller 1211d. If the message is supposed to go to crossbar 124, for one of its 1 to 2n-l outputs it will try line 1 of its set of busses. If ~here is no "busy" signal set by some v ou~put on line a of bus 19 the message will pass onto the bus lineb on output w. These v and w outputs are bi-directional (or may be paired inputs and outputs, thus 2 v's and 2 w's if desired).
If this bus line is busy, the message will be returned to node 1, incremented and resent on line 2 (not shown) to the next con~roller. The next controller 1212 (not shown but next in series) would try line 1 of its first Data/Bus Controller, which also connects to crossbar 124. This would go on until line 2n-1 of crossbar 1231 is tried, and a link attempted through Data and ~
Bus Controller 1212n ld is tried and its line 1. If s~ll unsuccessful the next ~ ~ :
increment can reset the header to try line 1 and start~the cycle over.
. . ...... ..
Enhancements to ~is concept, such as sending a signal backward beyond the - input node to cancel a message or resend later, should b~ apparent to one of ordinary shll in the art, without going outside the scope of this invention. ; ~
(Note also that there~ are single input/ou~put lines betwéen busses and v 20 ~ crossbar 124r and paired input and output lines between busses and crossbar - ~ 124,~ In~e drawing. This is~done~to show ~at it co~d be either way, - ~ , although it would be preferred if all ~e c~ossbars in a stage iare coMected~ to ; busses in the same manner).
The data/bus controller is next~descnbed for completeness.~ Refer first ~; ~ to~Fig. 14 in which a bus controller 140 is shown. This is essentially anexploded view of the box labeled 1211 m Fig. 12. However, the diagram also Dlust~ates a variation in embodiments in which a demultiplexer is used rather than an iDpUt and output line from and to each one of the first stage crossbars and thei~ return networks. Th:e input data coming in on line 141 is -: : :,. ,~.:.-W0 93/03s82 Pcr/us92/o6653 20~
demultiplexed by lool~g at ~e destination address, first part, and the request is sent to one of the bus control us~its, say bus con~ol 1. Bus control 1 ~en needs to seize databus 1 in order to send ~e message as data across the data line to databus 1. To do so it hrst checks the status of bus status 1 to determine if databus 1 is busy. This is done on line status l. If it is not busy, it places a high signal or other appr~riate signal on bus 1 status line across the set/reset 1 line of bus control 1. If the status is busy, bus control 1 willrerout the request out the lesponse line which is wire-ored to line 142. A tum ;i~
signal will be tagged onto this returned message so that it may find its way . , baclc through the return net. As was described in the earlier implementation, ,-.~: . ;, such a turn signal f~r return coming from a inter nediary level or s~ond stage crossbar was not required because each line went to the appropriate address in theretum net. ~ ~
FSM of the Bus Coll~rQl ~ ~ -1. Inputs: ParallelInputData/Headerbus,theBusStatus(0/l), and the Next_Stage Response, a single packet of data equal to the bus width, usually a header bit, and ehe source and destination addresses sent back from ~ i the next stage. The input Data/Header always contains 2 reserved bits for roubng.
. .
zo ~ 2. Qh~s: Dat~Bus, paralle!outputheader/datatothenext ; ~ stage,~ Set/Reset control for ~e bus~status line, Res~onse_Baclc lines that contains a bit (available or note available) ~ sourc~/destination information which may also be obtained forrn the Next Stage Response.
Note the bus lin~es are tri-state ~ow, high, high impedance) to 2s~ distinguish between no signal and ~valued signal.
3. State T}ansitions:
IF Input Data/Header bit = 00 /*leading bit denotes header or message packet*/
AND Bus Status = 1 I*Bus occupied or busy*/
; ,,`' Wo 93/035~2 PcrJus92/~6653 2I ~ 3~7 ~
THENResponse_Back = O!InputData/Header;
/~send a not available signal in respon~e*/
ELSE : ~:
Response Back = 1!InputDal~
/*send available signal in ~esponse~/
Bus_Status o 1 /~set bus status busy reserve the bus :
lines*/
ELSE IF Input Data/Header bit = 10 /*message packet received after the source received an available signal*/
~ Data Bus = Input Da~a/Header /*output bus = inputbus - transmitdata*/ ~ :IF Input Data/Header bit = 11 /*special code for end of message*/ `: :
~IEN Bus Status = O /*free the bus* /
Data Bus = XXX /*set bus back to high ~impedance state*/ :; ;
` We note that this implementation in routing through the second stage .
assumes that the source node operates in two modes: first it sends out the requesting header packet. }f the rosponse received from the routers is ;.
:egalive, itretransmits the headers until it obtains a connec~on. Second, if a : posi~ve response is received, ;the sourrc node transmits its message in packets ; ~ :
a o~ ; unt~it;~s~`heiat hesswi~hesint ecrossbasor~ebus sithadu d.
RQ~tip~ys We ~will only consider the average delays in~ set~ng up the routing path :~
in the bus-based;self-~outrllg Clos net~vork. The Average:delays in the first and , ,. .. '.. "'' .',',' ird~ stage as before i9 (n+ 1) for th~ Xl implementation and~ n/'7 for the X2 ~ :~
2 s implementation. The delay in the second stage is not as simple. Consider the delay in determining a conflict using the Data/Bus Controller architecture.
From Pigure 14, it will be clear that determining tXe availability of a .
connection is tA (access bme) = (log2r+k) where log2r is delay in the demultiplexer~while k is a constant due the latching delay in the FSM of the ;~
WO 93/03582 P~r/USs2/066~3 bus control and response logic.- This delay tA is repeated at most n times if inthe worst call all free n crossbars i~ the second stage are scann~d. Thus, ~e worst case delay in set~ng up the rou~ng path in this implementation is 0(2n~nlog2r). Once the routing path is set up, the switching delay is smaller -since no conflicts ha~e to be further resolved. Since the optimal Yalue of r=n~N, the worst case delay in sefflng up the routing path is O(~N(2+10g2~/N))(onlyO(~fN)forllormalroutingdelay). Webelievean optical implementation may have lower delays than the electronic one when implementing theswiteh version.
Broadcasting in CN's follows very simply from the above self-routing scheme. To implément broadcasting CN's, broadcasting switches discussed above must be used for the in~dividual crossbars in thg CN. The only difference in the broadcasting is the limit on ~e maximurn number addresses to~which a message~can be broadcast. I f the original Clos design is used, the limit is (n-l). This limit is imposed by the maximum number of c~ssbars ; ~ , used in the second stage. ~ ~ ~
, , : :`
:i .' .
. '. `': ''~
:
: '
-1- ' "'' ~
(~ROSSBAR WlT~ ~TURN N13:T ~R SCALABL~ SELF-ROUTING ~ ~ :
NON-BLOÇKIN(;~ l~ESS~GE SVVll C~ AND ROUI~NG SYSTEM
This invention relates to a self-routing, non-blocl~ng switch system for routing messages f~om any one of a number of inputs to any one of a number of outputs in an efficient and ccst effective manner.
BACKRO~ND QFT~IE~NVENI~ON
As data ~ansfer becomes more and more complex, the numbers of the messages which need to be t~ansferred across a nehvork from one location to ~ ~
another, or one area of a computing device to another, has grown remarkably. . ::
Accordingly, any system or subcomponen~ thereof which allows ~or ease of routing messages from any of a large number of inputs to any palticular~one of a large number of outputs is imponant. The invention: described herein allows for the routing of such messages wi~h very little overhead, at high speed.
The invention herein empioys a reh~rn netw~lc, is capable of using ;
different hnds of switches In the switching network, may operate in a ~ -broadc~ ast mode, may reduce the number of switches necessary by using a Clos network or by reliance on busses and controllers, and uses a self-routing algorithm, elinrinabng ~e need for~p~ocessing overh~.
2 0 ~ Self routing algorithms use information contained in the message itself ;
t~;route the message;at th~e swi~h~l~vel to the desi ed location. ~
Prior self-routing alf,orithms such as ~e ~one dosGribed in the,1~90 IEEE (Vol. 78, No. l),~entitled 'Pastp~çket Switch Archi~for ~
Broa~-Band Inte~rat~d~Sçr~ices Digital Networks", by Tobagi, do not indicate how these algo~ms can actually opeRe~ and do not descnbe how their ~ "
operation can be accomplished when using a Clos-tS pe networlc.
Clos networb ~e prefelTed for larger networks b~ause they reduce the number of switches necessary to accomplish a N x N message switching ;
system to a much smaller number than N2. See, for example, U.S. Patent , :.. :~ ~
..
WO 93/03582 Pcr/US92/~6~53 No. 4,696,000, issued to Payne, which refers to ~e Clos network and -rea~angeable n~tworlcs.
The inven~on described herein simplifies all these ideas.
. . , ,. , , ~ . :
Th,e switches may be arran,ged, in a standard crossbar network for small scale implementations, a Clos network or a va~ation of a Clos network using busses, and the types of switches and routings, dependent on ~}r,,e switch type -:., :, ~.. .
used, may also valy depending on the implementation. As described in the ~;
detailed description, a preferred implementation employs a return network for all blocked, messages that wish to address the same output node and returns lo them to the source or sending node. Due to the nature of the routing system ~-itself and the construction of the switching network, essential,ly no overhead or ` -preprocessing is required to route messages through the sys~m descnbed from the input nodes to the output nodes.
BRIEF DESCRIPIIQN OF l~IE DRA~INGS ;
Fig. 1 is a tw~dimensional layout of the switches in an N x N ;
crossbar. i ~ ~
Flg. 2 is a block diagram of a trap network for removing connection ; ~ `
request conflicts.
, . . . .
, : ~,: :...
Fig. 3 is a layout of a second implementation of an n by m crossbar.
2~ Fig. 4 is a layout~of a~ n by m crossbar wi~ an as~ return : ~ ~ , . .
network.~
. Sa is a~hema~c ~diag~ of 2 x 2 cro~b~ $witch.
Fig. Sb~ is a diagram of a gate-level implementalion of the switch descnbed in ~Fi~. 5a.
~ Fig. 6a (i), (ii),~ and ~l~i) are three possible m~des~ of operation of the 2 x 2 broadcast switch.
Fig. 6a is a diagram of a gate-level impl,ementation of a broadcast switch.
,,, ''~, ;' "
. .. ..
.. . .: : , Fig. 7 is an electroni~-level diagram of an implementation of a crossbar swiuh.
Fig. 8 is a 36 x 36 Clos network diagram.
Fig. 9 is a 16 x 16 Clos network diagram.
Fig. 10 is a layout of a non-blocking nr x nr self-routing Clos network with re~urnnetworks.
Figs. 11a, b and c are diagrams of the inputs and outputs of the crossbars used m.Fig. 10.
Fig. 12 is a schematic diagram of the bus implementation of a truly non-blocking network in accord with the invention.
Fig. 13 is a block diagram of a finite-state machine model of the bus controllerwhich may be used with the invention as described in Fig. 12.
Fig. 14 is a detailed schematic diagram of the databus con~roller blocks .: . . , employed in one of the 2n-1 databus controller blocks used with the invenuon as ;~
described with reference to Fig. 12.
Fig. 15 is a drawing of a complete 4 x 4 crossbar wlth associated return net.
~ ~ Fig. 16 is a block diagram of one message embodiment M for the invention havmg a header H and a data porlion D.
~ ; SUMM~RY OF THE ~lENTIoN ~ - ;
Crossbar With~Return Nçt This invention teaches a crossbar for use in a switching system w~uch switches messages and returns ~messages which do not!get through, on the first ~. I has ; headers in the messages wherein the headers have source and destination address and -25~ ~two~turn~;signal indicators~ which are bc~th set at the hme the message is sent. The crossbar has a number of inputs, n, and a nu~nber of outputs, m, wherein the m inputs are connected by an n X m array of 2 X 2 switches organized into logical colurnns and logical rows labeled 1 to m and 1 to n, respectively, and wherein the swltches are comlected between the n input sources and the m output destinations. The two-by-two -~
switches W0 93/03s82 ~ Pcr/us92~ 3 5 ~ 4 ~ ~ ~ :
may be set into a pass or exchange mode (but are all set into the s~ne mode, called a default mode, at the outset) based on the coincidence of two events, first ~at the message header des~nation address matches the column address of the switch and second that the turn signal has not a Iready been YeSet by a previous switch having a matching column address. Ihe crossbar has in communicative association therewith a ~eturn network ha~ing similar or identical 2 X 2 switches organized into logical diagonals with respect to the logical organization of the crossbar switches. The longest diagonal having the highest number of switches has addresses in its switches matching row 1. `
o Succeedingly smaller diagonals have succeeding larger addresses, and less switches. The ultimate diagonal has only one switch of address "n" (The longest diagonal may be shortened by one switch, as mentioned in the detailed description.) ',, These 2 x 2 switches are also setable into pass or exchange mode based on Ihe coincidence of two events, bu~ all set into the one mode, called default mode, at the outset. The f1rst event is that the message header source address '~ ;
matches the diagonal address of the return net switch. The second event is ' ,' ~ ,~,;' ' that the turn signal has not already been reset by a previous switch with the , ' , same column address.,, ~EA~ DES~I~ON ()F I~IE INVENIION , ~ -, , j~ Several conce~ts a~e, included he,re ,which are all centejIed on,the idea -, ~at ,self-routing algonthms may be employed to provide local control of 2 x 2 , ~ ' "~ switches which comprise various networks. In doing this, the n~ for a 2s ~ , global controller mondtonng all of the inputs and all of the outputs, and ~elr ; ~, ;
connections, is not required. Where plain crossbars are used, a total of NM of ' ~
*e 2 x 2 switches would be required for an N input and M output network. ,~ ,'" ,' ~ ' Because the switches are relatively simple in functionality and layout of the ' ' design would be extremely regular, such networks can be fabricated easily in . ~ , .
WO 93/03582 ~ ~ ~ 3 6 ~ ~ PCr/USs2/06653 electronic as well as optical implementations. Using a Clos variety of truly non-blocking switehing network, even the number of switches may be substantially reduced. Even further reductions in ~e num~er of switches required is possible using the bus design. This kind of savings improves the scalability of the inventive concepts described herein.
For e~sample, in a 36 x 36 netwo~k using a s~ight crossbar design, 1,296 2 x 2 switches would be required. Using a Clos network design, only 1,188 switches would be required. When moving to a network of approximately 128 x 128 design, 16,384 switches would be ~equired, whereas 0 wi~ a Clos design, only 99936 switches would be required.
An additional and important part of this set of inventions is the use of a return network. The return network operates to return messages that attempt to ~oute themselves to outputs which are already occupied. l~is re-routing or routing back is automatic, and takes less time than a bme~ut, especially in larger networks, to inform the sending node tbat ~e message has not gotten ~ ;~
through. (A timeout is a counter or timer that waits some increment longer than the maximum delay, and upon finding the time expired or the counter full, a new action is begun.) In providing this description, some terminology ought first be defined.
20 ~ ~ ~ The~firstdef~initionwouldbenTrulyNon-Blocl~ng~Networks" (lNBs). A
truly non-b ~ing ne~rk is one in which reanangement of existing , ; conditions is not requi~ed in order that ,every message ~get throug~. It ~equires a set of features, including ~e follow~g~
(i) set~ng the appropriate network switches for all requested ;~25~ connections, (ii) ensuring that no conflict occurs (a conflict occurs when ; -multiple inputs are connected to the same output), (iii) in the event of a conflict, only one connection may be ~ -satisfied while the other requests must be lcept walting or informed, ; - -'"~ ' `
WO 93/03582 ~ Pcr/us92/~6653 - 6 ~
(iv) in case of broadcasting networks, the control must also allow a single source to be connected to mul~ple outputs. ~` :
The inventive concep~s desc~ibed in this ap~lication show how TNBs ean provide for practical implementation for high ~peed and high band width ap~lications using a distribut0d local routing control. (l~ey are principally ~;:
oriented toward elec~ical switching but additional benefits can be gained using vptical switching, such as no sepa~ate line being required for a return signal.) ~:
The crossbar switch in general can be thought of as a square or rectangular tw~dimensional array connecting a set of inputs to an equal or, if rectangular, unequal, number of outputs. Thus, for N inputs and M outputs, a crossbar has NM crosspoints and can simultaneously provide any combination of input and output one-t~one connections in a non-blocking manner. In practice, other methods besides using an a~ray of switches, where each switch rel)resents a crosspoint, have been used. These include for example, in ~ ~ -electronics, bus arbitrated architecture used to simpli~y the quadratic complexity of switches and control and, in optics, outerproduct matrix ;
multiplying architectures have been used. The primary approach here is to use switch level design for self-routing. `
A TNB that requires less switches than are required in a crossbar was 20 ~ first proposed by Clos in ~Studv of Non-Blocldn~ Switçking Nrt~o h, Bell ;;
System Technical Journal, 32, ~. 4~24, 1953. A general Clos ne~worlc (CN) bas an odd numb~r, say, 2k+1 (where k-1,2,.~...), of stages ~nd is built in a modular desigh e ploy-ng a ~multiplicity of c~ossbars which are :
substan~ly smaller than the total N x N size desired. Thus, the number of 25 ~ ~ switches which would be r~equired for a Clos network would be on the order of ~N x ~/N, when a comparably sized crossbar would be N x N. (See above paragraph.) Also, networks of more than three stages can be built from -.. ,.. ~ ;. .
three-stage networks by successiYely replacing the center-stage by another three-stage CN.
.
~: :
WO 93/03582 Pcr/US92/o66~3 2 ~ 13 ~
It is hlown ~at in a N x N Clos network, if ~e input stage is built of r crossbars of dimension n x m where rn=N, the second stage is made up of m, r ~ r crossbars, and the third stage is made up of r, m x n crossbars. The size of a crossbar can be designated with the sequence (n,m,r). ~e Clos network will be ~uly non-bloel~ng if m 2 2n-1. The CN is not truly non-blocking but may be rearrangeably non-blocking if m < 2n~
It is not particularly desirable to have a rearrangeably non-blocking network in that messages may be required to be interrupted to rea~ange the network when such rearrangement becomes necessary. Por applications, therefore, requiring long duration or large messages, interruptions are extremely undesirable.
Refer first to Fig. 1 in which a simple c~ssbar network (10) is shown. ~ -It hlas inputs 1 through n and outputs 1 through m as shown, and each one of the 2 x 2 switches may be labeled as points in a mat~c or, 1,1 to n,m. The switchos may be label~ with their column address only since ~is is the one . . which will be compared wi~ the message header as will be explained later.
In Fig. 8, a Clos network is drawn for a 36 x 36 array of inputs to out~uts. The network is comprised of 6, 6 x ll crossbars in column 1, plus 11, 6 x 6 crossbars in column 2, plus another 6, 11 x 6 crossbars in column 3.
2o ~ Thus, an input on line 81 may be directed at switch or c~osspoint 82 to - .. connect ~to: the first top line 83 of crossbar 84 in oolurnn 2, row l, which may .
again ~e di~d by sw~tch 85 to output 86 of the sarne crossbar9 c~nnecting it ~ .
to inpult 87 of crossbar 88. At ~is point it m~y on!y~be rout~d to one of ~e six outputs of c~ossbar 88 by one of the six switches in that column of crossbar~ 88. It will occur~to the reader at this point that three crosspoints are mvolved ;. . ~
in transferring the message from input 81 to one of ~e outputs in cr~ssbar 88. . ~ :
The Clos network, it will also be noted, is generally cons~ucted having square root of N rows ~lus m rows of center crossbars, not labeled~ by three columns. The network may be expanded and the number of switches further - -W0 93/03582 Pcr/uss2/o6653 c~ .t~ - 8 ~
~, ~
reduced by splitting the central portion into three in acc~rdance with the ; .. - ;. ,:
teachings of Clos.
Refer now ~o Fig. 2 in which a trap network scheme is used to remove conneetion ~equest conflicts befor~ they happen, according bo the design 20. ;
s This design is avoided by the invention. In Fig. 25 inputs 1-n are provided to a parallel sorting network 21 and provided, sorted, to a compalator stage 22.
The comparator stage 22 provides an indication of a conflict to the duplicate ;
router stage 23 along with each message being passed. The duplicate router stage 23 returns messages, along lines 24 or 25, to the inputs prior to the ..
s~rting network a, to be handled in accordance with whatever scheme is desired. For instance, either the message header itself may go back to ~e input node where ~e message header and the message may be directed to a buffer, or any number of other things may be desired for messages which may require resending at some other ~ime. The crossbar 27 ~en allows its switches : to be set in accordance with the instruc~ons carried in the header which indicate the output address for each message routed through it. Using the teachings described herein, such an implementation, including units 21, 22, 23 ~ :
and outputs 24 and 25, is not required.
. ;
~ ~ Switches 2 0 ~ Before mo~ing on to a: description of the algoAthm for self-routing, and . : .~ because it would create some cornplicttions to describe ~em later, a ~;
descnption of the switcheslwhich may be used with this device, or at least their functionality, is now descnbed.
While this invention may be~well suited to light switches, only the ?5 :~ electronic versions are shown, however9 it should be clear to one of ordinary skill in the art that light swltches will prwide enhanced flexibility allowing for ~;
return signals over any path once established. Higher speed communications, higher bandwidth and so forth become available when using light switching :' -' WO 93/03582 P~r/Us92/o~6~3 2 ~ ~. 3 ~ ~1 7 devices. Also, by not requiring messages to get a return routing, the return network desc~ib0d herein may even be avoided when using light switches. :
The basic 2 x 2 switch can be seen in Fig. Sa, 50, as hav~ng two inputs, IN 1 and I~ 2, and two outputs, OIJT 1 and O~ 2. It may be ~ :
generally set in either a pass or e~change mode. Under the pass mode, lines :~ ~:
51 and 52 will be open and lines 53 and S~ will be closed. In the exchange mode, thereverse1s~ue.
In Fig. Sb, the switch 50 is shown in logic diagram form. The "change" in input 55 will control whether the switch is in the pass or exchange mode. Thus, when the input to SS is zero, OUT 1 is open to I 1 and closed to I 2, and OUI 2 is open to I 2 and dosed to I 1. When SS is one, the reverse is true. OIJT 1 is open to IN 2 and our 2 is open to IN 1.
Refening now to Fig. 6a, three modes of operation of a brbadcast sw;tch logical connection are shown, i, ii, and iii. In mode i, one input may ;
s be ~connected to both outputs. In mode ii, the inputs::are connected to their ` ` ~
direct outputs. In mode iii, the inputs are connected to their crossed outputs. ~ :
In Fig. 6b, a gate-level implementation of such a switch is shown.
Thus, where input 66 is zero and input 65 is hi (Gr "one"), the output of a' :
will be b and not a, and the output of b' urill be a and not b, thus, a switch 2 o condition. If input 66 is ~æro, or low, and output 65 is also low, there wlll be ~ -a bwadcast pass aondibon in which both outputs will be b. ~ - -Lik~wise, if ~e inpu~ 66 is hi and the output 65 is lqw, the ijnRut b u ill :~ be passed on a', iand the input b will also be pasised on the output b'. In ~e ,-.
case where both 66 and~65 are hi, the output of a' will be b, and the output of : 2s b' will be b, another broadcast condition. -~
The switch may be redesigned to eliminate broadcasit conditions if desir~d. As is also true with the gate-level implementation of the switches described previously, this switch may be designed differently as is well h~own to those of ordiniary skill in the art.
- ,- ~ , Wo 93/03582 Pcr/uss2/o66~3 ~r$~ - 10~
In set~ng the switches,.or not seffing them, or setting them into a broadcast state, this invention re~quires reference to the intended address of the message. ~us, it requires a d~vice similar to that described with reference to Fig. 7, 70, which contains the 2 ~ 2 switch, sw. Input data containing the s address is received by ~e crossbar switch device 70 across datalines 71 and ~ ; .;
72~ Depending on the overall implementation, either a serial or pa~allel input may be desired. Either way, a delay buffer 73a and 73b is provided before the data reaches the switch sw. The address label part of the message is read into the address compa~ator 74 at line 75. The signal output by comparator 74 on .
line 75 is positive if the switch address matches the appropriate part of the .
destination address. Such a positive signal on line 75 latches And-g~e 78. In : ~; :
this way, if the And 78 is not disabled by line 76 or reset by a signal from line 77, the control output will provide input C to switch sw.: The C signal also . ~
:: changes the output on 77a fIom "turn" to "no-turn' by means of latched And . . ;.; . . ;.
~ : 78. ~hus, all succeeding switches would receive ~e changed turn signal,: .. ~,', ' ,'' ,. "'','.!;'' "' ~ allowing the message to continue down the column. :Different schema for thisswitch which perform similar fimctions may be developed easily within the skill of those of : ~:
, ,, .. ::, . . ~
..... .
.:. : : ., : ~ : ' ' ' ~ ~ .
, . .
' ~:
:
WO 93/03s87 Pcr/uss2/Q66s3 ~3~7 ordinary skill in the art. This par~cular design is preferred for the nPtwor~
and algorithms desc~ibed, but ~rariations will ~ apparent to ~ose of ordinary ~ -sl~ll in the art. For example, line 76 could be removed without any loss of function desc~ibed herein. There may be paIallel input paths if desired. May other variations may be constructed without circumventing this invention as ~ ~
daimed. ~; -The Basic Sçlf-routin~ Al~rithm Please refer to Fig. 1. The switches should all have a default setting, either pass or exchange, and all should be set in that mode when beginning to ;
make any connec~ons. For the preferred embodiment, the default setting is exchange. (If desired, the default sefflngs and switches could all be reversed andl appropriate adjustments made to accommodate ~e reversed order. Such adjustments would be principally to reverse the input lines to each switcb.
The switch itself may require redesign.) Whcn a message arnves at any switcb, the heador hav~g a destinabon address will be routed along the row corresponding to the source from which it is sent. Thus, if it enters from input port or node 1, it will be routed along switches 1,1; 1,2; 1,3; .. l,m. The message wiD be accompanied with a ~ -se~ate, active signal which will be denoted for the purposes of this explanation "turnn. The label on the switch will be compared with the - des~nabon address in the header of the message at each switch at which the message arnf ves. If ey are ~e ~same and the turn signal is high, the switch will be set into,~ for ~ample, the pass mode, and the turn signal will be reset by the ~switch (see Fig. 73. If a destina~don label or indicator does no~ match .
~; ~25 ~ the address of ~e switeh, or if the tum signal is low, the switch will remain in .
- ~ the default modo (in the preferred embodiment, the case exchange mode).
Thus, if a message is to be transferred from node 1 to output node 3, at switch 1,1, it will be told to exchange to switch 1,2 across line a'. The turn signal ' ` ' wo s3/o3582 Pcr/us92/o6653 c~ 3~ 12~
will not have been reset since it is ~o continue to t~avel across this row. No ~ -:
matcb has been made so the switch will remain in an e~change condition.
On arriving at switch 1,2, the message header will again be compared with the address of the switch. On again finding no match, and the turn signal :~
again not reset, the message again will be e~changed through to switch 1,3 on line a'2. When the message reaches switch 1,3, the turn signal will be reset.
When the conhol signal is provided to switch sw (see Fig. 7) b~cause the ~ ::
address matches, the switch position will be changed to pass, and the message ;~
header and ensuing message will be ~ansferred from input line a'2 to output line b'3 of switch 1,3. The same control signal C, will reset the tum signal to ; ~ : .
its opposite.
There are seYeral ways to reset turn signals when the message is ;
complete, such as resetting the entire network toge~er at con~rolled intervals determined based on predeterrnined message sizing. However, the inventive concepts~ herein will work with nume~ous global or pass message reset - ~
techniques so these are not explained further here. `
Refe~ now to Fig. 3 in which a network 30 ~s shown having nearly the identical layout as the network of Fig. 1. lhe difference here is that each one of the input nodes broadcasts its message across all of the columns of switches - ~
2 0 ~ so that; each one may compare its address at ~e same time, providing for much ~ ~
. .
faster~connections. These broadcast lines are indicated as lines 31-34.
Return Networks The key concept for elimina~ng the ne~d ~vr a conflict resolution device is the provision of the return network for each input node or line. A
~ ~dmplified implement~tion is described with reference to Fig. 4 in which a crossbar network 40 and an associated retum net 41 are shown. For each node 42 or input line 1, an output as well as the input line is provided. A
modification is also required in the header: both the destination address and the source address are required in order to route back the message which has WO 93/03582 Pcr/US92/o6653 2 ~ ~ 3 ~ d ;~ 3 ~
found a contict. Each should also havbe its own turn si~nal. The ISrst part of the header should contain the des~nation address in the pre~erred embodiment. ;
Thus, for a message from node 1, it is ` 1' of 1 to m outputs. If all goss well,~he header routes itself to ~e proper ou~ut as described previously.
However, if the destination output line is already busy, the header will be ~ -diverted to the return network 41. Consider, for example, a case where input node 42 using input line 1, aind a second input node using input line 2, have connection requests for output nocte 3 (and thus send messages with headers to nocle 3). The header from the second node, for example, reaches the switch 2,3 before the header from nocte 42, if the two headers are clocked out together. Thus, switch 2,3 will be set (in the preferred embodiment) into the pass mocte by the header from the second node and, therefore, the header from nocte 42 (input line 1~ on arriving at nocte 2,3 will be routed through switch ~;
2,n into the return net 41.
The switches in ~e return net wïtl look only at the source a~tdress part of the header and its associated turn signal. Since the source has sent a message request, it should not be busy when it receives the request back, in this case across tine 44. To get there it will have come through switch s from switch t. The routing through the switches in the return net may work exactly ~ ~ i .
the same way as in the forward net, with a separate turn signal for the sending ~: ~ddress, which;is only reset when the switch address matches the origin addIess, and it then stays in reset condition through the ~switches in that ~ ;
(diagonal) return switch line until the header with its message (if any) is -~
I . ~
~; returned across one of ~e long honzontal lines in the crossbar (see Fig. 15~.
2s ~In cases where it is~b~eing routed back to a previous stage crossbar, the turn signal must change again before it is processed in the lower stage.) See Fig. 15 (or Fig. 4) for a complete 4 x 4 array of switches 150 and an associated retum networ~c. If Node 1 sends a message Ml at the same time Node 2 sends a message M2 the routing through both the cross~ars and ~he W0 93/03s~2 Pcr/us92/o6653 return net lSl can be ~ced on this diagram as labelled by following the M~
and M2 labels. (Note that to ~e right of dotted line 153, the switches may be shorted permanently if a different b~rn signal scheme is used than that describ~d in ~e preferred embodiment.) (You can also eliminate the :
connection from the topmost IOW switch in the crossbar to the return net and the lower right corner switch in the reh~rn net beeause no output from row 1 will ever need to be returned unless the crossbar receives messages back from a later stage crossbar by bus.) Th~Clos ~mplemçntation This self-routing algorithm may be applied to any N x N Clos network with r n x m crossbars in the first and third stages, and where m=N,~ m rXr -crossbars in the second stage. Also, because we are interested in TNBs, m 22n-1. -Previously, solving a self-routing crossbar network problem in a dis'aibuted manner, permutation of the rn requests such that the requests (to route a message) routed to addresses in different crossbars in the third stage are ~outed from different crossbars in the input stages. See, for example, Lin et al, T~Dimçn~ional Optical (:~los Interconnection Network and Its Uses, Applied Optics, Volume 27, No. 9, May 1, 1988, pp. 1734-1741. This solution,~ how~ver, requi~es perrnutating or rea~anging all nr requests ~simultaneous]y and cannot be implemented in the self-routing distributed manner used; by this inven~n. ~ j f Ihe self-routing approach described previously works fine for crossbars but not for Clos net vorks. The important inventive aspect for Clos networks .~ ,.
2 s for achieving non-blocking self-routing is in selecting the crossbars to be used in routing the requests through the second stage of the network~ Therefore, when conflicts are detected in the first n shared crossbars of the second stage crossbars, these messages are re-routed using the remaining n-l crossbars in the second stage.
W0 93/03582 2 ~ 1 3 ~ ~ ~ P(~r/US92/~ 3 "G~eedy" Se1f-Ro~ting Al~onthm For Multista~e or Clos Networks ;
Refer now to Fig. 9 which illustrates a crossbar network similar to that illustrated in Fig. 8 but srnaller. Each input line C1 has an address, in this network being a 16 X 16 Clos network there add~esses being (for base lû, 1 ~ ~-through 16 but as shown) for base 2, 0000 through 1111. I~e destinadon addresses in column C3 have the same designation and C3 is likewise comprised of the same number of crossbar switches. C2 has a greater numbPr of smaller crossbar switches in accordance with the requirements of a Clos -network. ~he crossbars in columns C1 and C3 are of size n by 2n-1). For 10 ~ the pulposes of the preferred embodiment if the number of crossbar switchesin C1 and C3 is not a power of 2 (thus r is not a power of 2) the number should be inc~eased until a power of 2 is achieved.
l~us it ean: be seen that, for ~e preferred embodiment, with four digits the: number of bits and the label for each address is I log2r ~
In the preferred embodiment again, no switches are considered latched . :, ` thus they are all in the pass position. ;
When any message arrives at any switch, the header containing the : ~
~: ~ destination addressisrouk~d following theself-routing algorithm described ~: ;
above for the first column. ~ The destination address~will bc nude up of two ~: ~20 ~ ts: ~ the output crossbar:address of llog~l bits and a lo~al destinadon address ¦log2nl~ :bits vvithirl ~e indiYidual crosslbar. This matching is .
~'` '~ i'~l .;``' ~'~ f ~ formed~onlycn thefirst llog2rl bits.
:If ~e messago header finds that the desired address ( l log2r l, say OO -) ~' ' . ' output is busy in the first of the inte~nediary switches in column C2, it will be -:
2 5 ~ ~ : routed to the next intennediary switch and so on until a ~ath to the cr~ssbar switch in column C3 desired is achieved. Thus the~ algorithm is terrned greedy in that any unused column may be used for routing. In the illustrated e~ample ::~ of Fig. 9 the address 00 - was free and a connection made on 9 l to the second - ~
stage crossbar is C2. The other message that wants this crossbar is coming in : j: :
~ ~.
Wo93/~3~8~ Pc~r~us92/o6653 16- ~ ~ -on 92. If they are going to different final crossbars as shown, across lines 93 and 94 there is no problem, otherwise the second message ~hrough will be blocked and have to be rerouted. This is why the greedy algorithm was created. ' .
The key to resolving conflicts in each stage is the use of the crossbar with the return network for conflict detection and resolution. Stage 1 and 3 conflicting requests, due to tme input request conflicts (where two input reques~s specify the same output address) are returned to the source node.
Ho~ever, conflicts in the second stage or second column that arise due to shaling of the crossbars in the second stage are resolved by detecting conflicts ~ ;
in those labelled crossbars and then, on receiving the return header, scanning available connections in the remaining second column crossbars. In the worst case then, a ~equest may scan through n outputs in its stage one or Cl crossbar before its request is met by the (2n-l)th output.
Another way to say this is there are two forms of 10 conflict that can occur, the first occurs in the first or third stages due to direct input conflicts.
This can be where two or more inputs request the same output address in the third stage or subaddresses in the first stage. These are handled by tbe conflict resolu~on as described in the case of the single crossb~. The second Idnd of 2 o c onflict aIises due to ~e ~aiing of crossbars m ~e middle ~tages and requires ; ~ -sc~ning the ~st n~ e crossbars to resolve the conflicts.
Because of ~ese multiple stage conflic~s, crossbars 2û in the second or middle and ~ird stages must provide return paths so that when the header part of the message is sent, the response can be sent back to the source to confinn that a routing path is available. Thus, the multi-stage network acts more like arouter ~an a pure interconnection network. ~ ;
Please refer to Fig. 10 which describes such a system, including the ;
return nets and extra return paths Wo 93/03582 ~ ~ ~ 3 ~ 7 PCr/USs2/~653 Again, if light switches are used, retum paths may be obviated because the light can ~avei back over ~e same open switch that it traveled to get to ~hedestination node. It should also be noted here that electronic switches with -~
. ~:
return paths opened at the same time and location as sending paths would also ~ ~ -obviate the necd for a return net o~ separate return pa~ system.
Crossbars la to ra establish ~e first stage, lb to 2n-lb ~e second and lc to rc the third. Each has a retum net 102, similar to the one described in detail with reference Fig. 15. For eve~r connection between crossbars like line 103 there is a return line like 103r. Please re~er briefly back to Pig. 15 ~o and line 103r thereon.
.~ . .
For ease of explanation, assume the network is populated with 4 x 4 crossbars in the first stage like 150 of Flg. 15. Output 1 goes to the first crossbar in the second stage, output 2 to the second, etc. The line returmng from the first crossbar of the second stage would be line 103r, ~rom the second crossbar of the second stage would be line 104r, from the third crossbar of ~e s~cond stage would be return line 105r, and from the fourth crossbar of the second stage, the return line would be 106r.
Reviewing the two message state of c~ossbar 150 described earlier, two : . , ,.. . ~
- ~ messages, Ml and M2, have ~ied to reach output -3, only message M2 made ~ it. ~ Ml was reùlmed. Let us assume M2 did not make it through the third ~ ;
c rossbar of the second stage. It~would~then be returned on line lO~r. Switch 3 in ~e firs~ row is still held in the pass mode and the network message is, ~ ~ `
~routed through across to the retum net. ~e return net routes ~e message back to node 2. (All switches, once changed from the default state, should be 2 5 held in that state for the maximum delay required for the message to return if ;
blocked at the last stage.) M2 then would be routed through the second row, 4th column switch, into the return net which would forward the header back to node 2, using the return address. As in the explanation which follows, destination address can then be incremental and the header resent.
".' ~.. -.`.,, Wo 93/~3582 PCr/US92/06653 Figs. 11a, b, and c detail the number of lines used in each of the crossbars of Fig. 10.
The remaining detail is how to scan to the ne~t or subsequent second -stage crossbar when the original choice is blocl~d. When the message is returned to its onginating node, the header's destination address is simply incremented and the header reset to ~y the next second s~age switch. The incrementing is only done to the f~st part of the destination address, however.
So, for example, (with refaence to Fig. 9 assuming it has return nets), if output node at receiving address 0000 wants to send a message to a receiving ~:
node at 0000, the incrementing can be done to the first two digits of the 0000 address, maldng ~e next attempt through line 01-of c~ossbar 9S. This would send the header to the second crossbar in C2 (not show~) whose switches would look at the second part of the destination address, thus ~outing~ the ;~
mesxage to the u~rrect stage 3 or C3 crossbar, in this case crossbar 96. If a : :
third~ crossb~ in s~e 3 is tried, because the second part of ~e da~nadon address remains unchanged, the message will still be outed to crossbar 96 if the output address is 0000.
:Use of Busses ~
~-,. ..
Instead of Cloi networks having an intermediary stage or set of st es of ~20 ~ crossbar~s,~they can use~busses~for routing messages. This~embodiment fur~er reduces the number of switches but;adds complexity in that it lequires :bus ontrollers., Refer to Fig. ~12 in which:a networlc 120 is shown~ having again l through r erossb~rs with return ~nets- for both tile first and third stages. Again, 2s ~ ~:: as~in Fig. 10, ~e return lines are shown attached near the sending line even ; ~ though they are located as described with reference to Fig. 15 on ~e side :
opposite the output. The difference in structure between this Fig. and Fig. 10 IS that there are 2n-1 controllers 1211 - 1212n l. Each controller itself has a : set of r data bus controllers 1211d - 121rd. ~
~: ,;~., .:
WO 93/03582 ~ ? Pcr/us9~/o66s3 The algonthm, however, worlcs about the same way on ~is structure as it does on the ldnd illustrated in Fig. 10. Thus, a signal or message sent from node 1 of crossbar 123, passes on output line 1 to Data and Bus Controller 1211d. If the message is supposed to go to crossbar 124, for one of its 1 to 2n-l outputs it will try line 1 of its set of busses. If ~here is no "busy" signal set by some v ou~put on line a of bus 19 the message will pass onto the bus lineb on output w. These v and w outputs are bi-directional (or may be paired inputs and outputs, thus 2 v's and 2 w's if desired).
If this bus line is busy, the message will be returned to node 1, incremented and resent on line 2 (not shown) to the next con~roller. The next controller 1212 (not shown but next in series) would try line 1 of its first Data/Bus Controller, which also connects to crossbar 124. This would go on until line 2n-1 of crossbar 1231 is tried, and a link attempted through Data and ~
Bus Controller 1212n ld is tried and its line 1. If s~ll unsuccessful the next ~ ~ :
increment can reset the header to try line 1 and start~the cycle over.
. . ...... ..
Enhancements to ~is concept, such as sending a signal backward beyond the - input node to cancel a message or resend later, should b~ apparent to one of ordinary shll in the art, without going outside the scope of this invention. ; ~
(Note also that there~ are single input/ou~put lines betwéen busses and v 20 ~ crossbar 124r and paired input and output lines between busses and crossbar - ~ 124,~ In~e drawing. This is~done~to show ~at it co~d be either way, - ~ , although it would be preferred if all ~e c~ossbars in a stage iare coMected~ to ; busses in the same manner).
The data/bus controller is next~descnbed for completeness.~ Refer first ~; ~ to~Fig. 14 in which a bus controller 140 is shown. This is essentially anexploded view of the box labeled 1211 m Fig. 12. However, the diagram also Dlust~ates a variation in embodiments in which a demultiplexer is used rather than an iDpUt and output line from and to each one of the first stage crossbars and thei~ return networks. Th:e input data coming in on line 141 is -: : :,. ,~.:.-W0 93/03s82 Pcr/us92/o6653 20~
demultiplexed by lool~g at ~e destination address, first part, and the request is sent to one of the bus control us~its, say bus con~ol 1. Bus control 1 ~en needs to seize databus 1 in order to send ~e message as data across the data line to databus 1. To do so it hrst checks the status of bus status 1 to determine if databus 1 is busy. This is done on line status l. If it is not busy, it places a high signal or other appr~riate signal on bus 1 status line across the set/reset 1 line of bus control 1. If the status is busy, bus control 1 willrerout the request out the lesponse line which is wire-ored to line 142. A tum ;i~
signal will be tagged onto this returned message so that it may find its way . , baclc through the return net. As was described in the earlier implementation, ,-.~: . ;, such a turn signal f~r return coming from a inter nediary level or s~ond stage crossbar was not required because each line went to the appropriate address in theretum net. ~ ~
FSM of the Bus Coll~rQl ~ ~ -1. Inputs: ParallelInputData/Headerbus,theBusStatus(0/l), and the Next_Stage Response, a single packet of data equal to the bus width, usually a header bit, and ehe source and destination addresses sent back from ~ i the next stage. The input Data/Header always contains 2 reserved bits for roubng.
. .
zo ~ 2. Qh~s: Dat~Bus, paralle!outputheader/datatothenext ; ~ stage,~ Set/Reset control for ~e bus~status line, Res~onse_Baclc lines that contains a bit (available or note available) ~ sourc~/destination information which may also be obtained forrn the Next Stage Response.
Note the bus lin~es are tri-state ~ow, high, high impedance) to 2s~ distinguish between no signal and ~valued signal.
3. State T}ansitions:
IF Input Data/Header bit = 00 /*leading bit denotes header or message packet*/
AND Bus Status = 1 I*Bus occupied or busy*/
; ,,`' Wo 93/035~2 PcrJus92/~6653 2I ~ 3~7 ~
THENResponse_Back = O!InputData/Header;
/~send a not available signal in respon~e*/
ELSE : ~:
Response Back = 1!InputDal~
/*send available signal in ~esponse~/
Bus_Status o 1 /~set bus status busy reserve the bus :
lines*/
ELSE IF Input Data/Header bit = 10 /*message packet received after the source received an available signal*/
~ Data Bus = Input Da~a/Header /*output bus = inputbus - transmitdata*/ ~ :IF Input Data/Header bit = 11 /*special code for end of message*/ `: :
~IEN Bus Status = O /*free the bus* /
Data Bus = XXX /*set bus back to high ~impedance state*/ :; ;
` We note that this implementation in routing through the second stage .
assumes that the source node operates in two modes: first it sends out the requesting header packet. }f the rosponse received from the routers is ;.
:egalive, itretransmits the headers until it obtains a connec~on. Second, if a : posi~ve response is received, ;the sourrc node transmits its message in packets ; ~ :
a o~ ; unt~it;~s~`heiat hesswi~hesint ecrossbasor~ebus sithadu d.
RQ~tip~ys We ~will only consider the average delays in~ set~ng up the routing path :~
in the bus-based;self-~outrllg Clos net~vork. The Average:delays in the first and , ,. .. '.. "'' .',',' ird~ stage as before i9 (n+ 1) for th~ Xl implementation and~ n/'7 for the X2 ~ :~
2 s implementation. The delay in the second stage is not as simple. Consider the delay in determining a conflict using the Data/Bus Controller architecture.
From Pigure 14, it will be clear that determining tXe availability of a .
connection is tA (access bme) = (log2r+k) where log2r is delay in the demultiplexer~while k is a constant due the latching delay in the FSM of the ;~
WO 93/03582 P~r/USs2/066~3 bus control and response logic.- This delay tA is repeated at most n times if inthe worst call all free n crossbars i~ the second stage are scann~d. Thus, ~e worst case delay in set~ng up the rou~ng path in this implementation is 0(2n~nlog2r). Once the routing path is set up, the switching delay is smaller -since no conflicts ha~e to be further resolved. Since the optimal Yalue of r=n~N, the worst case delay in sefflng up the routing path is O(~N(2+10g2~/N))(onlyO(~fN)forllormalroutingdelay). Webelievean optical implementation may have lower delays than the electronic one when implementing theswiteh version.
Broadcasting in CN's follows very simply from the above self-routing scheme. To implément broadcasting CN's, broadcasting switches discussed above must be used for the in~dividual crossbars in thg CN. The only difference in the broadcasting is the limit on ~e maximurn number addresses to~which a message~can be broadcast. I f the original Clos design is used, the limit is (n-l). This limit is imposed by the maximum number of c~ssbars ; ~ , used in the second stage. ~ ~ ~
, , : :`
:i .' .
. '. `': ''~
:
: '
Claims (5)
1. A crossbar for use in a switching system which switches messages having headers, wherein the headers have source and destination addresses and two turn signal indicator means which are associated with these respective addresses, said turn signal being both set at the time the message is sent, and wherein the crossbar has a number of inputs, n, and a number of outputs, m, wherein the m inputs are connected by an n X m array of 2 X 2 switches organized into logical columns and logical rows labeled 1 to m and 1 to n, respectively, and wherein said switches are connected between said n input sources and m output destinations and said switches may be set into a pass or exchange mode (but are all set into the same mode, called a default mode, at the outset) based on the coincidence of two events, first that the message header destination address matches the column address of the switch and second that the turn signal has not already been reset by a previous switch having a matching column address, and wherein said crossbar has in communicative association therewith a return network having similar or identical 2 X 2 switches organized into logical diagonals with respect to the logical organization of the crossbar switches, the diagonal having the number of switches that is highest for all diagonals having addresses matching row 1 and succeedingly smaller diagonals having succeeding large addresses and less switches, with the ultimate diagonal having one switch of address "n", and wherein these switches are setable into pass or exchange mode based on the coincidence of the two events but all set into the same mode, called default mode, at the outset, the first event being that the message header source address matches the diagonal address of the switch, and second, that the turn signal has not already been reset by a previous switch with the same column address.
2. A crossbar as set forth in claim 1 wherein the default mode is exchange.
3. A crossbar as set forth in claim 1 wherein the default mode is pass.
4. A crossbar as set forth in claim 1 wherein the switches are broadcast switches.
5. A crossbar as set forth in claim 1 wherein every switch in the crossbar is maintained in the non-default condition for at least as long as the maximum delay required for a message header to be returned to that switch from its destination address in the switching system.
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| US74026191A | 1991-08-05 | 1991-08-05 | |
| US07/740,261 | 1991-08-05 |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CA2113647A1 true CA2113647A1 (en) | 1993-02-18 |
Family
ID=24975735
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CA002113647A Abandoned CA2113647A1 (en) | 1991-08-05 | 1992-08-05 | Crossbar with return net for scalable self-routing non-blocking message switching and routing system |
Country Status (5)
| Country | Link |
|---|---|
| US (1) | US5319639A (en) |
| EP (1) | EP0598027A1 (en) |
| JP (1) | JPH06509919A (en) |
| CA (1) | CA2113647A1 (en) |
| WO (1) | WO1993003582A1 (en) |
Families Citing this family (16)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5469154A (en) * | 1991-02-19 | 1995-11-21 | Dva Ltd. | Multiconnection switching networks |
| FR2704999B1 (en) * | 1993-05-04 | 1995-07-21 | Thomson Csf | TELECOMMUNICATIONS NETWORK FOR PACKET DATA TRANSMISSION. |
| WO1995030315A1 (en) * | 1994-04-29 | 1995-11-09 | Honeywell Inc. | Design of a fault-tolerant self-routing crossbar |
| US5634004A (en) * | 1994-05-16 | 1997-05-27 | Network Programs, Inc. | Directly programmable distribution element |
| US5590123A (en) * | 1995-05-23 | 1996-12-31 | Xerox Corporation | Device and method for use of a reservation ring to compute crossbar set-up parameters in an ATM switch |
| KR100278016B1 (en) * | 1995-12-26 | 2001-01-15 | 윤종용 | Switching device and method of asynchronous transfer mode switching system |
| US5862340A (en) * | 1996-05-24 | 1999-01-19 | International Business Machines Corporation | Method operating in each node of a computer system providing and utilizing special records for collective communication commands to increase work efficiency at each node |
| US5758161A (en) * | 1996-05-24 | 1998-05-26 | International Business Machines Corporation | Testing method for checking the completion of asynchronous distributed collective operations |
| US5748959A (en) * | 1996-05-24 | 1998-05-05 | International Business Machines Corporation | Method of conducting asynchronous distributed collective operations |
| US5930256A (en) * | 1997-03-28 | 1999-07-27 | Xerox Corporation | Self-arbitrating crossbar switch |
| US6087958A (en) * | 1997-07-14 | 2000-07-11 | Arzt; Lawrence J. | Multi-stage routing switchers with sequential and non-repetitive distributive circuit interconnections |
| US6680939B1 (en) * | 2000-09-14 | 2004-01-20 | Nvision, Inc | Expandable router |
| US7012895B1 (en) | 2000-11-17 | 2006-03-14 | University Of Kentucky Research Foundation | Packet-switching network with symmetrical topology and method of routing packets |
| US20030149757A1 (en) * | 2000-12-27 | 2003-08-07 | Reiko Ueno | Identification code management system for home network |
| US6999453B1 (en) | 2001-07-09 | 2006-02-14 | 3Com Corporation | Distributed switch fabric arbitration |
| JP4755050B2 (en) * | 2006-08-18 | 2011-08-24 | 富士通株式会社 | Data processing apparatus, mode management apparatus, and mode management method |
Family Cites Families (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US4542497A (en) * | 1983-03-28 | 1985-09-17 | At&T Bell Laboratories | Wideband digital switching network |
| US4516238A (en) * | 1983-03-28 | 1985-05-07 | At&T Bell Laboratories | Self-routing switching network |
| US4635250A (en) * | 1984-04-13 | 1987-01-06 | International Business Machines Corporation | Full-duplex one-sided cross-point switch |
| JPS61214694A (en) * | 1985-03-18 | 1986-09-24 | インタ−ナショナル ビジネス マシ−ンズ コ−ポレ−ション | Switching unit for data transmission |
| US4725835A (en) * | 1985-09-13 | 1988-02-16 | T-Bar Incorporated | Time multiplexed bus matrix switching system |
| US4706240A (en) * | 1985-11-29 | 1987-11-10 | American Telephone And Telegraph Co., At&T Bell Labs | Switching system having multiple parallel switching networks |
| US4696000A (en) * | 1985-12-12 | 1987-09-22 | American Telephone And Telegraph Company, At&T Bell Laboratories | Nonblocking self-routing packet and circuit switching network |
| US4817082A (en) * | 1987-03-09 | 1989-03-28 | American Telephone And Telegraph Company, At&T Bell Laboratories | Crosspoint switching system using control rings with fast token circulation |
| US4833468A (en) * | 1987-10-14 | 1989-05-23 | Unisys Corporation | Layered network |
| JP2659421B2 (en) * | 1988-02-17 | 1997-09-30 | 日本電信電話株式会社 | Self-routing channel |
| US4914429A (en) * | 1988-12-09 | 1990-04-03 | Transwitch Corp. | Switch components and multiple data rate non-blocking switch network utilizing the same |
| US4991168A (en) * | 1989-05-08 | 1991-02-05 | At&T Bell Laboratories | Concurrent multi-stage network control arrangement |
| US4984237A (en) * | 1989-06-29 | 1991-01-08 | International Business Machines Corporation | Multistage network with distributed pipelined control |
| US5001702A (en) * | 1989-09-26 | 1991-03-19 | At&T Bell Laboratories | Packet switching network for multiple packet types |
-
1992
- 1992-08-05 CA CA002113647A patent/CA2113647A1/en not_active Abandoned
- 1992-08-05 JP JP5503881A patent/JPH06509919A/en active Pending
- 1992-08-05 EP EP92917711A patent/EP0598027A1/en not_active Withdrawn
- 1992-08-05 WO PCT/US1992/006653 patent/WO1993003582A1/en not_active Application Discontinuation
-
1993
- 1993-04-19 US US08/049,646 patent/US5319639A/en not_active Expired - Lifetime
Also Published As
| Publication number | Publication date |
|---|---|
| US5319639A (en) | 1994-06-07 |
| JPH06509919A (en) | 1994-11-02 |
| WO1993003582A1 (en) | 1993-02-18 |
| EP0598027A1 (en) | 1994-05-25 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CA2113647A1 (en) | Crossbar with return net for scalable self-routing non-blocking message switching and routing system | |
| EP0997050B1 (en) | Method and apparatus for multipoint-to-point transmission in an atm network | |
| US6026073A (en) | Route finding in communications networks | |
| JP2988972B2 (en) | Message routing method, computer network and routing switch | |
| JP3174046B2 (en) | Message routing method, computer network, routing switch | |
| US7787449B2 (en) | Butterfly network with switches set for two node disjoint paths and method for forming the paths | |
| CA1274304A (en) | Crosspoint circuitry for data packet space division switches | |
| EP0405208B1 (en) | Multistage network with distributed pipelined control | |
| US4891802A (en) | Method of and circuit arrangement for controlling a switching network in a switching system | |
| EP0868054A3 (en) | Self-arbitrating crossbar switch | |
| JPH06268683A (en) | Packet distribution network | |
| US6992984B1 (en) | Credit-based adaptive flow control for multi-stage multi-dimensional switching architecture | |
| JPH06268684A (en) | Packet switching arrangement | |
| JPH0750688A (en) | Packet switching arrangement | |
| EP1113627B1 (en) | Method of determining network paths in a three stage switching matrix | |
| US5132965A (en) | Nonblocking parallel banyan network | |
| ES2331941T3 (en) | TRANSPORT MECHANISM OF SIGNALING MESSAGES. | |
| KR0170493B1 (en) | Non-blocking fault tolerance gamma network for multi-processor system | |
| WO1993006675A1 (en) | Nonblocking multicast fast packet/circuit switching networks | |
| Chen | A survey of multistage interconnection networks in fast packet switches | |
| CA2113725A1 (en) | Scalable self-routing non-blocking message switching and routing network | |
| WO1993003581A1 (en) | Message structure for scalable self-routing non-blocking message switching and routing system | |
| WO1993006676A1 (en) | Nonblocking point-to-point fast packet/circuit switching networks | |
| JPH0570345B2 (en) | ||
| JPS63287140A (en) | Self-routing controller |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| FZDE | Discontinued |