CA2211083C - Address transformation in a cluster computer system - Google Patents
Address transformation in a cluster computer system Download PDFInfo
- Publication number
- CA2211083C CA2211083C CA002211083A CA2211083A CA2211083C CA 2211083 C CA2211083 C CA 2211083C CA 002211083 A CA002211083 A CA 002211083A CA 2211083 A CA2211083 A CA 2211083A CA 2211083 C CA2211083 C CA 2211083C
- Authority
- CA
- Canada
- Prior art keywords
- address field
- processor
- cluster
- memory
- bits
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F12/00—Accessing, addressing or allocating within memory systems or architectures
- G06F12/02—Addressing or allocation; Relocation
- G06F12/0223—User address space allocation, e.g. contiguous or non contiguous base addressing
- G06F12/0284—Multiple user address space allocation, e.g. using different base addresses
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F12/00—Accessing, addressing or allocating within memory systems or architectures
- G06F12/02—Addressing or allocation; Relocation
- G06F12/08—Addressing or allocation; Relocation in hierarchically structured memory systems, e.g. virtual memory systems
- G06F12/0802—Addressing of a memory level in which the access to the desired data or data block requires associative addressing means, e.g. caches
- G06F12/0806—Multiuser, multiprocessor or multiprocessing cache systems
- G06F12/0811—Multiuser, multiprocessor or multiprocessing cache systems with multilevel cache hierarchies
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F12/00—Accessing, addressing or allocating within memory systems or architectures
- G06F12/02—Addressing or allocation; Relocation
- G06F12/08—Addressing or allocation; Relocation in hierarchically structured memory systems, e.g. virtual memory systems
- G06F12/0802—Addressing of a memory level in which the access to the desired data or data block requires associative addressing means, e.g. caches
- G06F12/0806—Multiuser, multiprocessor or multiprocessing cache systems
- G06F12/084—Multiuser, multiprocessor or multiprocessing cache systems with a shared cache
Abstract
To integrate a plurality of processors (11-14), each capable of directly addressing a limited internal storage range, with a large external memory, processors are organized into clusters, each having a plurality of processors and a common secondary cache (7). An address translator (18) is provided to transform internal main memory space (8) addresses to external memory space addresses. External memory space is divided into private and shared areas. An internal address indicator bit, in conjuction with the cluster number form a requesting processor primary cache, is employed to set up the transformation either to the private external space of the cluster or the shared external space. In reverse external-to-internal transformation, a pair of indicator bits are used to generate the internal address and define the shared and private space of the designated cluster. A cluster member number assigned to each processor is used by the secondary cache to track with processor sends/receives information to/from the external memory.
Description
ADDRESS TRANSFORI!'IAT10N
IN A CI,I1STER CO~'IPL1TER SYSTEM
' 3 Field of the Invention This invention relates to the art of data processing and, more s particularly, to addressing in a cluster computer system employing multiple 6 clusters, each cluster having multiple processors, each processor being ~ incapable of directly addressing all information stored in the total main s memory.
y Back~rl-ound of the Invention is In the design of central processing units (CPUs) used in modern i i computing systems, improvements must take into account the con vnand i2 stmcture and tl~e word Length which often must remain the same as used in i3 the predecessors to a ''new" CPU. This is particularly true in the case of m systems running under proprietary software operating systems. The m fimdamental reason for this constraint is the often enonmous investment in 16 software which a user may have. In some instances, there may be , m represented three decades of software development in a given user system, . is and a user typically wishes to merely load its old software on a new, m improved hardware system and experience immediate higher performance I results. As a result, CPU designers go to gTl-eat lengths to maintain 2 compatibility with earlier CPUs in a given operating system family.
Thus, in CPU desigm based on existing systems, even limited ' 4 modification of the address range capability (i.e., allocation of address fields s in a CPU instruction word) of each CPU would have a major impact on ~s hardware design and, more especially, on the software which can or must be reused. This dlhe1111na 1S faced in view of the ongoing increase in density and capacity of memory chips and ocher technologies that have resulted in the availability of mass memories capable of storing information which is one to to two times gl-eater than was contemplated at the time the address field t l Iengths/word lengths were originally estabhished in the operating systeln/CPU
i2 family.
13 Another factor involved in the desigm of CPUs, particularly CPUs 14 intended to fimction under a long-established proprietary operating system, is Is the fact that ever-new technology allows an existing CPU design to be I6 physically implemented into a much smaller space. However, again, the I~ advantages of adhering to the fundamental specifications of existing operating Is system software (in order to permit the reuse of existing application software) 19 directs the retention of the operating system, ilOthWIthStandlng its limiting I effects on CPU design. Finally, and perhaps most importantly, those skilled ' a in the art are aware of inefficiencies which occur when too many CPUs are 3 operated in a single operating system environment.
IN A CI,I1STER CO~'IPL1TER SYSTEM
' 3 Field of the Invention This invention relates to the art of data processing and, more s particularly, to addressing in a cluster computer system employing multiple 6 clusters, each cluster having multiple processors, each processor being ~ incapable of directly addressing all information stored in the total main s memory.
y Back~rl-ound of the Invention is In the design of central processing units (CPUs) used in modern i i computing systems, improvements must take into account the con vnand i2 stmcture and tl~e word Length which often must remain the same as used in i3 the predecessors to a ''new" CPU. This is particularly true in the case of m systems running under proprietary software operating systems. The m fimdamental reason for this constraint is the often enonmous investment in 16 software which a user may have. In some instances, there may be , m represented three decades of software development in a given user system, . is and a user typically wishes to merely load its old software on a new, m improved hardware system and experience immediate higher performance I results. As a result, CPU designers go to gTl-eat lengths to maintain 2 compatibility with earlier CPUs in a given operating system family.
Thus, in CPU desigm based on existing systems, even limited ' 4 modification of the address range capability (i.e., allocation of address fields s in a CPU instruction word) of each CPU would have a major impact on ~s hardware design and, more especially, on the software which can or must be reused. This dlhe1111na 1S faced in view of the ongoing increase in density and capacity of memory chips and ocher technologies that have resulted in the availability of mass memories capable of storing information which is one to to two times gl-eater than was contemplated at the time the address field t l Iengths/word lengths were originally estabhished in the operating systeln/CPU
i2 family.
13 Another factor involved in the desigm of CPUs, particularly CPUs 14 intended to fimction under a long-established proprietary operating system, is Is the fact that ever-new technology allows an existing CPU design to be I6 physically implemented into a much smaller space. However, again, the I~ advantages of adhering to the fundamental specifications of existing operating Is system software (in order to permit the reuse of existing application software) 19 directs the retention of the operating system, ilOthWIthStandlng its limiting I effects on CPU design. Finally, and perhaps most importantly, those skilled ' a in the art are aware of inefficiencies which occur when too many CPUs are 3 operated in a single operating system environment.
4 All these factor are aggravated in a multiprocessor cluster environment s in which multiple gn-oups of multiple CPUs are incorporated into a large 6 computer system which includes a large main memory containing stored information at more addresses than can be directly addressed by the s individual CPUs, with the addressing constraint being imposed because of the address field/word length characteristics which are the natural heritage of the to historical hardware and operating system/application software of a given II computer system family. It is to the solution of this dilemma that the subject 12 invention is directed.
i3 Objects of the Invention It is therefore a broad object of tlIlS IIlVellt1011 to provide an improved Is mufti-processor computer SySte111 II1COI~7oratlllg extended memory addressing 16 with total system memory beyond that seen by a single CPU or operating I~ system.
Is It is a more specific object of this invention to provide an improved I~ multiprocessor system employing a cluster architecture in which an external WO 97/i3191 PCT/US96/15937 1 memory space, larger than that inherently directly addressable by the 2 processors employed, is divided into private spaces for each multiprocessor 3 cluster and shared spaces directly accessible by all the clusters according, to an address transformation which incorporates cluster member number s information for identifying each cluster and each member thereof.
Summary of the Invention Briefly, these and other objects of the invention are achieved, in an s exemplary system which includes an external memory space (e.g., main memory) a power of two larger than the internal memory space which is io inherently directly addressable by an address field generatedlinterpreted by i 1 the individual processors, by dividing the processors into multiple clusters i2 with multiple processors in each. Each CPU in a cluster of processors, fom t3 on a single multiprocessor board in the example, is assigned a cluster number m which is set, for example, by a two-bit lunber (thus identifying four clusters).
m In addition, each CPU is assiy ed a two-bit number (thus identifying four i6 CPUs) setting which CPU it is in its cluster. In the example, the two two-bit m numbers are concatenated to provide each CPU with a unique four-bit is identification number in the system.
i In an internal-to-external address transformation mode, the cluster ' 2 number is sent, along with an internal address which includes an indicator bit s specifying private/shared classification, from a requesting cluster to all 4 address translator. In the address translator, the indicator bit is used to s control the set up of the transformation.
6 In the exemplal-y system in which each CPLl can inherently directly address one gigabyte (30 bit address field identified as bits 0-29 with bit 0 being the most significant), the available external space is two grigabytes and the length of tl3e external space address field is 32 bits (bits 0-3 l , capable of I(1 SpeCIfyIilg fOllr gigabytes), bit U of the internal address is used as the I ~ indicator. If bit 0 is a ''0", tile cluster member is requesting information from 12 its private area of external space; conversely, if it is a '' I ", the cluster member i3 is addressing information in the shared regions of external space.
I4 If the indicator bit is a "0", the CILISter IlIIIIIbeI' is copied into bits l, 2 of 1s the external address, bits l, 2 of tile internal address are copied into bits 3, 4 I6 of the external address and the lower order address bits 3-29 of the internal a address are copied directly to bits ~-31 of the external address. The resulting ~s external address points to the specified pI-ivate region in external space.
l If, however, the indicator bit 0 in the internal address is a "I"
2 indicating a call to a shared region of external space, then a different transfonnation takes place. CNO, CN 1 are ilnlored. Bit 1 of the internal address is copied to bit 1 of the external address while bits 2, 3 of the external s address are each forced to logic "I ". The resulting external address points to 6 the specified shared region in external space.
In an external-to-internal address transformation mode, bits l, 2 of the s external address are copied into positions CNU, CNI of the internal address The external address indicator bits are bits 3, 4. If one or the other or both lo are "0", then an address from the private space of one of the clusters is l l indicated. Bit 0 of the internal address is forced to "0", and bits 3, 4 of the i2 external address are copied to bits I , of the internal address. The lower order bits 5 - 31 of the external address are copied directly to bits 3 - 29 of the m internal address. Thus, the developed internal address fully identifies an is inforniation block in external memory as that of the private space reserved to i6 the cluster identified by CNO, CNl .
m If both bits 3, 4 of the external address are "I ", then the external space is is known to be shared. Bit 0 of the internal address is forced to "1", bits 1, 2 t9 of the external address are copied to bits I , 2 of the internal address, and the lower order address bits are copied as before. The resulting internal address identifies an information block in external memory that is available to all the clusters.
In accordance with the present invention, there is provided a cluster computer system comprising: A) a plurality of clusters, each said cluster including: 1) a plurality of processors, each said processor having inherent limitations on the range of memory which is directly addressable by a processor address field, said processor address field being divided into higher order and lower order segments, each said processor including: a) a primary cache; and b) an address translator; and 2) a secondary cache; B) a memory having a capacity which exceeds the range each said processor is inherently capable of directly addressing, said memory being addressable by a memory address field, said memory address field exceeding said processor address field, said memory address field being divided into higher order and lower order segments; C) means establishing a cluster number for each said cluster; D) means coupling said plurality of clusters, via said secondary caches, to said memory; and E) each said address translator in each said proces~~or in each said cluster being coupled intermediate said primary cache in its said processor and said secondary cache in its said cluster, said address translator including first address translation logic responsive to processor address field and cluster number information in said primary cache in its said processor, said first translation logic comprising means responsive to the digital state of a predetermined indicator bit in said higher order segment of said processor address field by effecting one of two address transformations; 1) if said predetermined indicator bit is a first digital value; a) forcing the two lowest order bits of said higher order segment of said memory address field to a first logic level;
b) copying the two lowest order bits of said higher order segment of said processor address field to two higher order bits of said higher order segment of said memory address field; and c) copying the contents of said lower order segment of said processor address field to said lower order segment of said memory address field; thereby specifying a region of memory addressable by all said clusters; and 2) if said predetermined indicator bit is a second digital value:
a) copying the two lowest order bits of said higher order segment of said processor address field to the two lowest order bits of said higher order segment of said memory address field; b) copying the cluster number to two higher order bits of said higher order segment of said memory address field; and c) copying the contents of the lower order segment of said processor- address field to said lower order segment of said memory address field; thereby specifying a region of memory addressable only by the cluster identified by said cluster number.
In accordance with the present invention, there is provided a cluster computer system comprising: A) a plurality of clusters, each said cluster including: 1) a plurality of processors, each said processor having inherent limitations on the range of memory which is directly addressable by a processor address field, said processor address field being divided into higher order and lower order segments, each said processor including: a) a primary cache; and b) an address translator; and 2) a secondary cache; B) a memory having a capacity which exceeds the range each said processor is inherently capable of directly addressing, said memory being addressable by a memory address field, said memory address field exceeding said 7a processor address field, said memory address field being divided into higher order and lower order segments; C) means establishing a cluster number for each said cluster; D) means coupling said plurality of clusters, via said secondary caches, to said memory; and E) each said address translator in each said processor in each said cluster being coupled intermediate said primary cache in its said processor and said secondary cache in its said cluster, said address translator including second address translation logic responsive to memory address field information, said second translation logic comprising means responsive to the digital states of predetermined indicator bits in said memory address field, which predetermined indicator bits are of a lower order than the most significant bit thereof, by effecting one of two address transformations; 1) if said predetermined indicator bits are both a first digital value:
a) forcing a predetermined bit of said higher order segment of said processor address field to a first logic level; b) copying two higher order bits of said higher order segment of said memory address field to the two lowest order bits of said higher order segment of said processor address field;
and c) copying the contents of the lower order segment of said memory address field to said lower order segment of said processor address field; thereby specifying a memory region containing information accessible by all said clusters; and 2) if either of said predetermined indicator bits are a second digital value: a) copying the two lowest order bits of said higher order segment of said memory address field to the two lowest order bits of said higher order segment of said memory processor address field to the two lowest order bits of said higher order segment of said processor address field; b) copying higher bits, representing a cluster number, in said higher order section of said memory address field to said cluster number at said 7b common cluster interface; and c) copying the contents of said lower order segment of said memory address field to said lower order segment of said processor address field thereby specifying a memory region containing information accessible only by the cluster specified by said cluster number.
In accordance with the present invention, there is also provided a cluster computer system comprising: A) a plurality of clusters, each said cluster including: 1) a plurality of processors, each said processor having inherent limitations on the range of memory which is directly addressable by a processor address field, said processor address field being divided into higher order and lower order segments, each said processor including: a) a primary cache; and b) an address translator; and 2) a secondary cache; B) a memory having a capacity which exceeds the range each said processor is inherently capable of directly addressing, said memory being addressable by a memory address field, said memory address field exceeding said processor address field, said memory address field being divided into higher order and lower order segments; C) means establishing a cluster number for each said cluster; D) means coupling said plurality of clusters, via said secondary caches, to said memory; and E) each said address translator in each said processor in each said cluster being coupled intermediate said primary cache in its said processor and said secondary cache in its said cluster, said address translator including first address translation logic responsive to processor address field and cluster number in said primary cache in its said processor, said first translation logic comprising means responsive to the digital state of a first predetermined indicator bit in said higher order segment of said processor address field, which said 7c first predetermined indicator bit is of a lower order than the most significant bit thereof, by effecting one of two address transformations; 1) if said first predetermined indicator bit is a first digital value: a) forcing the two lowest order bits of said higher order segment of said memory address field to a first logic level; b) copying the two lowest order bits of said higher order segment of said processor address field to two higher order bits of said higher order segment of said memory address field; and c) copying the contents of said lower order segment of said processor address field to said lower order segment of said memory address field; thereby specifying a region of memory addressable by all said clusters; and 2) if said first predetermined indicator bit is a second digital value: a) copying the two lowest order bits of said higher order segment of said processor address field to the two lowest order bits of said higher order segment of said memory address field; b) copying the cluster number to two higher order bits of said higher order segment of said memory address field; and c) copying the contents of the lower order segment of said processor address field to said lower order segment of said memory address field; thereby specifying a region of memory addressable only by the cluster identified by said cluster number; and F) said address translator further including second address translation logic responsive to memory address field information, said second translation logic comprising means responsive to the digital states of second and thirds predetermined indicator bits in said memory address field, which second and third predetermined indicator bits are of a lower order than the most significant bit thereof, by effecting one of two address transformations; 1) if said second and third predetermined indicator bits are both a first digital value: a) forcing a predetermined bit of said 7d higher order segment of said processor address field to a first logic level; b) copying two higher order bits of said higher order segment of said memory address field to the two lowest order bits of said higher order segment of said processor address field; and c) copying the contents of the lower order segment of said memory address field to said lower order segment of said processor address field; thereby specifying a memory region containing information accessible by all said clusters; and 2) if either of said second and third predetermined indicator bits are a second digital value: a) copying the two lowest order bits of said higher order segment of said memory address field to the two lowest order bits of said higher order segment of said processor address field; b) copying higher bits, representing a cluster number, in said higher order section of said memory address field to said cluster number at said common cluster interface; and c) copying the contents of said lower order segment of said memory address field to said lower order segment of said processor address field thereby specifying a memory region containing information accessible only by the cluster specified by said cluster number.
Description of the Drawin The subject matter of. the invention is particularly pointed out and distinctly claimed in the concluding portion of the specification. The invention, however, both as to organization and method of operation, may best be understood by reference to the following description taken in conjunction with the subjoined claims and the accompanying drawing of which:
FIG.1 is a high level block diagram of a cluster computer system in which the present invention may be practiced;
7e FIG. 2 is a block diagram of the system shown in FIG. 1 illustrating more detail of the individual clusters and CPUs;
FIG. 3 is an exemplary representation of the allocation of internal address space as allocated to the individual mufti-processor clusters and external space representing the address space of a main memory device;
FIG. 4 is an exemplary transformation diagram illustrating an embodiment of the invention in which each CPU is inherently capable of 7f 1 addressing one gigabyte of information, and a main memory device in communication with all the CPUs is capable of storing two grigabytes of s information;
4 FIG. 5 is a representation of the two types of addresses, internal and s external, which are transformed according to the invention;
6 FIG. 6 is a simplified logic diagram illustrating the apparatus within an address translator component of each CPU operating in an internal-to-external x mode; and FIG. 7 is a simplified iog.~ic dia~,~ram of an address translator operating io in an exten~al-to-internal mode.
1 i Description of the Preferred Embodiments) t2 Referring first to the exemplary cluster computer system of FIG. l, it i3 will be observed that the system includes first l, second 2, third 3 and fourth is 4 multiprocessor circuit boards, each multiprocessor board including four m CPUs effecting a cluster, each multiprocessor board being connected to a i6 system bus 5 via an inte~~ral secondary cache 7. The system bus ~ couples the multiprocessor boards 1, 2, 3, 4 to one another and to various other .
is system components (including a service processor ''SP") represented by the 1 > block 6. In addition, system bus ~ couples the multiprocessor boards 1, 2, 3, i 4 to a memory contrbller 10 which, in turn, interfaces via a memory bus 9 2 with a main memory 8 representative of one or more main memory devices.
As illustrated for multiprocessor board I, each multiprocessor board l, :~ 2, 3, 4 includes four CPUs 1 l, 12, I3, 14. Tl~e four CPUs I l, 12, 13, 14 are s provided with respective primary caches 11 C, I 2C, 13C, 14C. In the 6 exemplary system, the primary caches on each multiprocessor board are each ~ disposed between its directly associated CPU and a common secondary cache s 7 and utilize a the classical ''store-into-cache" configuration. The secondary ~ cache 7 interfaces with the system bllS J. Interposed intermediate each of the u> primary caches 1 I C, 12C, 13C, 14C and the common (to tile cluster) n secondary cache 7 are address translators 11 A, 12A, I 3A, I 4A which are l2 also constituents of their respective processors 11, I 2, 13, 14.
i3 In accordance with a preferred aspect of the invention, each of the m multiprocessor boards I, 2, 3, 4 constitutes a cluster, and each cluster nzns is under its own operating system to improve system efficiency.
i6 Attention is now directed to F1G. 2 which illustrates the environment of m the subject invention in more detail. Thus, it will be seen that CPU I1 on is multiprocessor board 1 (board "0") communicates with the system bus 5 via i9 an address register l ~, address translator I 8 and secondary cache 7.
Register i I S is merely a convenient representation of address interface circuitry in the 2 primary cache of the CPU 11 by which an address generated by the CPU 11 may be transmitted, transformed in the address translator I8, as a request to 4 the secondary cache 7. This condition occurs when information required by s the CPU 11 is not resident in at least one of the primary caches of the CPUs 11, 12, 13, 14 on the multiprocessor board 1. (Those skilled in the art will understand that, in many such multiprocessor configurations, it is possible for one CPU to "siphon" information from another CPU's primary cache.) y Corresponding CPUs 21, 22, ?3, ?4 are found on multiprocessor board io 2 (board ''1"). Similarly, corresponding CPUs 31, 32, 33, 34 and 41, 42, 43, i i 44 are found on multiprocessor boards 3 (board "2"), 4 (board "3"), i2 respectively. In addition, registers ?~, 3~, 4~, con-esponding to register 1~, i3 are respectively found on multiprocessor boards 2, 3, 4. Also, address m translators 28, 38, 48, correspondinb to address translator 18, are respectively is found CPUO on each of multiprocessor boards 2, 3, 4.
i6 Also shown incorporated in the primary cache of CPU 11 on the m multiprocessor board 1 are f rst and second two-bit identification registers, m 16, 17, respectively. (It will be understood that the re~~isters 16, 17 are m representative of any convenient means for establishing a two-bit cluster to 1 identification for the multiprocessor board 1 and a two-bit cluster member identification for the CPU 11. For example, a cluster/cIuster member identification could be hard-wired in or exist as four switches which can be 4 set to a complete cluster member number.) In the case of CPU 11 in s multiprocessor I, the foul- stages of the registers 16, 17 are each set to the 6 logric ''0" position, thereby providing a cluster identification of "00"
(board "0") and a cluster member identification of "00" (CPU ''0"). As a practical matter, the regristers I6, 17 may be concatenated and may be set upon system initialization under control of the service processor- SP as represented by the to dashed line 20.
11 COITeSpOlldlllg register pairs, 26, 27; 36, 37; and 46, 47 are provided 12 on multiprocessor boards 2, 3, 4, for respective CPUOs 21, 3I, 41 and are 13 respectively set to provide cluster identifications of ''0l ", "10" and ''I
1" and, m in each case, cluster member identification of "00"
is In the succeeding discussion, the description will be directed to 16 multiprocessor board 1 (board "0"), it being understood that multiprocessor 1~ boards 2, 3, 4 operate identically, taking into account their differing cluster is number identifications as stored in the registers 26, 36, 46. Le., the cluster m i member number for CPU? 33 on multiprocessor board 3 is ''1010" (board "~"CPU ''~").
3 The 30-bit register 1 ~ in the primary cache of the CPU I I can specify one gigabyte of address locations which is the family Iimit of the exemplary s CPUs. There is value in resem~ing a portion of the addressable space in main 6 memory 8 to the private use of the CPUs on each multiprocessor board for, among other reasons, permitting each multiprocessor board to operate under x an independent operating system. (These independent operating systems may be the same or different operating systems.) On the other hand, there is a m need for establishing direct communication among the clusters and CPUs i i operating under the different operating systems in order that they can function i2 with the fiill power of a cluster system architecture. One way in which this i3 feature can be achieved is by pro~riding shared memory space in main m memory 8.
is The present invention provides for the enjoyment of all these features i6 while also permitting the collective direct and indirect address of more m memory spaces in main memory than the CPUs in the system are individually i8 inherently capable. -l In the fOIIOWIIIg dlSCliSS1011, the term "internal space" means the 2 memory addressable by the CPUs on each multiprocessor board while the ' 3 term "external space" means the total memory addressable; i.e., the memory 4 space available in main memory 8. These two types of spaces exist at tvvo s interfaces of, and are reconciled by, the address translator 18 which, in the 6 exemplary system, is disposed in the primary cache of each CPU.
The concept of internal and external spaces and private and shared x memory in the present context may be better understood with reference to FIG. 3. In this example, it may be assumed that each CPU has a 30-bit lo address field and can therefore directly address one gigabyte. It may further l l be assumed that the available external space in the exemplary main memory is iz two gigabytes and that the external address field is 3?-bits long; i.e., capable i3 of addressing four gigabytes.
i4 A key aspect of the present invention the mamer in which is is addressable memoryforeachmultiprocessor is allocated. For board the 16 external memory space to be accessible, as a result of address transformation, m by each multiprocessor board 1, 2, 3, 4, three-eighths of the inherently is addressable internal space is private and one-half is shared. Comment on the m apparently remaining one-eighth is reserved for discussion below.
1 Under these conditions, the memory allocation may be appreciated 2 from an examination of FIG. 3. Multiprocessor board 1 enjoys 3/8th gigabyte private memory space identified as //8th gigabyte segments P1 I, P12, PI3.
In addition, Multiprocessor board I has access to I/2 I,~igabyte shared s memory space in I /8th gigabyte segments SA, SB, SC, SD. Similarly, 6 Multiprocessor board 2 enjoys 3/8th gigabyte private memory space identified as 1 /8th gigabyte se~nnents P21, P22, P23. In addition, Multiprocessor board 2 has access to I /2 gigabyte shared memory space in 9 1/8th gigabyte selunents SA, SB, SC, SD. The internal space for m multiprocessor boards 3 and 4 are correspondingly assil,~~ed as shown in FIG.
a 3.
l? This internal space address information is provided to the address 13 translation block which, as previously described, is present in the primary m cache of each CPU, but is shown separately as a single block in FIG. 3 for is clarity. In the addresstranslation this addressinformation block, is i6 transformable, accordingthe invention, obtainexternalspace address to to m segments distributed as shown schematically in FIG. 3.
is Consider now an implementation of this example with reference to .
m FIGS. 2, 4, 5, 6 and 7. FIG. 4 shows the higher order address bit distribution 1 for the four multiprocessor boards for both the internal and external space addresses as they are transformed by the address translation block (e.g., the ' s address translator 18 in FIG. 2). In each case, there are three columns representing, respectively, the cluster number (already set by the registers 16, s 26, 36, 46; F1G. 2), the three most sitnificant bits of the 30-bit intental 6 address and the second, third, fourth and fifth most significant bits of the bit transformed address. As will be discussed in more detail below, the most s significant bit of the 32-bit transformed address is always a ''0" in the y example.
Referring also to FIG. ~ and examining first the internal address pattern i ~ for multiprocessor board 1, the cluster number is preset to "00", the private i2 space three most significant bits are ''000", "001" and "010" and the shared t3 space three most significant bits are "I 00", "1 O l ", '' 110" and "111 ".
It will ~ be noted that, in this example, the combination "011" for the three most Is significant bits is reserved. The three highest order internal address bits may i6 be deemed the higher order segment of the processor address field and the j~ remaining bits 3-29 may be deemed the lower order setnnent thereof.
Because the address field for external addresses has 32-bits and the iy main memory stores only two gigabytes, the most sitmificant bit in the 1 external addresses is always ''0" in the example. (It may be noted, however, 2 that this most significant bit is available to denote one of two complete systems which is to be granted access to a shared main memory.) Thus, the ' a corresponding addresses, transfonned into the five most significant bits of the private external space addresses to be transformed (i.e., bits l, 2, 3, 4), are, 6 respectively, "00000", "00001 ", "000 I 0" (or P I I , P 12, P 13 ) and those for the shared external space addresses are, respectively, "00011 ", "0001 I" , x "01011" and "01111" (or SA, SB, SC, SD). The five highest order irite~~
address bits may be deemed the higher order se~nent of the memory address to field and the remaining bits ~-31 may be deemed the lower order segment i i thereof.
12 For multiprocessor board 2, the cluster number is preset to "O1", the I3 private space three most si~,mificant bits are, main, "000'', ''001" and ''010"
m and the shared space three most significant bits are, again, "I 00", "101 ", is "1 I 0" and "111 ". (Except for the cluster number, here is no difference, and m this is also true for multiprocessor boards 3, ~ which have respective cluster m numbers of "10" and 1 I ".) The corresponding addresses, transforn~ed into ~s bits l, 2, 3, 4 of the private external space addresses are, respectively, , 19 ''00100", "00101 ", "00110" (or P21, P22, P23) and those for the shared 1 external space addresses are, respectively, "0001 1 ", "00011", "0I011" and "0l 111" (or, again, SA, SB, SC, SI~).
Similarly, tile internal addresses of multiprocessor board 3 and 4 are respectively transformed to external addresses "01000" (P31 ), "01001"
s (P32), "01010" (P33) "00011" (S~), "0001 t" (S$), "01011" (SC). "0l 111"
(SD) and ''01100"(P41), "01101" (P42), "01110" (P43) "00021" (SA), "0001 t" (SB), "01011" (SC), "0l I I 1" (SD).
s The basis of this transformation, which is different for the private and shared addresses, is not readily evident and requires filrther explanation.
to First, it will be noted that the most significant bit of each internal private 11 address is "0" while the most S1~711f1Callt bit of each internal shared address is 1? "I ". In the case of the private addresses, the CN of each microprocessor 13 board (cluster) is utilized as bits 1 and 2 of the external address. Bit I
, m 111ZOWn to be a "0", 1S IlOt used directly in the transformation. Bits ? and 3 of is the internal address are, respectively, used as bits 3 and 4 of the external 16 address.
m In the case of the shared addresses, bit 1 of the internal address, known is to be a "1", is not used directly in the transformation. Bits I and 2 of the m i internal address are used as bits 1 and 2 of the external address. Finally, bits 2 3 and 4 of the external address are set to "11 ".
3 In the discussion above, the teen "used as" should only be understood 4 as representative of the internal-to-external transformation process. The s external-to-internal transformation, of course, reverses the process.
6 It will now be understood why the ''O l I " internal address is not used with this particular addressing scheme; it would result in duplicate external s address ranges for private and shared space according to the transformation process.
in Exemplary transformation apparatus for the address translator 18 of ii CPUO 1l is shown in FIGS. 6 and 7. FIG. 6 illustrates this apparatus 12 operating in the internal-to-external transformation mode in which an internal address, including a cluster number, is sent to the address translator 18 as a m request from the primary cache directed to the secondary cache 7 (which, of Is course, will forward the request to main memory if the requested information i6 is not resident in the secondary cache at the time of the request). The internal m address is thus present in register 1 S to serve as an input to the address is translator iii conjunction with the cluster number in register 16.
For this transforniation, bit () of the internal address is key. If it is a 2 "0", the address is identified as private to the requesting multiprocessor board s which itself is identified by CNO, CN l in regrister I 6. Bit 0 is inverted to a 4 "1" by inverter ~5, and the output of inverter ~5 therefore enables one input s each of AND-gates 61, 63, 6~, 67. The other input to AND-gate 61 is CNO
6 which is therefore copied throu~~h OR-gate 56 into bit I of the external 7 address in exemplary address translator output register 54. The other input to AND-gate 63 is CNl which is therefore copied through OR-gate 67 into bit 2 of the external address. The other input to AND-gate 65 is bit 1 of the m internal address which is therefore copied, via OR-gate 58, to bit 3 of the i i external address in register ~4. Similarly, the other input to AND-gate 67 is i2 bit 2 of the internal address which is therefore copied, via OR-gate 59, to bit i3 4 of the external address in register ~4. The lower order address bits 3 -14 Of the llltenlal address in register 18 are directly copied to bits 5 - 31 of the is external address in register ~4. Bit 0 of the external address in register 54 is 16 forced to logic "0" because external space is limited, in the example, to two m gigabytes. As previously mentioned, this bit is available to accommodate two is separate systems having access to the main memory.
1 ASSElllllllg by way Of example that the address being transformed has 2 been supplied by the primary cache of CPUO l 1; i.e., CNO, CN 1 are set to s "00" and that bits 0,1,2 of the internal address are "0l 0". In this example, 4 the external space addressed after the transformation by the higher order s address bits ''00010" will be the 1 /8 gigabyte private block identified in FIG.
6 3 as P13.
In another example, assume that bit 0 of an internal address sent to the s address translator 18 is a logic ''I" indicating a call to shared space.
Since bit y 0, in this instance, will be inverted to a logic "0" by the inverter 5~, the AND-io gates 61, 63, 65, 67 will be disabled. Instead, AND-gates 60, 62, 64, 66 will r r each have one input enabled as a result of the logic "I" present in bit 0 of i2 register 53. The other input to AND-gate 60 comes from bit 1 of register 53 i3 such that the logic level of bit I in the internal address is copied, via OR-gate m ~6, into bit I of the external address in register ~4. The other input to AND-is gate 62 comes from bit 2 of register ~3 such that the logic level of bit 2 in the t6 internal address is copied, via OR-gate ~7, into bit 2 of the extenlal address in t~ register 54. ' is The other inputs to each of the AND-~:ates 64, 66 are directly tied to 1> logic "1" such that bits 3 and 4 of the external address in register 54 are set to 1 ''1" via OR-gates ~8, ~9, respectively. Again, bit 0 of the external address in 2 regnster 54 is forced to ''0" because of the two gigabyte limitation on external 3 space in the example. Lower order address bits 3 - 29 of the internal address 4 are directly copied into bits 5 - 31 of the external address as before.
s Assume now that bits 0, I ,2 of the internal address supplied by the 6 multiprocessor board l are "1l l''. In this example, the external space addressed after the transformation by the higher order address bits "01111"
will be the 1/8 gigabyte shared block identified in FIG. 3 as SD.
IIl the. example, the CPLJ on a ~,nven multiprocessor board making a m request to the secondary cache 7 is conventionally tracked by the secondary r r cache. Thus, the contents of the register 17 containing the cluster member i2 number is sent to the secondary cache for that purpose.
13 FIG. 7 illustrates the address translator 18 operating in the external-to-is internal address transforniation mode. An external address from the is secondary cache is placed in register 68, representative of any appropriate i6 logic for the purpose. Bit 0 of the external address is not used because of the m two gigabyte capacity of the main memory. Bits 1 and 2 are applied to la corresponding inputs to multiplexer pair 84, 85 which selectively drive inputs m to register 86 which is representative of any appropriate lo~,~ic for interfacing the primary cache with the address translator. In addition, bits 1 and 2 in the 2 relrister 68 are applied as respective inputs to comparators 81, 82 which have CNO and CNI in regrister 16 as their respective second inputs. If there is a 4 match in both comparators 81, 82, this state conflnns that cluster number s "00" is the destination of the external address identified as private space 6 reserved for one of xnliltlprocessur board 1 (board ''0") as determined by the status of bits 3 and 4 of the external address; i.e., if one or both of bits 3 and s 4 are logic "0". With both comparators 81, 82 indicating a compare state, y two of three inputs to AND-gate 83 are enabled. The third input to AND-io gate 83 is driven by the output fi-om AND-gate 89 which, in turn, is driven by 1 i the two outputs from comparators 87, 88. First respective inputs to the i2 comparators 87, 88 are CMO, CMl from register 17 while the second i3 respective inputs thereto are the cluster member number received fi-om the i4 secondary cache which has conventionally previously recorded that CPUO in is the local cluster made the original request for access to the memor5l block i6 identified by the external address. Thus, if there is a compare condition in all m the comparators 81, 82, 87, 88, AND-gate 83 is fully enabled to enable the m multiplexer 84 to transfer the transformed external-to-internal address to i9 register 86.
i Since private space is indicated in the example, one or the other of bits 2 3 and 4 in register 68 are "0", and AND-gate 70 is disabled such that its 3 output is "0" and the output of inverter 71 is "1 ". This condition enables 4 AND-gate 72 to transfer a logic "0", present at One Of 1tS lIlplItS, through OR-s gate 78, through multiplexer 84 and into bit 0 of register 86. Similarly, the output of inverter 71 enables AND-gate 7 4 to transfer bit 3 in register 68 through OR-gate 79, muItiplexer 84 and into bit 1 of relrister 86. The output s of inverter 71 also enables AND-gate 76 to transfer bit 4 in register 68 through OR-gate 80, multiplexer 84 and into bit 2 of register 86. Finally, bits 1o S-31 in register 68 are transfewed through multipiexer 84 to bits 3-?9 in i i register 86 to complete the external-to-internal transformation of the address i2 information and the selection of the con-ect CPU in the correct cluster to i3 receive requested information in the private storage area of the mass memory.
Tf both bits 3 and 4 of the external address are ''1 ", a shared address is is indicated, and AND-gate 70 is fiilly enabled. The resulting '' I " at the output of AND-gate 70 enables one input each of AND-gates 73, 75, 77, 90. Since m the indication is that the information is not private, there is no need to is consider the cluster number. Thus, if the cluster member number from the m secondary cache identifies CPUO as previously described, the output from r AND-gate 89 frilly enables the AND-gate 90 to enable multiplexer 8~ to 2 transfer inputs thereto into the register 86.
Because the other input to AND-gate 73 is driven by a lot,~ic ''1", a ''1" ' 4 is copied into bit 0 of register 86, via OR-gate 78 and multiplexer 85, to provide the inten gal address indicator for shared space. The other inputs to AND-gates 7~, 76 are, respectively, from bits 1, 2 in register 68. As a result, the contents of bits l, 2 in register 68 are copied into bits l, 2 of register x via OR-gates 79, 80, respectively, and multiplexer 85. Bits ~-31 of the external address in register 68 are copied through multiplexer 8~ into bits 3-n> 29 of the internal address in register 86 as the lower order address bits r r common to the internal and external addresses.
i2 It will now be understood, in the exemplary embodiment of the i3 invention shown in FIGs. 6 and 7, that each multiprocessor board (cluster) m loses the capacity to address 1 /8 gigabyte of the one gigabyte inherently i5 addressable by a 30 bit address field. However, what is gained is the ability i6 to directly communicate among all the multiprocessor boards via the shared m space. Further, there is achieved the ability of any multiprocessor board to m indirectly adduess the private space of another multiprocessor board by m sending a suitable request via shared space which results in the desired 1 infornlation being moved from private space into shared space. In addition, 2 the previously discussed advantage of permitting each multiprocessor board 3 to operate under its own operating system is achieved. Still further, the 4 capability to enjoy the benefits of rumling separate operating systems in each s cluster is facilitated and eWanced.
The example of FIGs. 3 - 7, inclusive is only one of many address ~ transfonnations which may be established according to the present invention.
s For example, consider briefly a system in which each multiprocessor board can, as before, inherently directly address one gigabyte while the main IllelilOry can store four gigabytes. Then, bit 0 of the external address would i 1 come into play, aad the logric of FIGS. 6 and 7 would be accordingly adjusted i2 and expanded. In one simple configuration, the cluster number for each is multiprocessor board may be extended to three bits with the most sigluficant m cluster number bit copied to and from bit 0 of the external address. This arraugen~ent will expand the ncunber of fom-multiprocessor clusters which i6 can be accommodated as well as expand the directly and indirectly m addressable external memory. Alternatively, the clusters can be reduced to is two multiprocessors per cluster (i.e., two cluster per multiprocessor board in m the exemplary system). The result will be to use more external space as i private space, each private space block being used by fewer processors.
2 Thus, it will be understood that the address transformations available according to the invention are many and that the foregoing example treated in ' 4 detail is only representative of one found to be especially applicable to a s specific system in which the available external space was two gigabytes 6 whereas the processors are limited to directly addressing one gigabyte. It has also been previously mentioned that bit 0 of the external address can be used a switch between tu~o separate mufti-cluster systems which access common y mass memory.
io Thus, while the principles of the lIlV2Ilt1011 have now been made clear m in an illustrative embodiment, there will be immediately obvious to those 12 skilled in the art many modifications of structure, arrangements, proportions, i3 the elements, materials, and components, used in the practice of the invention m which are particularly adapted for specific environments and operating is requirements without departing fi~om those principles.
i3 Objects of the Invention It is therefore a broad object of tlIlS IIlVellt1011 to provide an improved Is mufti-processor computer SySte111 II1COI~7oratlllg extended memory addressing 16 with total system memory beyond that seen by a single CPU or operating I~ system.
Is It is a more specific object of this invention to provide an improved I~ multiprocessor system employing a cluster architecture in which an external WO 97/i3191 PCT/US96/15937 1 memory space, larger than that inherently directly addressable by the 2 processors employed, is divided into private spaces for each multiprocessor 3 cluster and shared spaces directly accessible by all the clusters according, to an address transformation which incorporates cluster member number s information for identifying each cluster and each member thereof.
Summary of the Invention Briefly, these and other objects of the invention are achieved, in an s exemplary system which includes an external memory space (e.g., main memory) a power of two larger than the internal memory space which is io inherently directly addressable by an address field generatedlinterpreted by i 1 the individual processors, by dividing the processors into multiple clusters i2 with multiple processors in each. Each CPU in a cluster of processors, fom t3 on a single multiprocessor board in the example, is assigned a cluster number m which is set, for example, by a two-bit lunber (thus identifying four clusters).
m In addition, each CPU is assiy ed a two-bit number (thus identifying four i6 CPUs) setting which CPU it is in its cluster. In the example, the two two-bit m numbers are concatenated to provide each CPU with a unique four-bit is identification number in the system.
i In an internal-to-external address transformation mode, the cluster ' 2 number is sent, along with an internal address which includes an indicator bit s specifying private/shared classification, from a requesting cluster to all 4 address translator. In the address translator, the indicator bit is used to s control the set up of the transformation.
6 In the exemplal-y system in which each CPLl can inherently directly address one gigabyte (30 bit address field identified as bits 0-29 with bit 0 being the most significant), the available external space is two grigabytes and the length of tl3e external space address field is 32 bits (bits 0-3 l , capable of I(1 SpeCIfyIilg fOllr gigabytes), bit U of the internal address is used as the I ~ indicator. If bit 0 is a ''0", tile cluster member is requesting information from 12 its private area of external space; conversely, if it is a '' I ", the cluster member i3 is addressing information in the shared regions of external space.
I4 If the indicator bit is a "0", the CILISter IlIIIIIbeI' is copied into bits l, 2 of 1s the external address, bits l, 2 of tile internal address are copied into bits 3, 4 I6 of the external address and the lower order address bits 3-29 of the internal a address are copied directly to bits ~-31 of the external address. The resulting ~s external address points to the specified pI-ivate region in external space.
l If, however, the indicator bit 0 in the internal address is a "I"
2 indicating a call to a shared region of external space, then a different transfonnation takes place. CNO, CN 1 are ilnlored. Bit 1 of the internal address is copied to bit 1 of the external address while bits 2, 3 of the external s address are each forced to logic "I ". The resulting external address points to 6 the specified shared region in external space.
In an external-to-internal address transformation mode, bits l, 2 of the s external address are copied into positions CNU, CNI of the internal address The external address indicator bits are bits 3, 4. If one or the other or both lo are "0", then an address from the private space of one of the clusters is l l indicated. Bit 0 of the internal address is forced to "0", and bits 3, 4 of the i2 external address are copied to bits I , of the internal address. The lower order bits 5 - 31 of the external address are copied directly to bits 3 - 29 of the m internal address. Thus, the developed internal address fully identifies an is inforniation block in external memory as that of the private space reserved to i6 the cluster identified by CNO, CNl .
m If both bits 3, 4 of the external address are "I ", then the external space is is known to be shared. Bit 0 of the internal address is forced to "1", bits 1, 2 t9 of the external address are copied to bits I , 2 of the internal address, and the lower order address bits are copied as before. The resulting internal address identifies an information block in external memory that is available to all the clusters.
In accordance with the present invention, there is provided a cluster computer system comprising: A) a plurality of clusters, each said cluster including: 1) a plurality of processors, each said processor having inherent limitations on the range of memory which is directly addressable by a processor address field, said processor address field being divided into higher order and lower order segments, each said processor including: a) a primary cache; and b) an address translator; and 2) a secondary cache; B) a memory having a capacity which exceeds the range each said processor is inherently capable of directly addressing, said memory being addressable by a memory address field, said memory address field exceeding said processor address field, said memory address field being divided into higher order and lower order segments; C) means establishing a cluster number for each said cluster; D) means coupling said plurality of clusters, via said secondary caches, to said memory; and E) each said address translator in each said proces~~or in each said cluster being coupled intermediate said primary cache in its said processor and said secondary cache in its said cluster, said address translator including first address translation logic responsive to processor address field and cluster number information in said primary cache in its said processor, said first translation logic comprising means responsive to the digital state of a predetermined indicator bit in said higher order segment of said processor address field by effecting one of two address transformations; 1) if said predetermined indicator bit is a first digital value; a) forcing the two lowest order bits of said higher order segment of said memory address field to a first logic level;
b) copying the two lowest order bits of said higher order segment of said processor address field to two higher order bits of said higher order segment of said memory address field; and c) copying the contents of said lower order segment of said processor address field to said lower order segment of said memory address field; thereby specifying a region of memory addressable by all said clusters; and 2) if said predetermined indicator bit is a second digital value:
a) copying the two lowest order bits of said higher order segment of said processor address field to the two lowest order bits of said higher order segment of said memory address field; b) copying the cluster number to two higher order bits of said higher order segment of said memory address field; and c) copying the contents of the lower order segment of said processor- address field to said lower order segment of said memory address field; thereby specifying a region of memory addressable only by the cluster identified by said cluster number.
In accordance with the present invention, there is provided a cluster computer system comprising: A) a plurality of clusters, each said cluster including: 1) a plurality of processors, each said processor having inherent limitations on the range of memory which is directly addressable by a processor address field, said processor address field being divided into higher order and lower order segments, each said processor including: a) a primary cache; and b) an address translator; and 2) a secondary cache; B) a memory having a capacity which exceeds the range each said processor is inherently capable of directly addressing, said memory being addressable by a memory address field, said memory address field exceeding said 7a processor address field, said memory address field being divided into higher order and lower order segments; C) means establishing a cluster number for each said cluster; D) means coupling said plurality of clusters, via said secondary caches, to said memory; and E) each said address translator in each said processor in each said cluster being coupled intermediate said primary cache in its said processor and said secondary cache in its said cluster, said address translator including second address translation logic responsive to memory address field information, said second translation logic comprising means responsive to the digital states of predetermined indicator bits in said memory address field, which predetermined indicator bits are of a lower order than the most significant bit thereof, by effecting one of two address transformations; 1) if said predetermined indicator bits are both a first digital value:
a) forcing a predetermined bit of said higher order segment of said processor address field to a first logic level; b) copying two higher order bits of said higher order segment of said memory address field to the two lowest order bits of said higher order segment of said processor address field;
and c) copying the contents of the lower order segment of said memory address field to said lower order segment of said processor address field; thereby specifying a memory region containing information accessible by all said clusters; and 2) if either of said predetermined indicator bits are a second digital value: a) copying the two lowest order bits of said higher order segment of said memory address field to the two lowest order bits of said higher order segment of said memory processor address field to the two lowest order bits of said higher order segment of said processor address field; b) copying higher bits, representing a cluster number, in said higher order section of said memory address field to said cluster number at said 7b common cluster interface; and c) copying the contents of said lower order segment of said memory address field to said lower order segment of said processor address field thereby specifying a memory region containing information accessible only by the cluster specified by said cluster number.
In accordance with the present invention, there is also provided a cluster computer system comprising: A) a plurality of clusters, each said cluster including: 1) a plurality of processors, each said processor having inherent limitations on the range of memory which is directly addressable by a processor address field, said processor address field being divided into higher order and lower order segments, each said processor including: a) a primary cache; and b) an address translator; and 2) a secondary cache; B) a memory having a capacity which exceeds the range each said processor is inherently capable of directly addressing, said memory being addressable by a memory address field, said memory address field exceeding said processor address field, said memory address field being divided into higher order and lower order segments; C) means establishing a cluster number for each said cluster; D) means coupling said plurality of clusters, via said secondary caches, to said memory; and E) each said address translator in each said processor in each said cluster being coupled intermediate said primary cache in its said processor and said secondary cache in its said cluster, said address translator including first address translation logic responsive to processor address field and cluster number in said primary cache in its said processor, said first translation logic comprising means responsive to the digital state of a first predetermined indicator bit in said higher order segment of said processor address field, which said 7c first predetermined indicator bit is of a lower order than the most significant bit thereof, by effecting one of two address transformations; 1) if said first predetermined indicator bit is a first digital value: a) forcing the two lowest order bits of said higher order segment of said memory address field to a first logic level; b) copying the two lowest order bits of said higher order segment of said processor address field to two higher order bits of said higher order segment of said memory address field; and c) copying the contents of said lower order segment of said processor address field to said lower order segment of said memory address field; thereby specifying a region of memory addressable by all said clusters; and 2) if said first predetermined indicator bit is a second digital value: a) copying the two lowest order bits of said higher order segment of said processor address field to the two lowest order bits of said higher order segment of said memory address field; b) copying the cluster number to two higher order bits of said higher order segment of said memory address field; and c) copying the contents of the lower order segment of said processor address field to said lower order segment of said memory address field; thereby specifying a region of memory addressable only by the cluster identified by said cluster number; and F) said address translator further including second address translation logic responsive to memory address field information, said second translation logic comprising means responsive to the digital states of second and thirds predetermined indicator bits in said memory address field, which second and third predetermined indicator bits are of a lower order than the most significant bit thereof, by effecting one of two address transformations; 1) if said second and third predetermined indicator bits are both a first digital value: a) forcing a predetermined bit of said 7d higher order segment of said processor address field to a first logic level; b) copying two higher order bits of said higher order segment of said memory address field to the two lowest order bits of said higher order segment of said processor address field; and c) copying the contents of the lower order segment of said memory address field to said lower order segment of said processor address field; thereby specifying a memory region containing information accessible by all said clusters; and 2) if either of said second and third predetermined indicator bits are a second digital value: a) copying the two lowest order bits of said higher order segment of said memory address field to the two lowest order bits of said higher order segment of said processor address field; b) copying higher bits, representing a cluster number, in said higher order section of said memory address field to said cluster number at said common cluster interface; and c) copying the contents of said lower order segment of said memory address field to said lower order segment of said processor address field thereby specifying a memory region containing information accessible only by the cluster specified by said cluster number.
Description of the Drawin The subject matter of. the invention is particularly pointed out and distinctly claimed in the concluding portion of the specification. The invention, however, both as to organization and method of operation, may best be understood by reference to the following description taken in conjunction with the subjoined claims and the accompanying drawing of which:
FIG.1 is a high level block diagram of a cluster computer system in which the present invention may be practiced;
7e FIG. 2 is a block diagram of the system shown in FIG. 1 illustrating more detail of the individual clusters and CPUs;
FIG. 3 is an exemplary representation of the allocation of internal address space as allocated to the individual mufti-processor clusters and external space representing the address space of a main memory device;
FIG. 4 is an exemplary transformation diagram illustrating an embodiment of the invention in which each CPU is inherently capable of 7f 1 addressing one gigabyte of information, and a main memory device in communication with all the CPUs is capable of storing two grigabytes of s information;
4 FIG. 5 is a representation of the two types of addresses, internal and s external, which are transformed according to the invention;
6 FIG. 6 is a simplified logic diagram illustrating the apparatus within an address translator component of each CPU operating in an internal-to-external x mode; and FIG. 7 is a simplified iog.~ic dia~,~ram of an address translator operating io in an exten~al-to-internal mode.
1 i Description of the Preferred Embodiments) t2 Referring first to the exemplary cluster computer system of FIG. l, it i3 will be observed that the system includes first l, second 2, third 3 and fourth is 4 multiprocessor circuit boards, each multiprocessor board including four m CPUs effecting a cluster, each multiprocessor board being connected to a i6 system bus 5 via an inte~~ral secondary cache 7. The system bus ~ couples the multiprocessor boards 1, 2, 3, 4 to one another and to various other .
is system components (including a service processor ''SP") represented by the 1 > block 6. In addition, system bus ~ couples the multiprocessor boards 1, 2, 3, i 4 to a memory contrbller 10 which, in turn, interfaces via a memory bus 9 2 with a main memory 8 representative of one or more main memory devices.
As illustrated for multiprocessor board I, each multiprocessor board l, :~ 2, 3, 4 includes four CPUs 1 l, 12, I3, 14. Tl~e four CPUs I l, 12, 13, 14 are s provided with respective primary caches 11 C, I 2C, 13C, 14C. In the 6 exemplary system, the primary caches on each multiprocessor board are each ~ disposed between its directly associated CPU and a common secondary cache s 7 and utilize a the classical ''store-into-cache" configuration. The secondary ~ cache 7 interfaces with the system bllS J. Interposed intermediate each of the u> primary caches 1 I C, 12C, 13C, 14C and the common (to tile cluster) n secondary cache 7 are address translators 11 A, 12A, I 3A, I 4A which are l2 also constituents of their respective processors 11, I 2, 13, 14.
i3 In accordance with a preferred aspect of the invention, each of the m multiprocessor boards I, 2, 3, 4 constitutes a cluster, and each cluster nzns is under its own operating system to improve system efficiency.
i6 Attention is now directed to F1G. 2 which illustrates the environment of m the subject invention in more detail. Thus, it will be seen that CPU I1 on is multiprocessor board 1 (board "0") communicates with the system bus 5 via i9 an address register l ~, address translator I 8 and secondary cache 7.
Register i I S is merely a convenient representation of address interface circuitry in the 2 primary cache of the CPU 11 by which an address generated by the CPU 11 may be transmitted, transformed in the address translator I8, as a request to 4 the secondary cache 7. This condition occurs when information required by s the CPU 11 is not resident in at least one of the primary caches of the CPUs 11, 12, 13, 14 on the multiprocessor board 1. (Those skilled in the art will understand that, in many such multiprocessor configurations, it is possible for one CPU to "siphon" information from another CPU's primary cache.) y Corresponding CPUs 21, 22, ?3, ?4 are found on multiprocessor board io 2 (board ''1"). Similarly, corresponding CPUs 31, 32, 33, 34 and 41, 42, 43, i i 44 are found on multiprocessor boards 3 (board "2"), 4 (board "3"), i2 respectively. In addition, registers ?~, 3~, 4~, con-esponding to register 1~, i3 are respectively found on multiprocessor boards 2, 3, 4. Also, address m translators 28, 38, 48, correspondinb to address translator 18, are respectively is found CPUO on each of multiprocessor boards 2, 3, 4.
i6 Also shown incorporated in the primary cache of CPU 11 on the m multiprocessor board 1 are f rst and second two-bit identification registers, m 16, 17, respectively. (It will be understood that the re~~isters 16, 17 are m representative of any convenient means for establishing a two-bit cluster to 1 identification for the multiprocessor board 1 and a two-bit cluster member identification for the CPU 11. For example, a cluster/cIuster member identification could be hard-wired in or exist as four switches which can be 4 set to a complete cluster member number.) In the case of CPU 11 in s multiprocessor I, the foul- stages of the registers 16, 17 are each set to the 6 logric ''0" position, thereby providing a cluster identification of "00"
(board "0") and a cluster member identification of "00" (CPU ''0"). As a practical matter, the regristers I6, 17 may be concatenated and may be set upon system initialization under control of the service processor- SP as represented by the to dashed line 20.
11 COITeSpOlldlllg register pairs, 26, 27; 36, 37; and 46, 47 are provided 12 on multiprocessor boards 2, 3, 4, for respective CPUOs 21, 3I, 41 and are 13 respectively set to provide cluster identifications of ''0l ", "10" and ''I
1" and, m in each case, cluster member identification of "00"
is In the succeeding discussion, the description will be directed to 16 multiprocessor board 1 (board "0"), it being understood that multiprocessor 1~ boards 2, 3, 4 operate identically, taking into account their differing cluster is number identifications as stored in the registers 26, 36, 46. Le., the cluster m i member number for CPU? 33 on multiprocessor board 3 is ''1010" (board "~"CPU ''~").
3 The 30-bit register 1 ~ in the primary cache of the CPU I I can specify one gigabyte of address locations which is the family Iimit of the exemplary s CPUs. There is value in resem~ing a portion of the addressable space in main 6 memory 8 to the private use of the CPUs on each multiprocessor board for, among other reasons, permitting each multiprocessor board to operate under x an independent operating system. (These independent operating systems may be the same or different operating systems.) On the other hand, there is a m need for establishing direct communication among the clusters and CPUs i i operating under the different operating systems in order that they can function i2 with the fiill power of a cluster system architecture. One way in which this i3 feature can be achieved is by pro~riding shared memory space in main m memory 8.
is The present invention provides for the enjoyment of all these features i6 while also permitting the collective direct and indirect address of more m memory spaces in main memory than the CPUs in the system are individually i8 inherently capable. -l In the fOIIOWIIIg dlSCliSS1011, the term "internal space" means the 2 memory addressable by the CPUs on each multiprocessor board while the ' 3 term "external space" means the total memory addressable; i.e., the memory 4 space available in main memory 8. These two types of spaces exist at tvvo s interfaces of, and are reconciled by, the address translator 18 which, in the 6 exemplary system, is disposed in the primary cache of each CPU.
The concept of internal and external spaces and private and shared x memory in the present context may be better understood with reference to FIG. 3. In this example, it may be assumed that each CPU has a 30-bit lo address field and can therefore directly address one gigabyte. It may further l l be assumed that the available external space in the exemplary main memory is iz two gigabytes and that the external address field is 3?-bits long; i.e., capable i3 of addressing four gigabytes.
i4 A key aspect of the present invention the mamer in which is is addressable memoryforeachmultiprocessor is allocated. For board the 16 external memory space to be accessible, as a result of address transformation, m by each multiprocessor board 1, 2, 3, 4, three-eighths of the inherently is addressable internal space is private and one-half is shared. Comment on the m apparently remaining one-eighth is reserved for discussion below.
1 Under these conditions, the memory allocation may be appreciated 2 from an examination of FIG. 3. Multiprocessor board 1 enjoys 3/8th gigabyte private memory space identified as //8th gigabyte segments P1 I, P12, PI3.
In addition, Multiprocessor board I has access to I/2 I,~igabyte shared s memory space in I /8th gigabyte segments SA, SB, SC, SD. Similarly, 6 Multiprocessor board 2 enjoys 3/8th gigabyte private memory space identified as 1 /8th gigabyte se~nnents P21, P22, P23. In addition, Multiprocessor board 2 has access to I /2 gigabyte shared memory space in 9 1/8th gigabyte selunents SA, SB, SC, SD. The internal space for m multiprocessor boards 3 and 4 are correspondingly assil,~~ed as shown in FIG.
a 3.
l? This internal space address information is provided to the address 13 translation block which, as previously described, is present in the primary m cache of each CPU, but is shown separately as a single block in FIG. 3 for is clarity. In the addresstranslation this addressinformation block, is i6 transformable, accordingthe invention, obtainexternalspace address to to m segments distributed as shown schematically in FIG. 3.
is Consider now an implementation of this example with reference to .
m FIGS. 2, 4, 5, 6 and 7. FIG. 4 shows the higher order address bit distribution 1 for the four multiprocessor boards for both the internal and external space addresses as they are transformed by the address translation block (e.g., the ' s address translator 18 in FIG. 2). In each case, there are three columns representing, respectively, the cluster number (already set by the registers 16, s 26, 36, 46; F1G. 2), the three most sitnificant bits of the 30-bit intental 6 address and the second, third, fourth and fifth most significant bits of the bit transformed address. As will be discussed in more detail below, the most s significant bit of the 32-bit transformed address is always a ''0" in the y example.
Referring also to FIG. ~ and examining first the internal address pattern i ~ for multiprocessor board 1, the cluster number is preset to "00", the private i2 space three most significant bits are ''000", "001" and "010" and the shared t3 space three most significant bits are "I 00", "1 O l ", '' 110" and "111 ".
It will ~ be noted that, in this example, the combination "011" for the three most Is significant bits is reserved. The three highest order internal address bits may i6 be deemed the higher order segment of the processor address field and the j~ remaining bits 3-29 may be deemed the lower order setnnent thereof.
Because the address field for external addresses has 32-bits and the iy main memory stores only two gigabytes, the most sitmificant bit in the 1 external addresses is always ''0" in the example. (It may be noted, however, 2 that this most significant bit is available to denote one of two complete systems which is to be granted access to a shared main memory.) Thus, the ' a corresponding addresses, transfonned into the five most significant bits of the private external space addresses to be transformed (i.e., bits l, 2, 3, 4), are, 6 respectively, "00000", "00001 ", "000 I 0" (or P I I , P 12, P 13 ) and those for the shared external space addresses are, respectively, "00011 ", "0001 I" , x "01011" and "01111" (or SA, SB, SC, SD). The five highest order irite~~
address bits may be deemed the higher order se~nent of the memory address to field and the remaining bits ~-31 may be deemed the lower order segment i i thereof.
12 For multiprocessor board 2, the cluster number is preset to "O1", the I3 private space three most si~,mificant bits are, main, "000'', ''001" and ''010"
m and the shared space three most significant bits are, again, "I 00", "101 ", is "1 I 0" and "111 ". (Except for the cluster number, here is no difference, and m this is also true for multiprocessor boards 3, ~ which have respective cluster m numbers of "10" and 1 I ".) The corresponding addresses, transforn~ed into ~s bits l, 2, 3, 4 of the private external space addresses are, respectively, , 19 ''00100", "00101 ", "00110" (or P21, P22, P23) and those for the shared 1 external space addresses are, respectively, "0001 1 ", "00011", "0I011" and "0l 111" (or, again, SA, SB, SC, SI~).
Similarly, tile internal addresses of multiprocessor board 3 and 4 are respectively transformed to external addresses "01000" (P31 ), "01001"
s (P32), "01010" (P33) "00011" (S~), "0001 t" (S$), "01011" (SC). "0l 111"
(SD) and ''01100"(P41), "01101" (P42), "01110" (P43) "00021" (SA), "0001 t" (SB), "01011" (SC), "0l I I 1" (SD).
s The basis of this transformation, which is different for the private and shared addresses, is not readily evident and requires filrther explanation.
to First, it will be noted that the most significant bit of each internal private 11 address is "0" while the most S1~711f1Callt bit of each internal shared address is 1? "I ". In the case of the private addresses, the CN of each microprocessor 13 board (cluster) is utilized as bits 1 and 2 of the external address. Bit I
, m 111ZOWn to be a "0", 1S IlOt used directly in the transformation. Bits ? and 3 of is the internal address are, respectively, used as bits 3 and 4 of the external 16 address.
m In the case of the shared addresses, bit 1 of the internal address, known is to be a "1", is not used directly in the transformation. Bits I and 2 of the m i internal address are used as bits 1 and 2 of the external address. Finally, bits 2 3 and 4 of the external address are set to "11 ".
3 In the discussion above, the teen "used as" should only be understood 4 as representative of the internal-to-external transformation process. The s external-to-internal transformation, of course, reverses the process.
6 It will now be understood why the ''O l I " internal address is not used with this particular addressing scheme; it would result in duplicate external s address ranges for private and shared space according to the transformation process.
in Exemplary transformation apparatus for the address translator 18 of ii CPUO 1l is shown in FIGS. 6 and 7. FIG. 6 illustrates this apparatus 12 operating in the internal-to-external transformation mode in which an internal address, including a cluster number, is sent to the address translator 18 as a m request from the primary cache directed to the secondary cache 7 (which, of Is course, will forward the request to main memory if the requested information i6 is not resident in the secondary cache at the time of the request). The internal m address is thus present in register 1 S to serve as an input to the address is translator iii conjunction with the cluster number in register 16.
For this transforniation, bit () of the internal address is key. If it is a 2 "0", the address is identified as private to the requesting multiprocessor board s which itself is identified by CNO, CN l in regrister I 6. Bit 0 is inverted to a 4 "1" by inverter ~5, and the output of inverter ~5 therefore enables one input s each of AND-gates 61, 63, 6~, 67. The other input to AND-gate 61 is CNO
6 which is therefore copied throu~~h OR-gate 56 into bit I of the external 7 address in exemplary address translator output register 54. The other input to AND-gate 63 is CNl which is therefore copied through OR-gate 67 into bit 2 of the external address. The other input to AND-gate 65 is bit 1 of the m internal address which is therefore copied, via OR-gate 58, to bit 3 of the i i external address in register ~4. Similarly, the other input to AND-gate 67 is i2 bit 2 of the internal address which is therefore copied, via OR-gate 59, to bit i3 4 of the external address in register ~4. The lower order address bits 3 -14 Of the llltenlal address in register 18 are directly copied to bits 5 - 31 of the is external address in register ~4. Bit 0 of the external address in register 54 is 16 forced to logic "0" because external space is limited, in the example, to two m gigabytes. As previously mentioned, this bit is available to accommodate two is separate systems having access to the main memory.
1 ASSElllllllg by way Of example that the address being transformed has 2 been supplied by the primary cache of CPUO l 1; i.e., CNO, CN 1 are set to s "00" and that bits 0,1,2 of the internal address are "0l 0". In this example, 4 the external space addressed after the transformation by the higher order s address bits ''00010" will be the 1 /8 gigabyte private block identified in FIG.
6 3 as P13.
In another example, assume that bit 0 of an internal address sent to the s address translator 18 is a logic ''I" indicating a call to shared space.
Since bit y 0, in this instance, will be inverted to a logic "0" by the inverter 5~, the AND-io gates 61, 63, 65, 67 will be disabled. Instead, AND-gates 60, 62, 64, 66 will r r each have one input enabled as a result of the logic "I" present in bit 0 of i2 register 53. The other input to AND-gate 60 comes from bit 1 of register 53 i3 such that the logic level of bit I in the internal address is copied, via OR-gate m ~6, into bit I of the external address in register ~4. The other input to AND-is gate 62 comes from bit 2 of register ~3 such that the logic level of bit 2 in the t6 internal address is copied, via OR-gate ~7, into bit 2 of the extenlal address in t~ register 54. ' is The other inputs to each of the AND-~:ates 64, 66 are directly tied to 1> logic "1" such that bits 3 and 4 of the external address in register 54 are set to 1 ''1" via OR-gates ~8, ~9, respectively. Again, bit 0 of the external address in 2 regnster 54 is forced to ''0" because of the two gigabyte limitation on external 3 space in the example. Lower order address bits 3 - 29 of the internal address 4 are directly copied into bits 5 - 31 of the external address as before.
s Assume now that bits 0, I ,2 of the internal address supplied by the 6 multiprocessor board l are "1l l''. In this example, the external space addressed after the transformation by the higher order address bits "01111"
will be the 1/8 gigabyte shared block identified in FIG. 3 as SD.
IIl the. example, the CPLJ on a ~,nven multiprocessor board making a m request to the secondary cache 7 is conventionally tracked by the secondary r r cache. Thus, the contents of the register 17 containing the cluster member i2 number is sent to the secondary cache for that purpose.
13 FIG. 7 illustrates the address translator 18 operating in the external-to-is internal address transforniation mode. An external address from the is secondary cache is placed in register 68, representative of any appropriate i6 logic for the purpose. Bit 0 of the external address is not used because of the m two gigabyte capacity of the main memory. Bits 1 and 2 are applied to la corresponding inputs to multiplexer pair 84, 85 which selectively drive inputs m to register 86 which is representative of any appropriate lo~,~ic for interfacing the primary cache with the address translator. In addition, bits 1 and 2 in the 2 relrister 68 are applied as respective inputs to comparators 81, 82 which have CNO and CNI in regrister 16 as their respective second inputs. If there is a 4 match in both comparators 81, 82, this state conflnns that cluster number s "00" is the destination of the external address identified as private space 6 reserved for one of xnliltlprocessur board 1 (board ''0") as determined by the status of bits 3 and 4 of the external address; i.e., if one or both of bits 3 and s 4 are logic "0". With both comparators 81, 82 indicating a compare state, y two of three inputs to AND-gate 83 are enabled. The third input to AND-io gate 83 is driven by the output fi-om AND-gate 89 which, in turn, is driven by 1 i the two outputs from comparators 87, 88. First respective inputs to the i2 comparators 87, 88 are CMO, CMl from register 17 while the second i3 respective inputs thereto are the cluster member number received fi-om the i4 secondary cache which has conventionally previously recorded that CPUO in is the local cluster made the original request for access to the memor5l block i6 identified by the external address. Thus, if there is a compare condition in all m the comparators 81, 82, 87, 88, AND-gate 83 is fully enabled to enable the m multiplexer 84 to transfer the transformed external-to-internal address to i9 register 86.
i Since private space is indicated in the example, one or the other of bits 2 3 and 4 in register 68 are "0", and AND-gate 70 is disabled such that its 3 output is "0" and the output of inverter 71 is "1 ". This condition enables 4 AND-gate 72 to transfer a logic "0", present at One Of 1tS lIlplItS, through OR-s gate 78, through multiplexer 84 and into bit 0 of register 86. Similarly, the output of inverter 71 enables AND-gate 7 4 to transfer bit 3 in register 68 through OR-gate 79, muItiplexer 84 and into bit 1 of relrister 86. The output s of inverter 71 also enables AND-gate 76 to transfer bit 4 in register 68 through OR-gate 80, multiplexer 84 and into bit 2 of register 86. Finally, bits 1o S-31 in register 68 are transfewed through multipiexer 84 to bits 3-?9 in i i register 86 to complete the external-to-internal transformation of the address i2 information and the selection of the con-ect CPU in the correct cluster to i3 receive requested information in the private storage area of the mass memory.
Tf both bits 3 and 4 of the external address are ''1 ", a shared address is is indicated, and AND-gate 70 is fiilly enabled. The resulting '' I " at the output of AND-gate 70 enables one input each of AND-gates 73, 75, 77, 90. Since m the indication is that the information is not private, there is no need to is consider the cluster number. Thus, if the cluster member number from the m secondary cache identifies CPUO as previously described, the output from r AND-gate 89 frilly enables the AND-gate 90 to enable multiplexer 8~ to 2 transfer inputs thereto into the register 86.
Because the other input to AND-gate 73 is driven by a lot,~ic ''1", a ''1" ' 4 is copied into bit 0 of register 86, via OR-gate 78 and multiplexer 85, to provide the inten gal address indicator for shared space. The other inputs to AND-gates 7~, 76 are, respectively, from bits 1, 2 in register 68. As a result, the contents of bits l, 2 in register 68 are copied into bits l, 2 of register x via OR-gates 79, 80, respectively, and multiplexer 85. Bits ~-31 of the external address in register 68 are copied through multiplexer 8~ into bits 3-n> 29 of the internal address in register 86 as the lower order address bits r r common to the internal and external addresses.
i2 It will now be understood, in the exemplary embodiment of the i3 invention shown in FIGs. 6 and 7, that each multiprocessor board (cluster) m loses the capacity to address 1 /8 gigabyte of the one gigabyte inherently i5 addressable by a 30 bit address field. However, what is gained is the ability i6 to directly communicate among all the multiprocessor boards via the shared m space. Further, there is achieved the ability of any multiprocessor board to m indirectly adduess the private space of another multiprocessor board by m sending a suitable request via shared space which results in the desired 1 infornlation being moved from private space into shared space. In addition, 2 the previously discussed advantage of permitting each multiprocessor board 3 to operate under its own operating system is achieved. Still further, the 4 capability to enjoy the benefits of rumling separate operating systems in each s cluster is facilitated and eWanced.
The example of FIGs. 3 - 7, inclusive is only one of many address ~ transfonnations which may be established according to the present invention.
s For example, consider briefly a system in which each multiprocessor board can, as before, inherently directly address one gigabyte while the main IllelilOry can store four gigabytes. Then, bit 0 of the external address would i 1 come into play, aad the logric of FIGS. 6 and 7 would be accordingly adjusted i2 and expanded. In one simple configuration, the cluster number for each is multiprocessor board may be extended to three bits with the most sigluficant m cluster number bit copied to and from bit 0 of the external address. This arraugen~ent will expand the ncunber of fom-multiprocessor clusters which i6 can be accommodated as well as expand the directly and indirectly m addressable external memory. Alternatively, the clusters can be reduced to is two multiprocessors per cluster (i.e., two cluster per multiprocessor board in m the exemplary system). The result will be to use more external space as i private space, each private space block being used by fewer processors.
2 Thus, it will be understood that the address transformations available according to the invention are many and that the foregoing example treated in ' 4 detail is only representative of one found to be especially applicable to a s specific system in which the available external space was two gigabytes 6 whereas the processors are limited to directly addressing one gigabyte. It has also been previously mentioned that bit 0 of the external address can be used a switch between tu~o separate mufti-cluster systems which access common y mass memory.
io Thus, while the principles of the lIlV2Ilt1011 have now been made clear m in an illustrative embodiment, there will be immediately obvious to those 12 skilled in the art many modifications of structure, arrangements, proportions, i3 the elements, materials, and components, used in the practice of the invention m which are particularly adapted for specific environments and operating is requirements without departing fi~om those principles.
Claims (12)
1. A cluster computer system comprising:
A) a plurality of clusters, each said cluster including:
1) a plurality of processors, each said processor having inherent limitations on the range of memory which is directly addressable by a processor address field, said processor address field being divided into higher order and lower order segments, each said processor including:
a) a primary cache; and b) an address translator; and 2) a secondary cache;
B) a memory having a capacity which exceeds the range each said processor is inherently capable of directly addressing, said memory being addressable by a memory address field, said memory address field exceeding said processor address field, said memory address field being divided into higher order and lower order segments;
C) means establishing a cluster number for each said cluster;
D) means coupling said plurality of clusters, via said secondary caches, to said memory; and E) each said address translator in each said processor in each said cluster being coupled intermediate said primary cache in its said processor and said secondary cache in its said cluster, said address translator including first address translation logic responsive to processor address field and cluster number information in said primary cache in its said processor, said first translation logic comprising means responsive to the digital state of a predetermined indicator bit in said higher order segment of said processor address field by effecting one of two address transformations;
1) if said predetermined indicator bit is a first digital value:
a) forcing the two lowest order bits of said higher order segment of said memory address field to a first logic level, b) copying the two lowest order bits of said higher order segment of said processor address field to two higher order bits of said higher order segment of said memory address field; and c) copying the contents of said lower order segment of said processor address field to said lower order segment of said memory address field;
thereby specifying a region of memory addressable by all said clusters; and 2) if said predetermined indicator bit is a second digital value:
a) copying the two lowest order bits of said higher order segment of said processor address field to the two lowest order bits of said higher order segment of said memory address field;
b) copying the cluster number to two higher order bits of said higher order segment of said memory address field;
and c) copying the contents of the lower order segment of said processor address field to said lower order segment of said memory address field;
thereby specifying a region of memory addressable only by the cluster identified by said cluster number.
A) a plurality of clusters, each said cluster including:
1) a plurality of processors, each said processor having inherent limitations on the range of memory which is directly addressable by a processor address field, said processor address field being divided into higher order and lower order segments, each said processor including:
a) a primary cache; and b) an address translator; and 2) a secondary cache;
B) a memory having a capacity which exceeds the range each said processor is inherently capable of directly addressing, said memory being addressable by a memory address field, said memory address field exceeding said processor address field, said memory address field being divided into higher order and lower order segments;
C) means establishing a cluster number for each said cluster;
D) means coupling said plurality of clusters, via said secondary caches, to said memory; and E) each said address translator in each said processor in each said cluster being coupled intermediate said primary cache in its said processor and said secondary cache in its said cluster, said address translator including first address translation logic responsive to processor address field and cluster number information in said primary cache in its said processor, said first translation logic comprising means responsive to the digital state of a predetermined indicator bit in said higher order segment of said processor address field by effecting one of two address transformations;
1) if said predetermined indicator bit is a first digital value:
a) forcing the two lowest order bits of said higher order segment of said memory address field to a first logic level, b) copying the two lowest order bits of said higher order segment of said processor address field to two higher order bits of said higher order segment of said memory address field; and c) copying the contents of said lower order segment of said processor address field to said lower order segment of said memory address field;
thereby specifying a region of memory addressable by all said clusters; and 2) if said predetermined indicator bit is a second digital value:
a) copying the two lowest order bits of said higher order segment of said processor address field to the two lowest order bits of said higher order segment of said memory address field;
b) copying the cluster number to two higher order bits of said higher order segment of said memory address field;
and c) copying the contents of the lower order segment of said processor address field to said lower order segment of said memory address field;
thereby specifying a region of memory addressable only by the cluster identified by said cluster number.
2. The cluster computer system of Claim 1 in which said higher order segment of said processor address field comprises three bits and in which said higher order segment of said memory address field comprises at least five bits.
3. The cluster computer system of Claim 2 in which the most significant bit of said higher order segment of said processor address field is employed as said predetermined indicator bit.
4. The cluster computer system of Claim 3 which further includes means for establishing a cluster member number for each said processor in each said cluster and in which said cluster member number of a given said processor is sent to said secondary cache in said cluster in which said given said processor resides when a given memory request is made by said given said processor such that said secondary cache in said cluster in which said given said processor resides can identify which of said processors in its said cluster a response to said given memory request is to be made.
5. A cluster computer system comprising:
A) a plurality of clusters, each said cluster including:
1) a plurality of processors, each said processor having inherent limitations on the range of memory which is directly addressable by a processor address field, said processor address field being divided into higher order and lower order segments, each said processor including:
a) a primary cache; and b) an address translator; and 2) a secondary cache;
B) a memory having a capacity which exceeds the range each said processor is inherently capable of directly addressing, said memory being addressable by a memory address field, said memory address field exceeding said processor address field, said memory address field being divided into higher order and lower order segments;
C) means establishing a cluster number for each said cluster;
D) means coupling said plurality of clusters, via said secondary caches, to said memory, and E) each said address translator in each said processor in each said cluster being coupled intermediate said primary cache in its said processor and said secondary cache in its said cluster, said address translator including second address translation logic responsive to memory address field information, said second translation logic comprising means responsive to the digital states of predetermined indicator bits in said memory address field, which predetermined indicator bits are of a lower order than the most significant bit thereof, by effecting one of two address transformations;
1) if said predetermined indicator bits are both a first digital value:
a) forcing a predetermined bit of said higher order segment of said processor address field to a first logic level;
b) copying two higher order bits of said higher order segment of said memory address field to the two lowest order bits of said higher order segment of said processor address field; and c) copying the contents of the lower order segment of said memory address field to said lower order segment of said processor address field;
thereby specifying a memory region containing information accessible by all said clusters; and 2) if either of said predetermined indicator bits are a second digital value:
a) copying the two lowest order bits of said higher order segment of said memory address field to the two lowest order bits of said higher order segment of said processor address field;
b) copying higher bits, representing a cluster number, in said higher order section of said memory address field to said cluster number at said common cluster interface; and c) copying the contents of said lower order segment of said memory address field to said lower order segment of said processor address field thereby specifying a memory region containing information accessible only by the cluster specified by said cluster number.
A) a plurality of clusters, each said cluster including:
1) a plurality of processors, each said processor having inherent limitations on the range of memory which is directly addressable by a processor address field, said processor address field being divided into higher order and lower order segments, each said processor including:
a) a primary cache; and b) an address translator; and 2) a secondary cache;
B) a memory having a capacity which exceeds the range each said processor is inherently capable of directly addressing, said memory being addressable by a memory address field, said memory address field exceeding said processor address field, said memory address field being divided into higher order and lower order segments;
C) means establishing a cluster number for each said cluster;
D) means coupling said plurality of clusters, via said secondary caches, to said memory, and E) each said address translator in each said processor in each said cluster being coupled intermediate said primary cache in its said processor and said secondary cache in its said cluster, said address translator including second address translation logic responsive to memory address field information, said second translation logic comprising means responsive to the digital states of predetermined indicator bits in said memory address field, which predetermined indicator bits are of a lower order than the most significant bit thereof, by effecting one of two address transformations;
1) if said predetermined indicator bits are both a first digital value:
a) forcing a predetermined bit of said higher order segment of said processor address field to a first logic level;
b) copying two higher order bits of said higher order segment of said memory address field to the two lowest order bits of said higher order segment of said processor address field; and c) copying the contents of the lower order segment of said memory address field to said lower order segment of said processor address field;
thereby specifying a memory region containing information accessible by all said clusters; and 2) if either of said predetermined indicator bits are a second digital value:
a) copying the two lowest order bits of said higher order segment of said memory address field to the two lowest order bits of said higher order segment of said processor address field;
b) copying higher bits, representing a cluster number, in said higher order section of said memory address field to said cluster number at said common cluster interface; and c) copying the contents of said lower order segment of said memory address field to said lower order segment of said processor address field thereby specifying a memory region containing information accessible only by the cluster specified by said cluster number.
6. The cluster computer system of Claim 5 in which said higher order segment of said processor address field comprises three bits and in which said higher order segment of said memory address field comprises at least five bits.
7. The cluster computer system of Claim 6 in which the two least significant bits of said higher order segment of said memory address field are employed as said predetermined indicator bits.
8. The cluster computer system of Claim 7 which further includes means for establishing a cluster member number for each said processor in each said cluster and in which said cluster member number of a given said processor is sent to said secondary cache in said cluster in which said given said processor resides when a given memory request is made by said given said processor such that said secondary cache in said cluster in which said given said processor resides can identify which of said processors in its said cluster a response to said given memory request is to be made.
9. A cluster computer system comprising:
A) a plurality of clusters, each said cluster including:
1) a plurality of processors, each said processor having inherent limitations on the range of memory which is directly addressable by a processor address field, said processor address field being divided into higher order and lower order segments, each said processor including:
a) a primary cache; and b) an address translator; and 2) a secondary cache;
B) a memory having a capacity which exceeds the range each said processor is inherently capable of directly addressing, said memory being addressable by a memory address field, said memory address field exceeding said processor address field, said memory address field being divided into higher order and lower order segments;
C) means establishing a cluster number for each said cluster;
D) means coupling said plurality of clusters, via said secondary caches, to said memory; and E) each said address translator in each said processor in each said cluster being coupled intermediate said primary cache in its said processor and said secondary cache in its said cluster, said address translator including first address translation logic responsive to processor address field and cluster number in said primary cache in its said processor, said first translation logic comprising means responsive to the digital state of a first predetermined indicator bit in said higher order segment of said processor address field, which said first predetermined indicator bit is of a lower order than the most significant bit thereof, by effecting one of two address transformations;
1) if said first predetermined indicator bit is a first digital value:
a) forcing the two lowest order bits of said higher order segment of said memory address field to a first logic level;
b) copying the two lowest order bits of said higher order segment of said processor address field to two higher order bits of said higher order segment of said memory address field; and c) copying the contents of said lower order segment of said processor address field to said lower order segment of said memory address field;
thereby specifying a region of memory addressable by all said clusters; and 2) if said first predetermined indicator bit is a second digital value:
a) copying the two lowest order bits of said higher order segment of said processor address field to the two lowest order bits of said higher order segment of said memory address field;
b) copying the cluster number to two higher order bits of said higher order segment of said memory address field;
and c) copying the contents of the lower order segment of said processor address field to said lower order segment of said memory address field;
thereby specifying a region of memory addressable only by the cluster identified by said cluster number; and F) said address translator further including second address translation logic responsive to memory address field information, said second translation logic comprising means responsive to the digital states of second and thirds predetermined indicator bits in said memory address field, which second and third predetermined indicator bits are of a lower order than the most significant bit thereof, by effecting one of two address transformations;
1) if said second and third predetermined indicator bits are both a first digital value:
a) forcing a predetermined bit of said higher order segment of said processor address field to a first logic level;
b) copying two higher order bits of said higher order segment of said memory address field to the two lowest order bits of said higher order segment of said processor address field; and c) copying the contents of the lower order segment of said memory address field to said lower order segment of said processor address field;
thereby specifying a memory region containing information accessible by all said clusters; and 2) if either of said second and third predetermined indicator bits are a second digital value:
a) copying the two lowest order bits of said higher order segment of said memory address field to the two lowest order bits of said higher order segment of said processor address field;
b) copying higher bits, representing a cluster number, in said higher order section of said memory address field to said cluster number at said common cluster interface; and c) copying the contents of said lower order segment of said memory address field to said lower order segment of said processor address field thereby specifying a memory region containing information accessible only by the cluster specified by said cluster number.
A) a plurality of clusters, each said cluster including:
1) a plurality of processors, each said processor having inherent limitations on the range of memory which is directly addressable by a processor address field, said processor address field being divided into higher order and lower order segments, each said processor including:
a) a primary cache; and b) an address translator; and 2) a secondary cache;
B) a memory having a capacity which exceeds the range each said processor is inherently capable of directly addressing, said memory being addressable by a memory address field, said memory address field exceeding said processor address field, said memory address field being divided into higher order and lower order segments;
C) means establishing a cluster number for each said cluster;
D) means coupling said plurality of clusters, via said secondary caches, to said memory; and E) each said address translator in each said processor in each said cluster being coupled intermediate said primary cache in its said processor and said secondary cache in its said cluster, said address translator including first address translation logic responsive to processor address field and cluster number in said primary cache in its said processor, said first translation logic comprising means responsive to the digital state of a first predetermined indicator bit in said higher order segment of said processor address field, which said first predetermined indicator bit is of a lower order than the most significant bit thereof, by effecting one of two address transformations;
1) if said first predetermined indicator bit is a first digital value:
a) forcing the two lowest order bits of said higher order segment of said memory address field to a first logic level;
b) copying the two lowest order bits of said higher order segment of said processor address field to two higher order bits of said higher order segment of said memory address field; and c) copying the contents of said lower order segment of said processor address field to said lower order segment of said memory address field;
thereby specifying a region of memory addressable by all said clusters; and 2) if said first predetermined indicator bit is a second digital value:
a) copying the two lowest order bits of said higher order segment of said processor address field to the two lowest order bits of said higher order segment of said memory address field;
b) copying the cluster number to two higher order bits of said higher order segment of said memory address field;
and c) copying the contents of the lower order segment of said processor address field to said lower order segment of said memory address field;
thereby specifying a region of memory addressable only by the cluster identified by said cluster number; and F) said address translator further including second address translation logic responsive to memory address field information, said second translation logic comprising means responsive to the digital states of second and thirds predetermined indicator bits in said memory address field, which second and third predetermined indicator bits are of a lower order than the most significant bit thereof, by effecting one of two address transformations;
1) if said second and third predetermined indicator bits are both a first digital value:
a) forcing a predetermined bit of said higher order segment of said processor address field to a first logic level;
b) copying two higher order bits of said higher order segment of said memory address field to the two lowest order bits of said higher order segment of said processor address field; and c) copying the contents of the lower order segment of said memory address field to said lower order segment of said processor address field;
thereby specifying a memory region containing information accessible by all said clusters; and 2) if either of said second and third predetermined indicator bits are a second digital value:
a) copying the two lowest order bits of said higher order segment of said memory address field to the two lowest order bits of said higher order segment of said processor address field;
b) copying higher bits, representing a cluster number, in said higher order section of said memory address field to said cluster number at said common cluster interface; and c) copying the contents of said lower order segment of said memory address field to said lower order segment of said processor address field thereby specifying a memory region containing information accessible only by the cluster specified by said cluster number.
10. The cluster computer system of Claim 9 in which said higher order segment of said processor address field comprises three bits and in which said higher order segment of said memory address field comprises at least five bits.
11 The cluster system of Claim 10 in which the most significant bit of said higher order segment of said processor address field is employed as said predetermined indicator bit and in which the two least significant bits of said higher order segment of said memory address field are employed as said predetermined indicator bits.
12. The cluster computer system of Claim 11 which further includes means for establishing a cluster member number for each said processor in each said cluster and in which said cluster member number of a given said processor is sent to said secondary cache in said cluster in which said given said processor resides when a given memory request is made by said given said processor such that said secondary cache in said cluster in which said given said processor resides can identify which of said processors in its said cluster a response to said given memory request is to be made.
Applications Claiming Priority (3)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| US08/540,106 | 1995-10-06 | ||
| US08/540,106 US5590301A (en) | 1995-10-06 | 1995-10-06 | Address transformation in a cluster computer system |
| PCT/US1996/015937 WO1997013191A1 (en) | 1995-10-06 | 1996-10-04 | Address transformation in a cluster computer system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CA2211083A1 CA2211083A1 (en) | 1997-04-10 |
| CA2211083C true CA2211083C (en) | 2003-05-20 |
Family
ID=24154013
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CA002211083A Expired - Fee Related CA2211083C (en) | 1995-10-06 | 1996-10-04 | Address transformation in a cluster computer system |
Country Status (6)
| Country | Link |
|---|---|
| US (1) | US5590301A (en) |
| EP (1) | EP0855057B1 (en) |
| JP (1) | JPH11512857A (en) |
| CA (1) | CA2211083C (en) |
| DE (1) | DE69635865T2 (en) |
| WO (1) | WO1997013191A1 (en) |
Families Citing this family (29)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JPH08235141A (en) * | 1995-02-28 | 1996-09-13 | Kofu Nippon Denki Kk | Information processing system |
| US5884098A (en) * | 1996-04-18 | 1999-03-16 | Emc Corporation | RAID controller system utilizing front end and back end caching systems including communication path connecting two caching systems and synchronizing allocation of blocks in caching systems |
| US5940870A (en) * | 1996-05-21 | 1999-08-17 | Industrial Technology Research Institute | Address translation for shared-memory multiprocessor clustering |
| US5933857A (en) * | 1997-04-25 | 1999-08-03 | Hewlett-Packard Co. | Accessing multiple independent microkernels existing in a globally shared memory system |
| EP1010275A1 (en) * | 1997-07-02 | 2000-06-21 | Creative Technology Ltd. | Audio effects processor having decoupled instruction execution and audio data sequencing |
| JP3866426B2 (en) * | 1998-11-05 | 2007-01-10 | 日本電気株式会社 | Memory fault processing method in cluster computer and cluster computer |
| US6314501B1 (en) | 1998-07-23 | 2001-11-06 | Unisys Corporation | Computer system and method for operating multiple operating systems in different partitions of the computer system and for allowing the different partitions to communicate with one another through shared memory |
| JP3976432B2 (en) * | 1998-12-09 | 2007-09-19 | エヌイーシーコンピュータテクノ株式会社 | Data processing apparatus and data processing method |
| US6801937B1 (en) | 2000-05-31 | 2004-10-05 | International Business Machines Corporation | Method, system and program products for defining nodes to a cluster |
| US7769823B2 (en) * | 2001-09-28 | 2010-08-03 | F5 Networks, Inc. | Method and system for distributing requests for content |
| US6868483B2 (en) * | 2002-09-26 | 2005-03-15 | Bull Hn Information Systems Inc. | Balanced access to prevent gateword dominance in a multiprocessor write-into-cache environment |
| US8185602B2 (en) | 2002-11-05 | 2012-05-22 | Newisys, Inc. | Transaction processing using multiple protocol engines in systems having multiple multi-processor clusters |
| US7698483B2 (en) | 2003-01-21 | 2010-04-13 | Nextio, Inc. | Switching apparatus and method for link initialization in a shared I/O environment |
| US7953074B2 (en) | 2003-01-21 | 2011-05-31 | Emulex Design And Manufacturing Corporation | Apparatus and method for port polarity initialization in a shared I/O device |
| US7664909B2 (en) | 2003-04-18 | 2010-02-16 | Nextio, Inc. | Method and apparatus for a shared I/O serial ATA controller |
| US7103064B2 (en) | 2003-01-21 | 2006-09-05 | Nextio Inc. | Method and apparatus for shared I/O in a load/store fabric |
| US8346884B2 (en) | 2003-01-21 | 2013-01-01 | Nextio Inc. | Method and apparatus for a shared I/O network interface controller |
| US7046668B2 (en) * | 2003-01-21 | 2006-05-16 | Pettey Christopher J | Method and apparatus for shared I/O in a load/store fabric |
| US7836211B2 (en) | 2003-01-21 | 2010-11-16 | Emulex Design And Manufacturing Corporation | Shared input/output load-store architecture |
| US8102843B2 (en) | 2003-01-21 | 2012-01-24 | Emulex Design And Manufacturing Corporation | Switching apparatus and method for providing shared I/O within a load-store fabric |
| US8032659B2 (en) * | 2003-01-21 | 2011-10-04 | Nextio Inc. | Method and apparatus for a shared I/O network interface controller |
| US7917658B2 (en) | 2003-01-21 | 2011-03-29 | Emulex Design And Manufacturing Corporation | Switching apparatus and method for link initialization in a shared I/O environment |
| US7617333B2 (en) | 2003-01-21 | 2009-11-10 | Nextio Inc. | Fibre channel controller shareable by a plurality of operating system domains within a load-store architecture |
| JP2007115223A (en) * | 2005-09-20 | 2007-05-10 | Sharp Corp | Processor and method of configuring multiprocessor |
| TWI301270B (en) * | 2006-06-30 | 2008-09-21 | Winbond Electronics Corp | Semiconductor memory and circuit and method of decoding address for the same |
| WO2014190486A1 (en) * | 2013-05-28 | 2014-12-04 | 华为技术有限公司 | Method and system for supporting resource isolation under multi-core architecture |
| WO2016159765A1 (en) * | 2015-03-27 | 2016-10-06 | Recore Systems B.V. | Many-core processor architecture and many-core operating system |
| US20170031601A1 (en) * | 2015-07-30 | 2017-02-02 | Kabushiki Kaisha Toshiba | Memory system and storage system |
| CN112804335B (en) * | 2021-01-18 | 2022-11-22 | 中国邮政储蓄银行股份有限公司 | Data processing method, data processing device, computer readable storage medium and processor |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5197130A (en) * | 1989-12-29 | 1993-03-23 | Supercomputer Systems Limited Partnership | Cluster architecture for a highly parallel scalar/vector multiprocessor system |
| US5265235A (en) * | 1990-11-30 | 1993-11-23 | Xerox Corporation | Consistency protocols for shared memory multiprocessors |
| JPH04246745A (en) * | 1991-02-01 | 1992-09-02 | Canon Inc | Memory access system |
| US5394555A (en) * | 1992-12-23 | 1995-02-28 | Bull Hn Information Systems Inc. | Multi-node cluster computer system incorporating an external coherency unit at each node to insure integrity of information stored in a shared, distributed memory |
| WO1995025306A2 (en) * | 1994-03-14 | 1995-09-21 | Stanford University | Distributed shared-cache for multi-processors |
-
1995
- 1995-10-06 US US08/540,106 patent/US5590301A/en not_active Expired - Lifetime
-
1996
- 1996-10-04 DE DE69635865T patent/DE69635865T2/en not_active Expired - Lifetime
- 1996-10-04 EP EP96934045A patent/EP0855057B1/en not_active Expired - Lifetime
- 1996-10-04 CA CA002211083A patent/CA2211083C/en not_active Expired - Fee Related
- 1996-10-04 WO PCT/US1996/015937 patent/WO1997013191A1/en active IP Right Grant
- 1996-10-04 JP JP9514461A patent/JPH11512857A/en active Pending
Also Published As
| Publication number | Publication date |
|---|---|
| EP0855057A1 (en) | 1998-07-29 |
| JPH11512857A (en) | 1999-11-02 |
| EP0855057A4 (en) | 1999-12-08 |
| WO1997013191A1 (en) | 1997-04-10 |
| EP0855057B1 (en) | 2006-03-01 |
| DE69635865D1 (en) | 2006-04-27 |
| CA2211083A1 (en) | 1997-04-10 |
| US5590301A (en) | 1996-12-31 |
| DE69635865T2 (en) | 2006-09-14 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CA2211083C (en) | Address transformation in a cluster computer system | |
| CA1161962A (en) | Dual port exchange memory between multiple microprocessors | |
| US4470113A (en) | Information processing unit | |
| EP0497600B1 (en) | Memory access method and apparatus | |
| CA1129110A (en) | Apparatus and method for providing byte and word compatible information transfers | |
| EP0306702A2 (en) | Virtual input/output commands | |
| US5404474A (en) | Apparatus and method for addressing a variable sized block of memory | |
| KR900002894B1 (en) | Data processing system using address conversion | |
| JPH0576060B2 (en) | ||
| EP0229932B1 (en) | High-capacity memory for multiprocessor systems | |
| US6092136A (en) | Multi-processor central processing unit | |
| US4344130A (en) | Apparatus to execute DMA transfer between computing devices using a block move instruction | |
| US5517633A (en) | System for controlling an internally-installed cache memory | |
| JPS6133225B2 (en) | ||
| JPH08161216A (en) | Information processor provided with high-speed memory clear function | |
| JPH01298457A (en) | Computer system | |
| US20060230233A1 (en) | Technique for allocating cache line ownership | |
| USRE38514E1 (en) | System for and method of efficiently controlling memory accesses in a multiprocessor computer system | |
| JPH0227696B2 (en) | JOHOSHORISOCHI | |
| JPH0581133A (en) | Information processor | |
| JPS6022777B2 (en) | Data transfer method | |
| CN1299099A (en) | Biodge device | |
| JPH01226056A (en) | Address converter | |
| JPS59218692A (en) | Control system of logical buffer memory | |
| JPH04111153A (en) | Information processor |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| EEER | Examination request | ||
| MKLA | Lapsed |