|Publication number||US20040050231 A1|
|Application number||US 10/243,708|
|Publication date||18 Mar 2004|
|Filing date||13 Sep 2002|
|Priority date||13 Sep 2002|
|Also published as||US6714412|
|Publication number||10243708, 243708, US 2004/0050231 A1, US 2004/050231 A1, US 20040050231 A1, US 20040050231A1, US 2004050231 A1, US 2004050231A1, US-A1-20040050231, US-A1-2004050231, US2004/0050231A1, US2004/050231A1, US20040050231 A1, US20040050231A1, US2004050231 A1, US2004050231A1|
|Inventors||Richard Chu, Michael Ellsworth, Roger Schmidt, Robert Simons|
|Original Assignee||International Business Machines Corporation|
|Export Citation||BiBTeX, EndNote, RefMan|
|Patent Citations (5), Referenced by (29), Classifications (9), Legal Events (4)|
|External Links: USPTO, USPTO Assignment, Espacenet|
 The present invention relates in general to the cooling of computer electronic components by liquid systems. More particularly, the invention relates to a scalable design for liquid cooling of electronics systems, utilizing removable modular pumping units and an integrated plate heat exchanger/expansion tank.
 Heat fluxes dissipated by electronic equipment, such as microprocessors and power supplies, are reaching levels that preclude air cooling as a means to control component temperature. Liquid cooling (e.g. water cooling) is a very attractive technology to manage the higher heat fluxes. The liquid absorbs the heat dissipated by the components/modules in a very efficient matter (i.e. with minimal temperature rise from the liquid to the component being cooled). The heat ultimately has to be transferred from the liquid and out of the data center (i.e. room containing the electronic equipment) environment, otherwise, the liquid would continuously rise in temperature. From the 1970s through the early 1990s, IBM accomplished this task back by circulating the cooling liquid (i.e. system water) via a coolant distribution unit (FIG. 1). The system water would flow through a liquid/liquid heat exchanger that was cooled by relatively low temperature water, known as site service or customer water, which was provided by the customer facility. This unit stood separate from the electronics frames and would supply system water (maintained at about 22 C) to one or more electronics frames.
 Back when the cooling distribution unit (CDU) was used, a single computer system could fill the entire data center. There was only a need for one CDU design point in terms of heat removal and system water flow rates. However, with current and future systems occupying a single frame, a cooling unit may be called upon to support anywhere from 1 to n number of systems. More importantly, computer customers customarily choose to scale up their computing requirements as their needs grow by adding more electronics within a frame or adding additional electronics frames. It is highly desirable, therefore, to be able to scale up the function of a cooling distribution unit.
 Power levels in computer equipment (primarily processors) have risen to the level where they can no longer be air cooled. These components will likely be water cooled. Heat dissipated by the processor will be transferred to the water via a water cooled cold plate. Water typically available at customer locations (i.e. data centers) is not suitable for use in these cold plates. First, condensation formation is a concern as the temperature of the data center water, ranging from 7 C to 15 C, is far below the room's dew point (typically 18-23 C). Second, the relatively poor quality of the water (its chemistry, cleanliness, etc.) impacts system reliability. It is therefore desirable to utilize a water cooling/conditioning unit that circulates high quality water to/from the electronics to be cooled and rejects the heat to the data center water.
 It is also desirable to provide the water cooling function in a considerably smaller volume, preferably within a single 19″ or 24″ rack. It would help to utilize a plate heat exchanger in lieu of the bulky shell and tube heat exchangers used in past systems, but something more is needed in terms of volume reduction. Furthermore, it is desirable to avoid the extra expense and volume associated with insulating the heat exchanger to prevent condensation formation. While some attempt at space consolidation has been made in the past (e.g., as disclosed in patent application entitled “Cooling System for Portable Electronic and Computer Devices” by Richard C. Chu et al., Ser. No. 09/893,135, Attorney Docket No. POU920010049) filed Aug. 17, 2001, and assigned to the assignee of the present invention, wherein an expansion space was provided within a heat exchanger in a personal computer environment), these did not deal with the rack-mounted frame environment nor take the novel approach presented herein.
 Disclosed herein is the concept of a Scalable Coolant Conditioning Unit (SCCU) which provides the means to scale the function of a CDU. This is accomplished by utilizing modular pumping units that can be added to the SCCU. Additionally, the SCCU takes advantage of an intergral heat exchanger/expansion tank that is sized to handle the maximum design point. The modular pumping units are arranged in a parallel flow configuration; additional pumps provide additional flow at a consistent pressure drop. The alternative to this concept would be to design, build, and inventory multiple CDUs and swap in successively larger units as the customer's requirements grow. It is far more cost effective to apply the concept disclosed here utilizing one common unit with the capability of accommodating multiple pumping units to scale-up flow and cooling capability as the customer's requirements grow. The SCCU makes cooling water a customer supplied utility providing conditioned water (in terms of temperature and cleanliness) for cooling each frame as needed, much like a municipal water utility distributes water to each home as needed.
 Also disclosed herein is the concept of physically integrating a plate heat exchanger within the expansion tank in order to reduce volume and prevent condensation formation on the heat exchanger without having to add bulky insulation.
 The recitation herein of desirable objects which are met by various embodiments of the present invention is not meant to imply or suggest that any or all of these objects are present as essential features, either individually or collectively, in the most general embodiment of the present invention or in any of its more specific embodiments.
 The subject matter which is regarded as the invention is particularly pointed out and distinctly claimed in the concluding portion of the specification. The invention, however, both as to organization and method of practice, together with further objects and advantages thereof, may best be understood by reference to the following description taken in connection with the accompanying drawings in which:
FIG. 1 is a cooling distribution unit of the prior art;
FIG. 2 is a schematic of the Scalable Coolant Conditioning Unit (SCCU) of the present invention;
FIG. 3 shows an SCCU having a minimal number of pumps;
FIG. 4 shows an SCCU having an intermediate number of pumps;
FIG. 5 shows an SCCU having a maximum number of pumps;
FIG. 6 is a modular pumping unit (single in-line pump);
FIG. 7 is a modular pumping unit (multiple in-line pumps);
FIGS. 8A, 8B, and 8C, respectively, show top, side, and front views of a modular pump unit in position to be connected within an SCCU;
FIG. 9 is a conceptual drawing of an integrated plate heat exchanger and expansion tank (customer water side not shown);
FIG. 10 is an exploded view (isometric) of an integral plate heat exchanger/expansion tank;
FIG. 11 is a plate heat exchanger with supports for mounting a heat exchanger inside the tank; and
FIG. 12 is an isometric of an assembled integral plate heat exchanger/expansion tank.
 A cooling unit similar to that depicted in FIG. 1 was used to cool IBM's large bipolar systems back in the 1980s and early 1990s. The cooling unit 11 was relatively large and occupied more than what would now be considered as two full electronics frames. Within the cooling unit was a power/control element 12, a reservoir/expansion tank 13, a heat exchanger 14, a pump 15 (often accompanied by a redundant second pump), customer water (or site or facility service water or coolant) in 16 and out 17 supply pipes, a supply manifold 18 directing water to the electronics frames 20, and a return manifold 19 directing water from the electronics frames 20.
 In accordance with preferred embodiments of the present invention, FIG. 2 illustrates elements of the scalable SCCU 21 of the present invention. Within the unit is a bulk power regulator and controls 22. The coolant returning from the electronics frame 20 (“system coolant”) is collected by a return manifold 19 and directed through the expansion tank section of the integral heat exchanger/expansion tank 23 (described more fully below) and to another manifold 24 which supplies the coolant to multiple modular pumping units (MPUs) 27. The higher pressure discharge of the MPUs is collected in another manifold 25 and directed to the “hot side” of the heat exchanger within the integral heat exchanger/expansion tank. The MPU's are connected to the manifolds via an insertion facilitation mechanism comprising automatic coupling assemblies 53 which are connected via flexible hoses to an isolation valve mechanism comprising a plurality of solenoid operated isolation valves 26. Alternatively, the isolation valves could be manually operated either locally or remotely, and the automatic coupling assemblies could be replaced by manually operated quick disconnects. Isolation valves 26 are connected to manifolds 24 and 25 for isolating MPU's from the manifolds during installation or removal. (Note: FIG. 2 is a schematic and is not meant to show the actual location of the quick disconnects on the MPU's. This will be shown in detail later, e.g., in FIGS. 7 and 8.) Having been cooled by the site (or “facility”) service water flowing through the “cold side” of the heat exchanger (16, 17), the system liquid is sent to the supply manifold 18 which distributes the conditioned coolant to multiple electronics frames requiring cooling. Although not shown here, the SCCU may also incorporate means to filter the system water and automatically add a corrosion inhibitor such as benzotriazole (BTA) as needed. A two-way control valve 28 is used to regulate the flow rate of the customer water supplied to the heat exchanger within the integral heat exchanger/expansion tank, thereby controlling the temperature of system water delivered to the electronics frames 20. A thermistor temperature sensing element 29 located at the inlet of the system water supply manifold 18 supplies an electronic signal to circuitry 30 controlling the operation of two-way valve 28. If the supply water temperature is higher than desired, two-way valve 28 is commanded to open more allowing an increased flow of customer water through the heat exchanger resulting in a decrease in the temperature of the system water directed to the electronic frames from supply manifold 18. Alternatively, if the supply water temperature is lower than desired, two-way valve 28 is commanded to close more providing a decreased flow of customer water through the heat exchanger resulting in an increase in the temperature of the system water directed to the electronic frames from supply manifold 18.
FIGS. 3, 4, and 5 illustrate different ranges of operation for the SCCU. FIG. 3 shows a minimal number of MPU's 27 coupled to manifolds 24 and 25, to accommodate a low system flow requirement (note the minimal number of connections to manifolds 18 and 19 because of low number of electronic frames 20 and the low heat load associated with these frames). FIG. 4 shows a greater number of MPU's 27 coupled to manifolds 24 and 25, to accommodate a moderate coolant flow requirement (note the greater number of connections to manifolds 18 and 19 because of an increased number of frames 20 and the greater heat load associated with these frames 20). FIG. 5 shows the maximum number (for this configuration) of MPU's 27 coupled to manifolds 24 and 25, to accommodate the high coolant flow (note the maximum (for this configuration) number of connections to manifolds 18 and 19 and maximum heat load associated with the maximum number of frames this configuration will support.
 An important element to the scalable SCCU is the modular pumping unit 27. One or multiple pumps are housed in a package as illustrated in FIGS. 6 and 7, respectively. As shown in FIG. 6 (illustrating a single in-line centrifugal pump within an MPU), the pump motor 42 is disposed below the centrifugal pump 43. An example of the pump motor 42 and centrifugal pump 43 would be the Bell & Gossett (8200 N. Austin Ave, Morton Grove, Ill., 60053) Series 90 in-line mounted centrifugal pump. The suction 40 and discharge 41 of the pump are brought out to the outer boundary of the pumping unit 27, where they terminate in a male quick-disconnect fitting 50. An electrical connection 44 to the pump motor is in turn brought out to an external connection 46 on the outer boundary of the pumping unit.
FIG. 7 illustrates a modular pumping unit 27 having multiple in-line pumps (each comprising a pump motor 42 and centrifugal pump 43). Each pump motor has its own electrical connection 44, and all electrical connections are connected to external connection 46.
 FIGS. 8A, 8B,and 8C show a modular pumping unit 27 in position to engage automatic coupling assembly 53 within an SCCU (top, side, and front views, respectively). The pump here is a volute centrifugal pump and is configured so that suction and discharge are on the same side of the MPU. It can be readily appreciated that multiple such pumps can be configured within the MPU as was described above in the case of the in-line configuration. The MPU is fitted with carrying handles 45 to facilitate transportation. The MPU is positioned within the SCCU atop an MPU mounting track 54, which is in turn connected to the SCCU body by a connection mechanism (shown as shock-absorbers 55 in FIGS. 8B and 8C). To facilitate seating the MPU atop the mounting plate, the MPU and mounting plate are outfitted with cooperating seating mechanism (shown in FIG. 8C as a set of rollers 56 affixed to the MPU, seated within mounting track 54 affixed to the mounting plate). Rollers 56 allow the MPU to be rolled into position to engage or disengage automatic coupling assembly 53 comprised of mounting bracket 48 affixed to SCCU, female quick-disconnect fittings 51, actuation plate 47, and actuation solenoid 49. Female quick-disconnect fittings 51 are held in a stationary position by mounting bracket 48. Flexible hoses (not shown) attached to hose barbs 52 connect to solenoid-operated isolation valves (26 in FIG. 2 on manifolds 24 and 25). The locking-release collar 55 of female quick disconnects 51 is retained in actuation plate 47 which is connected to the shaft of actuation solenoid 49. Energizing actuation solenoid 49 causes actuation plate 47 and locking collar 55 of the female quick-disconnects 51 to move towards mounting bracket 48. Movement of locking-release collars 55 to the right in FIG. 8A permits male quick-disconnects 50 to engage or disengage female quick-disconnects 51. Alternatively, electrically operated actuation solenoid 49 could be replaced with a manually operated actuation mechanism in FIGS. 8A, 8B, and 8C.
 Following is a description of the sequence of actions/events involved in installing or removing an MPU (utilizing the insertion facilitation mechanism comprising an electrically actuated automatic coupling assembly 53 and isolation valve mechanism comprising electrically operated solenoid isolation valves 26) while an SCCU continues to operate:
 1. Status of relevant SCCU components prior to modular pumping unit (MPU) installation or removal:
 Applicable solenoid-operated isolation valves (SOIV's) 26 are in their normal (non-actuated) position which is open.
 Applicable actuation solenoid (AS) 49 is in its normal position (when the actuation solenoid 49 is in its normal position, the locking-release collars on the female portion of the quick disconnects are in their normal locked position.)
 2. Sequence of events when an MPU is installed in an operating SCCU:
 a) Stimulus is applied to ready MPU location for MPU install. This stimulus could be the entering of a computer command (when microcode controlled) or manual operation of an electrical switch. Two things happen when this stimulus is provided (in order):
 i) Applicable SOIV's 26 are electrically energized (closed)
 ii) Applicable AS 49 is electrically energized (unlocking locking-release collar of female quick disconnect).
 b) Install MPU by manually pushing (from left-side in FIG. 8A) into position (resulting in insertion of male portion of quick disconnect into female portion of quick disconnect)
 c) Apply separate stimulus to ready MPU for operation in the SCCU resulting in (in order)
 i) Applicable AS 49 being electrically de-energized (locking-release collar on female portion of quick disconnect returns (under spring load) to its normal locked position)
 ii) Applicable SOIV's 26 electrically de-energized (opened)
 3. Sequence of events when a MPU is removed while SCCU is operating.
 a) Stimulus is applied to ready MPU location for MPU removal resulting in (in order)
 i) Applicable SOIV's 26 being electrically energized (closed)
 ii) Applicable AS 49 being electrically energized
 b) Remove MPU (by manually pulling towards left-side in FIG. 8A)
 c) Apply separate stimulus to acknowledge MPU has been removed resulting in (in order)
 i) Applicable AS 49 electrically de-energized
 ii) Applicable SOIV's 26 electrically de-energized (closed)
 It may be appreciated that alternatively install and un-install of MPUs may be accomplished by manually coupling and uncoupling male quick disconnects 50 and female quick disconnects 51 and replacing of solenoid-operated isolation valves 26 with manually operated valves.
 It will be appreciated by those skilled in the art that wherein the Scalable Cooling Conditioning Unit has been described herein with respect to water-cooling, the concept is readily adapted to use other types of coolant (e.g. brines, fluorocarbon liquids, or other similar chemical coolants) on the system-side while maintaining the advantages and unique features of this invention.
 The overall concept of a physically integrated plate heat exchanger/expansion tank can be seen in FIG. 9. Closed liquid loops typically require an expansion space or tank to account for the volumetric expansion of the liquid under varying environmental temperatures and/or the volumetric expansion of flexible (i.e., rubber) hoses when exposed to operating pressures. It should be noted that the size and cooling capacity of the integrated plate heat exchanger/expansion tank is set to accommodate the maximum heat load of the SCCU system (i.e., the load with all possible MPU's attached to the appropriate manifolds). The plate heat exchanger 1001 exists completely within the tank 1002. The piping to 1003/from 1004 the heat exchanger penetrates the tank wall. The structure can be assembled in a number of ways An example is shown in FIG. 10. A cover 1005 is attached to the tank after the heat exchanger is placed within the tank, and the piping 1003/1004 is attached to the heat exchanger after the heat exchanger is placed inside the tank. Since the plate heat exchanger and tank are made of stainless steel, the pipe can be welded to the tank wall to prevent leakage.
 As shown in FIG. 10, a heat exchanger sub-assembly is made up of a plate heat exchanger 1001 with supports mounted to it. The detailed supports can be seen in FIG. 11. The axial (bottom) support 1201 would be designed to act as a spring so as to provide vertical compliance in the assembly. The lateral support 1202 would be designed to prevent lateral motion of the heat exchanger. A rubber gasket around the periphery of the lateral support (not shown) would provide lateral compliance during assembly. A top mounting plate 1203 would be bolted to supports provided on the tank to anchor the heat exchanger within the tank. FIG. 11 shows the completed assemblage of the expansion tank with the heat exchanger within.
 It has been shown, with a finite element thermal conduction analysis, that the warm water return from the electronics frame(s) prevents the tank walls from dropping below the dew point temperature therefore preventing condensation from forming on the outer walls. Additionally, it can be shown that to contain a 24.33″ long by 7.52″ wide by 4.41″ deep heat exchanger in an expansion tank 28.33″ long and provide 10 gallons worth of expansion volume requires a 12″ diameter tank versus a 10-gallon expansion tank (alone) 28.33″ long that must be 10.3″ in diameter. Factor in a minimum of 1″ insulation that would have to surround the heat exchanger if not mounted inside the expansion tank and the volume benefit associated with the integral concept becomes self evident.
 While the invention has been described in detail herein in accord with certain preferred embodiments thereof, many modifications and changes therein may be effected by those skilled in the art. Accordingly, it is intended by the appended claims to cover all such modifications and changes as fall within the true spirit and scope of the invention.
|Cited Patent||Filing date||Publication date||Applicant||Title|
|US2151733||4 May 1936||28 Mar 1939||American Box Board Co||Container|
|CH283612A *||Title not available|
|FR1392029A *||Title not available|
|FR2166276A1 *||Title not available|
|GB533718A||Title not available|
|Citing Patent||Filing date||Publication date||Applicant||Title|
|US7257000||7 Jul 2004||14 Aug 2007||Amphenol Corporation||Thermally enhanced pressure regulation of electronics cooling system|
|US7315448 *||1 Jun 2005||1 Jan 2008||Hewlett-Packard Development Company, L.P.||Air-cooled heat generating device airflow control system|
|US7355852||30 Sep 2004||8 Apr 2008||Amphenol Corporation||Modular liquid cooling of electronic assemblies|
|US7466549 *||20 Nov 2004||16 Dec 2008||Rittal Gmbh & Co. Kg||Cooling arrangement for server blades|
|US7507066||27 Mar 2006||24 Mar 2009||Koenig Kevin J||Pump header body and modular manifold|
|US7715194||6 Apr 2007||11 May 2010||Cooligy Inc.||Methodology of cooling multiple heat sources in a personal computer through the use of multiple fluid-based heat exchanging loops coupled via modular bus-type heat exchangers|
|US7730731||1 Nov 2005||8 Jun 2010||Hewlett-Packard Development Company, L.P.||Refrigeration system with serial evaporators|
|US7746634||7 Aug 2008||29 Jun 2010||Cooligy Inc.||Internal access mechanism for a server rack|
|US7775762||27 Mar 2007||17 Aug 2010||Koenig Kevin J||Pump header body and modular manifold|
|US7806168||30 Oct 2003||5 Oct 2010||Cooligy Inc||Optimal spreader system, device and method for fluid cooled micro-scaled heat exchange|
|US7895854||1 Jun 2005||1 Mar 2011||Hewlett-Packard Development Company, L.P.||Refrigeration system with parallel evaporators and variable speed compressor|
|US7903404 *||29 Apr 2009||8 Mar 2011||Hewlett-Packard Development Company, L.P.||Data centers|
|US8202040||1 Jul 2010||19 Jun 2012||Koenig Kevin J||Pump header and implementation thereof|
|US8276397 *||25 Jun 2008||2 Oct 2012||Exaflop Llc||Cooling and power paths for data center|
|US8526183 *||15 Apr 2011||3 Sep 2013||Exaflop Llc||Data center cooling circulation|
|US8561418||31 Jan 2011||22 Oct 2013||Hewlett-Packard Development Company, L.P.||Refrigeration system with parallel evaporators and variable speed compressor|
|US8824143 *||12 Oct 2011||2 Sep 2014||International Business Machines Corporation||Combined power and cooling rack supporting an electronics rack(S)|
|US8879257 *||3 Dec 2012||4 Nov 2014||International Business Machines Corporation||Combined power and cooling rack supporting an electronics rack(s)|
|US20040182551 *||18 Aug 2003||23 Sep 2004||Cooligy, Inc.||Boiling temperature design in pumped microchannel cooling loops|
|US20040188066 *||30 Oct 2003||30 Sep 2004||Cooligy, Inc.||Optimal spreader system, device and method for fluid cooled micro-scaled heat exchange|
|US20050084385 *||18 Oct 2004||21 Apr 2005||David Corbin||Micro-fabricated electrokinetic pump|
|US20050183445 *||20 Apr 2005||25 Aug 2005||Mark Munch||Remedies to prevent cracking in a liquid system|
|US20050183845 *||20 Apr 2005||25 Aug 2005||Mark Munch||Remedies to prevent cracking in a liquid system|
|US20050211417 *||29 Jun 2004||29 Sep 2005||Cooligy,Inc.||Interwoven manifolds for pressure drop reduction in microchannel heat exchangers|
|US20050211427 *||29 Jun 2004||29 Sep 2005||Cooligy, Inc.||Method and apparatus for flexible fluid delivery for cooling desired hot spots in a heat producing device|
|US20060007657 *||7 Jul 2004||12 Jan 2006||Teradyne, Inc.||Thermally enhanced pressure regulation of electronics cooling systems|
|US20130091706 *||3 Dec 2012||18 Apr 2013||International Business Machines Corporation||Combined power and cooling rack supporting an electronics rack(s)|
|US20130094139 *||12 Oct 2011||18 Apr 2013||International Business Machines Corporation||Combined power and cooling rack supporting an electronics rack(s)|
|WO2007120662A2 *||9 Apr 2007||25 Oct 2007||Cooligy Inc||Methodology of cooling multiple heat sources in a personal computer through the use of multiple fluid-based heat exchanging loops coupled via modular bus-type heat exchangers|
|U.S. Classification||83/574, 361/699, 83/614, 30/317|
|Cooperative Classification||H05K7/2079, Y10T83/8763, Y10T83/8822|
|13 Sep 2002||AS||Assignment|
|13 Jul 2007||FPAY||Fee payment|
Year of fee payment: 4
|15 Jul 2011||FPAY||Fee payment|
Year of fee payment: 8
|30 Jun 2015||FPAY||Fee payment|
Year of fee payment: 12