Reasons to Use Two-Phase Refrigerant Cooling

Water-cooling at the rack appears to be the new short-term technological solution for server cooling in datacenters, with its advantages over traditional cold aisle/hot aisle air-cooling being clear, as can be seen in [1]. Bringing chilled water to the rack and then cooling the air inside the rack aids with the transport of energy across the datacenter, making the necessity for large air-handling systems to cool large, unnecessary, volumes of air a thing of the past. This has an overall advantage in that the servers can now be cooled locally (only if the rack is in operation), which saves energy and opens the opportunity for more densely packed rack units, but still requires chillers for the cold water cooling supply. Long-term, with microproces­sor performance increasing, the limits of air-cooling technol­ogy are being reached, i.e. forced-air heat sinks have become significantly larger, more expensive and more complex [2], imposing a challenge for the development of a more drastic cooling approach.   

One solution attracting a lot of interest, at least in the aca­demic sphere, is direct “two-phase on-chip cooling”, that is, placing a micro-evaporator heat sink directly on the back of the chip, since most of the energy generated in a datacenter is accounted for by the microprocessor, and simpler evaporator cooling elements on the other heat generating units (memory, etc.). In fact, microchannel cooling is a promising technol­ogy for high heat flux removal that has a low pumping power requirement relative to the quantity of heat to be removed. Figure 1 shows some typical microchannel coolers tested in the LTCM lab at the EPFL, having channel widths from 50 μm to 200 μm and fin heights from 50 μm to 2 mm. Included is a silicon microchannel-evaporator having a fin height of 560 μm, fin width of 42 μm and a channel width of 85 μm. A first­of-a-kind prototype was tested early in 2009, which removed a heat flux of 180 W/cm2 using refrigerant R134a at a saturation temperature of 60˚C, while maintaining the chip junction temperature below 85˚C has been described by Madhour et al. [3]. Hence, cooling can be achieved by evaporating a fluid significantly above ambient temperature and dissipated to the ambient (or recovering the waste heat for other uses) and thus water chillers are no longer needed.   

Other technologies can also be found in the literature, such as liquid immersion [4] and spray cooling [5], each one with its advantages, challenges to be implemented and specific field of application. For example, Tuma [4] evaluated the liquid im­mersion technology, which proved to be simpler, denser, much less expensive and more efficient than any other liquid cooling technology. However, the technology is only suitable for low heat flux applications (12 W/cm2). Regarding spray cool­ing, Finch and Ballew [5] showed an interesting comparison between air natural, air forced and liquid forced convection, liquid immersion and direct spray. According to the authors, for a 30˚C of temperature difference between the chip and the fluid, the direct spray was able to remove from 6 to 90W against 0.02 to 0.09W, 0.06 to 1.2W, 0.3 to 4.8W and 2.4 to 30W for the other technologies, respectivelly. They charaterized such technologies for applications in harsh environments, such as manned and unmanned military platforms, capable of work­ing at temperatures of -65˚C or +60˚C.   

Today, there is a huge initiative to introduce “single-phase water on-chip cooling” using copper microchannel coolers, as attested to by the Aquasar project [6] (a joint project involving IBM, ETH and EPFL in Switzerland). In that project water at 60-70°C is used to cool the chips of a server with the waste heat being recovered into a heat pump system, illustrating that there are numerous advantages of using water as the cooling medium. On the other hand, “two-phase on-chip cooling” also has many thermal and energetic advantages. However, due to the perceived complexity of the two-phase flows by non-specialists, this solution is not yet well understood by the thermal packaging community. Considering here that “single­phase water on-chip cooling” is the leading technology both for cooling of high heat fluxes and for its low pumping power consumption compared to air-cooled systems, the comments about the advantages of two-phase cooling below are presented with water-cooling in mind. 

Compatibility with Electronics

 When working alongside electronics, water has obvious risks whilst refrigerants on the other hand are inherently dielectric fluids, meaning that even with a leak, the electronics will be safe from damage. In fact, some earlier generations of super computers, such as the CRAY-2 and CRAY T90, had their electronics submerged in refrigerants, making use of pool boiling as the main heat transfer mechanism. Likewise, there is significant experience with two-phase cold plates for cooling of the IBM Z-series mainframes. Hence, there is already a long industrial experience in the handling of refrigerants without leaks in computer cooling applications.  

Figure 2. Total pressure drop and pumping power consumption  

Figure 3. System volume and mass of working fluid.  

Pumping Power    

Simulations and experiments have shown that refrigerants have a clear advantage regarding pumping power relative to single-phase water [7]. This is due to refrigerants making use of latent heat removal (about 150 kJ/kg), unlike water (4.2 kJ/kg) that uses sensible heat removal, the former requiring about 1/10 of the flow rate. Hence two-phase cooling requires much less pump­ing power to remove the same amount of heat. Figure 2 shows the pressure drop and pumping power requirements simulated for a complete microchannel cooled rack [7]. The first plot is for R134a as the working fluid with cooling pipe diameters of 3 mm throughout, while the other plots are for water as the working fluid for various pipe diameters, ranging from 3 mm to 15 mm. For the single-phase watered cycle to have the same overall pressure drop as the two-phase refrigerant cycle (the simulations considered 30% of microevaporator outlet vapour quality), the inner diameter of the piping was more than double, 6.2 mm vs. 3.0 mm. However, even with the pressure drops being the same, the pumping power for the water cycle is still more than 9 times higher than for the refrigerant cycle because of its much larger flow rate. For the system using water to have the same pumping power as the two-phase system, cooling pipe diameters need to be 5 times that for R134a.  

Piping  

Small diameter pipes are advantageous as they are lighter in weight, require less coolant, are more flexible and can be easily installed in confined spaces, an aspect which is very important in electronic equipment. Figure 3 shows that the required system volume and mass of working fluid, when us­ing single-phase water (3 mm of piping diameter), is about 4 and 3.5 times more, respectively, than when using two-phase refrigerant.  

It is worth mentioning that the reason for a smaller system volume and mass of water when increasing the diameter from 3 mm to 6.2 mm (viz. Figure 3) is due to the size of the heat exchanger. As can be observed in Figure 2, there is a huge dif­ference in pumping power when considering the water cooling cycle with different diameters. This difference in power repre­sents the additional heat load that must to be removed in the condenser, assuming all energy from the pump is transferred to the fluid. When increasing the diameter even further, the opposite trend is seen since the piping becomes the dominant reason for the volume increase rather than the heat exchanger size due to the pumping power dissipation.   

Material Compatibility    

Refrigerants have a long and successful history in the refrigera­tion and air-conditioning industries with material compatibil­ity well understood. Refrigerants are mostly used with copper and aluminum, with some refrigeration systems running for over 30 years. The use of water, on the other hand, is more problematic, as it is known to attack metallic components un­less treated, with ultra-clean water being especially corrosive.    

Organic/Fouling    

Water is the essence of life, meaning here that organic material tends to grow inside the system. Thus, water is treated to try to prevent organic growth. Nevertheless, water tends to foul its containment, degrading the heat transfer performance of the cold plate and could eventually block the microchannels, rendering such a microchannel cooler unreliable. Refrigerants are not subject to organic growth or fouling.    

Erosion    

Erosion occurs due to the shear stress exerted by the flowing fluid on the channel wall, especially in joints. Guidelines for water flowing inside copper pipes suggest that water veloci­ties should be kept below 0.5-1.2 m/s, depending on the water temperature. Erosion can lead to the thinning of the piping or, in the case of the microchannel cooler, the thinning of the fins, which would degrade their performance. Furthermore, erosion contaminates the fluid with particles, which would need to be filtered out, or adversely affect the fluid pump. Due to the high mass flow rates required when using water for electron­ics cooling, erosion is a design concern. The flow rates (and velocities) of the refrigerants are typically much less as noted above and are known not to have significant erosive properties.    

Figure 4. Effect hotspots on heat transfer coefficients (left) and junction temperatures (right) for R134a, water, and a 50% ethylene-glycol water mixture along the length of a chip.     

Hot Spot Management    

Hot spots on microprocessors are a topic of much discussion due to their negative impact on the chip’s life cycle. Large temperatures and temperature gradients lead to thermal stresses between the layers of cooling materials, while also having a huge effect on the timing of the transistors. Refriger­ants, unlike single-phase fluids, have a self-enhancing effect when it comes to hot spots. This is shown in Figure 4, which is a comparison of R134a, water and a 50% ethylene-glycol mixture for a chip having a base heat flux of 50 W/cm2 and each hot spot having a heat flux of 250 W/cm2 (5-to-1 ratio), assuming parallel microchannels of 1.7 mm height/ 0.17 mm thickness and 0.17 mm channel width. The refrigerant reacts to the hot spot by increasing the local heat transfer coefficient (decreasing the local thermal resistance), whilst the single-phase liquids heat up (further exasperating the problem). This is evidenced in the junction temperatures shown at the right of Figure 4, with R134a having much lower temperature dif­ferences. This has also been experimentally observed at the LTCM lab for a pseudo-chip cooled with a micro-evaporator cooler with 35 local heaters and temperature sensors.    

Harsh Climates    

A problem often encountered in cooling systems exposed to the environment is that the cooling fluid needs to resist very cold and/or hot climates. Water freezes at temperatures below 0˚C, creating a problem in the winter on most places on the earth. However, glycol added to water, which decreases the freezing temperature, degrades the heat transfer performance and increases the required pumping power quite consider­ably (by about 50%). Hence, water cooled systems need to be charged on location. Refrigerants, on the other hand, have very low freezing temperatures (< -100˚C) and are ideal for harsh conditions. The critical point for most refrigerants is also on the order of 100˚C, which is the upper limit of a two-phase system. On the other hand, the saturation pressure of the refrigerant becomes very high at high temperatures, creating a maximum working pressure issue.   

Heat Dissipation    

Two-phase refrigerants also have the advantage of being able to dissipate heat to higher temperatures using a vapor compres­sion cooling cycle (heat pump). Therefore, chip or electronics temperatures can be maintained at a convenient operating temperature and heat can be discharged at temperatures above ambient. This allows for a longer life cycle of electronic components in harsh operating conditions.    

Environment    

Refrigerants are becoming much more environmentally friendly, with the development of so-called fourth generation refrigerants. One of these refrigerants, R1234yf, resulting from a joint development of DuPont and Honeywell, is considered to be one of these. Having thermal characteristics and properties similar to that of R134a, it is seen as a very good replacement. Environmentally, though, it is much greener, having a global warming potential of only 4 (1320 for R134a), an atmospheric lifetime of 11 days (14 years for R134a) and an ozone depletion potential of zero.    

Complexity    

Usually the most widely used reason not to consider a two-phase cooling solution in favor of a liquid cooling system is the “complexity” of two-phase flow. This “complexity” is pri­marily related to a lack of two-phase flow and heat transfer (boiling and condensation) expertise in the thermal packaging industry. Two-phase specialists have been successfully design­ing refrigeration and air-conditioning systems for nearly a century, including applications in demanding environments. Hence this issue is easily remedied with more training of new personnel with such experience.    

Above, the most prominent advantages of two-phase cool­ing have been addressed…one could also speak of the smaller size of the refrigerant pump with its 1/10 flow rate compared to a water pump. It is clear, though, that for industry to move 8. forward regarding electronics cooling, it needs to adapt a cooling technology that is reliable, efficient, well known and that provides a long-term energetically sound, green solution. 9. Two-phase on-chip cooling with microchannel refrigerant cooling technology appears to be a good long-term solution. For those interested in more details on two-phase flow, boil­ing and condensation in microchannels, please refer to the free online book of Thome [8] where over 200 two-phase flow videos are available in Chapter 1 while Chapters 20 and 21 deal specifically with the detailed state-of-the-art reviews of microscale two-phase flow and heat transfer.    

Finally, it is important to highlight that to implement such a new novel cooling technology some challenges must be overcome. We can list, for example, the control of micro-evap­orator outlet vapour quality (important to guarantee a high performance of the micro-evaporator and avoid the dry-out) and the flow distribution for several micro-evaporators operat­ing in a rack populated with several chips under different and variable heat loads. These problems, however, are not unique to two-phase microevaporator cooling and are also found in most of the other cooling technologies. We can also detach that nowadays, with the growth of the electronics field, new variable components (flow drivers, electrical valves etc) are in continuous development and are available in the market, allowing the design of smart control strategies for cooling systems, which not only saves energy but also offers better stability and control to the systems [9]. Actually, the LTCM lab is testing a hybrid cooling system capable of evaluating the performance of three different two-phase cooling cycles using micro-evaporators in parallel with cooling capacities up to 600 W. Until now the results are promising, showing good controllability for the micro-evaporators outlet vapour quality and good flow distribution for non-uniform heat loads, all this using very simple control strategies.    

References:    

1              Whitenack, K., Liquid Cooling for Datacom Equipment Centers, in ElectronicsCooling. August 2008.    

2              Mackey, S. and Hannemann, R., Advanced Cooling Using Meso-Scale Evaporative Cold Plates, in ElectronicsCooling. November 2007.    

3              Madhour, Y., Olivier, J.A., Costa-Patry, E., Paredes, S., Michel, B., and Thome, J.R., Flow Boiling of R134a in a Multi-Microchannel Heat Sink with Hotspot Heaters for Energy-Efficient Microelectronic CPU Cooling Applications. IEEE Transactions on Components and Packaging Technologies, Accepted for publication, 2010.    

4              Tuma, P.E., Open Bath Immersion Cooling In Data Centers: A New Twist On An Old Idea, in ElectronicsCooling. December 2010.    

5              Finch, A. and Ballew, E., Direct Spray Cooling and System-level Comparisons, in ElectronicsCooling. August 2009.    

6              Ganapati, P. Water-Cooled Supercomputer Doubles as Dorm Space Heater. 2009; Available from: http://www.wired.com/gadgetlab/2009/06/ ibm-supercomputer/    

7              Marcinichen, J.B. and Thome, J.R. New Novel Green Computer Two-Phase Cooling Cycle: A Model for Its Steady-State Simulation. in ECOS2010 -23rd International Conference on Efficiency, Cost, Optimization, Simulation and Environmental Impact of Energy Systems. 2010. Lausanne -Switzerland.    

Thome, J.R., ENGINEERING DATABOOK III, in Web-based reference book available for free at the website, 21 chapters so far, available at: http:// www.wlv.com/products/databook/db3/DataBookIII.pdf. 2010. Marcinichen, J.B., Holanda, T.N., and Melo, C. A Dual Siso Controller for a Vapor Compression Refrigeration System. in Internationl Refrigeration and Air Conditioning Conference at Purdue. July 14-17, 2008.