Research Papers

Energy-Efficient Cooling of Liquid-Cooled Electronics Having Temperature-Dependent Leakage

[+] Author and Article Information
Shawn A. Hall

e-mail: sahall@us.ibm.com

Gerard V. Kopcsay

e-mail: kopcsay@us.ibm.com
IBM T. J. Watson Research Center,
Yorktown Heights, NY 10598

Manuscript received February 10, 2012; final manuscript received May 13, 2013; published online October 25, 2013. Assoc. Editor: Ravi Prasher.

J. Thermal Sci. Eng. Appl 6(1), 011008 (Oct 25, 2013) (12 pages) Paper No: TSEA-12-1023; doi: 10.1115/1.4024843 History: Received February 10, 2012; Revised May 13, 2013

Energy conservation in data centers with liquid-cooled electronics is considered, taking into account the combined power consumption of both the electronics and the associated cooling equipment, particularly chillers. The energy-saving technique called “free cooling,” which is most effective at minimizing or eliminating the need for chillers when the temperature of the coolant delivered to the electronics is high, is shown to be at odds with the electronics itself, because of subthreshold leakage in complementary metal-oxide semiconductor (CMOS) transistors, which is minimized when coolant temperature is low. A mathematical model is developed to investigate this trade-off, to discover what liquid-coolant temperature is optimal overall, and thereby to determine what combination of free cooling and traditional chiller cooling should be used for minimal overall power consumption. As an example of how to apply the mathematical model, parameters are chosen in accordance with experimental measurements on an early prototype of a state-of-the-art, water-cooled computer (International Business Machines's (IBM) BlueGene/Q). Results for total data-center power (computer + cooling equipment) are obtained as a function of coolant temperature, the computational state of the computer (which affects CMOS leakage), and weather (which affects the ability to employ free cooling). For the type of system considered, the optimal coolant temperature is found to be a discontinuous function of the other parameters, and traditional chiller cooling is found sometimes to be more energy efficient than free cooling.

Copyright © 2014 by ASME
Your Session has timed out. Please sign back in to continue.


U.S. Environmental Protection Agency, 2007, Report to Congress on Server and Data Center Energy Efficiency, Public Law 109-431, Aug. 2.
Narendra, S. G., and Chandrakasan, A., eds., 2010, Leakage in Nanometer CMOS Technologies, Springer Science+Business Media, New York.
Hanson, Susanna, and Harshaw, Jeanne, 2008, “Free Cooling Using Water Economizers,” TRANE Engineers Newsletter, Vol. 37-3.
Breen, T. J., Walsh, E. J., Punch, J., Shah, A. J., Bash, C. E., Kumari, N., and Cader, T., 2011, “From Chip to Cooling Tower Data Center Modeling: Chip Leakage Power and Its Impact on Cooling Infrastructure Energy Efficiency,” Proceedings of the ASME 2011 Pacific Rim Technical Conference and Exposition on Packaging and Integration of Electornic and Photonic Systems, InterPACK2011, July 6–8.
Breen, T. J., Walsh, E. J., Punch, J., Shah, A. J., and Bash, C. E., 2011, “From Chip to Cooling Tower Data Center Modeling (Part 1): Influence of Server Inlet Temperature and Temperature Rise Across Cabinet,” J. Electron. Packag., 133(1), pp. 1–10. [CrossRef]
Walsh, E. J., Breen, T. J., Punch, J., Shah, A. J., and Bash, C. E., 2010, “From Chip to Cooling Tower Data Center Modeling (Part II): Influence of Chip Temperature Control Philosophy,” 2010 12th IEEE Intersociety Conference on Thermal and Thermomechanical Phenomena in Electronic Systems, June 2–5.
Moss, D., and Bean, J. H., 2009, “Energy Impact of Increased Server Inlet Temperature,” APC White Paper No. 138.
ASHRAE, 2005, Datacom Equipment Power Trends and Cooling Applications, American Society of Heating, Refrigerating and Air-Conditioning Engineers, Atlanta, GA.
Sherwood, S. C., and Huber, M., 2010, “An Adaptability Limit to Climate Change Due to Heat Stress,” Proceedings of the National Academy of Sciences, http://www.pnas.org/content/early/2010/04/26/0913352107.full.pdf
Kuppan, T., 2000, Heat Exchanger Design Handbook, 1st ed., CRC Press, Boca Raton.
ASHRAE, 2009, Thermal Guidelines for Data Processing Environments, 2nd ed., American Society of Heating, Air-Conditioning and Refrigeration Engineers (ASHRAE), Atlanta, GA.
2008 ASHRAE Environmental Guidelines for Datacom Equipment—Expanding the Recommended Environmental Envelope, a Supplement to Ref. [6], available at http://www.ashrae.org/publications/page/1900
Roy, K., Mukhopadhyay, S., and Mahmoodi-Meimand, H., 2003, “Leakage Current Mechanisms and Leakage Reduction Techniques in Deep-Submicrometer CMOS Circuits,” Proc. IEEE, 91(2), pp. 305–327. [CrossRef]
Liu, Y., Dick, R. P., Shang, L., and Yang, H., 2007, “Accurate Temperature-Dependent Integrated Circuit Leakage Power Estimation is Easy,” 2007 Design, Automation and Test in Europe Conference and Exposition, April 16–20.
Leakage in CMOS Nanometer Technologies, previously cited as Ref. [2], Figs. 1–9, p. 11.


Grahic Jump Location
Fig. 1

Data-center cooling

Grahic Jump Location
Fig. 2

Experimental determination of parameters for a prototype of BlueGene/Q (uninitialized state)

Grahic Jump Location
Fig. 3

Exponential parameter κ versus CMOS Gate length

Grahic Jump Location
Fig. 4

Parameter summary for three measured computational states of the prototype BlueGene/Q subsystem

Grahic Jump Location
Fig. 5

Subthreshold power fraction versus junction temperature (for prototype BlueGene/Q processing chips)

Grahic Jump Location
Fig. 6

Total power versus inlet coolant temperature and computational state, for T3≤12°CC

Grahic Jump Location
Fig. 7

Total power versus inlet coolant temperature and computational state, for T3=18°CC

Grahic Jump Location
Fig. 8

Total power versus inlet coolant temperature and computational state, for T3=24°CC

Grahic Jump Location
Fig. 9

Total power versus inlet coolant temperature and computational state, for T3=30°CC

Grahic Jump Location
Fig. 10

Total power versus inlet coolant temperature and computational state, for T3=36°CC

Grahic Jump Location
Fig. 11

Optimum coolant temperature T0* versus, T3andλ and λ, showing downshift locations

Grahic Jump Location
Fig. 12

Downshift boundary

Grahic Jump Location
Fig. 13

How does the downshift boundary depend on parameters?

Grahic Jump Location
Fig. 14

Downshift boundary versus CMOS exponent κ

Grahic Jump Location
Fig. 15

Downshift boundary versus thermal impedance ℜ¯

Grahic Jump Location
Fig. 16

Downshift boundary versus heat-exchanger figure of merit UA

Grahic Jump Location
Fig. 17

Downshift boundary versus chiller-overhead fraction α



Some tools below are only available to our subscribers or users with an online account.

Related Content

Customize your page view by dragging and repositioning the boxes below.

Related Journal Articles
Related eBook Content
Topic Collections

Sorry! You do not have access to this content. For assistance or to subscribe, please contact us:

  • TELEPHONE: 1-800-843-2763 (Toll-free in the USA)
  • EMAIL: asmedigitalcollection@asme.org
Sign In