Abstract

Data centers consume an extraordinary amount of electricity, and the rate of consumption is increasing at a rapid pace. Thus, energy efficiency in data center design is of substantial interest since it can have a significant impact on operating costs. The server cooling infrastructure is one area which is ripe for design innovation. Various designs have been considered for air-cooled data centers, and there is growing interest in liquid-cooled server designs. One potential liquid-cooled solution, which reduces the cost of cooling to less than 5% of the information technology (IT) energy use, is a chiller-less or warm water-cooled system, which removes the chiller from the design and lets the cooling water supply vary with changes in the outdoor ambient conditions. While this design has been proven to work effectively in some locations, environmental extremes prevent its more widespread implementation. In this paper, the design and analysis of a cold water storage system are shown to extend the applicability of chiller-less designs to a wider variety of environmental conditions. This can lead to both energy and economic savings for a wide variety of data center installations. A numerical model of a water storage system is developed, validated, and used to analyze the impact of a water storage tank system in a chiller-less data center design featuring outdoor wet cooling. The results show that during times of high wet bulb operating conditions, a water storage tank can be an effective method to significantly reduce chip operating temperatures for warm water-cooled systems by reducing operating temperatures 5–7 °C during the hottest part of the day. The overall system performance was evaluated using both an exergy analysis and a modified power usage effectiveness (PUE) metric defined for the water storage system. This unique situation also necessitates the development of a new exergy definition in order to properly capture the physics of the situation. The impacts of tank size, tank aspect ratio, fill percentage, and charging/discharging time on both the chip temperature and modified PUE are evaluated. It is determined that tank charging time must be carefully matched to environmental conditions in order to optimize impact. Interestingly, the water being stored is initially above ambient, but the overall system performance improves with lower water temperatures. Therefore, heat losses to ambient are found to beneficial to the overall system performance. The results of this analysis demonstrate that in application, data center operators will see a clear performance benefit if water storage systems are used in conjunction with warm water cooling. This application can be extended to data center failure scenarios and could also lead to downsizing of equipment and a clear economic benefit.

Introduction

Improvements in data center cooling are an important consideration in building energy system analyses due to the large and growing amount of electrical power they consume. U.S. data centers required 91 × 106 kWh of electrical power in 2013, and this amount is expected to grow by the year 2020 to over 140 106 kWh [1]. Despite recent improvements in system design, and reductions in over-provisioning, around 25–35% of the power consumed by data centers is still directed to the cooling infrastructure [2].

While many data centers are air-cooled designs, there is growing interest in liquid-cooled systems, which are more effective for today's higher power systems. Liquid cooling features the high heat transfer coefficients necessary to dissipate the heat from higher power chips, and can be implemented reliably in cold plate designs [3]. While not a new technology, liquid cooling is just now gaining widespread acceptance. Continually increasing demands for functionality are driving chip power density higher and higher, and several supercomputer designs over the past few years have featured liquid cooling including the IBM Power 575 and the Power 775, in which over 96% of the rack heat load is dissipated by water cooling [4,5].

The implementation of liquid-cooled servers has system benefits beyond the increased heat dissipation capacity. The overall infrastructure necessary for air-cooled systems feature either a computer room air conditioner (CRAC) or computer room air handler (CRAH). CRACs use a dedicated air-conditioning system for the server room, while CRAH systems condition the air by direct energy exchange from the air to a chiller. In both these cases, most of the required energy is used to drive either the air conditioning (AC) system or the air handlers and chillers. The implementation of liquid-cooled systems vastly reduces the demand on the CRAC, and replaces the CRAH with a cold-water distribution system instead. Water circulation is less energy intensive than air circulation, and the cooling water to each server is provided through a central closed-loop system that exchanges heat against the building chilled water or a chiller system [5].

Moving to a chiller-less or “warm water”-cooled system can reduce the energy demand even more substantially. In a chiller-less system, the water circulation to the servers is an indoor closed loop that exchanges heat through an intermediate heat exchanger against an exterior cold water supply loop with a cooling tower or dry-cooling system [6]. This two-loop system eliminates the chiller, creating a substantial energy savings. David et al. [6] designed, built, and tested a chiller-less dry cooling design and showed that the energy consumed by the cooling system was reduced to less than 5% of the information technology (IT) energy use. They stated that for a 1 MW data center, this would result in savings of $240,000 per year at an energy cost of $0.11/kWh.

However, the elimination of the chiller creates a situation in which the server cooling loop supply temperature floats with the exterior temperature, linking the server operation conditions to the environmental ambient conditions, generally about 5–6 °C over ambient [6]. A chiller-less system was tested for three full days covering days in both the summer and the fall in central New York to assess the impact of the environmental conditions on system performance [7]. The performance data for all three days reveal that the data center and device temperatures closely track the daily variations in temperature with a swing of about 6 °C. The August day was 10–15 °C warmer than the October days, but the only significant impact was the need for a slightly higher average fan speed for the dry cooler [7]. However, it has been clearly noted that the use of chiller-less designs in regions with more demanding environments may require additional evaporative cooling techniques, although the higher operating temperatures also make the implementation of waste heat recovery techniques more attractive [6]. The transient response time of the system components was analyzed by Gao et al. [8] to find that the server responds quickly to variations in temperature, indicating excellent responsiveness of the cold plates.

As can be seen here, the chiller-less design features excellent energy efficiency but can be subject to fast variations in external temperatures, making it ill-suited for certain environments without modification. In this work, the use of a water storage system is explored to extend the applicability of chiller-less data center designs to more challenging environments. This system is expected to be less expensive and easier to implement than an evaporative cooling system.

There are several possible designs for water storage systems, including simple tank storage systems and underground storage systems such as boreholes or aquifers [911]. While underground storage systems can store a large amount of water at a constant temperature, the large thermal mass inherent in these systems makes them more suitable for seasonal heating and cooling cycles rather than daily cycles. Additionally, their implementation can be limited by environmental factors. Thus, for greatest applicability, storage tanks are considered here. A single tank system is the most cost-effective system; thus, a stratified water storage tank system is a viable solution.

Stratified water storage tanks are commonly used in heating, venting and air conditioning applications to lower operating costs. At night, the AC unit has reduced loading and the system can be used to generate excess cooled water, which is routed into a storage tank. The cold water can then be used during the day to supplement the AC unit or to replace it for a certain amount of time [9]. The intention is to use the same principles here to reduce the loading on the data center cooling system during the day, and to replace it altogether during certain time periods as appropriate. The stored water can also serve a secondary purpose as a back-up cooling system in the event of failure, increasing reliability.

This paper will present the first ever analysis of the impact of the integration of a water storage system into a chiller-less data center design. The combination of the two should result in a high efficiency, low-cost cooling system with applicability to a wide range of potential locations. The system will be evaluated for typical August days in demanding environmental conditions. Additionally, this system design will include wet cooling design, which has not yet been analyzed for chiller-less systems. This is a unique application and its analysis is important for the data center industry. The data center industry requires high reliability to ensure near constant availability, and any design changes to the infrastructure must be thoroughly analyzed and vetted for new applications.

The storage tank analysis is unique in the fact that the water storage will occur at temperatures slightly above ambient, so heat losses to ambient are desirable. The lower the temperature of the stored water, the greater the effectiveness of the cooling system. This design is the opposite of virtually all stratified tank systems, in which heat losses to ambient are undesirable. In solar energy systems, for instance, heat transfer fluids store heat at a temperature far above ambient, and cold water is stored below ambient in the case of building cooling systems. In both cases, heat losses to ambient degrade the performance of the system.

Because there are limited existing analyses of systems where heat losses increase system performance, it is necessary to develop a unique model of the thermal energy storage of a water storage tank for use in a chiller-less data center. This system is thus a unique new application with significant interest and possible implementation in the data center industry.

System Overview

In its simplest form, the system for a chiller-less or warm water-cooled data center consists of the “warm” water loop with outdoor wet or dry cooling, the data center cooling water loop with closed loop water circulation to the cold plates on each server and an intermediate heat exchanger which links the two systems. This arrangement, as seen in Fig. 1 for wet cooling, isolates the external cold-water loop from the data center loop to prevent external contaminants from damaging the servers.

Fig. 1
Simplified system schematic with no water storage
Fig. 1
Simplified system schematic with no water storage
Close modal

Under most operating conditions, the outdoor ambient temperature will be lower than the desired cooling water temperature, leading to favorable conditions for the cooling tower. As the cooling tower produces cold water, some of the supply stream to the intermediate heat exchanger are diverted in order to fill the storage tank, as shown in Fig. 2. The cold water is then stored until it is needed—such as for a sudden loss of coolant situation, for peak loading situations, or for situations in which the cooling tower cannot meet the targeted cooling water temperature due to adverse climatic conditions. During discharging, the cold water is drawn from the tank and the cooling tower is shut off as seen in Fig. 3.

Fig. 2
Water storage system during charging, black lines represent pipes with no flow
Fig. 2
Water storage system during charging, black lines represent pipes with no flow
Close modal
Fig. 3
Water storage system during discharging, black lines represent pipes with no flow
Fig. 3
Water storage system during discharging, black lines represent pipes with no flow
Close modal

During the tank filling process, the fresh cold water enters the tank at its bottom and displaces warmer stored water out the top of the tank. This creates a stratified tank condition with a stratification layer (or thermocline) between the cold water and the warmer water. The tank is filled until the thermocline reaches the top of the tank. All displaced warm water is sent directly to the cooling tower.

During a tank discharge, cold water is pumped directly from the bottom of the tank below the stratification layer, and the thermocline moves to the bottom of the tank. As the stratification layer reaches the tank bottom, the discharge is complete and the tank is ready for recharging with cold water. The total discharge time of the tank is related to the tank volume and mass flowrate, and can be optimized based on data center usage scenarios.

System Modeling

The data center warm water-cooling system shown in Figs. 13 was modeled both with and without the water storage system. The modeling was completed using the in-house Villanova thermodynamic analysis of systems (vtas) software. vtas is a modeling tool that couples component models to each other using a flow network. The software is specifically designed for data center analysis. The component models in vtas are individual matlab functions that incorporate the physics of individual equipment such as a chiller or heat exchanger. Linking the component models together in a systematic framework enables calculations of the system's second law efficiency through an exergy destruction metric [12]. The steady-state scheme balances energy, humidity, and mass flows to enable sizing of equipment. For transient calculations, each sized component then calculates outlet thermodynamic properties based on inlet stream thermodynamic values, enabling system design under transient conditions such as those in this study.

Early published works on vtas centered around steady-state design of both legacy air [1214] and nontraditional cooling systems. These nontraditional systems include hybrid liquid–air systems such as rear door heat exchangers, in-row coolers, and overhead coolers [1517]; and direct liquid cooled systems [18]. These studies showed that legacy air systems had the largest exergy destruction, followed by hybrid liquid–air systems, and finally direct liquid cooled (cold plate) systems [18]. Furthermore, the largest portion of exergy destruction can be attributed to the air cooling of servers and direct expansion refrigerant units such as CRAC units or chillers [14]. These studies suggest that system exergy destruction can be minimized through the combined use of direct liquid cooling along with the removal of a chiller, which leads to the concept of warm water cooling explored in this study.

Four component models were used to create the systems for this study: storage tank, cooling tower, liquid–liquid heat exchanger, and a water-cooling system for the servers. Two of these models already existed within the vtas framework: the water-cooled system and the cooling tower. The water-cooling system is modeled as a single component that is placed within the chilled water loop. The system features automatic branching of supply water lines to one coolant distribution unit per row of racks, which then distributes water to each cold plate within that row. The mass flow rate of each branch is set proportional to the server heat output so the water temperature is constant in all return branches. The cold plates on each server are modeled using a standard effectiveness-number of transfer units (NTU) correlation for a constant chip temperature, where at steady-state, UA is calculated using a user-specified server heat output and chip temperature. In the transient mode, this UA (20 W/K) is used to calculate the cold plate effectiveness and the estimated operating chip temperature.

The cooling tower component uses the counterflow heat and mass exchange model by Khan et al. [1921], who discretize the tower into a one-dimensional (1D) system of finite volumes. Their model uses a Lewis factor calculation by Kloppers and Kröger [22] and is validated by experimental data by Simpson and Sherwood [23]. The results of the applied model were converted into a lookup function for a calibrated artificial neural network scheme, which reduced the computational time in the component model by several orders of magnitude with an error of 6% within the data set discussed in Ref. [23]. Finally, the cooling exergy destruction calculation, which includes heat and mass exchange terms, follows the formulation by Muagnoi et al. [24,25]. One limitation of the model, however, is that the calculated heat and mass exchange are independent of cooling tower physical size, and therefore the cooling tower size is fixed here to match those used in Ref. [23].

The heat exchanger component model is a steady-state plate-and-frame model with a constant UA value. An effectiveness-NTU correlation is used to calculate the outlet water temperatures for the heat exchanger, where the plate-and-frame heat exchanger correlation is identical to that for counterflow heat exchangers [26,27]
(1)

where ϵ is the heat exchanger effectiveness, Cr is the heat capacity ratio, and NTU is the number of transfer units. The UA value was calculated for a commercial coolant distribution unit from the provided inlet and outlet temperatures and flow rates [28].

The water storage tank component model was created for this study. The charging and discharging of the tank were developed using a plug flow model due to its accuracy and speed. In a work by Kleinbach et al. [29], several types of water storage models were evaluated. The models based on a multinode analysis were found to be accurate when developed using an optimal number of nodes, which must be determined in advance. In contrast, the plug flow models were shown to increase in accuracy with the number of nodes selected for the analysis, which allows the implementation of a sensitivity analysis to determine accuracy.

In a plug flow analysis, the fluid in the tank is divided using horizontal slices into a number of elements, each with a node at its center. The fluid within this domain is considered to act as a homogeneous “plug” of fluid. The transient computational analysis takes place over a number of discrete time steps. During each major time–step, a discrete volume of water flows into the tank and occupies one full element. The fluid in each tank element shifts correspondingly up or down to accommodate the inflow with the final element ejected from the tank. During charging, cold water is pumped into the bottom of the tank and warm water exits the tank top. During discharging, warm water flows into the top of the tank and the cold water exits at the bottom. The element volumes are uniformly sized such that the amount of water that flows into the tank during each major time-step is equal to the volume of each element. This means that the element size, flow rate, and time-step sizes are directly proportional.

The transient heat transfer equation can be discretized and written in nodal form as
(2)

where ρ is the mass density, v is the nodal volume, cp is the specific heat capacity, U is the overall heat transfer coefficient, A is the area, T is the temperature, and h is the nodal thickness in the vertical direction. The subscript j denotes the current node, Aj represents the nodal sidewall area, and Acs represents the nodal cross-sectional area. The left-hand side of Eq. (2) is the energy storage term, while the right-hand side is the heat transfer into and out of the element. The first term on the right-hand side represents losses to ambient air through the tank walls; the second term is conduction heat transfer from the node above; and the third term is conduction heat transfer from the node below.

The temperature distribution at the end of each major time-step is calculated by solving Eq. (2) using the Adams-Bashforth predictor–corrector method [30]. This method divides each major time-step into many minor time steps, where the major time steps are applied system wide, and the minor time steps are internal to the tank component model. The tank temperature distribution is calculated at each minor time-step until the minor time steps accumulate to equal one major time-step. Each major time-step allows the next volume of water to enter the tank.

Equation (2) does not account for mixing of the fluid within the tank, although there can be significant mixing, particularly at the inlets to the tank. Mixing occurs near the inlets because of the inertial forces associated with the inflow, and is a significant source of storage tank inefficiencies. This is a large factor in the formation of the thermocline [3133]. Once the thermocline is formed, its thermal shape remains relatively unchanged through the remainder of the charging and discharging process [3133] as only conduction through the water and heat transfer through the sides of the tank causes temperature changes away from the inlets. The heat transfer from ambient to the water is modeled using an overall heat transfer coefficient though the tank [29,31].

The mixing effect is included here by averaging the nodal temperatures closest to the inlet manifolds to reflect a mixing process. This is a simple yet effective way to represent mixing in a fast computational model and has been proven to yield sufficiently accurate results in previous work [31]. The water storage tank model makes the following assumptions: 1D heat transfer along the height of the tank, no mixing between nodes (except near the tank inlets), constant density, and constant heat-specific capacity.

The steps taken to solve the component model in one system-wide time-step are as follows:

  1. (1)

    Add volume of water to inlet, subtract volume from outlet;

  2. (2)

    Average temperatures of cells near inlet to represent mixing;

  3. (3)

    Incrementally step through minor time steps for the tank model to solve system of differential equations (Eq. (2)) until enough minor time steps have accumulated to equal one major time step, which is applied system wide.

The developed numerical model was compared to Wildin and Truman's experimental data [31] for water storage in a concrete square tank with dimensions of 3.65 m × 3.96 m × 4.27 m. Thermocouples were arranged throughout the tank, and the temperature distribution was measured during charging and discharging cycles.

The present work is validated against both Wildin and Trumans's Test A, which featured charging and discharging flow rates of 30 gpm and against Test B, which featured a charging flow rate of 40 gpm and discharging flow rate of 30 gpm [31]. All other factors were kept constant between the two experiments.

The accuracy of the developed model was characterized by both the developed temperature profiles within the tank, including the stratification layer, and the overall energy efficiency, where the energy efficiency of the processes is defined as the integrated discharge capacity over the integrated charge capacity
(3)
(4)

Here, Qd is the energy pumped out of the tank during discharging, Qc is the energy pumped into the tank during charging, Th is the water temperature at the top inlet/outlet of the tank, Tl is the water temperature at the bottom inlet/outlet of the tank, and v is the volume. It should be noted that this parameter is meaningful for the work of Wilden and Truman in which heat loss from the tank is undesirable, but this fact has limited applicability in the situation under consideration here. Thus, this parameter is used for validation of the model to Ref. [31] only, and is not used further in this analysis.

The mixing layer thickness was found to have a significant effect on the development of the temperature stratification within the tank. Different mixing layer thicknesses were considered in the model as seen in Figs. 4(a)4(c) where nondimensional tank position is defined as the height of the temperature measurement location over the tank height. These figures illustrate the temperature distribution of the tank throughout the charging process. The slope of the transition from the constant cold water temperature at the tank bottom to the constant hot water temperature at the tank top represents the thickness of the thermocline. The experimental data are from Ref. [4] and are the same in each case while the calculation of mixing layer thickness varies. The mixing process is simulated by averaging the nodal temperatures closest to the inlet manifolds to reflect mixing. The number of nodes considered is varied from 4% to 10% of the tank height.

Fig. 4
Comparison of numerical and experimental temperature distribution in tank during test A charging process: (a) 4% mixing layer thickness, (b) 7% mixing layer thickness, and (c) 10% mixing layer thickness
Fig. 4
Comparison of numerical and experimental temperature distribution in tank during test A charging process: (a) 4% mixing layer thickness, (b) 7% mixing layer thickness, and (c) 10% mixing layer thickness
Close modal

In Fig. 4(a), the mixing layer thickness is the smallest at only 4% of the tank height. As can be seen in comparison with the experimental data, the thermocline location is consistently underpredicted. Conversely, when a mixing layer thickness of 10% was used (Fig. 4(c)), the predicted thermocline slope is overpredicted. As such, the mixing near the entrance was chosen to occur over 7% of the tank height as shown in Fig. 4(b). This is not a perfect representation, as a discrepancy between the model and the experimental data can be seen at the onset of the charging process and near the bottom of the tank (nondimensional location <0.1). However, for a 1D simplified model as is desired here, the accuracy is acceptable.

A sensitivity analysis was completed to determine the necessary number of nodes in the water storage tank to ensure computational mesh independence. Mesh independence was achieved around 200 nodes. With 200 nodes, the computational time is just under 2 s for the simulation of a day's worth of charging and discharging. Thus, the model is found to balance both accuracy and speed to create a simple yet effective tool to analyze the first-order effects of water storage on warm water-cooled data centers.

Exergy Modeling

Exergy is the amount of energy, which is available to be converted into useful work. An exergy balance on a control volume is given by
(5)

where Ψ˙ is the exergy flow rate of a heat transfer process, m˙i is the inlet mass flow rate, m˙e is the exit mass flow rate, ψ is the specific energy per unit mass, W˙act is the actual work output of the control volume, and S˙gen is the entropy generation of the control volume. The last term of the equation is known as the exergy destruction. This is the amount of available work that is lost due to any irreversibilities. This value is always positive in real systems. An exergy balance, like an energy balance, can be taken on any system in order to calculate exergy destruction. Analyzing the exergy destruction of a component or system is a useful way of identifying inefficiencies. For example, a high value of exergy destruction often indicates a large temperature difference across a heat transfer device. For a cooling system, it is important to keep temperature differences as low as possible, and the efficiency of the water storage tank can be analyzed using exergy. The exergy control volume for this analysis includes both the storage tank and the air surrounding the tank as seen in Fig. 5. There are inlet and outlet mass flows m˙i and m˙e at temperatures Ti and To. Each tank element transfers heat to ambient (Q˙0X) at temperature T0.

Fig. 5
Control volume boundaries for storage tank with inlet and outlet heat transfer processes shown
Fig. 5
Control volume boundaries for storage tank with inlet and outlet heat transfer processes shown
Close modal
Starting with the exergy balance equation (Eq. (5)), and eliminating the output work term and work due to change in volume (which are not applicable here), the exergy balance becomes
(6)
The terms for exergy transfer due to heat transfer, the specific exergy of a fluid flow, and exergy of a control volume are
(7)
(8)
where E is internal energy. Incorporating these equations into Eq. (6) leads to
(9)
Note that the first term on the right-hand side term will become zero because the heat lost from the tank mixes with ambient air until it reaches the temperature T0 at the control volume boundary. This is simply a consequence of the control volume boundaries being drawn far enough outside the tank to include the surrounding air, which remains at ambient temperature T0. The potential and kinetic energy of the water are negligibly small, so the exergy balance becomes
(10)
where h is the specific enthalpy of the water or surroundings. For an incompressible liquid, the enthalpy is proportional to the specific heat and temperature
(11)
and the change in entropy can be written as
(12)
Substituting this into the exergy balance equation yields
(13)
Differencing with respect to time, where Δt is the time-step size yields
(14)
where E1 and E2 are the internal energy of the cell at the beginning and end of the time-step, respectively; S1 and S2 are the entropy of the cell at the beginning and end of the time-step, respectively; and Δt is the time-step. Substituting for the exergy storage term yields
(15)
Solving for exergy destruction
(16)

Equation (16) shows that the exergy destruction in the water storage tank is the difference between the net exergy pumped into the tank with the flow of water minus the change in exergy of the water in the tank. This equation allows us to identify the primary causes of exergy destruction. They are:

  1. (1)

    Heat transfer to the ambient;

  2. (2)

    Equilibration of water temperature due to conduction;

  3. (3)

    Mixing.

Minimization of component exergy destruction is achieved by minimizing these parameters.

Results

Impact of Cold Water Storage Systems.

Water storage systems are expected to be most beneficial for warm water-cooled data centers in environments where the cooling tower will not meet the cooling requirements under all environmental conditions. This will occur most often in climates with high wet bulb temperatures. For this reason, the test location was chosen to be a site subject to environmental extremes, including high wet bulb temperatures. A National Institute of Standards and Technology data base was used to identify Webster City, IA, in August as suitable site. Figure 6 depicts the average ambient conditions for this location on August 5, which was chosen as the starting day for the simulation as it is in the hottest, most humid part of the summer. It can be seen that, as expected, there is a daily sinusoidal-like swing in temperatures. The dry bulb varies from as high as 33 °C to a low of 25 °C. The relative humidity is seen to be quite high, ranging from 60% to 78%. This creates a wet bulb temperature variation from 27 °C to 21 °C.

Fig. 6
Average climatic conditions in Webster City, IA, starting on August 5
Fig. 6
Average climatic conditions in Webster City, IA, starting on August 5
Close modal

The variation in wet bulb temperature directly affects the performance of the cooling tower, and creates a corresponding swing in both the heat exchanger performance and the resulting water temperature for server cooling. These swings in temperature are unique to warm water cooled data centers since they do not have a chiller, which would have moderated these swings but at a substantial cost. The impact of these temperature swings will be examined here for data centers both with and without water storage capabilities.

The developed and validated model was first used to simulate a standard warm water cooling data center without water storage (Fig. 1) at this location using the operating conditions as defined in Table 1. The resulting temperatures throughout the system are seen in Fig. 7, which shows the inlet and outlet temperatures of the heat exchanger. A smaller temperature difference is seen on the data center side due to the higher mass flowrate.

Fig. 7
Intermediate heat exchanger temperatures on both cooling tower and data center sides
Fig. 7
Intermediate heat exchanger temperatures on both cooling tower and data center sides
Close modal
Table 1

Warm water-cooled data center operating conditions

Data center
270 servers, 2 CPUs per server
Total power output: 54 kW
Cold plate UA: 20 W/K
Heat exchanger UA: 8000 W/K
Data center loop flow rate: 0.008 m3/s
Cooling tower
Main loop flow rate: 0.0021 m3/s
Air flow rate: 1.8 m3/s
Feed water flow rate: 0.002 m3/s
Cooling tower loop flow rate: 0.0021 m3/s
Initial conditions
Cooling tower loop: 40 °C
Data center loop: 45 °C
Data center
270 servers, 2 CPUs per server
Total power output: 54 kW
Cold plate UA: 20 W/K
Heat exchanger UA: 8000 W/K
Data center loop flow rate: 0.008 m3/s
Cooling tower
Main loop flow rate: 0.0021 m3/s
Air flow rate: 1.8 m3/s
Feed water flow rate: 0.002 m3/s
Cooling tower loop flow rate: 0.0021 m3/s
Initial conditions
Cooling tower loop: 40 °C
Data center loop: 45 °C

Figure 8 shows the supply temperature of the cooling water to the server cold plates and the resultant cold plate temperature. The maximum cold plate temperature is found to be 56 °C. This number is well within the normal operating constraints in order to maintain a chip temperature of less than 85 °C.

Fig. 8
Data center inlet water temperature and cold plate interface temperature for the control case
Fig. 8
Data center inlet water temperature and cold plate interface temperature for the control case
Close modal

The impact of cold water storage is analyzed by adapting the system to include a water storage tank. The operating conditions remain the same as presented in Table 1, with slight modifications as seen in Table 2, to include the tank conditions and an increase in the number of operating servers. The number of servers is increased for the same operating conditions due to the impact of the storage tank. The charging and discharging times for the storage tank are chosen such that the center of the discharging period coincides with the highest wet bulb temperature of the day. This results in the use of cold stored water during the day, and the ability to idle the cooling tower during the hottest part of the day. The tests done were with a discharge duration of 7 h, with the tank recharging slowly over the remaining 17 h. The mass flow rates were 0.7 kg/s and 1.4 kg/s for charging and discharging, respectively. The water flow rates were chosen so that the tank would charge and discharge about 65% of its volume each cycle. This charge ratio helps maintain the thermocline within the tank and ensures that cold water is delivered to the data center. The impact of charge ratio on the performance of the system will be investigated. The initial temperature of the tank is set to the ambient temperature at the start of the simulation.

Table 2

Data center operating conditions with water storage

Data center
375 servers, 2 CPUs per server
Total power output: 75 kW
Storage tank
Height: 8 m
Diameter: 3 m
Usides: 5 W/m2K
Utop: 2 W/m2K
Data center
375 servers, 2 CPUs per server
Total power output: 75 kW
Storage tank
Height: 8 m
Diameter: 3 m
Usides: 5 W/m2K
Utop: 2 W/m2K

The analysis shows that the addition of water storage has an overall positive effect on the behavior of the baseline system. The cold plate temperature is seen in Fig. 9 to vary throughout the day from 46 to 51 °C, which is 5–7 °C cooler than without the water storage (53–56 °C) despite applying a power dissipation level almost 40% higher than for the condition without water storage.

Fig. 9
Response of cold plate interface temperature to the use of water storage
Fig. 9
Response of cold plate interface temperature to the use of water storage
Close modal

In this case, the parameters that have the greatest influence on the projected cold plate temperature are the outdoor ambient conditions and the temperature profile of the water stored in the tank. Due to the lack of natural convection modeled in the tank, the tank is found to serve as a mirroring mechanism as any water exiting the tank will be at roughly the same temperature than it was when it entered. The temperature within the tank at various heights during a charging cycle is seen in Fig. 10. Thus, the temperatures of the water supplied during the discharging process are almost mirror images of the tank water inlet temperatures during the charging process.

Fig. 10
Temperature distribution of storage tank during a charging cycle
Fig. 10
Temperature distribution of storage tank during a charging cycle
Close modal

However, because the charging and discharging times are unequal, the discharging temperature profile with respect to time features higher temperature gradients as seen in Fig. 9. While the tank design is selected to minimize mixing and thus maintain the stratification layer, the real-world condition will not have exact mirroring due to the presence of minimal natural convection.

Figure 9 depicts the inlet water temperature to the data center cooling loop and the projected cold plate interface temperature with respect to time. In this case, for the first 7 h of operation, the stored water in the tank is discharged and the cooling tower is idled. The tank was set to an initial temperature of 45 °C, so the water initially discharges at a constant temperature of 45 °C. Following the discharge of the tank, the flow switches to tank charging mode. During this mode, water from the cooling tower is used to cool the servers while a side stream refills the storage tank as depicted in Fig. 2.

The cooling water temperature is now determined by the cooling tower performance. This 17 h cycle occurs over the evening, night, and early morning hours. The ambient temperature during this time slowly drops through the evening and night, and then begins rising again in the morning hours. Thus, the temperature of the stored waster varies with this same profile. At the end of this cycle (at the 24 h point), the cooling tower switches off and the storage tank begins discharging. The temperature profile in the tank is now pumped out in reverse order to the data center, but over 7 h instead of 17, creating a steeper temperature profile. This process repeats until 3 days have passed. Under all operating conditions here, the projected cold plate interface temperature is lower than for the system without the water storage.

Exergy Analysis

A stand-alone exergy analysis was performed in order to determine the areas of greatest losses, and thus work, to maximize the efficiency of the water storage system. In order to most clearly see the impact of exergy on the system, an aggressive, worst case scenario is chosen for the operating conditions in which the temperature of the water being discharged from the tank is unusually high. Although the situation is a worst case scenario, the overall trends will hold true for more realistic operating conditions. The extent of exergy destruction in the system was analyzed using the operational scenario detailed in Table 3 in which a storage tank is filled and then discharged. The initial temperatures in this case are chosen to illustrate the effects of heat transfer to ambient during both charging and discharging. The effects of water equilibration and mixing are also quantified.

Table 3

Water storage exergy analysis

Storage tank
Height: 6 m
Diameter: 3 m
Usides: 8 W/m2K
Utop: 4 W/m2K
Temperatures
Ambient: 27 °C
Tank initial temperature: 47 °C
Tank charging: 37 °C
Tank discharging: 57 °C
Flow rates
0.1 kg/s charge and discharge
Storage tank
Height: 6 m
Diameter: 3 m
Usides: 8 W/m2K
Utop: 4 W/m2K
Temperatures
Ambient: 27 °C
Tank initial temperature: 47 °C
Tank charging: 37 °C
Tank discharging: 57 °C
Flow rates
0.1 kg/s charge and discharge

These values were chosen such that the relative effect that each parameter has on the performance of the system can clearly be identified. For both charging and discharging, the tank was set to an initial temperature of 47 °C and then was either charged with cold water from the bottom of the tank (expelling hotter water out the top), or the stored water was discharged from the bottom of the tank, (receiving hotter returning water at the top). Using Eqs. (7), (8), and (16), the resulting exergy flow rates are shown in Fig. 11(a) (charging) and Fig. 11(b) (discharging).

Fig. 11
Exergy flow rates during (a) charging and (b) discharging
Fig. 11
Exergy flow rates during (a) charging and (b) discharging
Close modal

In Fig. 11(a), both the flow exergy and the tank exergy are seen to be negative because cold water is replacing warmer water in the tank. From Eq. (16), the exergy destruction is the difference between these two curves. The exergy destruction is initially high but decreases rapidly over the first quarter of an hour of charging. This is because the initial mixing at the inlets creates a large exergy destruction when compared to the net flow exergy. However, after this initial mixing period, the exergy destruction levels off. This is because once the tank begins to fill steadily, mixing decreases and the exergy destruction is now due primarily to conduction in the tank and convection to ambient, because natural convection in the tank is not considered. Also, as the tank fills and as heat is transferred to ambient, the temperature difference between the tank and the ambient decreases. This causes the heat transfer rate to ambient to decrease, causing exergy destruction to decrease as well.

Figure 11(b) depicts the exergy flow rates for the discharging case. As with the charging case, the exergy destruction is initially high; however, the exergy destruction rate slowly increases afterward instead of decreasing. This is because the temperature difference between ambient and the tank water increases with time as the tank fills up with the returning warm water which leads to additional losses to ambient and thus additional exergy destruction. At the same time, heat losses to ambient are acting to decrease the tank temperature, creating an increasingly large temperature difference between the incoming warm water and the discharging cold water as time progresses. This causes the storage term and flow terms to diverge.

It can be confirmed from these trends that the storage tank will exhibit the lowest exergy destruction if mixing, conduction, and heat losses to the ambient are all minimized. Interestingly though, in this application, the water being stored is initially above ambient, but the overall system performance improves with lower water temperatures. Therefore, heat losses to ambient are beneficial to the overall system performance, which is a highly uncommon application for water storage tanks. In application, this could be maximized by designing a tank with fins to maximize heat losses.

For this situation, in which heat losses are beneficial, a new exergy performance parameter is defined in which exergy is calculated between the tank temperature and the temperature of the chip rather than between the tank and the ambient. This definition (Eq. (17)) causes the tank exergy to increase as the temperature difference between the tank and the cold plate increases
(17)
Solving for exergy destruction
(18)

Equation (18) now includes a term that represents the heat transfer to the ambient, and the control volume is changed so that it no longer includes the immediate surroundings of the tank. This modified definition provides a useful metric for characterizing data center cooling systems in general. According to this new definition, the cooling system is most efficient when the coolant is maintained at a temperature that is as low as possible, consistent with the goals of the system.

Equation (18) was applied to the same operating scenario described in Table 3. For the calculation of exergy, a reference (cold plate) temperature of 77 °C (350 K) was used. Figure 12 shows the new exergy flow rates of the tank during charging, where all three terms are now positive. It also shows a similar trend in exergy destruction. During the initial few minutes of the simulation, the exergy destruction rate is high due to mixing, and then it gradually tapers down. After about half an hour, the exergy destruction rate goes almost to zero. This is because the mixing in the tank has gone to zero and because the exergy term for convective heat loss is no longer a contributor of exergy destruction. At this point, the exergy flowing into the tank is being absorbed by the water with minimal losses. Comparing these results to those from the standard exergy definition shows that it is useful to use the modified definition because the results now show that the water in the tank is gaining cooling capacity as heat is leaving the tank.

Fig. 12
Exergy flow rates of storage tank during charging
Fig. 12
Exergy flow rates of storage tank during charging
Close modal

Exergy Economic Analysis

The response of the system for an 8-h simulated time period was performed to examine the total amassed exergy destruction from the various components. The sources of thermal exergy destruction are the cold plates, the heat exchanger, the cooling tower, the CRAC unit, and the storage tank. Any changes in flow exergy destruction are considered negligible. A cold plate may be approximated as a channel with a fixed wall temperature on one side (the other is adiabatic), designated as Tchip. Taking the server heat output of Q˙s, then the exergy destruction is
(19)
For the heat exchanger
(20)

where the subscripts hot and cold indicate the hot and cold fluid streams, respectively, and the subscript f indicates a water physical property.

For the cooling tower, the exergy destruction is determined using correlations by Muangnoi et al. [24,25], which include both thermal exergy destruction and chemical exergy destruction stemming from mass transfer. Finally, the CRAC unit has an exergy destruction of
(21)

where the subscript a designates an air property, and the subscripts in and out indicate the inside and outside air loops interacting with the CRAC unit, respectively.

The results show that the largest exergy destruction (O(107 J)) was associated with the cold plate assembly and CRAC unit. Much lower exergy destruction (O(105 J)) was seen for the storage tank, cooling tower, heat exchanger, pumps, and fans. The large exergy destruction for the cold plate is expected given that the difference between the chip temperature and cold plate outlet temperature is significant (O(10 °C)), resulting in the largest temperature drop in the system. It should be noted, however, that the temperature drop using cold plates is less than that using conventional air cooling, meaning that in terms of exergy economy cold plates provide more value than server fan-based cooling. For CRAC units, the large exergy destruction is due to the compressor work input requirement, which also suggests that the removal of other equipment containing refrigerant loops (i.e., the chiller) can have a great benefit in exergy savings. Therefore, this system-wide exergy analysis shows that a warm water-based system is superior to chiller-based or conventional air-cooled systems.

System Analysis.

In the analysis of a data center cooling system, the projected chip operating temperature (cold plate interface temperature is used as a proxy here to avoid biasing toward any one chip design) and the power consumption of the cooling equipment are important considerations. The power consumption of a data center is often analyzed using a metric known as power usage effectiveness (PUE). PUE is defined as
(22)
In this study, the calculation of PUE is adapted in order to isolate the effects of the water storage tank only. In this streamlined case, the heat dissipated by the chips is considered to be a proxy for the power consumption by the IT equipment, as most of the IT power is dissipated as waste heat. The total facility power is considered to be the heat dissipated plus that used to operate the circulation pumps and the cooling tower, which isolates the effects of the water storage tank on the system. This modified PUE will be referenced here as PUEcooling and will be calculated using Eq. (18)
(23)

where W˙p is the power required to run the circulation pumps and/or the cooling tower, and Q˙DC is the data center heat dissipation.

The effect of tank size, tank aspect ratio, fill percentage, and charging/discharging time on the cold plate interface temperature and PUEcooling are analyzed here against the baseline cold water storage case for comparison. Fill percentage is considered to be ∼60% or ∼90%. This value will affect the amount of stored water in use, and thus if the discharge time is held constant, the mass flow rate drops for lower fill ratio cases as less water is being pumped out over the same time period. For variations in charging/discharging time, the fill ratio will remain constant while time period changes, again affecting mass flow rate. The results are summarized in Table 4.

Table 4

Effect on PUEcooling and cold plate interface temperature of tank geometry and flowrate

Tank volume (m3)Pump power (W)PUEcoolingTcoldplate InterfaceTank Avg. ΔT (°C)Charging (kg/s)Discharging (kg/s)Charging fill (%)Discharging fill (%)Discharging time (h)Aspect ratio
283571.00552.918.90.410.9589.685.172.67
577701.01051.19.30.841.9091.385.172.67
11337951.05149.54.01.623.7887.884.472.67
283151.00457.126.20.290.7263.064.772.67
575531.00751.713.10.571.4761.765.572.67
11319811.02649.06.21.122.9460.965.872.67
577701.01051.19.30.841.9091.385.17.002.67
575371.00749.2212.20.931.588.785.29.002.67
574431.00650.3014.631.081.2290.085.411.002.67
575531.00751.7413.10.571.4761.765.575
575531.00751.7413.10.571.4761.765.578
Tank volume (m3)Pump power (W)PUEcoolingTcoldplate InterfaceTank Avg. ΔT (°C)Charging (kg/s)Discharging (kg/s)Charging fill (%)Discharging fill (%)Discharging time (h)Aspect ratio
283571.00552.918.90.410.9589.685.172.67
577701.01051.19.30.841.9091.385.172.67
11337951.05149.54.01.623.7887.884.472.67
283151.00457.126.20.290.7263.064.772.67
575531.00751.713.10.571.4761.765.572.67
11319811.02649.06.21.122.9460.965.872.67
577701.01051.19.30.841.9091.385.17.002.67
575371.00749.2212.20.931.588.785.29.002.67
574431.00650.3014.631.081.2290.085.411.002.67
575531.00751.7413.10.571.4761.765.575
575531.00751.7413.10.571.4761.765.578

As seen in Table 4, tank aspect ratio, tank geometry, and fill ratio are found to have little to no effect on PUEcooling. However, PUEcooling is affected as the tank charging/discharging time is varied. Shorter charging/discharging times require higher mass flow rates, which cause the pumps to consume more energy, increasing PUEcooling. In application, this will have to be considered in order to minimize energy demands as much as possible.

The parameters that affect the predicted cold plate interface operating temperature the most are tank size, fill percentage, and charging/discharging time. To understand these trends, it is necessary to take a look at the factors that contribute to the cold plate interface temperature. Figure 13 illustrates the three parameters that contribute to the operating cold plate interface temperature. The first is the temperature of the water supplied to the intermediate heat exchanger by either the cooling tower or the water storage tank. The second is the temperature difference across the heat exchanger that drives the temperature of the water exiting the intermediate heat exchanger on the data center side. The third is the temperature difference between the temperature of the water exiting this heat exchanger and the cold plate interface temperature. This temperature difference reflects any heat exchange on the data center side through cold plates and/or rear door heat exchangers. For this study, this temperature difference remains constant throughout the analysis as the flow rate in the data center loop is not varied.

Fig. 13
Breakdown of contributors to maximum cold plate interface temperature for the 90% fill cases
Fig. 13
Breakdown of contributors to maximum cold plate interface temperature for the 90% fill cases
Close modal

The temperature difference across the intermediate heat exchanger is determined by the inlet mass flow rates and the UA value of the heat exchanger. For this analysis, the UA value is held constant. While in reality this value with vary with flow rate, this is a simplified analysis. As the flowrate on the data center side is also held constant, the temperature difference across the heat exchanger is dependent only on the inlet mass flow rate on the cooling tower side. This flow rate is equal to the mass flowrate of the water exiting the tank during discharging. As can be seen in Fig. 13, as this flow rate increases from 0.95 to 3.78 kg/s for a larger tank, the temperature difference through the heat exchanger decreases from 19.6 to 10.6 °C.

The temperature of the inlet water to the heat exchanger on the cooling tower side is cyclical and is affected by flowrate through the cooling tower, ambient dry bulb temperature, relative humidity, and the length/timing of the charge/discharge cycles. With this in mind, Fig. 14 compares the cold plate interface temperature for two different fill percentages and three different tank sizes. Figure 15 compares the mass flow rates for those same cases. For all cases, the tanks charge and discharge over the same time period, regardless of size or fill ratio. Some interesting trends can be seen in these cases. For the smallest tank size, the mass flow rates are correspondingly low. These low flow rates lead to higher temperature differences across the intermediate heat exchanger, and the differences for the 60% and 90% fill ratios are amplified. The 90% fill ratio will have a higher flow rate (0.95 kg/s) than the 60% fill ratio (0.72 kg/s) due to the larger volume of water used from the tank. This higher flow rate leads to a lower predicted cold plate interface temperature (reduced 4 °C) for the 90% fill ratio case as the temperature difference across the intermediate heat exchanger is reduced.

Fig. 14
Comparison of maximum cold plate interface temperature and PUEcooling as a function of tank volume and fill percent
Fig. 14
Comparison of maximum cold plate interface temperature and PUEcooling as a function of tank volume and fill percent
Close modal
Fig. 15
Comparison of mass flow rates as a function of tank volume and fill percent
Fig. 15
Comparison of mass flow rates as a function of tank volume and fill percent
Close modal

For the largest tank size, the predicted cold plate interface temperature is higher for the 90% fill ratio case, as seen in Fig. 14. For this case, the higher mass flow rate for both cases leads to a narrowing of the temperature difference across the intermediate heat exchanger and it is no longer the dominant parameter. Instead, the water supply temperature becomes the dominant parameter. As the flow rate increases, the cooling tower effectiveness is reduced. At the higher flowrate for the 90% fill case, the inlet water temperature to the heat exchanger is higher by 1.2 deg. This is enough in this case to reverse the performance of the 60% and 90% fill ratios. This is important to note in application, as the effect of mass flowrate on both the intermediate heat exchanger and on the cooling tower must be evaluated when deciding on a water storage operating scenario.

Figure 16 compares the performance of the system for different discharge times. It appears that an optimum operating condition exists between the shortest and longest discharge times, which is because the shortest discharge time (7 h) leads to a higher inlet water temperature to the intermediate heat exchanger compared to two longer discharge cases. This temperature excursion is caused by two things: the high mass flow rate, which decreases the cooling tower efficiency, and the high ambient temperature at the beginning of the charging time. The 7-h discharge time leads to a 17 h recharge time. The longer recharge time means that the tank is being charged over a period that includes hotter ambient temperatures and thus includes hotter water temperatures.

Fig. 16
Maximum cold plate interface temperature and PUEcooling as a function of discharge time
Fig. 16
Maximum cold plate interface temperature and PUEcooling as a function of discharge time
Close modal

The 9-h discharge case results in the lowest predicted cold plate interface temperatures. This results from a balance between heat exchanger operation and cooling tower efficiency. The mass flow rate in this case is high enough to minimize the heat exchanger temperature difference, but low enough that the cooling tower efficiency is not affected. Additionally, the longer discharging time allowed for ambient conditions to be more favorable during the start of the charging process. A further increase in discharge time to 11 h results in an increase in maximum cold plate interface temperature. This is because the decrease in mass flow rate causes the heat exchanger temperature difference to increase while the cooling tower performance only increases slightly.

The results of the parametric study were used to determine “best-case” scenarios for the cold water storage system, which are compared to a data center without water storage in Table 5. The operating scenario, which resulted in the lowest energy use, was the small tank at a 60% fill ratio. This created the lowest mass flow rates situation, causing the pumps to use the least amount of energy. However, the maximum cold plate interface temperature was higher than the control case due to higher temperature differences in the intermediate heat exchanger. The operating scenario, which resulted in the lowest predicted cold plate interface temperature, was the largest tank filled at a 60% fill ratio. This case resulted in high mass flow rate, which reduced the temperature difference through the heat exchanger temperature without impacting the cooling tower performance. The predicted cold plate interface operating temperature here is 7.2 °C below the control case. This case does, however, requires the pumps to use more energy, which increases PUEcooling. A good compromise between predicted cold plate interface temperature and PUE is found when using the small tank at a 90% fill ratio. For this case, both the cold plate interface temperature and the PUE were significantly lower than that of the control case.

Table 5

Best case water storage and control cases

Tank volume (m3)Pump power (W)PUETcold plate interface. max (°C)
Small tank, 60% fill283571.00552.9
Large tank, 60% fill11319811.02649.0
Small tank, 90% fill283151.00457.1
No water storageNA4811.00956.0
Tank volume (m3)Pump power (W)PUETcold plate interface. max (°C)
Small tank, 60% fill283571.00552.9
Large tank, 60% fill11319811.02649.0
Small tank, 90% fill283151.00457.1
No water storageNA4811.00956.0

It is clear that there are significant benefits in both cooling effectiveness and in power usage to be gained from using a water storage system with a cooling tower design in certain environments. In order to get the most benefit from a water storage system, the charge/discharge times must be carefully matched to the environmental conditions, and the flow rates must be carefully matched to the cooling tower and intermediate heat exchanger design in order to optimize performance.

In application, this system can be used for various operating conditions that are not limited to those shown here where the tanks fully charge and discharge. In addition, the system can be used for various failure scenarios where a reliable supply of cold water is necessary at a moment's notice. Clear economic benefits can also be gained by using water tanks storage systems as a design element intended to cover the peak water supply conditions, allowing downsizing of the cooling tower and other design elements for the nominal cooling conditions instead of the maximum demand cooling conditions.

Conclusions

A computational model of a chiller-less or “warm water”-cooled data center was developed and validated against existing experimental data for stratified water storage tank systems. This model was then used to analyze the impact of a water storage tank system in a chiller-less data center design featuring outdoor wet cooling. The results show that during times of high wet bulb operating conditions, a water storage tank can be an effective method to significantly reduce chip operating temperatures for warm water-cooled systems. For the example case studied, the operating temperatures were reduced 5–7 °C during the hottest part of the day with the implementation of a water storage system.

A standard exergy analysis of the system showed that the main contributors to exergy losses are heat lost to ambient through the tank walls, mixing losses, and internal tank conduction. However, as in this case, heat losses to ambient actually improve the data center performance, a revised exergy definition was developed.

The overall system performance was evaluated using a modified PUE metric defined for the water storage system, which is the ratio of the sum of the system heat dissipated and the water cooling system power to the total heat dissipated by the system. The impact of tank size, tank aspect ratio, fill percentage, and charging/discharging time on both the cold plate interface temperature and modified PUE are evaluated. While the tank aspect ratio is not found to have much of an effect on system performance, the tank size, fill percentage, and charging/discharge time are all found to have a significant impact on performance. Each of these design variables (tank size, fill ratio, and charge/discharge time) can be related to system mass flowrate. The system mass flow rate is found to impact the effectiveness of the cooling tower performance, and the effectiveness of the intermediate heat exchanger performance, affecting overall cold plate temperature. Increases in mass flow rate increase the effectiveness of the heat exchanger, but at a certain point, begin to decrease the effectiveness of the cooling tower. Thus, there exists an optimum point at which the system will have the most impact on performance. Tank charging time must be carefully matched to environmental conditions in order to store water when the cooling tower is operating at its peak efficiency and the stored water is at its coolest.

This analysis featured the development of a unique stratified tank model in which heat losses to the ambient improve the system performance. This unique situation also necessitated the development of a new exergy definition in order to properly capture the physics of the situation. The results of this analysis demonstrate that in application, data centers operators will see a clear performance benefit if water storage systems are used in conjunction with warm water cooling. This application can be extended to data center failure scenarios and could also lead to downsizing of equipment and a clear economic benefit.

Acknowledgment

This material is based upon work supported by the National Science Foundation Center for Energy Smart Electronic Systems (ES2). Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.

Funding Data

  • Directorate for Engineering (IIP-1134810).

Nomenclature

A =

area, m2

cp =

specific heat at constant pressure, J/kgK

Cr =

heat capacity ratio

E =

energy, J

h =

height, m

k =

thermal conductivity, W/mK

m˙ =

mass flow rate, kg/s

NTU =

number of transfer units

P =

pressure, Pa

PUE =

power usage effectiveness

Q =

heat energy, J

Q˙ =

heat transfer rate, W

s =

specific entropy, J/kgK

S˙ =

rate of change of entropy, W/K

t =

time, s

T =

temperature, K

v =

volume, m3

V˙ =

rate of change of volume, m3/s

W˙ =

work rate, W

Greek Symbols
ϵ =

effectiveness

η =

efficiency

ρ =

density, kg/m3

ψ =

Specific exergy due to fluid flow, J/kg

Subscripts
0 =

ambient, reference

1 =

time step 1

2 =

time step 2

a =

ambient tank water

act =

actual

c =

charge

cs =

cross-sectional

cv =

control volume

chip =

chip

cold =

cold stream

DC =

data center

d =

discharge

des =

destroyed

e =

exit

f =

water (fluid) properties

gen =

generation

hot =

hot stream

i =

inlet

In =

inside air loop

j =

node designation

Out =

outside air loop

q =

heat transfer

tank =

relating to the entire tank

tot =

total

References

1.
NRDC
,
2014
,
America's Data Centers are Wasting Huge Amounts of Energy
,
Natural Resources Defense Council
,
New York
.
2.
ASHRAE
,
2009
, “
Datacom Series 6, Best Practices for DataCom Facility Energy Efficiency
,” 2nd ed., American Society of Heating, Refrigerating and Air-Conditioning Engineers, Atlanta, GA.
3.
Ellsworth, M. J.,
Campbell
,
L. A.
,
Simons
,
R. E.
,
Iyengar
,
M. K.
,
Schmidt
,
R. R.
, and
Chu
,
R. C.
,
2008
, “
The Evolution of Water Cooling for IBM Large Server Systems: Back to the Future
,”
Intersociety Conference on Thermal and Thermomechanical Phenomena in Electronic Systems
, Orlando, FL, May 28–31, pp. 266–274
4.
Ellsworth
,
M. J.
,
Goth
,
G. F.
,
Zoodsma
,
R. J.
,
Arvelo
,
A.
,
Campbell
,
L. A.
, and
Anderl
,
W. J.
,
2011
, “
An Overview of the Power 775 Supercomputer Water Cooling System
,”
ASME
Paper No. IPACK2011-52130.
5.
Ellsworth
,
M.
, and
Iyengar
,
M. K.
,
2009
, “
Energy Efficiency Analyses and Comparison of Air and Water Cooled High Performance Servers
,”
ASME
Paper No. IPACK2009-89248.
6.
David
,
M.
,
Iyengar
,
M.
,
Parida
,
P.
,
Simons
,
R.
,
Schultz
,
M.
,
Gaynes
,
M.
,
Schmidt
,
R.
, and
Chainer
,
T.
,
2012
, “
Experimental Characterization of an Energy Efficient Chiller-Less Data Center Test Facility With Warm Water Cooled Servers
,” 28th Annual IEEE Semiconductor Thermal Measurement and Management Symposium (
SEMI-THERM
), San Jose, CA, pp. 232–237.
7.
David
,
M.
,
Iyengar
,
M.
,
Parida
,
P.
,
Simons
,
R.
,
Schultz
,
M.
,
Gaynes
,
M.
,
Schmidt
,
R.
, and
Chainer
,
T.
,
2012
, “
Impact of Operating Conditions on a Chiller-Less Data Center Test Facility With Liquid Cooled Servers
,” 13th Intersociety Conference on Thermal and Thermomechanical Phenomena in Electronic Systems (
ITHERM
), San Diego, CA, May 30–June 1, pp. 562–573.
8.
Gao
,
T.
,
David
,
M.
,
Geer
,
J.
,
Schmidt
,
S.
, and
Sammakia
,
B.
,
2015
, “
Experimental and Numerical Dynamic Investigation of an Energy Efficient Liquid Cooled Chiller-Less Data Center Test Facility
,”
Energy Build.
,
91
, pp.
83
96
.
9.
Pinel
,
P.
,
Cruickshank
,
C.
,
Beausoleil-Morrison
,
I.
, and
Wills
,
A.
,
2011
, “
A Review of Available Methods for Seasonal Storage of Solar Thermal Energy in Residential Applications
,”
Renewable Sustainable Energy Rev.
,
15
(
7
), pp.
3341
3359
.
10.
De Ridder
,
F.
,
Diehl
,
M.
,
Mulder
,
G.
,
Desmedt
,
J.
, and
Van Bael
,
J.
,
2011
, “
An Optimal Control Algorithm for Borehole Thermal Energy Storage Systems
,”
Energy Build.
,
43
(
10
), pp.
2918
2925
.
11.
Dincer
,
I.
,
Sadik
,
D.
, and
Xianguo
,
L.
,
1997
, “
Performance Analyses of Sensible Heat Storage Systems for Thermal Applications
,”
Int. J. Energy Res.
,
21
(
12
), pp.
1157
1171
.
12.
Wemhoff
,
A. P.
,
del Valle
,
M.
,
Abbasi
,
K.
, and
Ortega
,
A.
,
2013
, “
Thermodynamic Modeling of Data Center Cooling Systems
,”
ASME
Paper No. IPACK2013-73116.
13.
Bhalerao
,
A.
,
Fouladi
,
K.
,
Silva-Llanca
,
L.
, and
Wemhoff
,
A. P.
,
2016
, “
Rapid Prediction of Exergy Destruction in Data Centers Due to Airflow Mixing
,”
Numer. Heat Transfer, Part A
,
70
(
1
), pp.
48
63
.
14.
Fouladi
,
K.
,
Wemhoff
,
A. P.
,
Silva-Llanca
,
L.
,
Abbasi
,
K.
, and
Ortega
,
A.
,
2017
, “
Optimization of Data Center Cooling Efficiency Using Reduced Order Flow Modeling Within a Flow Network Modeling Approach
,”
Appl. Therm. Eng.
,
124
, pp.
929
939
.
15.
Fouladi
,
K.
,
Schaadt
,
J.
, and
Wemhoff
,
A. P.
,
2017
, “
A Novel Approach to the Data Center Hybrid Cooling Design With Containment
,”
Numer. Heat Transfer, Part A
,
71
(
5
), pp.
477
487
.
16.
Wemhoff
,
A. P.
, and
Ortega
,
A.
,
2014
, “
An Exergy-Based Analysis of the Effects of Rear Door Heat Exchange Systems on Data Center Energy Efficiency
,”
IEEE Thermomechanical Phenomena in Electronic Systems Conference
(
ITHERM
), Orlando, FL, May 27–30, pp.
1129
1136
.
17.
Bhalerao
,
A.
,
Ortega
,
A.
, and
Wemhoff
,
A. P.
,
2014
, “
Thermodynamic Analysis of Hybrid Liquid-Air-Based Data Center Cooling Strategies
,”
ASME
Paper No. IMECE2014-38359.
18.
Bhalerao
,
A.
, and
Wemhoff
,
A. P.
,
2015
, “
Thermodynamic Analysis of Full Liquid-Cooled Data Centers
,”
ASME
Paper No. IPACK2015-48439.
19.
Khan
,
J. U. R.
,
Yaqub
,
M.
, and
Zubair
,
S. M.
,
2003
, “
Performance Characteristics of Counter Flow Wet Cooling Towers
,”
Energy Convers. Manage.
,
44
(
13
), pp.
2073
2091
.
20.
Khan
,
J. U. R.
, and
Zubair
,
S. M.
,
2004
, “
A Study of Fouling and Its Effects on the Performance of Counter Flow Wet Cooling Towers
,”
Proc. Inst. Mech. Eng., Part E
,
218
(
1
), pp.
43
51
.
21.
Khan
,
J. U. R.
, and
Zubair
,
S. M.
,
2001
, “
An Improved Design and Rating Analyses of Counter Flow Wet Cooling Towers
,”
ASME J. Heat Transfer
,
123
(
4
), pp.
770
778
.
22.
Kloppers
,
J. C.
, and
Kröger
,
D. G.
,
2005
, “
The Lewis Factor and Its Influence on the Performance Prediction of Wet-Cooling Towers
,”
Int. J. Therm. Sci.
,
44
(
9
), pp.
879
884
.
23.
Simpson
,
W. M.
, and
Sherwood
,
T. K.
,
1946
, “
Performance of Small Mechanical Draft Cooling Towers
,”
Refrig. Eng.
52
(
6
), pp.
574
576
.
24.
Muangnoi
,
T.
,
Asvapoositkul
,
W.
, and
Wongwises
,
S.
,
2007
, “
An Exergy Analysis on the Performance of a Counterflow Wet Cooling Tower
,”
Appl. Therm. Eng.
,
27
(
5–6
), pp.
910
917
.
25.
Muangnoi
,
T.
,
Asvapoositkul
,
W.
, and
Wongwises
,
S.
,
2008
, “
Effects of Inlet Relative Humidity and Inlet Temperature on the Performance of Counterflow Wet Cooling Tower Based on Exergy Analysis
,”
Energy Convers. Manage.
,
49
(
10
), pp.
2795
2800
.
26.
Haslego
,
C.
, and
Polley
,
G.
,
2002
, “
Designing Plate-and-Frame Heat Exchangers
,”
Chem. Eng. Prog.
,
98
(
9
), pp.
32
37
.
27.
Kays
,
W. M.
, and
London
,
A. L.
,
1984
,
Compact Heat Exchangers
, 3rd ed.,
McGraw-Hill
,
New York
.
28.
Motivair Cooling Solutions
,
2014
,
High Capacity Coolant Distribution Unit
,
Motivair Cooling Solutions
,
Amherst, MA
.
29.
Kleinbach
,
E. M.
,
Beckman
,
W. A.
, and
Klien
,
S. A.
,
1993
, “
Performance Study of One-Dimensional Models for Stratified Thermal Storage Tank
,”
Sol. Energy
,
50
(
2
), pp.
155
166
.
30.
Turner
,
P.
,
2001
,
Guide to Scientific Computing
,
CRC Press
,
Boca Raton, FL
.
31.
Wildin
,
M. W.
, and
Truman
,
C. R.
,
1985
, “
Evaluation of Stratifed Chilled Water Storage Techniques. Volume 1: Findings. Volume 2: Appendices
,” Electric Power Research Institute, Washington, DC.
32.
Jein
,
Y.
,
1986
, “
Initial Formation of a Thermocline in Stratified Storage Tanks
,”
ASHRAE Trans.
,
92
(
2
), p.
280292
.
33.
Wilden
,
M. W.
, and
Sohn
,
C. W.
,
1993
, “
Flow and Temperature Distribution in a Naturally Stratified Thermal Storage Tank
,” Construction Engineering Research Laboratory (U.S. Army), Champaign, IL, Report No. TR-FE-94/01.