WO2014007673A1

WO2014007673A1 - Server farm with an immersive cooling system

Info

Publication number: WO2014007673A1
Application number: PCT/RU2012/000817
Authority: WO
Inventors: Александр Александрович ЧИЧКОВСКИЙ; Сергей Михайлович АБРАМОВ
Original assignee: Chichkovskiy Alexander Alexandrovich; Abramov Sergey Mikhaylovich
Priority date: 2012-07-02
Filing date: 2012-10-09
Publication date: 2014-01-09
Also published as: RU2500012C1

Abstract

The invention relates to computer engineering. The technical result is an increase in the installation density of computational nodes. The server farm is comprised of an air-tight reservoir filled with cooling fluid and equipped with a lid, an internal heat-exchange unit, and an inlet pipe and an outlet pipe which communicate by means of pipelines with a circulation pump and an external heat-exchange unit. A perforated distribution pipe is installed inside the reservoir, parallel to the entire bottom of said reservoir, and a first printed circuit board, equipped with heating elements, is installed under said pipe. A second printed circuit board is installed flush against a wall of the reservoir and is adjoined by one of its edges to an edge of the first printed circuit board. Computational nodes are installed on the first printed circuit board, parallel to the second printed circuit board, and are comprised of a base printed circuit board separated by a divider into a narrow section and a wide section. The cooling system has two loops: the first is a cooling loop for cooling the computational nodes in the reservoir by means of a first cooling fluid, and the second is a loop for cooling the first cooling fluid by means of a second cooling fluid.

Description

Server farm with immersion cooling system The invention relates to the field of computer technology, namely server farms designed for high-performance computing and computer simulation, with cooling by completely immersing the heated electronic components in a cooling fluid.

Known apparatus for cooling electronic systems of the company Nortel Networks Corp., which is a set of printed circuit boards mounted vertically in a container partially filled with a dielectric fluid with a low boiling point (US patent 6019167, publ. 02/01/2000). A cooled electronic system is not a computing device. In addition, the disadvantages of this apparatus are the low density of the printed circuit boards, due to the presence of two container chambers, two container walls, an internal and external cooling plate system, the electronic components of the system cannot be removed from an absolutely sealed enclosure without violating the integrity of the entire apparatus.

Known computing installation company Cray Research Inc. (US patent 4590538, publ. 05.20.1986), consisting of a sealed cylindrical container, 16 blocks of motherboards radially mounted in it, oriented vertically in flat cells bounded by supporting frames, while the motherboards in the blocks are located horizontally, and between they form triangular-shaped compartments in which coolant circulates, forming upward and downward flows so that the motherboard blocks are completely washed by the coolant. Power supplies are installed at the bottom of the container and are also immersed in coolant. The disadvantages of this computing installation is the inability to operate the installation with negative ambient temperatures and the inability to hot replace failed electronic components.

Known computing installation company Isotermal System Research (US patent 6976528, publ. 12/20/2005), cooled by the injection of coolant onto heated electronic components. Such a computing installation can operate at a wide range of ambient temperatures, including at negative temperatures (-65 ° С ... +70 ° С). For operation at low temperatures, the unit is equipped with a heating element. The invention does not disclose a method of performing a heating element.

Known server farm company Iceotope Limited (US patent 7609518, publ. 10/27/2009). Computing nodes are installed on a standard server rack in an individual case, consisting of hard drives, a display, motherboards with memory and an evaporative coil. The motherboard and the evaporation coil are placed in an individual sealed container into which coolant cooled to a temperature of -30 ° C is supplied. Each container is connected externally to the cooling module, which includes a heat sink adjacent to the container, a heat insulation layer surrounding the container, two drying cartridges located on both sides of the heat removing plate, and two fans located on both sides of the drying cartridges.

The disadvantages of the server farm are the low density of the installation of computing nodes, due to the fact that the motherboards are enclosed in individual containers, the computing nodes are enclosed in individual cases, and the use of cooling modules, in addition, the invention uses standard server racks with a standard width of seats for computing nodes . also in such racks are provided for mounting and maintenance intervals. All this reduces the usable space that can be used to install additional computing nodes. Also, the hot swap of failed hard drives and power supplies, the ability to work in conditions of negative ambient temperatures are not provided.

Another server farm of the same company is known, described in UK patent 2467805, publ. 06/01/2011, US application 20100290190, publ. 12/11/2010 A standard server rack with equipment includes: a compartment for cooling units, additional shelves for equipment, one or more power supplies and a heat sink, shelves for a device to normalize pressure, a pump, an expansion tank, and a heat exchanger. The cooling unit consists of two sealed containers, each of which has one or more motherboards, AC DC converters, or a hard drive, or a power supply. The containers consist of two compartments: in the first, the first coolant circulates, and in the second, the second coolant. A heat sink plate is located at the border of the compartments, designed to transfer heat from the first coolant to the second.

The disadvantages of the server farm are the low density of the installation of computing nodes, due to the fact that the motherboards are enclosed in individual containers, the computing nodes are enclosed in individual cases, and a heat sink and devices are used to normalize the pressure, in addition, the invention uses standard server racks with a standard width Seats for computing nodes. Also, such racks provide for installation and maintenance intervals. All this reduces the usable space that can be used to install additional computing nodes. Also not provided the ability to hot replace failed hard drives and power supplies, the ability to work in conditions of negative ambient temperatures.

In addition, in both Iceotope Limited server farms, only motherboards with heating electronic components are immersed in the coolant, and the power supplies and hard drives are primarily cooled by air.

Hardcore Computer, Inc. server farms are known. with liquid cooling, in particular with the technology of cooling server farms by immersion in liquid LSS ™ 200 (Liquid Submerged Server) (http://www.hardcorecomputer.com/liquid-blade-video/index.html, for example, US patent 8089764, published on January 3, 2012, etc.). According to these decisions, computing nodes or server farms are placed in a standard server rack. Each computing node (or server farm) is enclosed in an individual case filled with coolant. Hard disks, one or several motherboards with daughter cards mounted on it, power supplies, processors, a video card, memory are placed in each case. Additionally, a pumping system may be present for pumping the hot liquid accumulating at the top of the container to an external heat exchanger or other heat dissipation systems. In one implementation of the invention provides for additional cooling of the heating components by a directed fluid flow through the dispersion chamber covering them.

The disadvantages of this server farm are the low density of the installation of computing nodes, due to the fact that the computing nodes are enclosed in individual cases, in addition, the invention uses standard server racks with a standard width of seats for computing nodes. Also in such racks provided for installation and maintenance intervals. All this reduces the usable space that can be used to install additional computing nodes. Also, the hot swap of failed hard drives and power supplies, the ability to work in conditions of negative ambient temperatures are not provided.

A known cooling system for various standard electronic devices and computing devices of the company Green Revolution Cooling, Inc., described in US application 20110132579 (published. 06/09/2011). The cooling system consists of a tank filled with coolant, in which a commercially available server rack is horizontally installed, to which pipes for supplying and leaving coolant for it are connected; a heat exchanger connected to the coolant outlet; a pump connected to a heat exchanger and a tank installed for pumping coolant in a closed circuit; a microcontroller for monitoring the temperature of the cooling liquid in a closed loop, which also controls the flow of liquid cooling dielectric in a closed loop in order to maintain an elevated temperature of the liquid cooling dielectric at the outlet of the second part of the closed loop. The tank may be configured to install two rows of computing nodes.

In some embodiments, the cooling system may include a second cooling device comprising a cooling liquid in a second closed loop for cooling the first cooling liquid.

The disadvantages of this cooling system are the inability to operate at negative ambient temperatures, the low density of the installation of computing nodes, due to the fact that this cooling system is intended, including for standard computing nodes, which are made for air cooling and, accordingly, are equipped with fans and ducts, which increases the size of the computing node and the size of the seats for the computing nodes in standard server racks that are mounted in the tank.

Closest to the claimed server farm is the server farm of Hardcore Computer, Inc., disclosed in independent clause 13 and its subordinate dependent clauses. 14-17 claims (US patent 8009419, publ. 30.08.2011,). The server farm is a sealed container filled with coolant and equipped with guides for installing blocks of computing nodes. Inside the container, a cooling mechanism is placed, which is a collection of tubes diluted on the motherboard. Hole tubes are in close proximity to heating components. Through these openings, coolant is injected onto such heating components, cools them and then mixes with the liquid filling the container.

The disadvantages of this server farm are the inability to hot-swap failed hard disks and power supplies, the server farm in negative environmental temperatures. Since this embodiment of the server farm is not described in the description section of the invention, it is difficult to assess the density of the installation of computing nodes inside the tank.

The objective of the invention is to expand the arsenal of server farms.

The technical result consists in expanding the arsenal of technical capabilities of server farms, namely, increasing the density of the installation of computing nodes, ensuring the functioning of the server farm at low temperatures environment, hot-swappable power supplies and storage devices.

The technical result is achieved by the fact that the server farm with an immersion cooling system consists of a sealed tank filled with coolant, with a heat exchanger installed in it, computing nodes consisting of a circuit board with motherboards mounted on it, power supplies, and information storage devices equipped with a cover, guides for the vertical installation of computing nodes, inlet and outlet nozzles communicating through a pipeline with a pump and an external heat exchange According to the invention, the server farm is provided with a distribution pipe, two printed circuit boards, a network switch, the distribution pipe being installed parallel to the entire bottom of the tank and formed by two parallel perforated pipes connected by a U-shaped connector, one end of the distribution pipe is plugged, and the other is connected with the first inlet pipe installed in the lower part of the tank wall, and the exhaust pipe is installed in the upper part of the same tank wall in which becoming the first inlet and through the first return isolation valve is connected via a first pressure line with an external circulation pump, coarse filter and an external heat exchanger, which on the other hand is connected via the reverse pressure line via a second shutoff valve with the inverse first inlet. A three-way solenoid valve is installed between the external circulation pump and the external heat exchanger, the third pipe of which is connected to the first end of the second pressure pipe, the second end of which is connected to the third pipe of the three-way splitter integrated in the return pressure pipe between the external heat exchanger and the second check valve. An internal heat exchanger is installed inside the tank, the first inlet of which is connected to the second inlet pipe, and the first outlet is connected via the pressure pipe to the internal circulation pump and the distribution pipe, the second input is connected via the pressure pipe to the first inlet pipe, and the second output is connected via the pressure pipe to the exhaust pipe branch pipe. The lid of the tank is formed by two leaves with different surface areas and is installed with the possibility of removal and lifting, and the large leaf is installed with the possibility of removal and lifting in the stop mode and the operation of the server farm, on the inner side of the larger leaf there is a network switch, and a hole is made in the smaller leaf for a loop or flexible printed circuit board having data connectors at the ends, the flexible board or loop connected to one end of an external data cable, the other end of which is connected connected to an external control device, and the network switch is connected to one end of the first data cable, the other end of which is brought out through an opening in the smaller leaf of the lid and connected to an external control device. The first printed circuit board is installed parallel to the plane of the tank bottom under the distribution pipe, it has end connectors for data and power, and data connectors are aligned along one edge, and power connectors are aligned along the opposite edge, between the power connectors and data connectors made heating elements. The second printed circuit board is installed close to one of the walls of the tank, to which a smaller flap of the lid is attached, perpendicular to the first printed circuit board and joined edge to edge with the help of mates for the data and power connectors of the first printed circuit board, and its upper part is connected via a connector data with a flexible printed circuit board or cable, passed through an opening in the smaller leaf of the lid of the tank, and with one end of the external power cable passed through the other hole in the smaller leaf of the lid, through power connectors, while the other end of the external power cable is connected to an external power source. The network switch is powered by a power connector located at the top of the second circuit board using a power cable. Computing nodes are installed using data and power connectors on the first printed circuit board parallel to the second printed circuit board and consist of a printed circuit board, which is divided by a longitudinal divider into narrow and wide compartments, two longitudinal holders are mounted symmetrically on both sides of the narrow compartment on both sides in the form of guides for fixing a removable printed circuit board on which the power supply unit, information storage devices and a block of power and data connectors are located, and in a wide compartment on both sides it mounts I'm on the motherboard. The motherboard is connected to the network switch via a second data cable, and the removable board is connected to the circuit board through the power and data connector blocks, and the wide compartment of the circuit board is connected via a flexible circuit board or a cable to the motherboard using data connectors. A rectangular cutout is made in the lower edge of the circuit board for passage of the distribution pipe. Half of the information storage devices located on the removable printed circuit board installed on one side of the circuit board are connected to the motherboard mounted on the same side of the mounting circuit board, and the remaining information storage devices are connected to the motherboard mounted on the other side of the mounting circuit board, between information drives connected to one motherboard, organized a complete data mirroring. Each power supply is connected to both motherboards, with both power supplies work in parallel mode.

In this case, it is possible that the server farm consists of n pressurized reservoirs connected in parallel through a system of inlet and outlet pipelines in communication with the outlet and inlet manifold pipes interconnected by a common pressure pipe.

In this case, it is possible that parallel to the general circulation pump, an additional general circulation pump is installed.

At the same time, it is possible that for vertical installation of the computing nodes, cuts are made parallel to the side edges of the tank.

At the same time, it is possible that solid-state information stores are used as information storage devices.

In this case, it is possible that heating elements are made in the lower part of the second printed circuit board.

It is possible that the heating elements are made in the form of load resistors.

It is possible that the internal heat exchanger is made in the form of a shell-and-tube countercurrent heat exchanger.

In FIG. 1 shows a server farm tank.

In FIG. 2 shows a distribution pipe.

In FIG. 3 shows the cover.

In FIG. 4 shows a first circuit board.

In FIG. 5 shows a second circuit board.

In FIG. 6 shows the circuit board.

In FIG. 7 shows a removable board.

In FIG. 8 is a diagram of a cooling circuit of a server farm including one tank.

In FIG. 9 is a diagram of a cooling circuit of a server farm including n tanks. AND

In one embodiment, the server farm is a single tank server farm. The server farm of FIG. 1, includes a sealed rectangular tank 1 filled with coolant and provided with a cover 2. On two opposite walls of the tank, plastic inserts with guides are mounted parallel to the side edges of the tank, or cuts are made for the vertical installation of computing nodes.

An opening is made in the upper part of one of the walls of the tank on the central axis, into which the outlet pipe 3 is installed. The tank is filled with liquid to the level of its overflow into the outlet pipe. An opening is made in the lower part of the same tank wall, into which an inlet pipe 4 of the same diameter as the outlet pipe is installed. The inlet pipe is connected to one end of the distribution pipe 5 (Fig. 2), mounted inside the tank on the supporting elements mounted on its bottom and consisting of two perforated pipes parallel to each other, connected by a U-shaped connector. The other end of the distribution pipe is plugged. The distribution pipe extends along the entire bottom of the tank.

In FIG. 3 shows a tank cap. The lid 2 of the tank consists of two flaps of different surface areas and fasteners, through which the flaps of the lid are attached to fasteners mounted on the upper edges of the walls of the tank. Fasteners are made with the possibility of removal and lifting of the wings. Both lid flaps have an opening angle of at least 90 degrees and are equipped with a mechanism for fixing them in this position. The upper edge of the tank around the perimeter and the lower edges of the lid flaps around the entire perimeter have a sealant layer that ensures a tight fit of the lid flaps to the tank. To provide quick access to the computing nodes inside the tank, the large lid flap opens and removed in the operating mode of the server farm. A smaller lid flap opens and is removed only after the server farm stops working.

A network switch 6 is located on the inside of the larger leaf. A first data cable is connected to the network switch, the other end of which is led out through an opening 7 made in the smaller leaf of the lid. Holes 8 are made in the smaller flap of the lid (one hole is shown in the drawing), through which a sealed input of power cables is arranged. An opening 9 is made in the smaller flap of the lid, into which a loop or flexible printed circuit board (not shown) is hermetically passed with two data end connectors. One end data connector is designed to interface with an end data connector located on a second circuit board. The other end data connector is designed to connect to an external data cable, the other end of which is connected to an external control device.

In the lower part of the tank, parallel to the plane of the tank bottom, under the distribution pipe 5, a first printed circuit board 10 is installed (Fig. 4), fixed on the supporting elements mounted on the tank bottom. Holes 11 are provided on the first printed circuit board for the passage of fasteners securing the distribution pipe.

Along one edge of the first printed circuit board, data connectors 12 are arranged coaxially between each other, made in an amount equal to the number of computing nodes installed in the tank. Along the opposite edge of the first printed circuit board are power connectors 13, made in an amount equal to the number of computing nodes installed in the tank. Between the data connectors 12 and power 13 mounted heating elements 14, made in the form of load resistors. The first circuit board is equipped with data end connectors 15 and power 16, designed to couple with the lower edge of the second printed circuit board 17. In this case, the data connectors 12 and 13 on the first printed circuit board are made along the edges perpendicular to the edge aligned with the second printed circuit board.

The second printed circuit board 17 (Fig. 5) is installed perpendicular to the first printed circuit board close to the tank wall, opposite to that on which the inlet and outlet pipes are mounted, and is rigidly fixed in this position by means of fasteners mounted on the tank wall. In the lower part of the second printed circuit board, the mating parts of the data connectors 18 and power 19 are made to the end data connectors 15 and power 16, respectively. In the upper part of the second printed circuit board, a mating part 20 of a data connector is provided for docking with an end data connector located on a cable or a flexible printed circuit board passed through an opening 9 in the smaller flap of the lid, and a power connector 21 for connecting external power cables, passed through the holes 8 in the smaller sash of the lid. The second printed circuit board is designed to supply power and transfer data from the data connectors 20 and power 21 located in its upper part to the first printed circuit board; in addition, it is used to install additional electronic components, such as, for example, an additional switch, etc. In some embodiments implementation of the invention in the lower part of the second printed circuit board placed additional heating elements 14.

Computing nodes are installed inside the tank on the first printed circuit board. The computing node consists of a mounting circuit board 22 (Fig. 6). A rectangular cutout is made in the lower edge of the mounting printed circuit board so that when it is installed on the first printed circuit board, the rectangular cutout falls on the distribution pipe. The mounting circuit board on the front and back sides is divided into two compartments - narrow and wide - with a longitudinal divider 23, made in the form plastic strips.

In the lower part of the wide compartment, the counterpart of the end data connector 24 is made to the end data connector 12, and in the lower part of the narrow compartment, the counterpart of the end power connector 25 is made to the end power connector 13.

Two longitudinal holders 26 are mounted symmetrically along the longitudinal edges of the narrow compartment of the printed circuit board on its front and back sides in the form of guides for fixing the removable printed circuit board 27 shown in FIG. 7, on which the power supply unit 28 and at least two information storage devices 29 are located. At the same time, part of the information storage devices located on the removable printed circuit board installed on the front side of the circuit board is connected to the motherboard mounted on the front side of the circuit board, and the other part of the information storage devices is connected to the motherboard mounted on the reverse side of the printed circuit board. Similarly connected information storage devices located on a removable printed circuit board mounted on the back of the circuit board. Between information storage devices connected to the same motherboard, complete data mirroring is organized. Each power supply is connected to both motherboards, while both power supplies operate in parallel. At the bottom of the plug-in board, a block of 30 power and data connectors is made.

In the lower part of the narrow compartment of the circuit board, there is a block 31 of power and data connectors designed to interface with a block of 30 power and data connectors located on a removable printed circuit board. The block 31 of the power and data connectors is connected to the power and data connectors made in the lower part of the narrow compartment of the circuit board (not shown in the drawing), which are used respectively for tracing power and data from the power supply and information storage devices, located on a removable circuit board, to the electronic components of the motherboard.

In the wide compartment of the circuit board, the motherboard 32 is mounted on the front and back sides. Thus, two motherboards and two removable printed circuit boards are mounted on the circuit board.

In the upper part of the wide compartment of the printed circuit board, a data connector 33 is provided for coupling by means of a flexible board or cable (not shown) with an end data connector 34 made in the upper part of the motherboard.

In the upper part of the motherboard there is a data connector 35, to which a second data cable is connected, the other end of which is connected to the network switch 6. The network switch receives power from the power connector 21 on the second printed circuit board using the power cable.

The cooling system of the server farm is double-circuit. The first circuit is the cooling circuit of the computing nodes in the tank by the first coolant, the second circuit is the cooling circuit of the first coolant by the second coolant.

In a first embodiment of the invention, the server farm consists of one reservoir 1 filled with coolant, provided with a lid, with computing units installed therein, a first printed circuit board 10, a second printed circuit board 17, and a distribution pipe 5. In FIG. 8 shows a cooling scheme of a server farm including one tank. The first cooling circuit includes an internal heat exchanger 36 mounted inside the tank and attached to the wall of the tank, on which the first inlet and outlet pipes are installed, by means of fasteners. The first input of the internal heat exchanger is connected to the second inlet pipe 37, and its first the outlet is connected via a pressure pipe 38 to an internal circulation pump 39 and a distribution pipe (not shown in the drawing). The internal heat exchanger is made in the form of a shell-and-tube countercurrent heat exchanger. Thus, a first closed loop of heat transfer by the first coolant from the computing nodes to the second coolant in the internal heat exchanger and from it back to the computing nodes is formed.

The second cooling circuit is formed by an internal heat exchanger 36, the second outlet of which is connected via a pressure pipe 40 to an outlet pipe installed in the tank wall, communicated through the first check valve 41 through a first pressure pipe 42 with an external circulation pump 43, a coarse filter 44 and an external heat exchanger 45. The external heat exchanger, on the other hand, is connected by means of a return pressure pipe 46 through a second check valve 47 with the outer end of the first th inlet pipe installed in the tank wall. The inner end of the first inlet pipe is connected via line 48 to the second inlet of the internal heat exchanger. Thus, a large second closed loop of heat transfer is formed by the second coolant from the first coolant to the external heat exchanger and from it the second cooled fluid back to the internal heat exchanger.

A three-way solenoid valve 49 is inserted between the coarse filter 44 and the external heat exchanger 45, the third pipe of which is connected to the first end of the second pressure pipe 50. The second end of the second pressure pipe is connected to the third pipe of the three-way splitter integrated in the return pressure pipe 46 between the external heat exchanger 45 and a second check valve 47. Three-way solenoid valve valve 49 is designed to shut off the flow of the second coolant entering the external heat exchanger and direct it through the second pressure pipe through the third pipe of the three-way splitter back to the internal heat exchanger. Thus, a small second closed loop of heat transfer by the second coolant from the first coolant from the internal heat exchanger to the second common pipe and the second cooled liquid back to the first coolant to the internal heat exchanger is formed, bypassing the external heat exchanger. To ensure fault tolerance, a backup circulation pump (not shown) is installed parallel to the external circulation pump 43.

In a second embodiment of the invention, the server farm consists of n tanks. In FIG. 9 shows a cooling scheme of a server farm including n tanks. The server farm consists of n sealed tanks 1 filled with coolant, with computing units installed in them, a first printed circuit board 10, a second printed circuit board 17, a distribution pipe 5, an internal heat exchanger 36. Each tank is equipped with a cover 2. In this embodiment, the first cooling circuit made similarly to the first embodiment of the invention. The second cooling circuit consists of an internal heat exchanger 36, the second outlet of which is connected by a pressure pipe 40 with a first exhaust pipe mounted in the tank wall, communicated through the first check valve 41 through a discharge pressure pipe 51 with an external circulation pump 43, a coarse filter 44 and an exhaust a manifold pipe 52, one end of which is plugged and the other connected via a first three-way splitter to a first common pressure pipe 53. To the first common pressure the th pipeline is then connected in series to an external common circulation pump 54, a common coarse filter 55 and an external heat exchanger 56. On the other hand, an external heat exchanger is connected to a return common pressure pipe 57. A second inlet of each internal heat exchanger 36 is connected via a pipe 48 through a first outlet pipe installed in the tank wall and a second shut-off check valve 47, with a supply pressure pipe 58, which is connected to the intake manifold pipe 59. One end of the intake manifold pipe is plugged, and the second is connected via a second three-way branch ator with reverse common pressure pipe 57. Thus, a large second closed heat transfer loop of the second coolant from the first coolant to the external heat exchanger and the second chilled liquid back to the indoor heat exchanger.

A three-way shut-off solenoid valve 60 is integrated between the common coarse filter 55 and the external heat exchanger 56, the third pipe of which is connected to the first end of the second common pressure pipe 61. The second end of the second common pressure pipe is connected through the third pipe of the third three-way splitter installed between the external heat exchanger and the inlet manifold pipe 59, with a return common pressure pipe 57. Shut-off three-way solenoid valve is designed to block the flow of second second coolant supplied to the outdoor heat exchanger, and direct it via the second common flow conduit and the third conduit a three-way splitter, which in turn communicates via the reverse common pressure line to the inlet manifold pipe, the supply line back to the indoor heat exchanger. Thus, a small second closed loop of heat transfer by the second coolant from the first coolant to the second common pressure pipe and back from the second cooled liquid from it to the internal heat exchanger bypassing the external heat exchanger.

To ensure fault tolerance, a duplicate common circulation pump (not shown) is installed parallel to the external common circulation pump 54.

During the operation of the server farm, the computing nodes are completely immersed in the coolant. In one embodiment of the invention, the first coolant heated by the computing units is poured into the second inlet pipe in communication with the first inlet of the internal heat exchanger and enters the internal heat exchanger, where it is cooled by the second cooling liquid. Then, the cooled first liquid through the first outlet of the internal heat exchanger enters the pressure pipe and is pumped by the internal circulation pump into the distribution pipe and fed through the openings made in it into the tank back to the computing nodes. Thus, the first coolant circulates in the first closed loop, starting from the computing nodes, continuing to the internal heat exchanger, the internal circulation pump and returning back to the computing nodes.

At the same time, the second coolant, heated by the first coolant in the internal heat exchanger, through its second outlet enters the pressure pipe connected to the outlet pipe installed in the tank wall, and then through the first check valve through the first pressure pipe enters the external circulation pump which pumps it through a coarse filter to an external heat exchanger. The second coolant cooled in the external heat exchanger enters the return pressure pipe and through the second check valve in communication with the first inlet pipe installed in the wall reservoir, enters the pipeline in communication with the second inlet of the internal heat exchanger. Thus, the second coolant circulates in a second large closed loop starting from the internal heat exchanger, continuing to the external circulation pump, the coarse filter, the external heat exchanger and returning back to the internal heat exchanger.

If the ambient temperature does not exceed 0 ° C, the shut-off three-way solenoid valve is closed. An external circulation pump through a coarse filter pumps a second coolant into the first pressure pipe to the shutoff three-way solenoid valve, which directs it to the second pressure pipe, where it is cooled. Then, the cooled second coolant through a three-way splitter, a second check valve and a first inlet pipe installed in the tank wall, enters the pipeline in communication with the second inlet of the internal heat exchanger. Thus, the second coolant circulates in a second small closed loop, starting from the internal heat exchanger, continuing to the external circulation pump, the coarse filter, the second pressure pipe and returning back to the internal heat exchanger, bypassing the external heat exchanger.

In another embodiment of the invention, the first coolant heated by the computing units circulates in a similar primary circuit. The second coolant, heated by the first coolant in the internal heat exchanger, through its second outlet enters the pressure pipe in communication with the exhaust pipe, and through the first check valve, overflows into the discharge pressure pipe to the external circulation pump, which pumps it through the coarse filter graduation the collector pipe and into the first common pressure pipe, where a common circulation pump through a common filter of rough cleaning pumps it to an external heat exchanger. The second coolant cooled in the external heat exchanger enters the return common pressure pipe and the inlet manifold, and then through the inlet pressure pipe through the second check valve and the first inlet pipe installed in the tank wall, enters the pipe in communication with the second input of the internal heat exchanger. Thus, the second coolant circulates in a second large closed loop, starting from the internal heat exchanger, continuing to the exhaust piping system, the exhaust manifold pipe, the external common circulation pump, the coarse filter and the external heat exchanger and returns back to the internal heat exchanger through the pipeline, the intake manifold pipe and piping system.

If the ambient temperature does not exceed 0 ° C, the shut-off three-way solenoid valve is closed. An external common circulation pump, through a common coarse filter, pumps a second coolant through the first common pressure pipe to the shutoff three-way solenoid valve, which directs it to the second common pressure pipe, where it is cooled. And then, through the third branch pipe of the third three-way splitter, by means of the return common pressure pipe, the intake manifold and the supply pipe system, the cooled second cooling liquid is returned back to each of the internal heat exchangers, bypassing the external heat exchanger. Thus, the second coolant circulates in a second small closed loop, starting from the internal heat exchanger, continuing to the system of the discharge pipes, the exhaust manifold pipe, the external common circulation pump, the common coarse filter and the second common pressure pipe and returns to the internal heat exchanger through the pipeline, the intake manifold pipe and the supply pipe system.

Due to the fact that the second small and second large cooling circuits, parallel connecting n tanks, are tight, the first additional technical result is achieved, which consists in providing the possibility of installing each of the tanks at different levels, starting from the same point on the earth's surface.

Due to the fact that the first cooling liquid, directly cooling the computing nodes, does not leave the tank, a second additional technical result is achieved, which consists in ensuring the quick launch of the inventive server farm.

The ambient temperature at which the operation of the inventive server farm is possible ranges from -50 ° C to +45 ° C.

The server farm receives power from an external power source through power cables through the holes 8 in the smaller flap of the lid, one ends of which are connected to an external power source, and the other ends are connected to the power connector 21 on the second circuit board.

Synthetic or natural oils are used as the first coolant. Ethylene glycol or its analogues are used as the second coolant.

The increase in the density of the installation of computing nodes is achieved due to the lack of an individual case in which the computing unit is located, and an individual container in which one or more motherboards are located, cooling devices used in known solutions, for example heat-removing boards and cooling modules, not using standard server racks with a standard width of seats for computing nodes and with mounting and operating intervals. In addition, motherboards and removable printed circuit boards on both sides are installed on the circuit boards. All this increases the usable space that can be used to install additional computing nodes. In addition, the invention does not provide for the use of standard computing nodes designed for air cooling, which reduces the size of the computing nodes installed in the inventive server farm.

Functioning at low ambient temperatures is achieved through the use of heating elements.

The ability to hot-swap power supplies and information storage devices is achieved by connecting part of the information storage devices located on a removable printed circuit board installed on one side of the mounting printed circuit board to a motherboard mounted on its other side, and organizing data mirroring between information storage devices connected to one motherboard. As well as the fact that both power supplies operate in parallel. As well as the fact that power supplies and information storage devices are located on a removable printed circuit board.

Claims

Claim

1. A server farm with an immersion cooling system, consisting of a sealed tank with a heat exchanger installed in it, computing nodes consisting of a circuit board with motherboards mounted on it, power supplies, storage media filled with coolant, equipped with a cover, guides for vertical installation computing nodes, inlet and outlet pipes, communicating via a pipeline with a pump and an external heat exchanger, characterized in that the server the farm is equipped with a distribution pipe, two printed circuit boards, a network switch, and

the distribution pipe is installed parallel to the entire bottom of the tank and is formed by two perforated pipes parallel to each other, connected by a U-shaped connector, one end of the distribution pipe is plugged, and the second is connected to the first inlet pipe installed in the lower part of the tank wall,

and the exhaust pipe is installed in the upper part of the same wall of the tank in which the first inlet pipe is installed, and through the first check valve is connected via the first pressure pipe to an external circulation pump, a coarse filter and an external heat exchanger, which on the other hand is connected via a pressure return pipeline through a second check valve with a first inlet pipe,

a three-way solenoid valve is installed between the external coarse filter and the external heat exchanger, the third pipe of which is connected to the first end of the second pressure pipe, the second end of which is connected to the third pipe of the three-way splitter integrated in the return pressure pipe piping between an external heat exchanger and a second check valve,

an internal heat exchanger is installed inside the tank, the first inlet of which is connected to the second inlet pipe, and the first outlet is connected by means of a pressure pipe to the internal circulation pump and distribution pipe, the second input is connected by a pressure pipe to the first inlet pipe, and the second output is connected by a pressure pipe to the exhaust pipe branch pipe

the tank lid is formed by two shutters with different surface areas and is installed with the possibility of removal and lifting, and a large shutter is installed with the possibility of removal and lifting in the stop mode and the server farm, on the inner side of the larger shutter there is a network switch, and a hole is made in the smaller shutter for a loop or flexible printed circuit board having data connectors at the ends, the flexible board or loop connected to one end of an external data cable, the other end of which is connected connected to an external control device, and the network switch is connected to one end of the first data cable, the other end of which is brought out through an opening in a smaller leaf of the lid and connected to an external control device,

the first printed circuit board is installed parallel to the plane of the tank bottom under the distribution pipe, it has end data and power connectors, and data connectors are aligned along one edge, and power connectors are aligned along the opposite edge, between the power connectors and data connectors made heating elements

the second printed circuit board is installed close to one of the walls of the tank, to which a smaller flap of the lid is attached, perpendicular to the first printed circuit board and docked with it edge using mates for the end data connectors and the power of the first printed circuit board,

and its upper part is connected via a data connector to a flexible printed circuit board or cable, passed through an opening in the smaller leaf of the lid of the tank, and to one end of the external power cable passed through the other hole in the smaller leaf of the lid, through power connectors, while the other end of the external the power cable is connected to an external power source,

moreover, the network switch is powered by a power connector located in the upper part of the second printed circuit board, using a power cable,

computing nodes are installed using data and power connectors on the first printed circuit board parallel to the second printed circuit board and consist of a printed circuit board, which is divided by a longitudinal divider into narrow and wide compartments, two longitudinal holders are mounted symmetrically on both sides of the narrow compartment on both sides in the form guides for fixing a removable printed circuit board on which the power supply unit, information storage devices and a block of power and data connectors are located, and in a wide compartment on both sides it mounts I'm on the motherboard,

the motherboard is connected to the network switch via a second data cable,

the plug-in board is connected to the circuit board through the power and data connector blocks,

a wide compartment of the circuit board is connected via a flexible printed circuit board or a cable to the motherboard using data connectors,

in the lower edge of the mounting circuit board, a rectangular cutout is made for the passage of the distribution pipe, at the same time, half of the information storage devices located on the removable printed circuit board installed on one side of the circuit board are connected to the motherboard mounted on the same side of the mounting circuit board, and the remaining information storage devices are connected to the motherboard mounted on the other side of the mounting circuit board , between the storage devices connected to the same motherboard, a complete data mirroring is organized,

each power supply is connected to both motherboards, while both power supplies operate in parallel.

2. The server farm according to claim 1, characterized in that it consists of n sealed tanks parallel connected to each other by means of a system of inlet and outlet pipelines in communication with the outlet and intake manifold pipes interconnected by a common pressure pipe.

3. The server farm according to claim 1, characterized in that an additional general circulation pump is installed parallel to the general circulation pump.

4. The server farm according to claim 1, characterized in that for the vertical installation of computing nodes parallel to the side edges of the tank, cuts are made.

5. The server farm according to claim 1, characterized in that solid-state information drives are used as information stores.

6. The server farm according to claim 1, characterized in that the heating elements are made in the lower part of the second printed circuit board.

7. The server farm according to claim 1, characterized in that the heating elements are made in the form of load resistors.

8. The server farm according to claim 1, characterized in that the internal heat exchanger is made in the form of a shell-and-tube countercurrent heat exchanger.