WO2014007672A1

WO2014007672A1 - Server farm with an immersive cooling system

Info

Publication number: WO2014007672A1
Application number: PCT/RU2012/000816
Authority: WO
Inventors: Александр Александрович ЧИЧКОВСКИЙ; Сергей Михайлович АБРАМОВ
Original assignee: Chichkovskiy Alexander Alexandrovich; Abramov Sergey Mikhaylovich
Priority date: 2012-07-02
Filing date: 2012-10-09
Publication date: 2014-01-09
Also published as: RU2496134C1

Abstract

The invention relates to computer engineering. The technical result is an increase in the installation density of computational nodes. The server farm is comprised of an air-tight reservoir filled with cooling fluid and equipped with a lid, an inlet pipe and an outlet pipe which communicate by means of pipelines with a circulation pump and a heat-exchange unit. A perforated distribution pipe is installed inside the reservoir, parallel to the entire bottom of said reservoir. A first printed circuit board, equipped with heating elements, is installed under said pipe. A second printed circuit board is installed flush against a wall of the reservoir and is adjoined by one of its edges to an edge of the first printed circuit board. Computational nodes are installed on the first printed circuit board, parallel to the second printed circuit board, and are comprised of a base printed circuit board separated by a divider into a narrow section and a wide section. A removable printed circuit board having a power unit and data storage units arranged thereon is affixed to both sides of the narrow section, and a motherboard is affixed to both sides of the wide section. Half of the data storage units on the removable printed circuit board installed on one side of the base printed circuit board is connected to the motherboard on this same side, while the remaining half is connected to the motherboard on the other side. Full data mirroring is set up between data storage units connected to a single motherboard.

Description

Server farm with immersion cooling system The invention relates to the field of computer technology, namely server farms designed for high-performance computing and computer simulation, with cooling by completely immersing the heated electronic components in a cooling fluid.

Known apparatus for cooling electronic systems of the company Nortel Networks Corp., which is a set of printed circuit boards mounted vertically in a container partially filled with a dielectric fluid with a low boiling point (US patent 6019167, publ. 02/01/2000). A cooled electronic system is not a computing device. In addition, the disadvantages of this apparatus are the low density of the printed circuit boards, due to the presence of two container chambers, two container walls, an internal and external cooling plate system, the electronic components of the system cannot be removed from an absolutely sealed enclosure without violating the integrity of the entire apparatus.

Known computing installation company Cray Research Inc. (US patent 4590538, publ. 05.20.1986), consisting of a sealed cylindrical container, 16 blocks of motherboards radially mounted in it, oriented vertically in flat cells bounded by supporting frames, while the motherboards in the blocks are located horizontally, and between they form triangular-shaped compartments in which coolant circulates, forming upward and downward flows so that the motherboard blocks are completely washed by the coolant. Power supplies are installed at the bottom of the container and are also immersed in coolant. The disadvantages of this computing installation is the inability to operate the installation with negative ambient temperatures and the inability to hot replace failed electronic components.

Known computing installation company Isotermal System Research (US patent 6976528, publ. 12/20/2005), cooled by the injection of coolant onto heated electronic components. Such a computing installation can operate at a wide range of ambient temperatures, including at negative temperatures (-65 ° С ... +70 ° С). For operation at low temperatures, the unit is equipped with a heating element. The invention does not disclose a method of performing a heating element.

Known server farm company Iceotope Limited (US patent 7609518, publ. 10/27/2009). Computing nodes are installed on a standard server rack in an individual case, consisting of hard drives, a display, motherboards with memory and an evaporative coil. The motherboard and the evaporation coil are placed in an individual sealed container into which coolant cooled to a temperature of -30 ° C is supplied. Each container is connected externally to the cooling module, which includes a heat sink adjacent to the container, a heat insulation layer surrounding the container, two drying cartridges located on both sides of the heat removing plate, and two fans located on both sides of the drying cartridges.

The disadvantages of the server farm are the low density of the installation of computing nodes, due to the fact that the motherboards are enclosed in individual containers, the computing nodes are enclosed in individual cases, and the use of cooling modules, in addition, the invention uses standard server racks with a standard width of seats for computing nodes . also in such racks are provided for mounting and maintenance intervals. All this reduces the usable space that can be used to install additional computing nodes. Also, the hot swap of failed hard drives and power supplies, the ability to work in conditions of negative ambient temperatures are not provided.

Another server farm of the same company is known, described in UK patent 2467805, publ. 06/01/2011, US application 20100290190, publ. 12/11/2010 A standard server rack with equipment includes: a compartment for cooling units, additional shelves for equipment, one or more power supplies and a heat sink, shelves for a device to normalize pressure, a pump, an expansion tank, and a heat exchanger. The cooling unit consists of two sealed containers, each of which has one or more motherboards, AC DC converters, or a hard drive, or a power supply. The containers consist of two compartments: in the first, the first coolant circulates, and in the second, the second coolant. A heat sink plate is located at the border of the compartments, designed to transfer heat from the first coolant to the second.

The disadvantages of the server farm are the low density of the installation of computing nodes, due to the fact that the motherboards are enclosed in individual containers, the computing nodes are enclosed in individual cases, and a heat sink and devices are used to normalize the pressure, in addition, the invention uses standard server racks with a standard width Seats for computing nodes. Also, such racks provide for installation and maintenance intervals. All this reduces the usable space that can be used to install additional computing nodes. Also not provided the ability to hot replace failed hard drives and power supplies, the ability to work in conditions of negative ambient temperatures.

In addition, in both Iceotope Limited server farms, only motherboards with heating electronic components are immersed in the coolant, and the power supplies and hard drives are primarily cooled by air.

Hardcore Computer, Inc. server farms are known. with liquid cooling, in particular with the technology of cooling server farms by immersion in liquid LSS ™ 200 (Liquid Submerged Server) (http://www.hardcorecomputer.com/liquid-blade-video/index.html, for example, US patent 8089764, published on January 3, 2012, etc.). According to these decisions, computing nodes or server farms are placed in a standard server rack. Each computing node (or server farm) is enclosed in an individual case filled with coolant. Hard disks, one or several motherboards with daughter cards mounted on it, power supplies, processors, a video card, memory are placed in each case. Additionally, a pumping system may be present for pumping the hot liquid accumulating at the top of the container to an external heat exchanger or other heat dissipation systems. In one implementation of the invention provides for additional cooling of the heating components by a directed fluid flow through the dispersion chamber covering them.

The disadvantages of this server farm are the low density of the installation of computing nodes, due to the fact that the computing nodes are enclosed in individual cases, in addition, the invention uses standard server racks with a standard width of seats for computing nodes. Also in such racks provided for installation and maintenance intervals. All this reduces the usable space that can be used to install additional computing nodes. Also, the hot swap of failed hard drives and power supplies, the ability to work in conditions of negative ambient temperatures are not provided.

A known cooling system for various standard electronic devices and computing devices of the company Green Revolution Cooling, Inc., described in US application 20110132579 (published. 06/09/2011). The cooling system consists of a tank filled with coolant, in which a commercially available server rack is horizontally installed, to which pipes for supplying and leaving coolant for it are connected; a heat exchanger connected to the coolant outlet; a pump connected to a heat exchanger and a tank installed for pumping coolant in a closed circuit; a microcontroller for monitoring the temperature of the cooling liquid in a closed loop, which also controls the flow of liquid cooling dielectric in a closed loop in order to maintain an elevated temperature of the liquid cooling dielectric at the outlet of the second part of the closed loop. The tank may be configured to install two rows of computing nodes.

In some embodiments, the cooling system may include a second cooling device comprising a cooling liquid in a second closed loop for cooling the first cooling liquid.

The disadvantages of this cooling system are the inability to operate at negative ambient temperatures, the low density of the installation of computing nodes, due to the fact that this cooling system is intended, including for standard computing nodes, which are made for air cooling and, accordingly, are equipped with fans and ducts, which increases the size of the computing node and the size of the seats for the computing nodes in standard server racks that are mounted in the tank.

Closest to the claimed server farm is the server farm of Hardcore Computer, Inc., disclosed in independent clause 13 and its subordinate dependent clauses. 14-17 claims (US patent 8009419, publ. 30.08.2011,). The server farm is a sealed container filled with coolant and equipped with guides for installing blocks of computing nodes. Inside the container, a cooling mechanism is placed, which is a collection of tubes diluted on the motherboard. Hole tubes are in close proximity to heating components. Through these openings, coolant is injected onto such heating components, cools them and then mixes with the liquid filling the container.

The disadvantages of this server farm are the inability to hot-swap failed hard disks and power supplies, the server farm in negative environmental temperatures. Since this embodiment of the server farm is not described in the description section of the invention, it is difficult to assess the density of the installation of computing nodes inside the tank.

The objective of the invention is to expand the arsenal of server farms. The technical result consists in expanding the arsenal of technical capabilities of server farms, namely, increasing the density of the installation of computing nodes, ensuring the functioning of the server farm at low temperatures environment, hot-swappable power supplies and storage devices.

The technical result is achieved in that the server farm with an immersion cooling system consists of a sealed tank filled with coolant, with computing units installed in it, consisting of a circuit board with a motherboard mounted on it, power supplies, information storage devices, equipped with a cover, guides for vertical installation of computing nodes, inlet and outlet pipes, communicating via a pipeline with a pump and heat exchanger, according to the invention It is equipped with a distribution pipe, two printed circuit boards, a network switch, the distribution pipe installed parallel to the entire bottom of the tank and formed by two parallel perforated pipes connected by a U-shaped connector, one end of the distribution pipe is plugged, and the second is connected to the inlet pipe installed at the bottom of the tank wall. An exhaust pipe is installed in the upper part of the tank wall and is connected through the first check valve to a coarse filter, a circulation pump and a heat exchanger, which, on the other hand, is connected via a check pressure pipe through a second check valve to the inlet pipe. A three-way solenoid valve is installed between the coarse filter and the heat exchanger, the third pipe of which is connected to the first end of the second pressure pipe, the second end of which is connected to the third pipe of the three-way splitter, which is built into the return pressure pipe between the heat exchanger and the second check valve. The lid of the tank is formed by two flaps with different surface areas and is installed with the possibility of removal and lifting during operation and stopping the server farm, on the inner side of the larger leaf there is a network switch, and in the smaller leaf there is a hole for a cable or a flexible printed circuit board with data connectors at the ends. A first data cable is connected to the network switch, the other end of which is led out through an opening in the smaller casement of the lid. The first printed circuit board is installed parallel to the plane of the tank bottom under the distribution pipe, it has end connectors for data and power, and data connectors are aligned along one edge, and power connectors are aligned along the opposite edge, with power and data connectors located between the connectors heating elements. The second printed circuit board is installed close to one of the walls of the tank, to which a smaller flap of the lid is attached perpendicular to the first printed circuit board and joined edge to edge with the help of mates for the end data and power connectors of the first printed circuit board, and its upper part is connected via a connector for data with a flexible printed circuit board or ribbon cable inserted through an opening in the smaller casement of the lid of the tank, and with an external power cable passed through another hole in the smaller casement of the lid, power connectors. The network switch is powered by a power connector located at the top of the second circuit board using a power cable. Computing nodes are installed using data and power connectors on the first printed circuit board parallel to the second printed circuit board and consist of a printed circuit board, which is divided by a longitudinal divider into narrow and wide compartments, two longitudinal holders are mounted symmetrically on both sides of the narrow compartment on both sides in the form of guides for fixing a removable printed circuit board on which the power supply unit, information storage devices and a block of power and data connectors are located, and in a wide compartment on both sides it mounts I'm on motherboard. In this case, half of the information storage devices located on the removable printed circuit board installed on one side of the circuit board are connected to the motherboard mounted on the same side of the mounting circuit board, and the remaining information storage devices are connected to the motherboard mounted on the other side of the mounting circuit board , between data storage devices connected to one motherboard, a complete data mirroring is organized, and each power supply is connected to both motherboards and both power supply units Italy works in parallel. The motherboard is connected to the network switch via a second data cable, and the plug-in board is connected to the circuit board through the power and data connector blocks. The wide compartment of the circuit board is connected via a flexible printed circuit board or a cable to the motherboard using data connectors. A rectangular cutout is made in the lower edge of the circuit board for passage of the distribution pipe.

It is possible that the inlet pipe is installed at the bottom of the same tank wall in which the outlet pipe is installed and the second circuit board is installed perpendicular to the first printed circuit board close to the same tank wall on which the inlet and outlet pipes are installed. In this case, the upper edge of the second printed circuit board is cut so as to provide direct communication of the outlet pipe with the tank, an opening is made in the lower part of the second printed circuit board at the level of the orthogonal projection of the inlet pipe so that the inlet pipe enters the tank.

It is also possible that the inlet pipe is installed in the lower part of the tank wall, opposite to the one in which the outlet pipe is installed, and the second printed circuit board is installed perpendicular to the first printed circuit board close to the same tank wall on which an inlet pipe is installed, and then in its lower part, at the level of the orthogonal projection of the inlet pipe, an opening is made so as to provide direct communication of the inlet pipe with the reservoir. Or the second printed circuit board is installed perpendicular to the first printed circuit board close to the same wall of the tank on which the outlet pipe is installed, and then its upper edge is made with a notch so as to provide direct communication between the outlet pipe and the tank.

It is also possible that an additional circulation pump is installed in parallel with the circulation pump.

It is also possible that for vertical installation of computing nodes parallel to the side edges of the tank cuts are made.

It is also possible that the heating elements are made in the form of load resistors.

It is also possible that solid-state information stores are used as information storage devices.

It is also possible that the server farm consists of n pressurized tanks connected in parallel through a system of inlet and outlet pipelines in communication with the outlet and inlet manifold pipes interconnected by a common pressure pipe.

It is also possible that additional heating elements are made in the lower part of the second printed circuit board.

In FIG. 1 shows a server farm tank.

In FIG. 2 shows a distribution pipe.

In FIG. 3 shows the cover.

In FIG. 4 shows a first circuit board.

In FIG. 5 shows a second circuit board.

In FIG. 6 shows the circuit board. In FIG. 7 shows a removable board.

In FIG. 8 is a diagram of a cooling circuit of a server farm including one tank.

In FIG. 9 is a diagram of a cooling circuit of a server farm including n tanks.

In one embodiment, the server farm is a single tank server farm. The server farm of FIG. 1 includes a sealed rectangular tank 1 filled with coolant and provided with a cover 2. On two opposite walls of the tank, plastic inserts with guides are mounted parallel to the side edges of the tank, or cuts are made for the vertical installation of computing nodes.

An opening is made in the upper part of one of the walls of the tank on the central axis, into which the outlet pipe 3 is installed. The tank is filled with liquid to the level of its overflow into the outlet pipe. An opening is made in the lower part of the same or opposite wall of the tank, into which an inlet pipe 4 of the same diameter as the outlet pipe is installed. The inlet pipe is connected to one end of the distribution pipe 5 (Fig. 2), mounted inside the tank on the supporting elements mounted on its bottom and consisting of two perforated pipes parallel to each other, connected by a U-shaped connector. The other end of the distribution pipe is plugged. The distribution pipe extends along the entire bottom of the tank.

In FIG. 3 shows a tank cap. The lid 2 of the tank consists of two flaps of different surface areas and fasteners, through which the flaps of the lid are attached to fasteners mounted on the upper edges of the walls of the tank. Fasteners are made with the possibility of removal and lifting of the wings. Both lid flaps have an opening angle of at least 90 degrees and are equipped with a mechanism for fixing them in this position. The upper edge of the tank around the perimeter and the lower edges of the lid flaps around the entire perimeter have a sealant layer that ensures a tight fit of the lid flaps to the tank. To provide quick access to the computing nodes inside the tank, a large lid flap is opened and removed in the server farm operating mode. A smaller lid flap opens and is removed only after the server farm stops working.

A network switch 6 is located on the inside of the larger leaf. A first data cable is connected to the network switch, the other end of which is led out through an opening 7 made in the smaller leaf of the lid. Holes 8 are made in the smaller flap of the lid (one hole is shown in the drawing) through which a sealed input of power cables is arranged. An opening 9 is made in the smaller flap of the lid, into which a loop or flexible printed circuit board (not shown) is hermetically passed with two data end connectors. One end data connector is designed to interface with an end data connector located on a second circuit board. The other end data connector is designed to connect to an external data cable, the other end of which is connected to an external control device.

In the lower part of the tank, parallel to the plane of the tank bottom, under the distribution pipe 5, a first printed circuit board 10 is installed (Fig. 4), fixed on the supporting elements mounted on the tank bottom. Holes 11 are provided on the first printed circuit board for the passage of fasteners securing the distribution pipe.

Along one edge of the first printed circuit board, data connectors 12 are arranged coaxially between each other, made in an amount equal to the number of computing nodes installed in the tank. Connectors are located along the opposite edge of the first PCB. power 13, made in an amount equal to the number of computing nodes installed in the tank. Between the data connectors 12 and power 13 mounted heating elements 14, made in the form of load resistors. The first printed circuit board is equipped with end connectors 15 data and power 16, designed to couple with the lower edge of the second printed circuit board 17. Moreover, the data connectors 12 and 13 on the first printed circuit board are made along the edges perpendicular to the edge docked with the second printed circuit board.

The second printed circuit board 17 (Fig. 5) is installed perpendicular to the first printed circuit board close to one of the walls of the tank and is rigidly fixed in this position by means of fasteners mounted on the wall of the tank. If the second printed circuit board is installed close to the wall of the tank on which the outlet pipe is located, then the upper edge of the second printed circuit board is cut so as to provide direct communication between the outlet pipe and the tank. If the second printed circuit board is installed close to the wall of the tank on which the inlet pipe is located, an opening is made in the lower part of the second printed circuit board at the level of the orthogonal projection of the inlet pipe so that the inlet pipe enters the tank. If the second printed circuit board is installed close to the wall of the tank on which the inlet and outlet pipes are located, then the upper edge of the second printed circuit board is cut so as to provide direct communication between the exhaust pipe and the tank, and at the bottom of the second printed circuit board at the orthogonal projection of the inlet pipe is made a hole so as to ensure the entry of the inlet pipe into the tank.

In the lower part of the second printed circuit board, the mating parts of the data connectors 18 and power 19 are made to the end data connectors 15 and power 16, respectively. At the top of the second circuit board is made the mating part 20 of the data connector, designed for docking with the end data connector, located on the cable or flexible printed circuit board, passed through the hole 9 in the smaller leaf of the lid, and the power connector 21, designed to connect external power cables, passed through the holes 8 in the smaller leaf covers. The second printed circuit board is designed to supply power and transfer data from the data connectors 20 and power 21 located in its upper part to the first printed circuit board; in addition, it is used to install additional electronic components, such as, for example, an additional switch, etc. In some embodiments implementation of the invention in the lower part of the second printed circuit board placed additional heating elements 14.

Computing nodes are installed inside the tank on the first printed circuit board. The computing node consists of a mounting circuit board 22 (Fig. 6). A rectangular cutout is made in the lower edge of the mounting printed circuit board so that when it is installed on the first printed circuit board, the rectangular cutout falls on the distribution pipe. The printed circuit board on the front and back sides is divided into two compartments - narrow and wide - with a longitudinal divider 23, made in the form of a plastic strip.

In the lower part of the wide compartment, the counterpart of the end data connector 24 is made to the end data connector 12, and in the lower part of the narrow compartment, the counterpart of the end power connector 25 is made to the end power connector 13.

Two longitudinal holders 26 are mounted symmetrically along the longitudinal edges of the narrow compartment of the printed circuit board on its front and back sides in the form of guides for fixing the removable printed circuit board 27 shown in FIG. 7, on which the power supply unit 28 and at least two information stores 29 are located. At the same time, part of the drives information located on a removable printed circuit board installed on the front side of the circuit board is connected to the motherboard mounted on the front side of the circuit board, and the other part of the information storage devices is connected to the motherboard mounted on the back side of the circuit board. Similarly connected information storage devices located on a removable printed circuit board mounted on the back of the circuit board. Between information storage devices connected to the same motherboard, complete data mirroring is organized. Each power supply is connected to both motherboards, while both power supplies operate in parallel. At the bottom of the plug-in board, a block of 30 power and data connectors is made.

In the lower part of the narrow compartment of the circuit board, there is a block 31 of power and data connectors designed to interface with a block of 30 power and data connectors located on a removable printed circuit board. A block of 31 power and data connectors is connected to the power and data connectors made in the lower part of the narrow compartment of the circuit board (not shown in the drawing), which are used to trace power and data from the power supply and information storage devices located on the removable printed circuit board, respectively electronic components of the motherboard.

In the wide compartment of the circuit board, the motherboard 32 is mounted on the front and back sides. Thus, two motherboards and two removable printed circuit boards are mounted on the circuit board.

In the upper part of the wide compartment of the printed circuit board, a data connector 33 is provided for coupling by means of a flexible board or cable (not shown) with an end data connector 34 made in the upper part of the motherboard. In the upper part of the motherboard there is a data connector 35, to which a second data cable is connected, the other end of which is connected to the network switch 6. The network switch receives power from the power connector 21 on the second printed circuit board using the power cable.

In FIG. 8 shows a cooling circuit of a server farm including one tank. The outer end of the exhaust pipe through the first non-return valve 36 is connected in series with the first pressure pipe 37 to the circulation pump 38, the coarse filter 39 and the heat exchanger 40. The heat exchanger is connected by the return pressure pipe 41 through the second check valve 42 with the outer end of the inlet pipe. Thus, a large closed loop of heat transfer from the computing nodes to the heat exchanger and the cooled liquid from the heat exchanger back to the computing nodes is formed.

A three-way solenoid valve 43 is integrated between the coarse filter and the heat exchanger, the third pipe of which is connected to the first end of the second pressure pipe 44. The second end of the second pressure pipe 44 is connected to the third pipe of the three-way splitter integrated in the return pressure pipe between the heat exchanger 40 and the second non-return valve valve 42. Three-way shut-off solenoid valve 43 is designed to block the flow of coolant entering the heat exchanger, and n directing it by means of a second pressure pipe through a third branch pipe of a three-way splitter back to the reservoir to the computing nodes. Thus, a small closed loop of heat transfer from the computing nodes to the second pressure pipe and the cooled liquid from the second pressure pipe to the tank, bypassing the heat exchanger, is formed. To ensure fault tolerance parallel to the circulation pump 38 installed backup circulation pump (not shown in the drawing).

In another embodiment of the invention, the server farm is a server farm with n tanks. In FIG. 9 is a diagram of a cooling circuit of a server farm including n tanks. The server farm consists of n tight rectangular tanks 1 filled with coolant, with computing units installed in them, the first printed circuit board 10, the second printed circuit board 17, distribution pipe 5. Each tank is equipped with a cover 2. Each tank 1 is provided with a discharge pressure pipe 45 through the exhaust pipe, the first check valve 36, the circulation pump 38 and the coarse filter 39 communicates with the exhaust manifold pipe 46, one end of which is plugged and the other is connected by means of a first three-way splitter with a first common pressure pipe 47. A common circulation pump 48, a common coarse filter 49 and a heat exchanger 40 are further connected in series to the first common pressure pipe 47. On the other hand, the heat exchanger is connected to the return common pressure pipe 50. Each tank 1 is the supply pipe 51 through the inlet pipe and the second check valve 42 communicates with the inlet manifold pipe 52, one end of which is plugged, and the second is connected by the outlet with the return common pressure pipe 50. Thus, a large closed loop is formed of heat transfer from the computing nodes to the heat exchanger and the cooled liquid from the heat exchanger back to the computing nodes.

Between a common coarse filter 49 and a heat exchanger 40, a three-way shut-off solenoid valve 43 is integrated, the third pipe of which is connected to the first end of the second common pressure pipe 53. The second end of the second common pressure pipe 53 is connected to the third pipe of the second three-way splitter, embedded between the heat exchanger 40 and the outlet. The three-way shut-off solenoid valve 43 is designed to shut off the flow of coolant entering the heat exchanger 40 and direct it through a second common pressure pipe 53 connected to the third pipe of the second three-way splitter, which in turn is communicated via a return common pressure pipe 50 with an inlet manifold pipe 52 and supply pipelines 51, to each of the tanks to the computing nodes. Thus, a small closed loop of heat transfer from the computing nodes to the second common pressure pipe and the cooled liquid from the second common pressure pipe back to the reservoir to the computing nodes, bypassing the heat exchanger, is formed. To ensure fault tolerance, a duplicate circulation pump (not shown) is installed parallel to the general circulation pump 48.

During the operation of the server farm, the computing nodes are completely immersed in the coolant. In one embodiment of the invention, the coolant heated by the computing units through the exhaust pipe and the first check valve is poured into the first pressure pipe connected to the circulation pump. The circulation pump through the coarse filter pumps the heated coolant through the first pressure pipe connected to it to the open shut-off three-way solenoid valve and heat exchanger. The heat exchanger removes heat from the incoming heated fluid. From the heat exchanger, the cooled coolant enters the return pressure pipe in communication with the second check valve and the inlet pipe, and then is fed back to the computing units via the distribution pipe. Thus, a large closed loop of coolant circulation is formed, starting from the computing nodes, continuing to the circulation pump, heat exchanger and returning to the tank back to the computing nodes.

If the ambient temperature does not exceed 0 ° C, the shut-off three-way solenoid valve is closed. The circulation pump through the coarse filter delivers coolant to the first pressure pipe, and the three-way shut-off solenoid valve directs the coolant to the second pressure pipe, where it is cooled, and then through the three-way splitter, the second shut-off valve and the inlet pipe, the coolant goes back to the tank to computing nodes. Thus, the coolant circulates through a small closed cooling circuit, starting from the computing nodes, continuing to the circulation pump and returning back to the reservoir to the computing nodes, bypassing the heat exchanger.

In another embodiment of the invention, the coolant heated by the computing units through the outlet pipe and the first check valve connected with each of the reservoirs is poured into the discharge pressure pipes connected to the exhaust manifold pipe. Then, the coolant enters the first common pressure pipe to the common circulation pump, which through the general coarse filter delivers it to the heat exchanger. In the heat exchanger, the heated coolant is cooled and fed through the return common pressure pipe to the intake manifold, then through the supply pipelines connected to it, through the check valves and inlet pipes back to the reservoirs to the computing nodes. Thus, the coolant circulates along a large closed cooling circuit, starting from the computing nodes, continuing to the heat exchanger and returning back to the reservoir to the computing nodes. If the ambient temperature does not exceed 0 ° C, the shut-off three-way solenoid valve is closed. The common circulation pump through the common coarse filter supplies coolant to the first common pressure pipe, while the three-way shut-off solenoid valve directs the coolant to the second common pressure pipe. And then through the third branch pipe of the second three-way splitter, by means of the return common pressure pipe, the intake manifold pipe and the supply pipes, the cooling liquid is returned back to each of the tanks to the computing nodes. Thus, the coolant circulates through a small closed cooling circuit, bypassing the heat exchanger.

The ambient temperature at which the operation of the inventive server farm is possible ranges from -40 ° C to +40 ° C.

The server farm receives power from an external power source through power cables through the holes 8 in the smaller flap of the lid, one ends of which are connected to an external power source, and the other ends are connected to the power connector 21 on the second circuit board.

As a cooling liquid, synthetic or natural oils are used.

The increase in the density of the installation of computing nodes is achieved due to the lack of an individual case in which the computing node is located, and an individual container, which contains one or more motherboards, cooling devices used in known solutions, for example, heat-removing boards and cooling modules, without using standard server racks with a standard width of seats for computing nodes and with mounting and operating intervals. In addition, circuit boards are installed on circuit boards. motherboard and removable circuit boards on both sides. All this increases the usable space that can be used to install additional computing nodes. In addition, the invention does not provide for the use of standard computing nodes designed for air cooling, which reduces the size of the computing nodes installed in the inventive server farm.

Functioning at low ambient temperatures is achieved through the use of heating elements.

The ability to hot-swap power supplies and information storage devices is achieved by connecting part of the information storage devices located on a removable printed circuit board installed on one side of the mounting printed circuit board to a motherboard mounted on its other side, and organizing data mirroring between information storage devices connected to one motherboard. As well as the fact that both power supplies operate in parallel. As well as the fact that power supplies and information storage devices are located on a removable printed circuit board.

Claims

Claim

1. A server farm with an immersion cooling system, consisting of a sealed tank with computing units installed in it, consisting of a circuit board with a motherboard mounted on it, power supplies, storage media filled with coolant, equipped with a cover, guides for the vertical installation of computing nodes , inlet and outlet pipes communicating via a pipeline with a pump and a heat exchanger, characterized in that the server farm is equipped with a distribution a single pipe, two printed circuit boards, a network switch, and

the distribution pipe is installed parallel to the entire bottom of the tank and is formed by two perforated pipes parallel to each other, connected by a U-shaped connector, one end of the distribution pipe is plugged, and the second is connected to the inlet pipe installed in the lower part of the tank wall,

and the outlet pipe is installed in the upper part of the tank wall and through the first check valve is connected via a first pressure pipe to a circulation pump, a coarse filter and a heat exchanger, which on the other hand is connected via a check pressure pipe through a second check valve with an inlet pipe,

between the coarse filter and the heat exchanger there is a three-way shut-off solenoid valve, the third pipe of which is connected to the first end of the second pressure pipe, the second end of which is connected to the third pipe of the three-way splitter integrated in the return pressure pipe between the heat exchanger and the second check valve,

the lid of the tank is formed by two flaps with different areas surface and installed with the possibility of removal and lifting, and the large leaf is installed with the possibility of removal and lifting in the stop mode and the server farm, on the inside of the larger leaf there is a network switch, and in the smaller leaf there is a hole for a cable or a flexible circuit board having at the ends are data connectors, a flexible board or cable connected to one end of an external data cable, the other end of which is connected to an external control device, and the network switch Inonii to one end of the first data transmission cable whose other end is led out through an opening in the lower sash cover and connected to an external control device,

the first printed circuit board is installed parallel to the plane of the tank bottom under the distribution pipe, it has end data and power connectors, and data connectors are aligned along one edge, and power connectors are aligned along the opposite edge, between the power connectors and data connectors made heating elements

the second printed circuit board is installed close to one of the walls of the tank, to which a smaller flap of the lid is attached, perpendicular to the first printed circuit board and joined edge to edge with the help of mates for the data and power end connectors of the first printed circuit board,

and its upper part is connected via a data connector to a flexible printed circuit board or cable, passed through an opening in the smaller leaf of the lid of the tank, and to one end of the external power cable passed through the other hole in the smaller leaf of the lid, through power connectors, while the other end of the external the power cable is connected to an external power source, moreover, the network switch is powered by a power connector located in the upper part of the second printed circuit board, using a power cable,

computing nodes are installed using data and power connectors on the first printed circuit board parallel to the second printed circuit board and consist of a printed circuit board, which is divided by a longitudinal divider into narrow and wide compartments, two longitudinal holders are mounted symmetrically on both sides of the narrow compartment on both sides in the form guides for fixing a removable printed circuit board on which the power supply unit, information storage devices and a block of power and data connectors are located, and in a wide compartment on both sides it mounts I'm on the motherboard,

the motherboard is connected to the network switch via a second data cable,

the plug-in board is connected to the circuit board through the power and data connector blocks,

a wide compartment of the circuit board is connected via a flexible printed circuit board or a cable to the motherboard using data connectors,

in the lower edge of the mounting circuit board, a rectangular cutout is made for the passage of the distribution pipe,

at the same time, half of the information storage devices located on the removable printed circuit board installed on one side of the circuit board are connected to the motherboard mounted on the same side of the mounting circuit board, and the remaining information storage devices are connected to the motherboard mounted on the other side of the mounting circuit board , between the storage devices connected to the same motherboard, a complete data mirroring is organized, each power supply is connected to both motherboards, while both power supplies operate in parallel.

2. The server farm according to claim 1, characterized in that the inlet pipe is installed in the lower part of the same wall of the tank in which the outlet pipe is installed.

3. The server farm according to claim 2, characterized in that the second printed circuit board is installed perpendicular to the first printed circuit board close to the same tank wall on which the inlet and outlet pipes are installed.

4. The server farm according to claim 3, characterized in that the upper edge of the second printed circuit board is cut so as to provide direct communication of the outlet pipe with the reservoir, an opening is made in the lower part of the second printed circuit board at the level of the orthogonal projection of the inlet pipe so as to provide entry of the inlet pipe into the tank.

5. The server farm according to claim 1, characterized in that the inlet pipe is installed in the lower part of the tank wall, opposite to that in which the outlet pipe is installed.

6. The server farm according to claim 5, characterized in that the second printed circuit board is installed perpendicular to the first printed circuit board close to the same wall of the tank on which the inlet pipe is installed.

7. The server farm according to claim 6, characterized in that an opening is made in the lower part of the second printed circuit board at the level of the orthogonal projection of the inlet pipe so as to provide direct communication of the inlet pipe with the reservoir.

8. The server farm according to claim 5, characterized in that the second printed circuit board is installed perpendicular to the first printed circuit board close to the same wall of the tank on which the outlet pipe is installed.

9. The server farm according to claim 8, characterized in that the upper edge of the second printed circuit board is cut so as to provide direct communication of the outlet pipe with the reservoir.

10. The server farm according to claim 1, characterized in that it consists of n pressurized tanks connected in parallel through a system of inlet and outlet pipelines in communication with the outlet and intake manifold pipes interconnected by a common pressure pipe.

11. The server farm according to claim 1, characterized in that an additional circulation pump is installed parallel to the circulation pump.

12. The server farm according to claim 1, characterized in that for the vertical installation of computing nodes parallel to the side edges of the tank cuts are made.

13. The server farm according to claim 1, characterized in that solid-state information drives are used as information storage devices.

14. The server farm according to claim 1, characterized in that the heating elements are made in the lower part of the second printed circuit board.

15. The server farm according to claim 1, characterized in that the heating elements are made in the form of load resistors.