Workstation with 4xGPU

Wednesday November 18th, 2020 Leave a comment By admin

In this article I will describe the creation of a Workstation with 4 video cards.

The goal is to create a computer for CG artists working with GPU graphic rendering engines. As simple as possible, efficient and reliable. Do not modify graphics cards for water cooling. Achieve operating temperatures of video cards in the region of 70 °C (158 °F ). With the possibility of modification and upgrade.

The first thing to decide was the PC – case. It would be ideal to develop a case yourself, which would exactly meet the tasks set, but it will be cheaper and faster if you can find a ready-made case.
Digging around on the internet, I came across the Vinga Starkad case. I was attracted by the number of expansion slots in it. There are as many as 10 of them horizontally and 2 more vertically. This gave a chance to place video cards at a sufficient distance without reworking the inside and back of the case. Purchased!

Asus X99-E WS motherboard. Firstly, I already had it, and secondly, it provides 4x PCI-E slots in 16x mode. Although it doesn’t matter for GPU rendering, but it’s still cool 🙂

Xeon 1650 v3. I considered this the best option. 6 cores are enough for GPU rendering in my experience. High frequency per core, which is important when working with CG applications. Supports server memory.

Memory 96Gb. Since I was faced with a lack of RAM in GPU tasks, I decided that 96Gb would be the right amount for such a computer. Server memory, with error correction, frequency 2133. Not fast, but the volume is more important, and the memory frequency has almost no effect on the rendering speed.

Power Supply. Such a powerful system requires a reliable power supply. I chose the 1600 Watt EVGA 1600 G2. Not a very well-known brand in our market, but this is a really very high quality power supply that can hold all its stated power. With a 10 year worldwide guarantee. The system showed its maximum consumption of 1100 watts, so the power supply has a lot of headroom for an upgrade.

Bequiet CPU cooler! dark rock pro 3. Since the processor will overclock, and the case will be quite cramped and hot, I decided to install the largest cooler.

Case fans. A lot of them. And I decided to buy them of high quality, with ball bearings, or more advanced bearings. The cooling system must be reliable, since 1100 watts is not easy to cool in a closed case. The fans will run at full capacity. We bought Deepcool UF 120 – 3 pieces, Deepcool UF 140 – 3 pieces, Noctua Industrial 2000 – 2 pieces. In total, I installed 8 fans in the case.

Video cards. So we got to the most basic component of our system. I chose the Nvidia GTX 1080ti as it is the best price / performance graphics card available at the time of the PC build. The manufacturer decided to take EVGA, so I had only positive experience with video cards from this manufacturer. I also like that they use ball bearing fans that last a long time. The size of the cards was also important. I ended up with 3 EVGA GTX 1080ti SC cards and one GTX 1080ti FTW3.

Time to put it all together!

I removed all the HDD bay enclosures, leaving only one at the bottom. The body has become much lighter. I inserted 3 cards into the motherboard, and took one out on a riser (EZDIY 300 mm) and placed it vertically. The gap between the cards is one slot. Tests have shown that the top and middle cards overheat both in a closed case and in an open one. Their temperatures were 77° C (170° F) degrees, despite the fact that I tried to blow air between them with 140 mm noctua at 2000 rpm. Installing a more powerful fan will be too noisy. It is necessary to increase the distance between the cards.

OK. We proceed to option B. To do this, we need another riser, holes in the lower partition for air intake by the lower video card. We disassemble everything and begin to shred the case.

I cut 2 holes with a dremel, right under the fans of the video card. I made a mount for the second riser. This is another HLX-Link riser, it is written on it, it is lateral and has a length of about 150 mm. To secure it, I drilled 2 holes and screwed in the racks for attaching the motherboard.

I assembled my computer and started testing. The results were encouraging. Temperatures on the upper and middle graphics cards dropped. Then it remained to organize the optimal air flow in the case. I found that the closer I place the fan to the graphics cards, then the better they will be cooling. I also added another fan higher to supply air to the processor and faster heat dissipation from the top of the case. Later I made special mounts for these two fans.

The front part of the case is deaf and the fans standing in the front do not create the required flow. I had to take air from the side cover. These cases come with a special plate for attaching three 120 mm fans instead of HDD cages. But there was another problem, the ventilation holes in the side cover are too small. Well, let’s take the grinder and cut holes in the shape of the purchased magnetic nets. At the same time, we solve the issue of the absence of anthers in the case.

The body is now “breathing”. I also screwed small rubber feet onto the side cover so that it was impossible to lean the case against the plane and block the ventilation holes.

Additional ventilation holes were drilled in the rear wall of the case, around a vertically standing video card, and slots were made in the top cover. This also improved the outlet of hot air from the case.

I replaced the standard legs with furniture wheels. This not only made the heavy chassis mobile, but also increased the gap between the PSU ventilation grill and the floor.

Everything worked out. Hot air comes out, cold air comes in, all components do not overheat. It’s time to do the final assembly. Disassemble everything again, paint all the custom parts from the spray can with black matte paint. I make mounts for fans on a milling machine from fiberglass, paint.

The project is finished. The workstation is assembled and passed all tests. Below you can see the photos.