GIGABYTE Server GA-7PESH3 Motherboard Review

A dual processor system sounds awesome to the home user but in reality it is almost entirely a professional market. The prosumer has to use Xeons at JEDEC memory speeds and then ensure that the software is NUMA aware, especially if it decides searching for data in the other processor's L3 cache. However now GIGABYTE Server is selling to the prosumer via Newegg, and they sent us the $640 GA-7PESH3 for review.

For most users, a dual processor system affords several issues, aside from the cost. Performance with a 2P system is very dependent on the software in use. With a single processor system, each core can ‘snoop’ into the other core cache in order to see how data is updated. In a 2P system, the latency of talking between the two CPUs is at least an order of magnitude higher. This means that memory accesses can be delayed causing branched and locked code to be slow. In order to get around this, the software has to be NUMA aware, and the majority of regular applications are not.

There is no overclocking with a 2P system, and single thread speeds can be lower. As a result, gaming often sees a hit in performance, as well as basic tasks. The optimal use case scenario, for most software that is not aware of dual processor architecture, is any workload that needs as few memory accesses as possible and is ‘embarrassingly parallel’ such as ray tracing, video editing, virtualization or particular types of scientific compute. 2P motherboards, particularly those built by server teams, also often come with system management tools not seen in the consumer space, allowing users to access their system as it processes data and monitor progress as well.

GIGABYTE GA-7PESH3 Overview

This motherboard is built with storage and GPUs/coprocessors in mind. Although each CPU socket has four DRAM slots (rather than a more usual complement of eight each), the PCIe is a full array of PCIe 3.0 slots offering x16/-/x16/-/x16/-/x16 or x16/x8/x8/x8/x8/x8/x8 whether the system is using four PCIe devices or seven. This means that the PCIe lane allocation comes from both CPUs, which in itself can produce issues for CrossfireX and SLI setups, although compute does not rely on those technologies and should suffer no issues. Interestingly GIGABYTE does not supply any extra power connectors for these PCIe slots, indicating that not each port might not be able to provide 75W when all are populated (e.g. seven graphics cards).

In terms of storage, the system has eight SAS 6 Gbps ports in white on the right hand side, rather than the mini-SAS connectors we saw on other C60x motherboards. For SATA storage there are two SATA 6 Gbps ports in red and four SATA 3 Gbps in black, each at the bottom of the board. The motherboard also has two USB 3.0 type A ports sticking out of the motherboard – this feature is useful for users with software that requires USB dongle verification.

We tested the motherboard in both dual E5-2697 v2 (2x 12C/24T, 2x 130W) and dual E5-2687W v2 (2x 8C/16T, 2x 150W), showing where more cores of the former and the higher base frequency of the latter pay off in our CPU benchmarking suite. This especially shows through in our rendering benchmarks, although not so much in the x264 video conversion. In these configurations, idle power consumption with a single GTX 770 installed gave 115-130W at idle and 300W-320W with 100% CPU load. The Realtek ALC892 codec is more a token gesture for basic audio; other server motherboards tend to eschew audio, but as this is more a workstation outfit then basic audio should be sufficient with dedicated audio professionals using their own PCIe sound cards. On the audio note, the DPC Latency was clocked in as 300+ microseconds, similar to other server products, but ultimately quite high compared to mainstream motherboards.

Server motherboards are designed to be left on for extended periods of time, and thus not a lot of effort is made in POST times – this is indicated by the SAS and two Intel NICs having their own OPROM POST sequences, making the total power on to Windows loading screen over 60 seconds. This reduces to 38 seconds when the SAS and NICs are disabled.

Our gaming benchmarks were limited by the motherboard not displaying an image when more than one GPU was used, but the single GPU benchmarks all showed a regression in performance against an i7-4770K setup. One could suggest that this is not a platform for gaming, although users who want to have this system in a dual work/home role should have no problems matching mainstream system performance.

Motherboards in this bracket are all about the feature set, and the GA-7PESH3 oozes capability, especially when pairing two 12 or 15 core CPUs with a four-way co-processor setup matched with a SAS RAID. The only downside is the cost to get there. Server motherboards are often limited in BIOS and Software, and the 7PESH3 does not move things into a more useful role there, indicating a potential gap in the server board market.

Visual Inspection

For those that are more used to the consumer sized motherboards, the SSI CEB standard is both longer and uses different fixing points to the case, so users who invest should ensure that their case supports the standard as most do not. One of the poignant features the GIGABYTE Server team pointed out to me was the socket implementation which on this board has the sockets side by side and level, compared to others (such as the 7PESH1 we reviewed) which are often offset due to PCB layout.

Because of this orientation, equipping each CPU with eight DRAM slots is a little unwieldy unless concessions are made, so GIGABYTE went with one DIMM per channel for each CPU. While this limits the system to eight slots, this does afford 64GB of UDIMMs or 256GB of quad-rank RDIMMs both at 1866 MHz rather than the 1600 MHz standard. Given how Xeon server and workstations are often memory speed limited, this might be a good positive step.

Each CPU needs its own 8-pin CPU power, and because there is no overclocking on this platform each of the power delivery segments is built to server specifications: several high-rated power phases built for 24/7 operation.

While there are six fan headers on board, the placement does seem to detract from any liquid cooler setups by providing each of the sockets only one header to play with. The other four headers are located in pairs – one pair above the SAS ports and the other pair below the PCIe sockets.

For storage the motherboard uses SAS ports directly on the PCB rather than through a mini-SAS interface. These eight ports are in white, with the SATA ports in red and black at the bottom of the board. The red ports are SATA 6 Gbps while the black ports are SATA 3 Gbps. The heatsinks above these ports are for the chipset and a controller. Next to the two black SATA ports at the bottom are two USB 3.0 type A ports (powered by an Etron controller) directly in the motherboard. These ports allow for permanent USB storage to be attached while in a case, or as is often the case with workstation builds, suitable for USB verification dongles for professional software that is sold on a per-system license. My experience is that these dongles are being phased out due to the prevalence of a central license server or the web, but this is a feature that makes it on to some consumer motherboards.

The GA-7PESH3 has a maximum complement of full-length PCIe slots, although the naming configuration is odd. The bottom one is labeled PCIE_1 and the top is PCIE_7, this is because PCIE_1/3/5/7 are all x16 (two from each CPU) with PCIE_2/4/6 being x8. When something is installed into an x8 slot, the slot underneath reduces down to x8 mode, ending up with two possible extremes:

Four PCIe devices: x16/-/x16/-/x16/-/x16
Seven PCIe devices: x16/x8/x8/x8/x8/x8/x8

For users with single slot RAID cards or single slot compute GPUs, such as AMD’s FirePro V7900 or S7000, each element of bandwidth is more than welcome. However for users interested in SLI or Crossfire, the card that gets the default video output is often the one at the bottom. It should be noted that there is no additional power delivery on the motherboard for the PCIe slots. This is a little worrying - should a user put in several devices that consume ~70W each and not require extra power plugs, the motherboard might have a hard time delivering 7x70W and cause the 24-pin ATX connector to burn out. This comes across as a design flaw, perhaps suggesting that while the system should be geared towards 4-way GPUs or 7-way Compute, only devices that have external power should be used. Even then, this might cause issues for any combination of cards requiring >200-250W. from the 24-pin power connector.

Like other motherboards in this segment, the 7PESH3 uses an Aspeed IC to provide system management tools. This allows the user to log in to the system via a dedicated network port, turn the machine on, and access the graphical output. The Aspeed also provides the VGA output on the back if the system is under a desk but filled with compute performance. The base design of an AST2300 is a 400 MHz ARM9 core, which GIGABYTE has paired with a small amount of onboard memory.

The rear panel on the motherboard contains a PS/2 combination port, four USB 2.0 ports, four USB 3.0 ports (from an Etron controller), a VGA port, a COM port, two Intel 82574L NICs, a management NIC from the Aspeed IC and audio jacks from the Realtek ALC892 audio codec.

Board Features

In The Box

Workstation and server motherboards are in that weird nexus where system admins might not want a lot of extras. Their systems are planned out and all the separate items are ordered separately. Now that the GIGABYTE Server division is selling to end-users, the packages have to represent what a home/prosumer might need in with the box. As a result, we get the following:

Driver DVD (Manual on Disk)
Quick Reference Guide
Rear IO Shield
3.5-inch USB 3.0 Front Panel with 2.5” bracket
Six SAS/SATA Cables
USB 2.0 Rear Bracket
2/3/4 way SLI Connectors
Three Crossfire Connectors

This is a nice amount of kit to get with a motherboard, especially with a server/workstation based product. One could levy an argument that for non-multi GPU implementations, the price could be reduced by $25 by removing all the additional SLI/Crossfire Connectors, but items like the USB 3.0 front panel are well received.