Building a home server (2024 edition)

Apr 05, 2024

For the past decade I've had a HP MicroServer Gen8 as a home server. This tiny box has been very performant and reliable and is among my favorite computers that I've ever owned. However, its hardware is getting a bit long in the tooth. It's equipped with a low-power Xeon E3-1220L V2 and the memory is maxed at 16GB. While it stills performs decently, it was overdue for an upgrade.

The MicroServer case is very compact and it uses mostly custom HP components such as the motherboard, so simply upgrading its components is not feasible. I therefore settled for building a new one. I didn't really have any concrete specifications in mind, but I knew I wanted some sort of out-of-band management as I've gotten used to this on the MicroServer (iLO) and ECC memory.

It's been a long time since I last built a machine from scratch, but luckily most of the process has remained the same. I started out with choosing a case. I like the cube factor of the MicroServer, but I wanted something with room for more 3.5" drives and generally big enough so that the initial build, and later upgrades, doesn't become annoying due to space constraints.

I settled on a Fractal Design Node 804 which is a micro-ATX chassis with room for up to 10 disks. It has an elegant cube-shaped look, and has neat solutions for cable management. Now for the hard part of any build: Which motherboard, CPU and memory combination can I buy which isn't prohibitively expensive? Historically ECC memory has been artificially limited to enterprise-grade CPUs and motherboards, such as the Intel Xeon CPUs and associated chipsets. In comparison, AMD has supported ECC memory on their CPUs for many years now.

AMD-based build

I initially came up with the following components:

Case: Fractal Design Node 804
Motherboard: ASRock B650D4U
CPU: AMD Ryzen 7 7700
Memory: 2 x Kingston DDR5 32GB ECC (KSM48E40BD8KM-32HM)
OS disk: WD Black SN850X 1TB
PSU: Seasonic Focus GX 550W

The hardest part was finding a motherboard that supported both ECC memory and out-of-band management. When I finally found one, I just went with memory and OS disk from the motherboard QVL list to keep things simple. However, the board wasn't in stock anywhere and Norwegian sellers were charging exorbitant prices for it. This is presumably because this is card is classified as "professional" or "enterprise" product and is thus priced differently from consumer boards.

Intel-based build (final choice)

During my research found out that Intel had (quietly?) started supporting ECC on their consumer processors. The only catch being that you need a motherboard with the W680 chipset. This meant that Intel was back in the running, so I started considering an equivalent Intel-based build.

After reading some benchmarks I ended up with the following components:

Case: Fractal Design Node 804
Motherboard: Asus PRO WS W680M-ACE
CPU: Intel Core i5-13500
Memory: 2 x Kingston DDR5 32GB ECC (KSM48E40BD8KM-32HM)
OS disk: WD Black SN850X 1TB
PSU: Seasonic Focus GX 550W

I had to buy the motherboard from Amazon US as it was half the price Norwegian sellers were charging (and none had it in stock, a recurring theme). Yet another case of being charged a tax for "professional" components.

Conclusion

Components with ECC memory support are becoming more readily available, but finding compatible motherboards and memory modules is a bit of a chore. Getting what you want requires fine-reading motherboard QVL lists and CPU data sheets. However, it's great that this feature is no longer being restricted to server-grade components.

Assembling the components into the Node 804 case was a breeze. The case has lots of room and is easy to work with. Stability of the build has also been excellent over the past couple months. I did however have to coerce Debian into supporting the new hardware properly. See appendix for more details.

My new server (left) now sits neatly next to the MicroServer, which has since become a router. Hopefully it will last a decade, like its predecessor!

Appendix: Linux woes

iGPU

The Asus PRO WS W680M-ACE motherboard has an integrated VGA controller. This is exposed through A) a physical VGA connector and B) a virtual VGA console, accessible via the BMC. It also has DisplayPort and HDMI connectors, these are connected to the iGPU on supported CPUs.

I wanted to use the iGPU for transcoding video, but I wanted the console to be available via the VGA controller. This turned out to be difficult because connecting anything to the VGA connector would disable the iGPU completely, and vice-versa.

The solution to this problem is as follows:

Update the kernel to atleast 6.5. On Debian Bookworm, it's sufficient to install the latest kernel from backports
In the BIOS, set Primary display: Auto and iGPU Multi-Monitor: Enabled
Boot the kernel with nomodeset i915.modeset=1. This forces the console to use a good old framebuffer, but explicitly enables video mode setting in the i915 driver (used by the iGPU)

RAID controller

Another issue was related to PCI-e power saving. I have a RAID controller connected to a PCI-e port, and occasionally the kernel would print a bunch of errors from the 3w-9xxx driver. For example:

3w-9xxx: scsi0: ERROR: (0x03:0x0100): SGL entry contains zero data:address=0x0, length=0x0, cmd=X

This turned out to be caused by something called Active State Power Management, which tries to power down the PCI Express serial link in order to save power. This is obviously not a good idea for a RAID controller. To solve this issue I added pcie_aspm=off to the kernel boot parameters.