File Server Builder's Guide
by Zach Throckmorton on September 4, 2011 3:30 PM ESTWhat is a file server?
Essentially, a file server is a computer that stores files, is attached to a network, and provides shared access of those files to multiple workstation computers. File servers do not perform computational tasks - that is, they do not run programs for client machines. Furthermore, they do not provide dynamic content like a web server. Still further, file servers are not like database servers in that the former do not provide access to a shared database whereas the latter do. File servers provide access to static files via a local intranet through Windows or Unix protocols as well as over the internet through file transfer or hypertext transfer protocols (FTP and HTTP).
What can you do with a file server?
The primary function of a file server is storage. For the home user, one central storage location can increase overall computing efficiency and reduce overall computing cost. By placing all of your important files in a single location, you do not need to worry about different versions of files you're actively working on, wasting disk space by having multiple copies of less-than-important files scattered on different systems, backing up the right files onto the right backup storage medium from the right computer, making sure every PC in your home has access to the appropriate files, and so on.
From a system builder's perspective, a file server can also liberate your various workstation computers from having to accommodate multiple hard drives, and decrease overall hard drive expenditures. With the rise of SSDs, which offer tremendous performance at a high cost per GB, a file server can free workstations from the performance shackles of platter-based disks - an especially useful consideration for laptops and netbooks, where the small capacity of an SSD is often a deal breaker since these mobile computers usually can house only one drive.
A dedicated file server allows every user in a home - whether they're at home or on the road - to access every file they might need, regardless of which particular device they might be using at any given time. Dedicated file servers also allow you to share your files with friends and coworkers - simply provide them with a URL, a login name and password, and specify what content they can access. For example, maybe you'd like to share your kids' camp photos with the in-laws - but your cloud storage capacity won't fit all of those photos plus all of the other stuff you have stored in your cloud drive locker. Maybe you'd like to share sensitive information with a colleague that you'd rather not upload to a server owned by Amazon or some other third party, but the files are too big to email. Or maybe you'd simply like to access your 200GB library of MP3s while you're holed up in a hotel on business with nothing but your 60GB SSD-based netbook. These few examples are really only the tip of the iceberg when it comes to the utility of a file server.
That said, there are alternatives to a file server for all of these needs. You could dump all of your photos onto a flash drive and give them to the in-laws the next time you see them - but you have to do this every time you want to share more photos - and who knows if you'll get your flash drives back? You could mail a DVD-R to your colleague - but perhaps a DVD-R's ~4GB capacity is insufficient, and snail mail takes days if not weeks to be delivered. If you're on the road, you could just bring along your portable external hard drive - which takes up space, and can be lost or stolen. A file server is a simple, singular solution to all of these problems. Home file servers do not require enterprise-grade hardware and can be very affordable. They can also be made from power-sipping components that won't spike your electrical bill.
What considerations are important in building a file server?
Because the primary role of a file server is storage, this is the most important aspect to think about. How much storage space do you need? Do you want to share 50GB of photos taken on a point and shoot digital camera? 500GB of music? 2TB of movie DVD ISOs? 30TB of mixed media and work-related files? Also, at what rate are your storage demands growing, and how easily do you want to be able to expand your file server?
How easily do you want to be able to administer your files? Many of the more powerful file server operating systems are unfortunately not particularly easy to run for the non-IT professional. However, there are file server OS's that are easy to run. What about being able to recover your files in the event of catastrophe? Placing your files in one computer is tantamount to putting all of your eggs in one basket, which can be risky. What about security? Anything on any sort of network is vulnerable to intrusion. While this guide answers all of these questions, it is aimed at home users and therefore necessarily makes some sacrifices to storage space, administration capabilities, recoverability, and security - simply because home users typically can neither afford nor require professional-grade file server solutions.
Why build a file server instead of using NAS?
Simply put, a NAS (networked attached storage) device is a computer appliance. It is built specifically to provide network-accessible storage. NAS devices typically offer easier administration than file servers (some are a few mouse clicks away from plug and play operability), but are often limited by proprietary software, and are neither as capacious nor as expandable as a dedicated file server. Further, higher-end NAS devices that can house as many hard drives as some of the builds outlined in this guide are more expensive than the file server alternative. Finally, because they are designed with only one purpose in mind, they are not as flexible as a file server, which in a multi-system home, might need to be co-opted into a basic workstation at a later point in time. That said, while NAS devices are outside the scope of this guide, they're worth investigating if you're not already familiar with them.
This guide is laid out differently than my previous builder's guides in that rather than detailing specific systems at specific price points capable of performing specific tasks, it instead discusses options for operating systems and types of components and how these different options are best suited to addressing different needs. That is, maybe you need a lot of storage space but you're not particularly concerned about backups. Or perhaps you don't need much storage space at all but want to use a very straightforward file server operating system. By mixing and matching recommendations to suit your needs, hopefully you'll be able to construct a file server with which you'll be pleased!
152 Comments
View All Comments
noxipoo - Sunday, September 4, 2011 - link
i'm looking for alternatives to drobo or the more expensive NAS devices so some raid card recommendations along with all the things that one needs would of been nice.bamsegrill - Sunday, September 4, 2011 - link
Yeah, some raidcard-recommendations would be nice.Rick83 - Sunday, September 4, 2011 - link
MY RAID card recommendation is a mainboard with as many SATA ports as possible, and screw the RAID card.For anything but high end database servers, it's a waste of money.
With desktop boards offering 10 to 12 SATA ports these days, you're unlikely to need additional hardware, if you chose the right board.
Otherwise, it's probably wisest to go with whatever chipset is best supported by your OS.
PCTC2 - Sunday, September 4, 2011 - link
But there's the fact that software RAID (which is what you're getting on your main board) is utterly inferior to those with dedicated RAID cards. And software RAIDs are extremely fickle when it comes to 5400 RPM desktop drives. Drives will drop out and will force you to rebuild... over 90 hours for 4 1.5TB drives. (I'm talking about Intel Storage Matrix on Windows/ mdadm on Linux).You could run FreeNAS/FreeBSD and use RAID-Z2. I've been running three systems for around 5 months now. One running Intel Storage Matrix on Windows, one running RAID-Z2 on FreeBSD, and one running on a CentOS box on a LSI2008-based controller. I have to say the hardware has been the most reliable, with the RAID-Z2 in a close second. As for the Intel softRAID, I've had to rebuild it twice in the 5 months (and don't say it's the drives because these drives used to be in the RAID-Z2 and they were fine. I guess Intel is a little more tight when it comes to drop-out time-outs).
A good RAID card with an older LSI1068E for RAID 5 is super cheap now. If you want a newer controller, you can still get one with an LSI2008 pretty cheap as well. If you want anything other than a giant RAID 0 stripe (such as RAID 5/6/10), then you should definitely go for a dedicated card or run BSD.
Rick83 - Sunday, September 4, 2011 - link
I've been using 5400 rpm disks and mdadm on linux for quite a while (6 years now?) and never had a problem, while having severly sufficient performance.If disks drop, that's the kernel saying that the device is gone, and could be jut a bad controller.
I've been on three boards and never had that kind of issue.
Windows is a bit more annoying.
Also, your rebuild time is over the top.
I've resynced much faster (2 hours for 400GB - so 10x faster than what you achieved. While also resyncing another array. Sounds like you may have a serious issue somewhere)
The compatibility advantage of software RAID outweighs any performance gain, unless you really need those extra 10 percent, or run extreme arrays (RAID-6 over 8 disks, and I might consider going with a dedicated card)
I think it might be the Intel/Windows combination that is hurting you - you might want to try the native windows RAID over the Intel Matrix stuff. Using that is the worst of both worlds, as you have the vendor lock in off a dedicated card and the slightly lower performance of a software solution.
Of course, you also mentioned mdadm, but I've never had a problem with that, with a bunch of different disks and controllers and boards.
I guess in two to three years, when I upgrade my machine again, I will have to look at a SATA controller card, or maybe sooner, should one of my IDE disks fail without me being able to replace it.
I think you may just have been unlucky with your issues, and can't agree with your assessment :-/
Flunk - Sunday, September 4, 2011 - link
I agree, I've used the Windows soft raid feature a lot and it trumps even hardware raid for ease of use because if your raid controller dies you can just put your drives in any windows system and get your data off. You don't need to find another identical controller. Performance is similar to matrix raid, good enough for a file server.vol7ron - Monday, September 5, 2011 - link
Wouldn't a RAID card be limited to the PCI bus anyhow? I would suspect you'd want the full speed that the SATA ports are capable ofvol7ron - Monday, September 5, 2011 - link
Even with 5400RPM drives, if you have a lot of data you're copying/transfering, you could probably saturate the full bandwidth, right?Rick83 - Monday, September 5, 2011 - link
PCI is wide enough to support gigabit ethernet, so if you don't have too many devices on the bus, you'll be fine until you have to build a RAID array.With PCI-X and PCIe these limitations are no longer of relevance.
Jaybus - Monday, September 5, 2011 - link
There are plenty of PCI-E x4 and x8 RAID cards.