This article/howto/recipe/whatever was inspired by a bunch of things:
- Me constantly running out of space for my media library.
- Me starting to use Apple’s Time Machine a lot.
- Drobo’s offerings.
- My desire to have a home server that could handle all the data I could throw at it, and then some, while also doing some automated tasks like downloading TV
torrentsfiles from Usenet and acting as my SSH tunnel, when needed.
Note: all prices are in NZD, unless otherwise stated.
UPDATE #1: Pictures are available here.
UPDATE #2: I’ve found a more up-to-date Netatalk guide here. For Ubuntu 9.10 or later. I’ve also had issues with ACPI using the factory BIOS of my motherboard. Luckily Intel had already posted an update for it, and it can be found here.
UPDATE #3: I’ve upgraded it without a hitch to Ubuntu 10.04. And it should stay this way for a while, since it’s an LTS release (Long Term Support).
UPDATE #4: Second part is now available here.
Feeling adventurous? Keep reading…
As you do with hobbies, you gotta start with the chassis. I’ve scoured the interwebs in search of options that would accommodate at least 4 x 3.5" HDDs while keeping the overall form-factor (minus the power supply) small. How small, you ask? Well, my intention was to be able to take it with me to the odd LAN party and hook it up to the host’s network for some good ol’ file document sharing. Yes, documents. 😉
Before I started, I already had two things already defined:
- For an ealier (postponed now) project, I bought an Intel D945GCLF2 Mini-ITX board. These things are great for little projects like this because they come with an embedded Atom processor and cost very little. In fact, I think I picked mine up for $170. At the time, I thought I was buying a single-core Atom 230. Imagine my surprise when I booted the system and found out I had dual-core Atom 330 running with double the cache size!
- And the system HDD would be an old 60GB SATA 2.5" disk I found lying around.
For the chassis, I ended up picking the Chenbro ES34069. This was due to the fact that it has 4 hot-swappable 3.5" HDDs bays. Now this is pretty much the crown jewel of my setup, mainly because of the cost. And there is no other way of saying this: it cost dearly.
Since the reseller for Chenbro in NZ was really, really crappy at customer service (yeah, no link love for him), I had to go the usual way: eBay. After a couple of weeks of research, I’ve found a seller that would ship to NZ. The case itself cost US$192.42. Shipping and handling added another US$109 to the bill. And then Customs took another ($100) bite out of it. Again, not cheap. :/
You can find cheaper versions of this case (the 120W one) but I decided for the 180W version after doing some calculations using eXtreme Power Supply Calculator Lite, which then told me my system’s estimated requirements would be less than 100W. Note, however, that at the time I did not know that my CPU was actually a dual-core Atom, but rather a single-core. After re-inputting the values now I get 113W as the estimated value, which would leave very little room to breathe on a 120W PSU. Definitely take all into consideration before buying the case and PSU!
All that was left now was to get some RAM (2GB DDR2 667Mhz module for $50) and the RAID disks. I’ve borrowed 2 x 160GB and 2 x 500GB disks until I manage to get 4 x 1TB disks. I’ve recently replaced the 2 x 160GB with 2 x 1TB, so I’m halfway there. Information on how to grow the array is further down.
Since I’m constantly surrounded by Ubuntu Servers, I decided to set it up with it as well. For this, I picked 9.04, even though 9.10 was about to come out. Why? I was anxious to get the server up and running. An upgrade will be done in due time.
Since my home is packed full of Apple gear (not literally, of course!), I needed AFP support. It would also be good if I could have Bonjour present the AFP shares automatically on the sidebar of the computers in the network. For this, I relied heavily on krema’s wonderfully written HowTo: Make Ubuntu A Perfect Mac File Server And Time Machine Volume article. Be sure to check the bottom comments as well, as there’s load of new info on it. A hint, though: If you go with Ubuntu 9.10 or later, no need to recompile Netatalk. There, saved you hours of work already.
Initial designs (hahaha!) had a RAID5 array setup. I liked the idea of cheaper (and bigger) storage, but I wanted more reliability since some of my own personal files (documents, pictures) would be stored in it.
So I went with the option that combined the highest performance with the most fault tolerant shape: RAID10. In this case, I went with Linux’s implementation of RAID10 using MD. I ended up with RAID5 after hosing my RAID10 array and losing all data on it. Not the best day ever…
Although having a RAID10 array means you get less storage space than RAID5 for the same amount of disks but greater redundancy, you can’t expand your array (not in MD’s current version) if you decide to add bigger disks.
Because I borrowed 2 x 500GB Seagate drives to put together with my own 2 x 1TB drives, I knew I’d have to give it back at some point. And if I wanted to buy more disks, I wouldn’t want to be limited to buying 500GB disks. Maybe when I finally decide to buy the other two disks, they’ll have 3 or 4T disks available. Growing (expanding the size) of the array was then considered a ‘must’. And only RAID5 growing is available in the current implementation of MD.
Since the main purpose of this project is to store and share TV shows on my private network, I need, well, TV shows. I’m a huge fan of House and Top Gear, amongst others. I occasionally watch a movie as well, but the bulk of my TV-watching is composed of TV shows.
For viewing this content, I have an Apple TV (don’t flame me, bro!). And since it is an Apple product, it only accepts videos encoded to Apple’s standards (h264, basically). Great. Except when ALL of the Internet (ok, not quite) only has episodes available in DivX/XviD/x264 formats. So to watch a downloaded episode of a TV show, I’d have to do the following:
- Convert the video to h264 using HandBrake (I’m using the CLI version for scripting purposes).
- Tag the video file using AtomicParsley and a clever Python script.
- Move the converted and tagged .M4V file to the Automatically Add to iTunes folder on my server.
- Wait until iTunes on my MacBook Pro picks the show up and adds it to its library, then syncs it to Apple TV.
From this point, I can watch the recently downloaded/converted/tagged/synced episode on my Apple TV at any time.
Now you can see that if the process above had to be done manually, I’d simply give up. Instead, being the geek I am, I decided to automate it. From when the TV show episode is first made available on the Internet to downloading to converting/tagging to it finally being available to watch on the Apple TV, I didn’t want it to have any manual input from me. This required work.
Stay tuned for part 2 of this (hopefully coming sooner than this one took to be published)! Second part now available!