Scalable Storage and Backup VI

Stripped.

I've renamed my 2 most recent posts with the "Scalable Storage and Backup" header so all 5 previous posts should be searchable if you want to read about this crazy journey.

You can also read about the original set of posts under "File Storage Upgrades" via the search bar on the right.

Let me reprise the premise. I have a lot of media that I need to access. My first and foremost concern is security and secondly speed.

The advent of transfer protocols such as USB 3.0 and Thunderbolt changes the existing playing field of USB 2.0, eSATA, and FireWire 800.

When I started my journey, Thunderbolt and USB 3.0 did not exist. Neither did 3 or 4TB hard disks. The set of scalable storage posts I'd previously written utilized a single OWC Qx2 enclosure (4-bay RAID 5) as the primary media server and 2 OWC Mercury Elite (2-bay) enclosures. The idea was to use the 2-bay enclosures as off-site backups that I would rotate every 2-weeks. The 4-bay enclosure contained 4 x 1.5TB drives and the 2-bay enclosures used 2 x 1.5TB drives.

Therein lies the initial pickle. You can't backup 4.5TB (1 drive is for redundancy) with only 3TB.

So that system quickly "broke". Initially I had less than 3TB of data to secure but as my media server content outgrew my backup capacity,  I had to make executive decisions about what data I wanted to omit from the 3TB off-site backups. It started with the backups of my boot drive and my SSD scratch disk. But quickly it became that I didn't backup the media server at all because the 2nd problem was that it was a pain in the ass to maintain off-site backups. Months would go by before I'd update the backup and rotate the 2-bay enclosures. The system was broken.

Not only was the system broken but the "scalable storage" system that I'd created was not very "scalable". The OWC Qx2 4-bay enclosures are typical RAID enclosures. You toss in some drives, format the RAID and use the enclosure as a finite non-expandable RAID array. Well, with video files I quickly outgrew the initial 4.5TB 4-bay Qx2 setup as well. Seeing a sale on OWC for refurbished units, I purchased a second OWC Qx2 4-bay enclosure. Then I stripped the old 2-bay enclosures of their drives and created a 2nd OWC Qx2 with 4.5TB of space.

But I had no off-site backup. Hadn't had off-site backups for over a year now.

And then the problems with the 2nd OWC Qx2 began. The drives in each OWC Qx2 were different. In the first Qx2 I had 4 x 1.5TB Western Digital Caviar Greens. In the second Qx2 I had 4 x 1.5TB Seagate Barracuda 7200.11 drives. Both are NOT RECOMMENDED on OWC's website. Without repeating my post, I have now returned 12 or so Seagate Barracudas back to Seagate. They've been nice enough to send me new drives.

But we are just getting started. Below you'll see what Seagate has been sending me. There are 3 drives in the image below and if you look closely they are 3 different model drives.



1 x 1.5TB Barracuda ST31500341AS 32MB cache
1 x 1.5TB Barracuda ST1500DM003 64MB cache
1 x 2.0TB Barracuda ST2000DM001 64MB cache

I started with 4 x 1.5TB Barracuda ST31500341AS 32MB cache

Most RAID arrays (with exception of the Drobo) require that you use the same drives with the same firmware for maximum reliability. Now, I'm sure there are exceptions to the rule, but surely you can not pop in a 2.0TB drive into an array built upon 1.5TB drives and expect proper redundancy. The drives need to be the same size (unless you have a Drobo).

But as long as only 1 drive failed at a time on my new Qx2 unit, I was happy to swap them out, send them back to Seagate and receive a new drive. I got a little suspicious when the drives they sent back to me showed up with new stickers saying "Certified Repaired HDD" but I didn't realize the drives were actually different until my last post.

But even that was tolerable as long as only 1 drive failed at a time. I'd gotten so used to hearing the Qx2 beeping every month or so and sending back the failed drive until...

Saturday morning June 29th catastrophic failure raised its ugly head. Hearing the typical beeping of a faulty drive from the Qx2, I walked into the office to find that 2 Seagate drives had failed at the same time. The 2 bays that I'd been repeatedly pulling broken drives from (bay A and bay B) had both taken a shit on me overnight. My heart dropped to my stomach and honestly didn't know what to do. This was one of the calculated risks I'd been running by not having a backup of the Qx2. But if you play with fire eventually you'll get burned. It's just a matter of time. I had no protection from losing the content in the enclosures. What if the enclosures had been stolen? What if the house burned down?

So I did my best. First I rebooted the enclosure and that bought me some time because the enclosure did not immediately recognize that the failed drives were faulty. This allowed the partitioned disks from the Qx2 to mount on Mac OS. I grabbed a few single-disk USB enclosures and began copying over all the data I could off the Qx2 and onto the single-disk drives. Fortunately this Qx2 housed "secondary" data. Mostly videos but also my entire archive of full-layered Photoshop files for the past 3 years.

2 days later, I had finished copying everything over to the single-disk external drives. I was very fortunate that I only lost the most recent webinar (2 videos) that I had already uploaded to Vimeo. So basically I didn't lose anything at all. I got lucky. But throughout the 2 days, the 2 faulty disks would "flare up" and and on several occasions cause the entire Qx2 enclosure to unmount. I later discovered that this only happened when I tried to access the 2 webinar files so by leaving them alone, I was able to backup everything else.

And all of this gets me only to the beginning of this post and the picture above. An empty Qx2 enclosure (I pulled the 2 good drives too). I just packaged the 2 faulty drives for warranty repair/replacement with Seagate. But I'm lacking redundancy for the saved content. I need more space. And the current "backup" system has long been broken.

So what do I have at my disposal?

5 x 1.5TB WD Caviar Green
2 x 1.5TB Seagate Barracuda ST31500341AS 32MB cache (7200.11)
1 x 2.0TB Barracuda ST2000DM001 64MB cache
2 x x.xTB Barracuda Certified Repaired HDD
2 x OWC Qx2 4-bay enclosures (eSata/FireWire800/FireWire400/USB2.0)
2 x OWC Mercury Elite 2-bay (eSata/FireWire800/FireWire400/USB2.0)
a few smaller capacity 2.5/3.5"HDD, and 2 external USB2.0 HDD

What have I decided?

Phase 1

4 x 3TB Western Digital Red in OWC Qx2 4-bay enclosure
1 x Drobo 5N with assortment of drives above

I'll keep 1 Qx2 unit and retool it with 4 x 3TB WD Red drives. These are supposed to be good in RAID arrays and should play well in either a Drobo or the Qx2. The new Qx2 drive will have 9TB capacity and can house everything originally on the 2 OWC Qx2 units.

I'll setup the Drobo 5N as the off-site NAS and outfit it with the largest capacity drives I have left. Even at 5 x 1.5TB I will have minimally 5.4TB of space. If 5.4TB isn't enough, I'll buy whatever I need to cover my backup needs. The benefit of the Drobo is that it's relatively fast (Gigabit Ethernet), expandable (swap in larger drives later), you can mix-and-match drives, and with Carbon Copy Cloner and Gigabit Ethernet I won't have to think about maintaining backups. A truly expandable and automated solution.

Whatever drives I have left will serve as external scratch drives. I will keep the other OWC Qx2 4-bay enclosure and the 2 x OWC Mercury Elite 2-bay enclosures until after Phase 2.

Phase 2

1 x Drobo 5D with 5 x 3TB Western Digital Red
OR 1 x Pegasus R4/R6 with 4/6 x 3TB Western Digital Red
AND
1 x Drobo 5N with 5 x 3TB Western Digital Red

The 5D is the same as the 5N but with Thunderbolt. The Pegasus R4/R6 is supposedly the reigning speed king of RAID Thunderbolt enclosures. I will be considering either of these units as the primary media server for storing and accessing photos and videos. Combined these units should suffice for media server and backup needs. At which point I'll have to figure out what to do with the Qx2 units as well as the 2-bay Mercury Elites along with all the 1.5TB drives. I suspect that I can find a use for them in our family computer and retool the Mercury Elites as Raid 0 eSata scratch drives.

Why not just skip all the fuss and go to Phase 2 right now? Simple. I don't have a Thunderbolt capable computer. I am using a Macbook Pro from mid-2010 and neither does it have eSata nor does it have Thunderbolt. Quite sad isn't it? I've been hooking up my drives via FireWire 800 and USB 2.0 *cringe. Since it's become imperative that I upgrade computers in the near future, I need a storage system that I can grow into. So buying the 5D or the R4/R6 right now won't do me any good because I'd be forced to connect to it via USB 2.0 which would very likely cause me to go mad. Actually the R4/R6 doesn't even have USB connectivity.

Why go back to Drobo? Ah, yes. I've had a Drobo before and it was nothing but good to me. Never a single problem except that it was slow so I replaced it with the Qx2. I'd still have the same predicament of not having an adequate backup solution though.

So there you have it. Another theoretical system that will hopefully stand the test of time. Hard to say really but it is what it is.