Not long ago…January 2007…I was at MacWorld in San Francisco. There was a LOT of buzz going around about a great many things. The iPhone was announced, Leopard was demoed, a company converted a MacBook into a tablet computer…there was no end to the cool things to see. But all the companies that were there showing off storage solutions…RAID solutions…were hit by one big question.
“Does this do Raid 5?”
Well, I can tell you, that a VAST majority of the answers were “Sorry, no. Only Raid 0 or Raid 1.” This turned many people away. The only people who could say “Why yes, we do!” were the Fibrechannel solutions. But, as you might suspect, those solutions were EXPENSIVE. Restrictively so for many of us.
My how technology flies.
Here we are…8 months later…and the companies who were at MacWorld (and NAB) heard the cry. One such company was CalDigit. Their offering, the HD PRO, offers what everyone desires…hardware Raid 5 support. For a device to offer “Hardware RAID” it must have 2 things: An IO Processor (CPU) and a Cache Memory. This unit has both.
What is Raid 5? Well, let us first start with the lower numbers.
RAID 0 is what is called PERFORMANCE RAID. What you do is take two or more drives and “stripe” them together so they act as ONE DRIVE. “Striping” essentially means you format multiple hard drives together as one large “virtual” hard drive for your system.” For example, if you take four 250GB drives and stripe them RAID 0, you’d have a 1TB Raid. In doing this, you spread out the information that is being written onto all the drives so that you get faster data transfers. This means that you can capture uncompressed video formats and play them back. The more drives you add, the faster the raid, the better the performance.
RAID 1 is known as a PROTECTION RAID. You take 2 or more drives (need to be in pairs) and group them so that they again appear as one drive, but this time the information is MIRRORED. If you took two 250GB drives and striped them as RAID 1, you’d have a 250GB drive, but the information would be secure because the information on one drive is copied to the other drive as well. The drives are exact duplicates of each other. The good thing is that your media is safe, the drawback being that your drive performance isn’t as great, and you cannot really work with uncompressed formats.
RAID 5 is the best of both worlds. Performance with protection. RAID 5 stripes both data and parity information across three or more drives. It doesn’t have one dedicated parity drive that contains all the information, but rather distributes the data and parity blocks across all the drives in the array. “What the heck did you just say Shane? What does all that mean?” To tell you the honest truth…I don’t understand the inner workings of the engineering. I am not an engineer, I am a creative editor. But I DO know that it basically means that your footage and the backup information is not on one drive, but spread across many. So that if one drive fails (and drives are known to do this, more often than we care to admit) your information is still safe.
If you want to know more about the RAID types, go to the Wikipedia. Enough with the schooling…let’s get to the HD Pro itself.
First off, I must mention that currently the HD Pro ONLY WORKS WITH PCIe MACHINES (PCI-X support due by mid-September I hear). This means it works only on the newer MacPros, and the older G5 Quads withe a PCIe slot. Now, I don’t have one of these machines, I have a PCI-X G5 with a Kona LH capture card. I needed a PCIe machine with capture card in order to do this test. So for the purpose of testing, I was loaned a MacPro (Dual-core 2.66Ghz model) with a Decklink Multibridge Pro capture card.
When I went to unpack the unit, I encountered something similar to what I encountered when I unpacked the S2VR Duo unit. The hard drives were not inside the unit. They were all tucked into little cut out sections of the packing foam that encased the drive chassis. This is clever, for it helps prevent damage that can occur by jostling the unit. We know how gently delivery services are with packages, right? So I pulled out all the drives from their safe cubbies, removed them from the plastic wrap, and installed them into the chassis. They were all conveniently labelled A0, A1, A2 so I knew what order to install them.
Another interesting thing is that the card that you install inside the MacPro is a VERY small and basic card. It is a 4-lane one port card, with no real electronics to speak of. The card doesn’t have the controller on it. Nope…it is simply a BRIDGE from the board to an external connection that a PCI Express cable connects to. The controller is INSIDE the chassis. To quote the website, ” This direct link eliminates latency introduced by the conversion of other interfaces to PCI Express and provides superior bandwidth, availability and deployment flexibility over earlier-generation SCSI and Fibre Channel technology.” The reason for this is two-fold. First, it doesn’t contribute to the heat inside your MacPro…since you will have plenty with your DOUBLE-WIDE graphics card and your capture card. And second, it allows for controlling MULTIPLE units with the aid of a PCIe switch (not out yet). Meaning that your machine can control more than one HD Pro box…you can add more units as the need arises. Just like I can now with my eSATA boxes and my port multiplier card. Room to grow.
The test unit I received was a 4TB unit. Eight 500GB drives. When I fired up the unit it appeared on the desktop, ready to go. It was already configured…Raid 5 and everything. So I ran the AJA test with the following result:
331 Write / 372 Read. RAID 5!! Plenty fast for uncompressed HD…a couple streams of uncompressed HD. Niiiiice. But, me being me and not trusting the integrity of a drive format after shipping…and, well, to see what is involved in setting up a RAID 5, I did it myself. CalDigit includes a handy little pamphlet on how to do this, so I won’t go into the boring details. But I will mention that it did take 3 hours to build the RAID…which is slightly faster than I hear RAID 5 takes to build on an XServe. Do note that the size of your Raid determines how long it will take to build. 8TB will take longer to build than 2TB. Makes sense. So, I let it do it’s thing while I ate dinner, played with the kids, and tucked them into bed.
When the building was done, I ran the AJA test again:
311 Write and 402 Read….again, niiiiice. The Write was a little slower, but who really needs more than 250 anyway? I am not CAPTURING multiple streams…so 300 to 340 is fine. The Read increased, which was cool. And I suppose the average will be about 320 Write and 385 Read…which is VERY nice indeed.
I captured a few different ways. First DVCPRO HD native via firewire. No sweat…but I can do that with my G-Raid. So then I captured uncompressed 8-bit 720p…no problems. Uncompressed 10-bit 720p…no problems. Uncompressed 10-bit HD…passed again without one dropped frame.
OK, time to abuse this thing. Again I set it up to capture uncompressed 10-bit, and about 5 minutes in…I YANKED A DRIVE OUT! Just so you know, this isn’t something you can do by accident. There is a pinhole that you need to push an included tool into (or a paperclip if you lost the tool) and that pops open the handle that you then use to disengage the drive and pull it from the chassis. When I did this it did give me this nice little warning:
But, it kept merrily match capturing…not missing a beat. When it was done I saved my project and set about fixing what I intentionally broke. I pushed the drive back in and went into the RAID SHIELD software and unlocked the drive and the software went about rebuilding the Raid. Again, that took about 3 hours. But the footage I captured was still intact and played fine.
OK, now let’s look at playback. I was able to get 9 streams of DVCPRO HD playback (both firewire and Decklink encoded DVCPRO HD)…I resized my footage to look like the opening to THE BRADY BUNCH. The first 4 streams gave me no render bar whatsoever. When I hit 5 streams I encountered the dark green render bar, which means high quality playback. Eight streams got me the light green render bar….low quality playback…and that held up until I added the 10th stream…when it went RED. So i stuck with nine. Since it was getting late, and I had work in the morning, I just let it play this footage as I went to bed, setting Final Cut Pro to loop the footage(did I mention I was testing this with Final Cut Pro? No? Sorry).
I woke up, went to work, came back home, had dinner, then went “oh yeah, I have something going on in my office.” I turned my monitors on and there the footage was, happily playing along. No dropped frames.
But hey…seriously….DVCPRO HD footage? Was that a REAL test? OF COURSE NOT! So I went to the 8-bit uncompressed 720p footage. I started layering that. I got the dark green render bar after two layers, light green after 3 and the red render bar when I hit five. So four streams it is. I tried 10-bit uncompressed and couldn’t get more than one. But really…uncompressed 10-bit…I am not offline editing with this codec. I am FINISHING with it, so one stream is plenty. I am offlining with DVCPRO HD, and getting nine RT streams is a nice thing to have. OH, and I was able to add the 3-way color corrector to seven streams and stay RT.
Time for bed again, so I resized the uncompressed 8-bit footage into four windows, and pressed play. This time I let it run for 3 days. Every time I came back, it was running. On the third day when I got back from work, I walked in, opened the unit, and yanked a drive. I turned on the monitor and again, saw the warning, but playback wasn’t affected. Kept going and going and going. So I let it go. Went to bed. Woke up the next morning to see it still humming along…footage playing. I stopped…saved…quit…and set the raid to rebuild again. And again, when it was done, all my footage was still there…safe and sound.
The HD Pro unit was running for the entire 8 days that I had it…I didn’t turn it off. When I finally did, and I removed a drive and felt it…is was warm to the touch, but not overly hot. The RAID SHIELD software has temperature monitoring capabilities, so you can always check and see how hot things are. OH, and I forgot to mention that you can set up e-mail notification so when a drive does happen to die and you are…somewhere else…you’ll get an e-mail telling you a drive failed. Very handy.
So I am rather impressed with this. And I would easily consider it for use in my edit bay. The piece of mind that RAID 5 gives you is something EVERYONE should experience. Currently I am running a RAID 0 box, and have all the footage backed up to a couple firewire drives. A sort of manual RAID 1. With this box I wouldn’t need to waste the drive space. It would be an invaluable asset to any edit bay.
OH! OH! OH! AND…oh I love this…AND CalDigit makes an Express34 card for this unit, so you can use it with your MACBOOK PRO! You won’t get nearly the numbers you get with the tower, but the numbers you do get are QUITE impressive:
168 Write and 195 Read. PLENTY for uncompressed SD and even uncompressed 8-bit HD work. And for DVCPRO HD and the new ProRes format, that would be plenty for several streams of playback…ON A LAPTOP!
Man…time for me to get a MacPro…or MacBook Pro.