Well, it had to happen sometime. I can only say that I must be blessed or something to have been editing on my own system for…what, 6 years now?… and not one time did I have a drive fail on me in the middle of an edit. I have had failures AFTER I delivered. Or when I was loading something. A G-Raid here, LaCie there…bare Hitachi internal one other time. But not once DURING an edit have I had a RAID fail.
Until today. Er…yesterday (It is 3:14 AM as I write this).
I was working on network notes from a project that I am doing for the History Channel. I have a copy of ALL the footage at home…because the office where the production is located is in Long Beach, and I didn’t want to do the commute every day. Luckily, I have understanding producers. So I copied all of the footage onto a four drive SOHO RAID…set up as RAID 0. Yeah, that’s dangerous. I know. But I happened to have four 1TB drives sitting on my shelf, and this project took up 3.24TB of space…so I couldn’t do the RAID 5 that I wanted to. So I went with the risky RAID 0. Until I could afford to get four 2TB drives in there.
OK…so I am working along when I’d get some random crash. It wouldn’t be consistent. Well, the ERROR would, but when it happened wasn’t. Playing in the timeline…crash. Opening a clip in the Preview monitor…crash. Scrolling through the clips in the bin in SCRIPT VIEW…crash. (I am running Avid Media Composer 5 on this…BTW).
But then something else started happening. My media was starting to vanish. Files there were there before suddenly started coming up MEDIA OFFLINE when I hit them in the timeline. I turned on CLIP COLOR to reflect OFFLINE MEDIA. Boom…holes everywhere. In all my sequences. Footage I KNOW was there the day before. It wasn’t 2 hours later when suddenly playback stopped. I’d see the footage move for about 2 seconds, and then the picture would freeze…I’d get 3 more seconds of audio…then beachball. Damn.
Well, to try to make this long story shorter…I ran Disk First Aid on the RAID that said that it was unrepairable by it. So I ran DISK WARRIOR…and that check ran really slow…and said it was due to hardware errors. The amount of hardware errors ticked off like seconds on a watch. 1…2…3… 56…57….58… 115…116…117. Wow. After about 30 min Disk Warrior stopped and said there were A LOT of issues. It gave me the option to mount a proxy directory to allow me to copy all of the footage off the raid. Which I did.
Or rather, attempted to do.
In copying over the OMFI media folder (I had OMFI and MXF folders)…a thing that was about 3.04TB of footage…I got an estimate of 71 hours to complete this process. WOW. Really? That hardware failure must be bad. I tried one of the MXF folders (I have eight), the smallest one that was 105MB. That took 25 min to copy. Man, at this rate I wouldn’t be done until Halloween maybe….Thanksgiving. I had 3.24TB of footage!
I know…this isn’t that bad. This was a BACKUP of the footage that is located at the office. But copying that took a long time. And made copying WHILE editing tough. And I couldn’t let it just go over night because I had to do it in small chunks. LARGE chunks popped up copying errors (I just mention this to head off the obvious COMMENT questions I feel I might get).
SO…I mention this situation to a friend of mine. A REAL tech head. And he gave me a tip.
“Open up Terminal…”
“Terminal? That old school text interface tool? Seriously?”
“Yeah…that’s it.”
“Dude…I hated computers back when I was forced to use MS-DOS and do all this command line stuff. I can’t picture how this all works. My mind doesn’t work that way.”
“Hey, this is simple. REALLY simple. Trust me.”
“OK…”
He proceeded to tell me to type in a command, then things into the terminal…
“Wait…DRAG and DROP into the Terminal? You can do that?”
“Yeah…it’s pretty cool. Now come on…follow me.”
I didn’t get it at first. He isn’t the best guy at describing what to do. So he made a quick screen capture of the task I needed to do. OH…it was simple. Just for yucks I’ll make one and put it at the end of this blog post…for those of you who are like me. Need to see it.
What he had me do is type in “cp -r”, then press the space bar, then drag the file I wanted to copy onto the terminal…then the location folder where I wanted it to go…and press ENTER.
BAM. Lightning fast. Well, not FAST…but a bit faster than dragging and dropping in the Finder works…when it works properly. Why? Well, as he puts it: “It doesn’t copy over the finder directory. Is just brings over the files.” OK man…as long as it works. And it does. VERY fast. Now, there is no PROGRESS BAR to show you, well, the progress. Files are just being moved and you just need to know that they are going to where they are going. In opening the destination folder I could see the item number increase, and the GB available number decrease. That’s how I knew it was working.
WHY do this? Well, when the Finder gets stuck on copying a file…it STAYS on that file until it can copy it, or until it crashes. The Terminal copy hits a bad file, shoves it aside and continues on with the rest of the files. It puts the bad file at the end of the line. When it gets to it again, if it can’t copy it…if the file is too damaged…it skips it. What a GREAT way to root out bad files!
NOTE: You should use this ONLY in the case of emergency…like mine was. Mucking about in the terminal, when you don’t know what the hell you are doing (like I don’t), can be disasterous. So…use this only as a last resort. Unless you are a Terminal Geek.
OK…as promised…the visual representation.
Copy files using the TERMINAL from Shane Ross on Vimeo.
