PDA

View Full Version : Photographer's G5 config & backup strategy



mhvik
04-02-2006, 11:22 PM
I'm posting because everyone seems to recommend MacGurus. I have a dilema which I know is not unique, but I haven't found many great answers elsewhere. So here goes:

As a working professional photographer I shoot 63.5 Mb RAW files using Capture One software. I don't shoot a huge number of files for each job, but enough to make me have storage concerns. I keep track of current and old assignments using iView Media Pro. I use Photoshop for light retouching, and I save the final files in layered TIFF's. After an assignment, I move the Capture One session folder with all of the original RAW files and final TIFF's to an external drive. Those drives are backed-up by a simple drag n' drop method to a duplicate drive.

One big problem is that I've been relying on external LaCie 250 d2 drives for the inactive/completed assignments. I've been learning about LaCie reliability the hard way, as 3 of my 8 d2 drives has either failed or is failing.

As I grow into a new workflow of file management, it seems that I need to find a better solution. I've read about RAID, large FireWire devices, NAS, and eSATA (like the Burly). I'm just not sure what my best approach is here.

I need a way of accessing old assignments a few times each week, but I don't necessarilly need constant access to those files. I'd like the solution to include one device or box, so I don't have to keep plugging and unplugging external HD's. I have been leaning towards the MacGurus Burly setup because it seems like a RAID would offer some protection, and the eSATA might give me a fast enough connection for a good PS scratch disk.

I've heard good things about NAS, but I'm not sure if I want to get into the configuration of such a thing, and I don't really need constant access to my files over a network or the internet. Gigabit ethernet seems fast, but it cant be faster than a Burly eSATA setup, right? The FireWire products seem expensive per Gigabyte.

I'm really leaning toward the MacGurus eSATA products, maybe the 1.6 TB or 2 TB Burly and a card. I'm just not sure about port-multiplier vs. not port multiplied? Which RAID, if any would make sense? If I needed the BUrly on a network, I could easily share the files through the G5, right?

I would appreciate any thoughts on a good approach here, or any advice from similarly situated person.

My setup is:
Dual-Core 2.3 GHz G5
5.5 GB RAM
nVIDIA GeForce 6600 driving a 30" display
2 internal 250 GB drives:
1 with applications,user profile, and current assignment.
1 other internal 250 GB drive for Photoshop scratch disk

Oh yes, sometimes I network the G5 to a G4 laptop...

Thanks in advance.

Damien
04-03-2006, 05:51 AM
Well first, Striped raid is very fast but not a good solution for storage/backup. Mirrored raid is no faster on writes than a single drive but you get an instant and always updated backup very easy! When you mirror remember that 2 400gig drives give you 1 400 gig mirror

Just because you have one big Burly box does not mean it has to be one big drive. Say you bought an 8 bay burly and had 8 400gig drives in it. You could make an 800gig stripe for VERY fast scratch space and use the other 6 drives as a mirrored raid of 1.2TB raid. Or use them as single drives for 2.4TB of space.

You will have to wait for Rick (or someone else) to get here for the details on Port Multipliers. My suspicion is that you will drastically cut your speed if you plan to have more than 3-4 drives in a raid stripe. (1-2 drives in you have SATA 3-4 if SATA II) simply because SATA is 150MB a sec max and SATAII is 300MB a sec and the PM boards run 4 drives through one SATA connection. That is my suspicion anyway... wait for those more knowledgeable than me on this one ;) PM tech is very new and I have never worked with it.

unclemac
04-03-2006, 11:23 AM
Hi and weclome.

Only have time for a quick thought. Besides storage space, which we will hash out, you should also have a backup plan too. RAIDs fail, hard drives fail, floods fire and theft happen. How important is the data, and what is it worth to you?

If it really valuable, then you need to consider a backup setup over and above your primary storage. Best options include something you can take off site on a regular basis, or at least disconnect and put in a fire-proof safe.

ricks
04-03-2006, 12:24 PM
First off, Port Multipliers are NOT slow. There is a maximum throughput per cable, about 225 MB/sec, but as drives fill up the 4 or 5 drives attached to a single data cable are going to be slower than that cable. WHile the maximum performance from EMPTY drive is a bit slower on a Port Multiplier as compared to a direct connected system, the drives themselves perform identically except in striped RAID configuration when empty. The only time you would notice much of a difference would be in a scratch disk RAID where every little bit of speed is essential. For everything else the speed is marvelous.

mhvik,

Please, first and foremost, RAID is not any kind of backup! A parity RAID gives very limited protection form a drive failure. There are loads and loads of other ways to lose data! You must plan a backup as the primary focus of a storage plan.

If it is backed up to another location a RAID0 is nearly as well protected as a RAID5. So we mostly sell pairs of drives for that purpose: for instance a 4 bay backed up to another 4 bay. Each in a RAID0 for both speed and volume capacity.

You can also do singe drives backed up to single drives. Just like you are doing with the LaCie drives only to more reliable SATA enclosures. Using Hotswap enclosures you can add in the ability to pull your backup and put it in storage as needed, replacing it with new empty drive and a replacement tray.

One way or the other we can get you set up with a workable and reliable system. The Port Multiplier enclosures are so versatile that they have really changed our conversations about storage. Because we can keep adding drives to a single host card it is able to expand with your needs.

Rick

Damien
04-03-2006, 05:16 PM
See I told you... Rick is a know it all err... I mean Rick knows it all yeah that's what I meant

A.D.
04-05-2006, 12:47 AM
I've heard good things about NAS, but I'm not sure if I want to get into the configuration of such a thing

Mark,
I second unclemac and Rick's insistence on a separate backup strategy. If you're otherwise interested in a NAS for the backups, take a look at http://www.digitalimagearchives.com/imagevaultnyc_how.html for a configuration-free solution.

unclemac
04-05-2006, 01:23 AM
That looks like a good option AD. Built in incremental backup is a "nice to have" feature for any SOHO.

One thing to note if we get into RAID 5 setups.....it takes at least three drives at a minimum *and* if you lose more than one drive at a time - you're done. Bye-bye data.....or hello very expensive data recovery. Part of the reason the enterprise grade stuff is commonly configured with a hot swap spare. One extra (empty) drive always waiting to automatically come on line to replace a failed drive. Cause once you lose one, there is no cushion, and no time to waste getting a replacement up and spinning.

Now, this is likely a really rare event...but... Say you lost two or more out of four (or 5, 6, whatever) drives in an array, then you would still need another backup. Out here in CA, a midsize earthqake could drop an unsecured running RAID box on the floor. The definition of a bad day.

Even though it is not really cheap or easy, I still lean towards tape for a real archive that is portable, and offline most of the time. In a perfect world, we would all have:

1) on line data (current local or network shared data)
2) near line data (daily backups to a local file server or NAS)
3) far line (remote file server with nightly/weekly backups in a differnt building location) or off line data in a tape vault some where safe

At a minimum.

Since most cannot aford such things, we have to come up with the safest, most cost effective compromise.

A.D.
04-05-2006, 04:03 AM
Good points. And in a really perfect world I'd add
#4 Someone to manage all this stuff (for free) so we don't have to devote so much of our creative time to being ITs :)

unclemac
04-05-2006, 08:53 AM
Absolutely.

mhvik
04-11-2006, 02:02 PM
If it is backed up to another location a RAID0 is nearly as well protected as a RAID5. So we mostly sell pairs of drives for that purpose: for instance a 4 bay backed up to another 4 bay. Each in a RAID0 for both speed and volume capacity.

You can also do singe drives backed up to single drives. Just like you are doing with the LaCie drives only to more reliable SATA enclosures. Using Hotswap enclosures you can add in the ability to pull your backup and put it in storage as needed, replacing it with new empty drive and a replacement tray.

One way or the other we can get you set up with a workable and reliable system. The Port Multiplier enclosures are so versatile that they have really changed our conversations about storage. Because we can keep adding drives to a single host card it is able to expand with your needs.

To everyone,
Thanks for all of your good advice. To take this concept to the next step, would it make sense to purchase something like the Sonnet Tempo E4P PCIe SATAII Host Card (http://www.macgurus.com/productpages/sata/PCIeSATAHostCards.php) and a 2 TB Burly 8 Bay Rack Mount PM Enclosure (http://www.macgurus.com/productpages/sata/BurlyPortMultiEncl.php) with 8 Hitachi 250 GB drives? From what I understand, I could then set it up as 2 separate RAID 0 volumes. The 1st RAID 0 volume could be backed up to the 2nd RAID 0 volume with something like ChronoSync (http://www.econtechnologies.com/site/Pages/ChronoSync/chrono_overview.html) ? ChronoSync'ing two volumes might help prevent accidental deletions or file corruptions possible with a mirrored RAID?

Optimal Configuration:
I could use one of my internal 250 G5 drives as a boot volume, with user profile, and current/active projects. Use my second internal 250 drive as Photoshop Scratch disk. Use one of the Burly's RAID 0 volumes for completed projects, and a completed projects backup in the Burly's other RAID 0 volume. I could then use my older Firewire 250's for offsite backups for extra backup protection?

Does this sound like a good way to address this?

Thanks again for the insight.

catfell
08-21-2006, 08:43 PM
mhvik,

I too am a professional photographer that uses Capture One Pro mostly, with minor photoshop fixes. I don't use any sort of cataloging/digital asset management program yet.

My "cheap" backup system is simply a two bay hot swappable external SATA enclosure. I have one drive plugged in all the time as my working "hot" drive, and another drive of the same capacity nearbym but not plugged in.

When I make any changes to the first drive, mostly after any shoot or after editing and processing a job, I mount the second drive and back up the first. Then I unmount the second drive.

I have had many lacie fw drives die on me, and have often been able to recover quite a bit of data from the dead drive. But if a raid fails, data recovery is a much more difficult thing to do. So I prefer (cost aside) single hard drives instead of having a RAID.

I have not tried using a striped RAID, but have been very happy with the speed of my processing and backing up, since adding the SATA drives to my main processing computer.

My hopeful strategy in the future is to get a Macbook Pro, and the SATA adapter for the expresscard3/4 slot, which would allow me to share any of the SATA drives in the external enclosure between my ancient G4 desktop and the newer (and not yet purchased) Mac Book Pro.

Once I fill up a primary hard drive to 85% capacity, I back it up and offsite the backup, and plug in the primary only when I need to access it.

As I am sure you are discovering, you are constantly buying hard drives to store all your work. I decided that it is easier to have matched pairs of drives, where I can buy more when I need them, instead of setting up a RAID that will fill up eventually, and then have to keep track of all the drives that make up the now full RAID.

This is just my 2, and I realize that there are a million ways to skin the same digital cat.

My other hope for the external hotswappable SATA drives is that since Capture One Pro is a Universal binary, processing with a Macbook Pro should be a lot faster than using my G4. But I don't want to keep a lot of my images/files on the laptop, so I hope with the new laptop to be able to have the following workflow:
1. Shoot
2. Download to laptop
3. Transfer job to external SATA
4. Edit and process job on SATA
5. Trash captures from laptop

I do wish I had bought a 4 bay SATA enclosure instead of the 2 that I got, so that I could have dedicated two bays to each computer.

Hope this helps.

-catfell

unclemac
08-21-2006, 11:23 PM
Thanks Catfell.

Always good to hear what real users in the real world are doing, and what works for them.

Disc2
08-22-2006, 02:00 AM
Image files are getting larger and larger everytime and sometimes also ran out of solution of backing up files. I have several raid which are almost full and no time to catalog or view the files. I shoot over 2000 images a weekend and go crazy locating files.
I started to use the Gold DVDs not too long ago which has archival quality for 60 to 100 years.
As a photographer you need 2 backups for your backup, hard drives fail on me especially those Big Lacie firewire drives, they get too hot. I still have them but usually unpluged if I dont need the files. Went back to my old Seagate SCSI raid that I use most of the time.
I would like to get those new SATA swappable to store jobs and backups. But for now my solution is 2 fireproof safes and 2 copies in Gold DVD for each client plus same files on 3 different SCSI raid.