PDA

View Full Version : Long-term archiving strategies



TeeH
01-17-2006, 11:57 AM
Wondered if we could get a discussion going about the merits of different "deep" storage strategies for a large image file archive?

I currently use removable SATA drives that backup the working drives, and am about to put in place a second tier of security, to be stored off-site.

DVDs are an option that's been suggested as more bulletproof than HDs. We're talking a collection in the tens-of-thousands of hi res images, so the burning time alone is daunting. Not to mention access and restoring in the event of a disaster.

I've also heard very a compelling case made that modern HDs are so reliable that their inherent vulnerabilities are almost moot, especially if treated carefully (key word: almost... ;) ) Without going completely obsessive/compulsive, remember there will be two backups of everything.

No simple answer, I know. And anything done now would obviously be re-visited in the future if new technology were to offer a better answer. Anyone have other ideas?

Tom

bif
01-17-2006, 12:30 PM
I am currently examing this very question regarding home video files. For myself, cost is also a consideration but I have already run into an issue that gave me a scare. I had backed up to an external FW drive and upon trying to reimport the video onto my powerbook for further editing, I found the data corrupted. The external hard drive copy was OK, I transferred it to another machine and edited there. It was somehow damaged during the FW transfer to the powerbook, and I tried this three times, the final time I erased free space prior to copying the file. My experience could have been an issue with the powerbooks HD, it is a 5400 rpm Toshiba, but I am going to use DVDs for back up if I can stand the time consuming process of burning all those DV data files.

eric
01-17-2006, 01:00 PM
I just read this article (http://computerworld.com/hardwaretopics/storage/story/0,10801,107607,00.html) the other day urging magnetic tape or UDO optical disks for really long term solutions. But as noted here (http://www.techworld.com/storage/features/index.cfm?featureid=2159), the expert does work for IBM who makes products for both those types of solutions -- and existed the HD market.

Nicolas
01-17-2006, 01:45 PM
Hello,

I am working at a prepress company so we have much of clientdata
which is needed twice a year (for making a new catalog for exapmle),
with all the images and stuff.

We had Optical storage arrays (MO's) also Tape libraries (streamer) but the best and reliable storage we have now.

One big/fast fibre>fibre RAID5 array and a "slower" SCSI>SATA RAID5 array both backuped by older RAID5 arrays. Good stuff but $45.000+ without a computer.

If you need the stuff online there are non other options than running a RAID.


Regards

Nicolas

unclemac
01-17-2006, 09:24 PM
Cheapest, biggest, fastest, portable soluiton I have found yet. $5000 (http://www.exabyte.com/products/products/get_products.cfm?prod_id=601&product=Magnum%20Tape%20Drive%20%28LTO%29) for the tape drive is very pricey for the SOHO market, but if you have enough data, or frequent enough backups....eventually it is a good value. Still need SCSI setup and software that will interface from a backup machine to the drive. Never used one myself, but planning on it for archiving data long-term off site.

For those us with more time than money, VXA (http://www.exabyte.com/products/products/get_products.cfm?prod_id=641&product=VXA%2D320%20Packet%20Tape%20Drive) may be the way to go, but you still will need about five tapes (and a week!) to get a full backup of a Gig array, assuming you loaded a tape last thing at the end of the day. VXA is well supported (as tape goes) for Macs from the likes of Retrospect. Have used the smaller VXA 1 and 2, and no issues to speak of.

Drives are cheaper up front, but if you keep adding at a few hundred bucks a pop, tape is cheaper in the long run. Big pill to swallow to get started though. And drives are a bit more fragile IMHO. If you swap 'em in and out of trays, there is always a slight chance of bending a pin or something. A bit nerve racking if the building burns down and everything comes down to a single drive you have to get to mount...... Been there done that. Much prefer to have a half dozen different tapes in different sets, any of which can get you 95% + of what you need.

That'll be $0.02, please. :)

jjlphoto
02-14-2006, 05:12 PM
Hello,


If you need the stuff online there are non other options than running a RAID.


Are you referring to a RAID 5 NAS?

TeeH
02-14-2006, 05:56 PM
Interesting suggestions. Tape isn't an option IMO. And RAID backing up RAID sounds good & fast, but the cost makes it prohibitive. For now, it looks like the removable SATA HD setup might be as good as any - fast, good access, probably about as reliable as anything. One thing for sure, I'm always VERY careful with the drive trays...

T

Leighgion
02-14-2006, 07:09 PM
I just read this article (http://computerworld.com/hardwaretopics/storage/story/0,10801,107607,00.html) the other day urging magnetic tape or UDO optical disks for really long term solutions. But as noted here (http://www.techworld.com/storage/features/index.cfm?featureid=2159), the expert does work for IBM who makes products for both those types of solutions -- and existed the HD market.

I found that article hilarious. It acts like magnetic storage is the end-all of stability when the fact is, it's the oldest form of storage with the most well-known and established vulnerabilities, which are by no means few. Not only is magnetic tape a medium subject to various mechanical issues, one good slug from a strong magnetic field and it can all be done.

The sad fact is, I don't think there is any affordable method of high capacity digital storage right now that's genuinely reliable for the long term. The best buzz I ever heard was magneto-optical, but that seems to have dropped off the consumer map so it's tough to find any MO solution that has decent capacity.

Nicolas
02-14-2006, 07:33 PM
Hello,

I saw MO cartridges failing like flies when used in a library!
No cheap stuff, IBM lib. and Philips/Hoechst/Maxwell cartridges.
Yesterday you saved your data on it, today you try to read tak tak tak tak cartridge and data gone south.

Also the drives are VERY sensitive and it doesn't matter which brand you have, Sony, Ricoh, IBM (which are SONY or Ricoh drives), Olympus or MaxOptix.

One MaxOptix cost 1.500 to 2.500EUR depends on capacity, back in 1998/9 the same drive was half of that.

@jjlphoto
No, RAID5 subsystems, you'll need a host for those with SCSI, FW or Fibrechannel. You can connect it to your workstation or to a server.
Or you can buy a NAS but I don't like the NAS subsystems cuz if something goes wrong, you have no tools to recover/repair the volumes. OK, big systems are using an entire "normal" server but those all in one boxes have no tools installed.


Regards

Nicolas

unclemac
02-14-2006, 08:45 PM
....I
The sad fact is, I don't think there is any affordable method of high capacity digital storage right now that's genuinely reliable for the long term...

My point too I guess. If you can break your data up into logical groups no more than 400 Gigs or so, then the hot-swap drive idea is prbably the fastest/easiest.

No easy way that I know of to span multiple drives to create a volume larger than one drive by itself - that I would trust for backups. What to do if you need one volume? I suppose any of the enterprise level backup software (Retrospect or better) could do this, but I have not tried it with HDs yet.

Did use Retrospect to push out a disk image of 10.2 once to remote computers once, some years back. Spanned the disk image over 3 CDs, and mailed out. Worked on 39 out of 40 imacs as I recall. And that was a bootable fully configured disk image, not merely data.

If you want to (*have* to) span multiple backup volumes, there are a few tools out there for Macs: Impression (http://babelcompany.com/impression/), Backuper (http://dobrysoft.com/products/backuper/), even backing up to a DV Handicam (http://www.coolatoola.com/). And yes, you can find support for MO if you still have some around....check out BRU (http://www.bru.com/)

I'm sure there are others.

Here is a pdf that covers the disk vs. tape dilema from BRU: http://www.bru.com/pdf/BackupDevices.pdf