PDA

View Full Version : a general raid question



tmxmnr
04-15-2002, 10:22 AM
This has probably been asked before and I may have blew right by it in the archives, but can you mirror a striped RAID to another drive?

In other words, if you have two 40 GBs striped, can you then mirror them to an 80 gb drive?

If so, why would you do this?

Thanks,

Anthony

ricks
04-15-2002, 10:36 AM
Anthony,

I have always wanted to do this. This is called RAID 0+1 and it gives you a increase of both speed and redundancy. I always wanted to take my four Atlas IIIs and RAID0 the fastest 1gig partition off of each drive for a 4 drive Photoshop scratch disk, it'll fly. Then I wanted to take and RAID0 the bulk of the drives that is left over into 2 two drive RAID0 arrays. Then RAID1 mirror the two 2 drive stripes.

Would be good combination of good speed and reasonable security for the data.

Alas, the only way I know how to do this is with a RAID controller and the only way to get one of them is to buy a commercially assembled RAID array. Said controller will have its own processor and takes the raid maintenance and overheads away from the computers cpu. Best have a few extra thousands of dollars sitting around to accomplish this.

Rick

tmxmnr
04-15-2002, 01:12 PM
So it would primarily be good for swap partitions for hardcore PS users? Does it have any application to regular use, such as speeding up your main RAID, then mirroring it all to the second drive for back-up purpose? Or say if you wanted to network you house with three or four Macs and run one was a server?

Not that I will ever bother to do something like this, I just got to thinking about my next upgrade http://macgurus.com/infopop/emoticons/icon_smile.gif and how far I could take it. Frankly, if it takes me to thousands of dollars, then I'd rather have a new 1.0+ ghz Mac when they come out! http://macgurus.com/infopop/emoticons/icon_smile.gif

Thanks again!

Anthony

ricks
04-15-2002, 02:30 PM
So, yes, striping is useful for all around acceleration of a computer. Any program that accesses disk a lot, and there are too many of them to list, will benifit from a stripe raid that speeds up reads and writes.

Any time the data is truly important though, RAID0 is the worst secure place to put it. Anything goes wrong, one drive has a critical index corruption, a drive dies, the software has a bad day and zaps your stripe, anything can cost you your data because with a true RAID0 stripe there is NO protections at all.

A good, and I mean comprehensive data backup regime must be followed and even that is not sufficient if your data is irreplaceable. If you have truly important data then a hardware raid 0+1, 3 or 5 is the way to go. Raid 3 and 5 use a drive to do a parity check while the rest of the drives are striped. Lose a drive and you can just slap a blank good one in place and the controller will backfill the new with data from the rest of the stripe. This is the way to set a network file system up that does a decent job of protection.

Barring needing to consider serious protection and redundancy striping does a good job with rendering video, most any graphics intensive work like Photoshop and any other drive intensive program. Most of the time the only data that should be stored on the pure stripe would be the work in progress stuff to limit your losses if you have a failure or data that can be backed up frequently to another place.

Speed is kewl.

Rick

unclemac
04-16-2002, 05:37 PM
Ricks:

While you are giving the RAID tips, I was just wondering. . .Which would be faster for database use: RAID 5 or 0+1?

How do the two compare? I see that most hardware RAID suppliers support both (and more) formats. When I fianlly get to RAID heaven, what should I consider when I have to choose between 5 and 0+1?

Of corse I will consult the supplier when the time comes, I was just curious if you had an opinion.

Thanks!

ricks
04-16-2002, 07:57 PM
Hey Newbie,

That would depend on the RAID configuration. As you add drives to a RAID5 you get a faster RAID that also has more overhead, less though per drive with each drive added. RAID5s read faster than write because they don't have to access the parity block during reads and they have to create the parity blocks during writes. RAID5 is considered the most secure storage system array.

With a 0+1 the speed would all depend on the speed of the striped arrays. That could, like the 5, be anything. I noticed in reading specs on a lot of commercial RAID boxes that speed is seldom of primary interest to manufacturers. Lots of them were only good to 40 or 80 MB/sec in RAID5, I never saw specs for RAID0+1 only the RAID5 specs, probably since it could be configured in so many ways.

High fault tolerance and capacity, hotswap-ability, multiple power supplies, multiple controllers seems to be the target with the commercial arrays I've seen. At least according to my definition of speed they aren't after speed. I have seen a few that would pass data advertised at 160 MB/sec but that's not common.

After building ATTO UL3D dual channel arrays that will consistently max the PCI bus both read and write at over 200MB/sec with SoftRAID striped arrays I have trouble finding 80MB/sec to be blazing. I just hope that SoftRAID for OSX will work as well.

I suppose that logically a RAID0+1 could be made faster. Writing a parity block takes processing, that doesn't occur in 0+1. However in a properly designed RAID for a given purpose, if speed was the big issue then you'd have caches set up to accommodate whatever speed you'd need, up to huge multi gigabyte ram caches if needed. They'll suck data off a LAN like a vacuum cleaner holding the data while the controller writes it to disks.

Be fun to have a big budget huh?

Rick

unclemac
04-17-2002, 10:59 AM
Ricks:

Now how did I know you would have some juicy tips? http://macgurus.com/infopop/emoticons/icon_smile.gif

We are gonna get there (RAID) one way or the other. Right now we are looking at the cost/benefit of using thin client tech (like Citrix) to serve a couple different DBs. Only problem is that we would have to switch to Win 2000 server with Terminal Services. . .the licensing crap alone gives me a headache! No luck yet finding a vender that will support a Mac server - only Mac clients.

Macs sure can spoil you. . .

Mac or Wintel, we will move to a least one RAID box. Either way, disc access speed will always be a significant part of the DB performance. If this project goes well, it could evolve into a sever farm or two.

BTW, did you get the email I sent you with some questions for/to your brother?

ricks
04-17-2002, 11:20 AM
New,

I did get and pass on the email. Most of which you wrote in greek to me. My taste for hardware that's fast doesn't qualify me to even sit down at the table on the real world issues that you two guys talk about. My brother came by last weekend and spoke in tongues for a while about your email, I know he is thinking on how to properly respond. He aso wanted input but I understood only every other word he spoke, all I could do was croak.

I did understand his one point though, "What's your definition of 24/7?" That question will determine everything you do with your network file system, everything. The fault tolerance, hardware backups, power..........This is where you have your work cutout for you, a simple definition, hah, no such thing.

The hard part actually appears that budget sets that definition instead of the definition sets the budget. Only way that isn't true is in our dreams.

Oh well, I'll check with him and see what's up.

Glad to see you're still out there. Look forward to the stories that come from your project.

Rick

unclemac
04-17-2002, 04:21 PM
still here . . .

Work is stacked up in big heaps, so I don't have much time to post, but I alway try to read everyday.

Trust me: you are plenty qualified to discuss these topics (If you are not then I/we are in trouble!). I just hope I know what I don't know - which is an awful lot.

Where is the best place to get him our proposed specs? I can do it here, directly to you, or to him if you would prefer? Can I pick his brain a little? Gosh I just scared myself. If you croaked, I may faint dead away. . .

Sometimes I do think Macs are a curse in a way. It's like I have been driving an automatic Honda Civc for the last 10 years, and now I have to teach myself how to drive an 18 speed Kenworth. Oh well, at least the first 10 years were a piece of cake!

Thanks, as always, for your help and knowledge.

ricks
04-17-2002, 04:56 PM
Hey I talked to dear old brother Kris and the poor guy had spent hours on a reply to your email and decided to pull a save and....pooooof, he'd timed out his network connection and lost the darn email. That has happened to me a bunch of times, so hard to ever write the thing the same again. Uhgggg.

Expect something soon from him. Collaborating has been a blast. Learn new stuff every day,that's the real joy. I have to agree with my brother though, when it comes down to real world industrial stregnth knowledge, experience is everything. I just haven't got any time in a commercial enviroment doing this stuff. When your job is on the line, make sure you're getting the best info available. I love to learn this stuff, but all my experience is small scale. I'm happy to help any way I can, but take it with a grain of salt and back it up from a professional.

Can't wait to hear what you two come up with.

Rick

unclemac
04-20-2002, 11:01 PM
Ricks:

Got his post - wow. I keep re-reading it and trying to digest. Very cool stuff...very intimidating, but very cool. I just added a bunch of things to my "stuff I don't know" list. I was tempted to post it here, but it is a bit much, and way, WAY off topic. Don't you think?

Thanks for the help (and to all of the Gurus), I will keep you posted as things develop with the RAID 5 project.

rhunt000
05-01-2002, 02:46 PM
Ricks,
I've read in several places that it is a good idea to set up the fastest partition of a RAID to exclusively be a scratch disk for Photoshop. Around 2 GB sufficient? What about the actual photo file you are working on? Does it matter so much where it is located or does Photoshop do its work on the scratch disk until it's time to save the file?

Thanks,

Richard

ricks
05-01-2002, 03:43 PM
I am starting to revise upward my estimate of what Photoshop actually uses for scratch space. I haven't finished testing Photoshop7 for scratch space yet, but it complained it was running out with the initial 2 gigs I gave it while I was working a 500MB photo.

I should actually say I'm barely getting started testing the array with Photoshop7, since I've only had the Photoshop OSX software for a week.

My best guess at this time is 5 gig scratch doubled up by making 2 of them on the same array as what I'll end up with. The rest of the drives capacity I would like to do a combined 0+1 but won't know if that's possible until SoftRAID comes out. 10 gig seems like a lot, but prices per gig are way down and better to have a bit more than you need than have to reconfigure full drives at a later date.

The faster we make the storage the faster we can both open files and save files and works in progress. But we really want security as a higher priority than speed when it comes to the irreplaceable data. I get nervous with pure RAID0 as storage for valuable (to me) data. So I have been storing on non-RAID drives and cloning them every night for backup with SwitchBack, a shareware sync program fro OSX and OS9.

I wrote the developer of SwitchBack and asked for a 'Hotbutton' to do a quick backup, he wrote back that the program will already do that just by using the save configuration command which places a saved folder in SwithBack, by double clicking that folder SwitchBack automatically performs a syncing. Then by placing an alias of that folder in the Doc, I can click it and save (sync) my Photoshop Save folder at any time to another drive or computer, instantly. Probably overkill and a bit anal. At least it's easy.

Backups are the key. RAID0 storage isn't bad if you have a good backup plan. Even storing on non-RAID drives without a backup plan will screw you in the end everytime, guaranteed, 100% of the time.

Rick

TZ
05-01-2002, 04:48 PM
If you are using one of the latest 10k or X15 Cheetahs, there just isn't the 'hit' using inner tracks as once was the case.

If you can afford to have a large area that can be erased at will, do it. One thing I like about SoftRAID is the ability to remove volumes and create mirror, raid, or plain volumes as needed w/o requiring an initialize.

Apple's Disk Utility looked like they might have plans down the road to delete, split, volumes, but so far doing so wipes out any data on all volumes.

And now RamBunctious has an OS X version to create a RAM disk, for those with an extra 500+ MB they want to use for scratch. Now all you need are 1GB DRAM chips, fat wallet, UL4D and some Ultra320 drives from Seagate. Maxtor has said it will have a new Atlas 10k III in Ultra320 and Seagate can't be far behind. And probably new cables to support them.

Gregory

ricks
05-01-2002, 07:00 PM
Gregory,

While we're on the wish list, could we ask Santa Jobs for a PCI bus that'll pass, uhh...maybe 450-500 MB/sec?

Rick

rhunt000
05-02-2002, 09:24 AM
I agree on the backup issue and I do have a plan. Where was your 500 MB file residing while you were working on it?

Thanks,

Richard

ricks
05-02-2002, 01:02 PM
"Where was your 500 MB file residing while you were working on it?"

Interesting thing, the newer the Photoshop version, the more copies of a work in progress Photoshop maintains. That fancy multiple undo must save a copy for each undo you can access.

With OSX we have a lot less control as to how much RAM is actually being used by Photoshop at any given time (and we don't want that control back), but it's really hard to know how much of a photo is in RAM and how much, as well as how many copies are on scratch and storage disks.

What I'm saying is, I don't know.

I had done a bunch of filters and sizing changes and just ran out of Drive space on a drive that I think started out with 2 gig available, you got me. I just purchased a 2000p photo printer and was trying to get the highest definition I could to test it out. I had to start over, the out of places to store warning left me with no alternatives eventually.

I probably could have designated another partition as scratch disk space and saved the process but I didn't think of it at the time. I also leave room for a mistake on my part in configuring Photoshop or a bug that caused the problem. The whole OSX and Photoshop thing is too new for me.

The fun is in the learning for me, but I don't make a living driving Photoshop. I can imagine this isn't nearly as fun for a professional.

Rick

gmidd
05-02-2002, 03:58 PM
PS wants a lotta room for scratch 'cause it's constantly updating its temp files as you work. I couldn't tell ya what its doing in X now as to how much in ram, how much to disk but I really noticed this big time when I had my cheetah stripe with 2 partitions: one scratch and one for graphic files. I was working on a 20mb file and this disk was crankin away even when I wasn't actually making any moves. The temp file for this particular session was pushing 300mb...
I think it tries to stick the history and multiple undo's in RAM, but I'm going to have to investigate more. I know when we all get SoftRAID X, I'll make probably a 10G scratch partition.... (especially for the initial 50-150mb hi-res film scans I'm planning to do)

George