PDA

View Full Version : RAID 5 for X!!!!



unclemac
07-03-2002, 03:30 AM
Here is the link for the announcement of a beta release:
http://www.nyro.com/nxraid/

Now for the important question. . which comrade wants to risk his/her data for the good of the collective?

kaye
07-03-2002, 09:36 AM
Comrade ricks emailed me yesterday about one of us signing up. I have too many other projects going on right now but I think he is interested. They are in Fairfield, CA, not too far from me.

The question I have is that I don't see any previous RAID experience there, not even in the bios of the employees. Maybe I missed something. But it would certainly be worth the effort for someone to try, someone with solid OSX knowledge which pretty much excludes me. I can only install X and watch it run. If it ever goes south on me, I will have to yell for help. k

ricks
07-03-2002, 11:08 AM
?I did in fact email nyro to volunteer for their beta test. I also have qualms. Mine are more the generic: do I want a software based RAID5 on my computer? I am still waiting for SoftRAIDX so at least my array is not actively being used to store data. (can you say guinea pig?)

?I wish the market were large enough for a PCI card controller that would hardware RAID 0+1 and 5 for Macs. I think the cpu overhead is going to be excessive no matter how well written a software solution is.

?The proof will be in the pudding though. At least the tools for cpu activity and bus through-put are in place already. That'll make this possible to test.

?I'll let you know how they respond at Nyro.

Rick

unclemac
07-03-2002, 12:23 PM
I hear ya Ricks.

My first thought was that RAID 5 is not where one would start, more like where you hope to get with years of experience (like SoftRAID). I know nothing about the process, but if it was easy it would have already been done, right?

Still, glad to see someone taking a stab at it.

Are you saying that the overhead for 5 would be much more than for 1?

ricks
07-03-2002, 01:36 PM
newbie,(ish) http://www.macgurus.com/ubb/tongue.gif

?Ya know, you really do need to put a smilie after your name. You walk the walk and talk the talk.

?The short answer is: yes. There is a ton more overhead in RAID 3 and 5 than in 0 or 1. In RAID 0 all the controller does, whether a dedicated RAID controller or the cpu, is portion out the data by switching it between the drives.

?In RAID 3 and 5 you not only are portioning out the data to at least three drives, but are creating the parity block and then checking the parity block. BTW, the parity block is not on a single drive in most RAID5 configurations, it rotates as you go, first one on the third drive, next one on the fourth drive next on first and so on. Creating the indexing takes more cpu time also.

?I have found that most stand-alone RAID devices are not very fast, not in comparison to software RAID0 anyway. That isn't true when you start spending tens of thousands of dollars on the array, there are some top manufacturers that produce REALLY fast as well as huge arrays, but in the sub $10,000 range I have yet to see one that was faster than 80MB/second. It probably exists, but it usually costs for that kind of speed in RAID5.

And all the fast RAID5 I have seen have dedicated CPU controllers. Unfortunately I have never found a RAID controller manufacturer that sells just the controller so we could build our own. That would be the way to really rock and roll. But they make their money selling RAID solutions and not just the controller card/software.

It too bad really, there are a bunch of controller card manufacturers for PC's. Makes ya wanta go ballistic.

Rick

unclemac
07-04-2002, 03:21 AM
Sure, rain on my parade. http://macgurus.com/infopop/emoticons/icon_biggrin.gif

I have been looking at those $5000 to $10,000 RAID boxes you mention (and drooling), and your right; it's about redundancy and the ability to rebuild data, not max speed (compared to RAID 0).

FWIW, my experience with our databases confirms that drive performance is not super critical. Moving from a factory ATA drive (on a G4 733 w/1 Gig of memory) to a nice Gurus supplied 15X Cheetah/ATTO UL3S/Graninte/SoftRAid SCSI 160 RAID 1 was not even noticable to our DB users. I'm sure that if I did some side-by-side testing that the SCSI hardware is indeed faster in the DB, like finding all records, or better yet many simultanious repeated find requests. But at the end of the day, nobody noticed the Mac Gurus great hardware, and all my hard work.

Although disapointing, the main reason we laid out the bucks was for reliability and redundancy. The good news, I guess, is that about 40 megs per second appears to be enough - at least for this DB currently - and paying for more speed may be wasted money. Of coarse, this is just my speculation based on how the DB "feels" to users.

I'll let you know when we get the DB moved over to an Xserve. At thise point I am planning to set up a RAID 1 for data only, and install the OS and App(s) on a third ATA drive. Curious to see how performance improves, but hard to tell exactly where the improvement will come from: faster G4, L3 cache, new bus architecture, different OS, DDR RAM, etc. I hope to have time to benchmak RAID 1 performance properly per your *MOST IMPRESSIVE* RAID chart on the Xserve. And let's not forget about the Xraid (or whatever it is called) still to come...all ATA as well?



[This message has been edited by newbie (edited 04 July 2002).]

Damien
07-07-2002, 09:38 AM
correct me if I am wrong.....

But in a network/database situation the speed of scsi comes into play when you get a LOT of users accessing the raid at the same time. Your individual speed limit is controlled by the speed of your network so you will never get a speed higher than gigabit network speed (or 100baseT if you are using that type) But when 20 ppl ask the array for data at the same time the ATA system will choke and die where the U160 scsi system will kick butt!

------------------
Damien,

Damien's Stuff (http://www.macmeisters.com/Damien/)

unclemac
07-09-2002, 12:04 PM
Damien -

I thought the same about Ultra 160 SCSI. But when we moved data files off the SCSI RAID 1 (all good Guru components) to the factory ATA while doing some troubleshooting and repair work, I did NOT see or hear any complaints of slowdowns.

Now in all fainess, or Filemaker DB is very slow over our WAN (37 remote sites, all on DSL or cable) and perhaps another few seconds more here or there don't really matter when some of the processes can be MINUTES long. Have not done any real testing - this is all very subjective.

Maybe Filemaker server queues the find/sort requests (one at a time) so that access/read/write time is all that matters... or maybe other aspects like processor load, app speed, or the process of the scripts and calcs are so time intensive that the "slowness" of the ATA drive is insignificant...

Right now we are still running on 9.1 so that we can use SoftRAID for mirroring. Soon we will moving to an Xserve with native OS X mirroring, so I can easily monitor processor load, etc. Have read FileMaker Server runs faster on X or NT/2000 compared to OS9, so that should help as well.

IF we can determine that there is a speed increase with SCSI (as everyone agrees there should be), then I lose the hot swap drives, the cool remote monitoring tools, and have to lay out cash for an external SCSI box!! I know I am swimming upstream, but I hope the new ATA set up (seperate buses, etc.) is adequate. Will find out in a few weeks...

Maybe I can sqeeze a couple Cheetahs and an ATTO card for a data mirror in an Xserve!?


[This message has been edited by newbie (edited 09 July 2002).]

[This message has been edited by newbie (edited 09 July 2002).]

kaye
07-09-2002, 12:19 PM
xlr8yourmac has a press release on a firmware update to the Acard ATA/133 HardRAID Controller to Add RAID 1 (Mirror) Support to the card's existing HardRAID 0. k

jorge
07-09-2002, 06:18 PM
newbie,

make sure FileMaker runs in the foreground. It slows down like a dog on a hot day when it is in the background.

Also, Filemaker doesn't beneift from DP or Altivec. It needs MHz, fast access drives and a fat pipe to the switch. Also, make sure your ethernet connection is running full duplex.


George



[This message has been edited by jorge (edited 09 July 2002).]

unclemac
07-09-2002, 09:35 PM
Jorge:

Thanks for the tips. No worries on the foreground issue; those of us who touch the box know too well, and we have an Applescript that will bring it to the front automatically. According to ASP, duplex @ 100.

Going out to the web on a 1.5 T1, and we have not used more than about 30% when we were monitoring bandwidth. Locally the speed is very good, but WAN (over IP) users are slow.

Example: time to login screen (opens about 12 files - some on the server,and some on the client machine)

local - about 10 seconds

remote - 2 to 4 minutes


About 6 months ago we moved our Gurus internal RAID from a G4 466 to a new 733, and tested from a remote site. No improvement. I was disapointed, but came to the conclusion that it just is not the server. I am off on a tangent here, but as far as we can tell Filemaker is just slow over a WAN. We have seen folks talking about this all over, whether competetors, developers, or even users in FM forums.

Thanks to all for questions and input. Always more to learn!


[This message has been edited by newbie (edited 09 July 2002).]

ricks
07-10-2002, 12:25 AM
Newbie,

?Wouldn't it be nice if the new FileMaker that started shipping today is a vast speed improvement over the last one? Maybe.

Rick

unclemac
07-10-2002, 02:08 AM
amen brother!

But me thinks it is only the client app, and my woes are mostly FM server related...I'm sure it will come along in a while.

Maybe I will hear something at the FM Devolper conference next month.

The other problem we have (it is such a saga I am stating to forget bits!) is that we get a FM error when trying to backup / transfer files over a certain size onto a SoftRAID mirror; finally gave up mirroring and unistalled the SR drivers = problem went away. Hope the problem was between FM Server and the SR drivers, not FM Server and RAID; I will find out when we try to use native OS 10.1.5 RAID mirror on the X serve. Woo Hoo! Fun for everyone!!

typing geting bad; msut sleap noww.

[This message has been edited by newbie (edited 10 July 2002).]

dkemme
07-30-2002, 10:49 AM
The problem is with FM Server, it cannot backup to a RAID drive or from a RAID drive. It must run on a single volume and backup to that same volume. I discussed this severe limitation with FM tech support just yesterday and no relief is in site. I feel so naked without having my large databases on a RAID drive.

unclemac
07-30-2002, 02:16 PM
Is this still true even on OS X native RAID? I have another OS 10.1.5/FMS 5.5 box working great with this setup, but maybe all the files are smaller than the magic limit . . I better verify backups!

I'm pretty sure we hashed trough this about 6-8 months ago here at the gurus, right? For some reason I was thinking it was a SoftRAID and/or OS 9 issue. Probably blind hope that X would fix every problem known to man. http://macgurus.com/infopop/emoticons/icon_smile.gif

Thanks for the tip. Did the FileMaker guy give you any hope for a fix?

magician
07-31-2002, 12:48 AM
oh, man.

that is not good.

thanks for letting me know.

unclemac
08-20-2002, 02:31 AM
Filemaker update:

I was lucky enough to attend the FM developer conference last week, and got some tidbits that might clear up this backup issue.

Spoke to several senior engineers, and all said the same thing: RAID 5 on SCSI hardware.

They also shocked me with the following: DO NOT use the built in backup option in Server. On all platforms (OS 9, 10, Linux, and Win) script your own backups. There are scripts available on the FM site; the scripts for 10 are in pearl. I can add a link if anyone is curious. If the scripts are so easy, the question is why doesn't FM just incorporate them into Server instead of what they are now using?? http://www.macgurus.com/ubb/mad.gif

Nobody seemed to know anything else about any RAID or backup limitations. I guess either the scripts and/or hardware RAID address the issue, or there is an FM conspiracy!!

Now for the testing...

Thanks to all!

karlerm
08-20-2002, 01:49 PM
Regarding FileMaker and RAIDS, there is no problem that I know of. I do BOTH running of my databases from an external GURU RAID 0 and backing up to an internal RAID 0. I backup every hour to the 2nd RAID and then back up the files from the end RAID to VXA drive every night. No problems.

FileMaker performance is a chain of limiting factors begining with the server, moving over the network, and finally the Client hardware speed is crucial since it performs much of the work. For WANs FileMaker recommends using Citrix or Terminal Server and I've heard fantastic speed reports.

HTH

- Karl

------------------

unclemac
08-26-2002, 07:12 PM
Ditto on Citrix - we are testing it over the next few weeks.

As far as your RAID goes, are you on OS 9 or 10? If 10, are you using Apple's native RAID? Are you running Server 5.5, and I can assume SCSI Ultra 160 (Cheetahs, ATTO card, etc.), right?

What is the size of your largest file? We had problems with files larger than about 150 megs on OS 9, or so I thought. I was pointed to file size by another FM server admin, and since our large files had the same symptom, it seemed reasonable that file size was the decisive factor. But maybe not...

Thanks for the help!

Dan Clark
08-31-2002, 02:14 PM
So you guys are talking about a mile over my head, but I need some basic RAID info. If I should post a new topic, I could do that.

I do Photoshop work on 350 and 400 B&W G3s, and want to know if RAID is an easy solution to speed up my work. Where do I go to educate myself about what HDs and SCSI cards to get?

Thanks in advance,
Dan Clark

TZ
08-31-2002, 02:59 PM
Dan,

Go ahead and start another topic - will make sure you get notified of relies.

You are limited to 53MB/s.
SCSI drives keep getting faster for LE$$.
10.2 offers some advantages.
I think one of the new dual 800's would be a better bet on investment.

Gregory

ricks
08-31-2002, 04:15 PM
Hi ya Dan,

?You have come to the right place! In fact, you are talking to the right people, both Gregory and I have B&W's running scsi RAIDs. If there is any way we can help give a shout.

?BTW, the reason I run a scsi RAID on my computer is for Photoshop scratch space, I have to admit I migrated the RAID over to a 867 Quicksilver so I could realize the much higher PCI bus transfer speeds and the acceleration of same. The B&W does have some significant limitations there, too bad since the rest of the computer is pretty much top flight.

?Again, I agree with Greg that you should probably start a new thread so you get the very closest attention to your needs and particulars.

?
Rick