PDA

View Full Version : Asymmetric RAID 0 on ATTO



rhunt000
10-19-2001, 05:35 PM
Would there by any problem to have 4 drives externally on bus 1 and 2 drives internally on bus 2 striped together? It's my understanding that there is a greater speed potential when using both channels of the ATTO. Would the above be a speed increase compared to just having 4 drives on 1 channel?

Richard

kaye
10-21-2001, 09:53 AM
Richard,

No definite answer to your question. I know that magician preaches symmetry with striped RAID and I recall someone way back having problems with an ATTO UL3D, SoftRAID, and an asymmetrical setup of drives internal and external on separate channels. There should definitely be a speed increase using both channels as opposed to 4 drives on one channel. But the difference might depend on what machine we are talking about, vintage or New World. k

kaye
10-21-2001, 10:04 AM
Richard,

I see from your post to another forum that we are talking G4-533DP. Running both channels on the ATTO UL3D would be a definite speed boost, but the number of drives matters too. For instance, my QS DP, with 2x G2 X15s, both on one channel is only very slightly slower than one on each channel. So I opted for both on one channel to cut down on all the extra Granite stuff cluttering the inside. Adding more X15s to each channel would clearly give the advantage to using both channels in a New World Mac, particularly with that bus speed of the G4-533DP. k

rhunt000
10-21-2001, 10:12 AM
Actually, I don't know what the heck I'm talking about. I can't afford two more drives anyway. I often visit dreamland and have trouble distinguishing it from reality. What I will do is pull two of the drives out of my 4 bay burly, put them internal on bus 2 and leave the other two external on bus 1. Seems like I read on a post that it's not a big deal to have an internal and external set up and I should reap the speed gain. I hope. Of course, this means getting another terminator.

Richard

[This message has been edited by rhunt000 (edited 21 October 2001).]

MacMikester
10-21-2001, 12:16 PM
Symmetry is a major principle of good disk array design. One clue to this is that when you set up a stripe on disks of unequal size, the RAID software will create volumes on the larger disks that are no larger than the smallest disks (assuming you are using the entire smallest disks for the stripe). In other words, all striped volumes must be of the same size.

The most symmetrical way to set up an array with two identical int. disks and two identical ext. disks on the UL3D card is to set up one int. and one ext. on each channel and this requires two terminators (one int. and one ext.) per channel. Each channel will be limited by the slower of the two disks on that channel but the two channels will have the same transfer rates. This is where asymmetry can cause cause problems---when you introduce different transfer rates on each channel. This creates a great dependency on the error checking routines of the driver software. For example, any instantaneous electrical error might affect only the stripe segments of a single stripe in a symmetrical situation, with a fairly easy error recovery by initiating a single resend message. In an asymmetrical situation, segments for several different stripes may be transferring at different points in the architecture, creating errors in multiple stripes at the same time. This could result in a much more complex error recovery or even an unrecoverable error situation (freeze). Any deviation from perfect symmetry gets you closer to the edge of eventual disaster.

To amplify the wisdom of Kaye's point: As you build up an array with more disks on a UL3D card in a particular system with that system's PCI bus, you saturate the controllers in a strict order which dictates when to switch to dual channel and when to switch to a dual card configuration. He shows that you nearly max out one 32-bit channel on the UL3D with two G2 X15s. The next optimal step is to add two G2 X15s to the other 32-bit channel on the UL3D and near saturate that channel. The UL3D has a 64-bit path to a 64-bit PCI bus like in the Sawtooth so it will shovel the near maxed out throughput from both of its onboard controllers up to the saturation point of the PCI bus. This will be somewhere under 200 MB/s as the Sawtooth PCI controller will not give the full 264 MB/s theoretical maximum bandwidth to any one PCI card slot. It is my understanding that that this will be somewhere over 200 MB/s in Quicksilver due to relaxed limitations on the throughput allowed from any single PCI card slot.

You probably can gain very very small speed increases by adding a third G2 X15 to each arm of the above array but each and every controller will likely be oversaturated and gain no inrease in throughput by going beyond that very expensive setup. At this point, the optimal step would be to add a second UL3D and build a symmetrical array of two or four G2 X15s on that card. You would then be able to bypass the limitation of throughput from a single PCI sot device, but you very quickly saturate the PCI bus and will see vastly diminishing returns for the addition of very expensive SCSI controller cards and disks. In summary, current SCSI HDs and controllers can easily reach the saturation point of a 64-bit, 33 MHz PCI bus and this bus is the limiting factor for designing your setup. There is nothing about the PCI standard that prevents a 66 MHz or 133 MHz bus (or even 128-bit width, I think) but the reality is what it is right now.

rhunt000
11-01-2001, 06:35 AM
4 drives striped with SoftRAID 2.2.2 on channel one of Atto UL3D on a DP533 using Intech QuickBench 1.5 Extended---10 MB Read Performance 136.6 MB/sec Write Performance 126.7 MB/sec. As soon as I get my GD 2015 terminator, I am pulling two drives from the burly and mounting them inside the sawtooth on channel 2 and see what my numbers are. Very pleased so far! I can't get over how snappy this thing is. It's like when I open an application, it explodes on my screen and my monitor vibrates.

kaye
11-01-2001, 07:49 PM
Very nice performance. What are the drives? Let us know the performance when you split the drives into two channels. k

[This message has been edited by kaye (edited 01 November 2001).]

rhunt000
11-02-2001, 11:41 PM
I have Quantum Atlas 10K II U160 drives. When I bought my first drive to upgrade my Beige, I knew that SCSI was fast and that's about it. I went to StorageReview.com for the scoop. After buying one, I stumbled across MacGurus.com and bought another for the RAID, since it was most economical. Now I have 4. I mounted the two internal and noticed it was difficult to get my GD Cable connector down to the drive nearest the floor and when I tried to make the connection I felt some resistance. I tried again thinking I didn't have it lined up well and still felt the resistance. I unscrewed the bracket to get the drive up off the floor and when I tried to connect, it went on crooked. I then remembered reading the Gurus pages about forcing these things on and breaking a pin. I looked and a pin on the edge was bent outward. I began to stress. I calmed down and took a flat edge screwdriver and began applying pressure to straighten it. You'd thought I was performing brain surgery. I got it straightened, both drives connected and terminated. I fired it up and had a SoftRAID error stating there was an incomplete stripe. I thought aww sh*t, I screwed that pin up. I looked down and I hadn't powered the burly up. Whew! After a restart, here's what I got
10 MB Read Performance 210.8
10 MB Write Performance 151.3 (3 MB = 189.5)
That is the fastest of the three volumes. The other two are in the 150-180 range. Random write is noticeably slower, sometimes almost half the speed of Sequential Write. I've been reading about the 200+ for months now and I must admit I feel pretty darn good about getting there. Whenever anyone e-mails me or asks me a Mac question, I send them to macgurus.com. You guys rock! Do you think I'm nearing a max or would you expect more drives to equal a steady climb? I am freaking addicted.

kaye
11-03-2001, 12:34 AM
Richard,

It is brain surgery when straightening a Granite pin, considering the cost, but then if it breaks, I'm sure Granite will fix it at no cost, except shipping to them.

Great performance and definitely a speed boost using both channels vs. all four drives on one channel. How far will it go? My guess would be that one more drive on each channel would give a significant and linear boost, two more on each channel would peak you out. But then, I think you will have to find out where that peak is, to satisfy that addiction. Don't forget the Burly button and BTW, which Burly, 4-bay? k

rhunt000
11-03-2001, 07:13 AM
I have the 4 bay. I described it here --> http://www.macgurus.com/ubb/Forum1/HTML/000176.html
Thanks for the help!

Richard

rhunt000
11-03-2001, 07:24 AM
I ran ATTO ExpressPro-Tools for the heck of it.
PR 229.8 SR 226.1 PW 144.3 SW 132.4
Do you know it the drop off in write is typical or is it drive dependent?

Richard

[This message has been edited by rhunt000 (edited 03 November 2001).]

kaye
11-03-2001, 12:01 PM
Richard,

Your EPT settings, Max Transfer Size 8MB, Sample Size 2, and System Disk Cache unchecked?

Yes the SW is drive dependent. A long while ago, a member said that the true measure of a drive's capability is SW. IMO, he is correct, tho at first I wondered about it. I'm not trying to put down your drives, your performance is great, but the latest drives will more evenly match SW with SR, as well as RR and RW, but of course at greater expense. Even with the latest drives, there is a small window where performance is state of the art, until the next generation drives appear, such as my G1 X15s (eight of 'em) compared to my G2 X15s (two of 'em).

A couple of thoughts, you could experiment with some of the settings within SoftRAID and also its Mode Page Parameters (or HDT's FWB Configure for Mode Page Parameters which has more settings but be careful with this, Mode Page 8 is what you are looking for). k

MacMikester
11-03-2001, 01:14 PM
Super speed!! I think you are probably already up against the wall with those numbers but 200+ MB/s is a great place to be stuck. Your setup is pretty symmetrical with two drives per channel and the GD terminator will go a long way towards keeping the impedences equal on the internal leg and the external leg of your array. As k mentions, tweaking your mode page parameters, cache segment numbers, buffer full/empty ratios and so forth can alter performance somewhat, but there is most often a significant difference in Read performance and Write performance with current SCSI drives.

rhunt000
11-03-2001, 04:52 PM
I played around with some of the settings with little change. I investigated my stripe volumes a little closer and realized at some point things got a little out of alignment. When I checked the first volume, it didn't point to the first partition of each drive. Second volume didn't line up either but the third did point to the same corresponding partition for each drive. I backed up and re initialized under SoftRAID 2.2.2 and created 3 volumes.
ATTO Tools
PR 229.9 SR 226.1 PW 163.0 SW 152.3
QuickBench
10 MB Read 213.5 Write 158.1
That seemed to help.
One thing I found curious is that the cache segment number always came up 27 no matter what I tried to input. Can't figure that one out.

Richard

I upgraded the Firmware for the ATTO card from 1.6.2 to 1.6.4 which helped some, but I also changed my Buffer Full Rat to 1 and Buffer Empty Rat to 255 (after more experimentation) and it looks like my write increased a little-QuickBench 10 MB Read 204.4 Write 169.9
ATTO PR 229.2 SR 228.3 PW 162.7 SW 152.7
I'm also going to move the ATTO from Slot 5 to Slot 4 and test it.
In slot 4 the 10MB dropped to 159, I put it back in slot 5 and it was back up to 169.

[This message has been edited by rhunt000 (edited 03 November 2001).]

[This message has been edited by rhunt000 (edited 04 November 2001).]

[This message has been edited by rhunt000 (edited 04 November 2001).]