PDA

View Full Version : How Fast Should it Be?



jlswartz
04-24-2001, 04:20 PM
I have my Burly/HomeBrewed/Guru spec RAID completed, and it works well as far as I can tell. It even seems quite quick. But... I can't keep from wondering whether there's a little more performance there I might be missing... It's not that I'm not satisfied... But what if it could do more and I was missing it?! There's a scary thought.

The RAID in question is 5 IBM ultrastars, Softraid 2.2, and the stock ATTO lvd card in a 500 mhz G4. (That card is dual channel, but only one is available externally, so I'm just using one.)

I've run ATTO tools on it, and got the following results:
(512k, sample size 2)
Pk Read: 68.08 Pk. Write: 65.86
sust. Read: 66.81 sust. Write: 64.22

Another question:

I've done some "burn in" with it, using an Applescript to copy it full of files over and over, duplicate them, delete them, etc. and it seems fine. What other steps do people take to make sure it's stable before putting it into service?

Thanks very much for the help. So far I'm new to a lot of this, I hope to be able to contribute answers instead of just questions some day!

------------------
==========================
John Swartz
Deskey Associates
==========================

[This message has been edited by jlswartz (edited 24 April 2001).]

kaye
04-24-2001, 04:58 PM
John,

Do your ATTO test at 8MB, sample size 2, and no disk cache. The reason for 8MB is that it does all of the sizes up to 8MB and the 8MB goes beyond the drive's onboard disk cache. Unchecking the ATTO Disk Cache option disables the Memory CP Disk Cache setting and gives a truer measure of the drive's performance.

What is the model number of the Ultrastars? Are they all exactly the same model? The latest version of SoftRAID is 2.2.2.

Your stability tests seem good to me. k

magician
04-25-2001, 01:02 AM
I like to use Integrity as a reliability tester....email me offline so I can hook you up with a demo copy of HDST, which includes Integrity and QuickBench. You need that stuff.

I gotta say....are those 7200 or 10k rpm drives? And what is their capacity? Are they LVD drives, and are they all matched part numbers?

I'm thinking, depending on the type of drives, that your performance is respectable but a little less than it could be, perhaps, if you are using 10k LVD drives.

doublecheck and make sure you have all firmware updates available for that card. That's the Apple version, right?

what cables and terminators are you using?

jlswartz
04-25-2001, 10:11 AM
Kaye, Magician,

Thanks for your help.

To answer the questions:

Drives are 10,000 rpm DDYS-T18350 IBM ultrastars, they're all exactly the same.
I'm using Softraid 2.2.2, and I've just (at your suggestion) upgraded the card firmware to version 1.6.3.

I've got the Granite Digital cables form you guys, and the GD 2015 terminator. It is the Apple version of the card.

Using the 8MB size:
PRead: 67.92 Peak Write: 69.61
susRead:50.67 susWrite: 51.60

One more thing... I've discovered that I get better performance with extensions off (should have thought of that right off, of course...) but I obvioiusly can't run that way. I can do all the process of elimination troubleshooting, but are there extensions which are known culprits? (tests above are sys 9 all set)

Extensions off:
Using the 8MB size:
PRead: 74.2 Peak Write: 76.12
susRead:64.7 susWrite: 61.25

Thanks again.


------------------
==========================
John Swartz
Deskey Associates
==========================

kaye
04-25-2001, 06:18 PM
The DDYS-T18350 is a U160 LVD 10k drive 4MB cache and a current model. So you are using Granite TPO LVD cabling and Granite LVD terminator? Jumper settings here http://www.storage.ibm.com/techsup/hddtech/ddys/ddysjum.htm

Only comparison I can make is I have an Ultrastar DMVS 09V (18LZX) U2W LVD 10k drive 2MB cache, older model. I am running it as a boot drive in an S900 G3-500, 50MHz system bus. It is being run off of a JackHammer UW card in a slow PCI slot. So the drive is running single-ended, not LVD, and at UW speed.

ATTO EPT 8MB, Sample Size 2, no Disk Cache:
PR 27.08, PW 27.15
SR 26.83, SW 26.63

I gotta think you can do better with 5 10k drives each with 4MB cache, running LVD off of an ATTO dual-channel card in a 500MHz G4.

The jumper settings for J4, no jumper on Force SE mode (a jumper on any one drive would take the whole chain out of LVD mode and into single-ended mode). No jumper on Term power (the ATTO card supplies Term power).

On J6, no jumper on Disable parity.

Let's start with this. That way we can be sure the array is running LVD and not single-ended. I run ATTO test with a minimal set, standard OS stuff, and whatever else I need for my PCI cards and processor, etc. k

[This message has been edited by kaye (edited 25 April 2001).]

magician
04-26-2001, 03:31 PM
there you go.

follow K.

he will steer you right.

jlswartz
04-26-2001, 05:57 PM
OK, I've been busy, I've done the following:

Took everything apart and made *certain* that there are no jumpers except "enable auto spin" on any drive. (and the scsi ids of course...). Made sure all cables are snug. I tried running with the terminator and scsi cable swapped to each other's ports. None of this had any affect.

The terminator has its "LVD" light lit.

I tried creating stripes of fewer drives, and found a pronounced improvement from the intial striping to two drives, then smaller incremental improvements after that, with some ambiguity about whether there's any improvement at all between 4&5.

Here're the results:

(EPT, 8MB, 2, all extensions off)

Each Drive by itself:

Drive # PR PW susR susW
1 39.4 69.70 35.24 34.56
2 42.74 69.79 35.29 32.77
3 45.44 70.36 35.22 34.53
4 42.45 69.63 35.25 35.18
5 42.5 69.86 35.23 35.44

Stripes:

# of Drives
1(just hfs) 39.4 69.70 35.25 34.56
2 68.1 73.02 65.80 52.83
3 72.30 74.80 56.06 67.15
4 67.29 76.63 64.75 72.51
5 72.02 76.11 64.15 67.02

So, it looks to me like I'm not getting the kind of improvement I should be from adding more drives... That first jump (if you combine read and write) is on the order of 70%, the second one is more like 3%, then about 11%, and finally a 5% or so drop... maybe some sampling error in there...

There are setting in Soft Raid, for block size, and some other stuff, I've tried adjusting that from 128 up to 512, and then back down to 32. No noticeable difference.

Oh, I've let integrity run overnight twice, and no errors...

Any other suggestions?

Thanks much.

note: you mention dual channel, and it is, but I've only got one channel in use because it's the weird in/out card from Apple. don't know if that matters.


------------------
==========================
John Swartz
Deskey Associates
==========================

[This message has been edited by jlswartz (edited 26 April 2001).]

magician
04-26-2001, 11:43 PM
I'm wondering if something couldn't be tweaked in the mode pages,K....maybe his cache segments or something?

kaye
04-27-2001, 02:30 AM
I think you're right magician. I want to be sure first that he has set the basic SoftRAID settings first.

jl, you mention SoftRAID 2.2, you do mean version 2.2.2, right? And in the SoftRAID application, highlight a drive, and select Driver Parameters from the Menu Bar (I'm doing this from memory), a windo opens, and you have set:
Maximum Concurrent IOs: 256
Memory Pool Size: 804
Write Acceleration: ON
Read Acceleration: ON
Photoshop Acceleration OFF

Check each of the drives for the same settings just to be sure they are all set the same. They should be, but you never know for sure until you check. Something not right.

If this is what is set, change Memory Pool Size from 804 to 1369. That is what I use for four X15s. Try that. Do either 4 or 5 drives and post results. I will get to some Mode Page Parameters later. I need to review my IBM drive Mode Page Parameters settings before I make some suggestions there.

Last questions, so far you are running this RAID with little software installed? You don't have a lot of files and apps loaded? When you striped the drives in SoftRAID, did you use the default stripe size of 128 SU (stripe units) which is 64KB size stripes? So 128SU=64KB stripes. k

jlswartz
04-27-2001, 09:30 AM
Kaye,

Thanks again for your help.

To answer your questions:

SoftRaid 2.2.2 yes, all three 2s.
SU=128, though, as I tried to say, I've tried it as low as 32 and as high as 512 with no noticeable difference.
The RAID has nothing on it at all (handy with all the re-striping I'm doing! ;-)

Under "install driver..." I have the settings you describe, Memory pool is 1369. An odd thing though, I move the slider to 768, but when I look at it next it's back at 256, even though the memory pool has expanded. The numbers I've been giving you were done with the memory pool at 1369, and write on, read on, photoshop off. I've checked all 5 drives, they're the same.

The mode parameters say:
buffer full 0
buffer empty 0
write cache is enabled
cache segments = 14



------------------
==========================
John Swartz
Deskey Associates
==========================

kaye
04-27-2001, 11:07 AM
John,

Excellent information. First, what MacOS are you running?

I have tested a bunch of stripe sizes and settled on SU=256 which is 128KB stripe sizes, but leave it at SU=128 for the moment. Try SU=256 later. But I do think SU=256 is better. And yes, later versions of SoftRAID that I have, both 2.2.1 and 2.2.2, you are stuck with Conncurrent IOs of 256, no matter what value above 256 you set. I confirmed this with SoftRAID Tech and was told more would not help anyway.

I checked my Mode Page Parameter settings for three IBM 10k Ultrastars that I have. They are all set at:
The mode parameters say:
buffer full 0
buffer empty 0
write cache is enabled
cache segments = 9

YMMV, but my various model 10k IBM Ultrastars, in testing, all benefit from a smaller number of cache segments (the 14 cache segments is optimized for PeeCee test utilities). There is a balance though. For pure I/O, such as the ATTO 8MB test, cache segments = 3 was best. However, in MacBench 5 Disk and Pub Disk, cache segments = 11 was best. So I tested every number between 3 and 11 in both EPT and MB5 to see what would, overall, give me a combination of best ATTO EPT and MB5, shading slightly towards MB5, since that would emulate my needs more than the pure I/O of EPT.

All that said, there is a caution. Once you set any cache segments below 11 in SoftRAID, you cannot get back above 11. I use HDT Configure to reset it above 11, if need be. And, in SoftRAID or Configure, you must set the number of cache segments for each drive. Let us know what the results are. k

kaye
04-27-2001, 11:44 AM
John,

Duh, let me back up a moment. This is a striped RAID, RAID 0, you have created? Not RAID 1? And is it HFS or HFS+ (just curious about that)?

Additionally, I have an email into Mark James at SoftRAID about one of your jumper pin sets, specifically the J6 block, TI sync negotiation, pins 9 and 10. I will wait to hear from him. k

jlswartz
04-27-2001, 12:24 PM
Kaye,

RAID 0, hfs+.

I'm using Mac OS 9.0.4. I'm planning to move this to another machine once I'm sure it's right, the hardware is identical, but the OS is 9.0.3. I could update that one when I install the RAID.

I'm getting the weirdest thing. When I try to set to 9 cache segments, it keeps going back ot 14... I'll mess around with it some more to see if I can get it to take.

Thanks again.



------------------
==========================
John Swartz
Deskey Associates
==========================

kaye
04-27-2001, 04:50 PM
John,

Heard back from Mark James of SoftRAID. Gave him all the info and link about your drives and also the ATTO card. He said no jumper on the J6 block, TI sync negotiation, pins 9 and 10, so you are good there.

He also mentioned, but was not sure, that Apple shipped some ATTO cards that were not LVD, they were UltraWide. Go to Apple System Profiler in the Apple Menu and tell me exactly what it says about the card. I know you are getting the LVD light but I want to give him the model of the card. I told him I would report to him on Monday including a link to this thread. I was in a rush to email him with the details and forgot the link.

If you are not able to set 9, try 2 or 3 or 4. Sorry, I know this is a stupid question, but I gotta ask, you are clicking OK to the change? If it will not change from 14, there is always the possibility that IBM set this parameter for this drive to be unchangable. Every drive that I have looked at for all of the parameters in other utilities has some parameters that the manufacturer will not allow changes.

If that is not resetable, try the SU=256. BTW, none of these drives are SCA with adapters, right? I keep asking these dumb questions because I can't imagine why 5 drives in striped RAID are only about twice as fast as one drive. Something screwy here, and probably something staring us in the face.

I know on your machine it should not make a difference but try the ATTO card in another PCI slot, preferably the first slot. k

kaye
04-27-2001, 05:26 PM
Another thought, the Granite stuff, I know that GD2015 is LVD. Look at your receipt and tell me the Granite numbers for the external cable (ATTO to Burly) and the internal cable inside the Burly.

Something else on my mind but forgot it now. I will be off for awhile. Got to test something. Maybe I'll think of it later.

Anyone got any ideas? I'm not trying to hog this thread. Something staring us in the face? Somebody itching to contribute? k

Santilli
04-27-2001, 06:00 PM
Hmmm. Is the card in the busmastering slot?
This is an LVD 80 MB/sec max card, right?
Seems to me 74-76 MB/sec is about all you are going to get.

I can't help but thinking you are running the card in a 32 bit/33 mhz pci slot, with a Grackle chipset. Looks just like my figures for my raid, except I have way more horsepower, and a ATTO U3 card, that should be much faster then the ceiling I hit in my G3. As for the increase from adding drives you expect, with my system, and the maxed out pci bus, after the 4th drive, my system goes slower...

How many slots on the G4 are supposed to be 32/33 and how many 64/33? Wonder if you have the wrong slot, or got the wrong chipset put on your mobo???

Weirder things have been known to happen.

gs


GS

Santilli
04-27-2001, 06:02 PM
One other thing. Are you running all the drives on one cable, on one channel of the card?

If so, you might get a bit of a benefit by putting two drives on each channel, on separate cables.

GS

Santilli
04-27-2001, 06:07 PM
If you are running your os off the raid, you might want to consider running two drives on separate channels, as the os drives, and use the others for storage.

You may have sufficent speed for the os with two drives, and each drive you add may increase access time enough so that it does not compensate for the added through put...


gs

jlswartz
04-28-2001, 09:40 AM
Kaye, and others, thanks for the help, here're the answers so far:

The card listing in system profiler:
SCSI-2 (is that the problem?)
ATTO, Express PCIProUL2
UL2D
1.6.3f1
7
1000

Yes, I'm clicking OK to the change, but have noticed that if the 14 is displayed, it won't let me click OK.

I'll try the other sizes when I can. I'll also try the other slots and get back to you.

part numbers gd3100, GD1005.

Thanks to everyone, I'll get back with more data when I have time.



------------------
==========================
John Swartz
Deskey Associates
==========================

kaye
04-28-2001, 11:04 AM
Santilli,

You are exactly right. Thanks. He has the UL2D U2W LVD card (80MB/s max) and not the UL3D U160 LVD (160MB/s) which I thought he had. He is running it on the external single (Apple version card) connector because he has a Burly and will be migrating the Burly over to another computer.

John,

Your Burly raid is maxed out on the UL2D LVD card on its single external connector. I thought you had the Apple version of the UL3D card, U160 LVD (160MB/s). I just don't know these Apple G4 variations well enough. So you are pretty much maxed out with 2, 3, or 4 drives on the UL2D. Your part numbers GD3100 and GD1005 are exactly correct. You can leave the number of cache segments at 14 or continue to attempt 9. You can leave your stripe size at SU=128 or go to SU=256. But the UL2D is maxed out.

When you migrate the Burly over, I guess it will be attached to another UL2D? Since your Burly is configured for single channel, the expense of converting the Burly to dual-channel is expensive, you might consider replacing the UL2D with a Gurus UL3D, but I have no stats on what that increas would be, given that you would still be operating single channel but with U160 LVD.

Sorry for having you jump thru hoops on this. Your responses and expertise in what you have done is excellent. But the ATTO card is throttling your striped raid. Any questions or comments? There might be someone with a UL3D in a G4 with results on the forums, but it would probably be dual channel configured. k

[This message has been edited by kaye (edited 28 April 2001).]

magician
04-29-2001, 02:48 AM
I'm pretty sure I stopped benchmarking single-channel at four drives, then moved to dual-channel. Those benchmarks should still be on the FTP site, and they should be somewhat relevant, considering they are for a UL3D with X15's in a Sawtooth 500MHz.

I'm a little confused by the Granite part numbers, probably because I had several beers with an amazingly good fajita dinner. Do you mean you have the GD1300 cable? And which terminator are you using again?

http://macgurus.com/infopop/emoticons/icon_smile.gif

kaye
04-29-2001, 01:46 PM
GD1005 is the LVD enclosure cable for his Burly and GD3100 Ext. 68-pin .8mm to 68-pin MicroD 6-ft. Cable is the external cable (Champ) for ATTO UL2D to Burly.

His terminator is GD2015. k

jlswartz
04-30-2001, 09:03 AM
To everyone, especially Kaye,

I really appreciate the benefit of your experience with these things. I'm a very experienced Mac user, but high performance SCSI is a new area for me. I've already learned a great deal, and expect to learn more. I'm sorry it wasn't apparent to me that I had reached the limitiations of the card, but that ties into the original question. Simply not enough experience on my part.

Thanks very much to all of you. The question is now answered, and I can go ahead and enjoy this new RAID (which is considerably faster than anything else we are running here, except maybe the HVD hardware raid, and I'm not sure about that...) without worrying that in my inexperience I've left throughput "on the table..."

I would still be interested in advice on extensions which could cause the 10-15% speed loss I'm seeing when my extensions are on. I'll start methodically troubleshooting them this afternoon, but if you know of any specific extensions which are likely to be the cause, please let me know.

One more time: Thanks very much to everyone who helped.



------------------
==========================
John Swartz
Deskey Associates
==========================

magician
04-30-2001, 11:15 AM
oops. Thanks, K.

just goes to show that I cannot trust my brain under the best of circumstances--much less after two 23oz Dos Equis.

kaye
04-30-2001, 11:58 AM
John,

A few specific examples of things that will slow down the whole box, Kensington software for their trackballs and mice (at least earlier versions were and I think partly because they were not native), AppleTalk on, networking on, ATM, a lot of TrueType fonts which do load at startup. For test purposes, I stick with just the fonts that come with the OS version I am testing with, no Kensington, AppleTalk off, networking off.

One thing I have noticed is that, after a few weeks of use, the drives are faster by about 2%. k

Santilli
04-30-2001, 05:34 PM
THANK YOU THANK YOU THANK YOU.

Still no answer if the card is in the busmaster slot...

GS

magician
04-30-2001, 07:56 PM
too many fonts loaded, for sure....applications which load "prevention" extensions or control panels....stupid stuff like DiskLight.....

kaye
04-30-2001, 08:19 PM
Santilli,

I thought the G4 500 with UniNorth controller chip created all PCI slots equal, not like the two BusMaster slots in 6-slot vintage Macs. Not knowing much about these New World boxes, I don't know. From what you are asking, is one slot faster, a BusMaster? k

Santilli
05-01-2001, 08:07 PM
Kaye:
Don't know much about the new macs.
If the mobo creates all slots equal, that is a nice change.

gs

Louie
05-01-2001, 08:37 PM
Greg:

I think from the beige up, all PCI slots feed thru one bridge making them equal.

jlswartz
05-29-2001, 09:47 AM
I just wanted to follow up on this, thanking everyone again. I've figured out (some time ago actually, but I've been slow to post this) that the "extension" related slow down was due to virtual memory. I made sure that the virtual memory file is on the non-RAID drive, but still, it slows it down considerably. I don't know *why* this is. My answer to this has been just to not use VM on that machine. Problem solved.



------------------
==========================
John Swartz
Deskey Associates
==========================

kaye
05-29-2001, 01:27 PM
Duh, John. I always forget to mention virtual memory because I never use it and I always assume that nobody living on the speed edge does either. This old brain is fried with age. Thanks for posting your results. k

Santilli
05-30-2001, 05:24 AM
Louie, if you are correct about all the slots being the same, this explains why my beige works great with Magicians;'s 2940B, 2940 UW, and SBS box, all in the wrong slots;-)

gs

Louie
06-11-2001, 04:41 PM
After the 9600, all PCI slots were created equal. They all feed to a controller thru the same bridge.

magician
06-11-2001, 06:03 PM
well, I used to believe that most Macs had one slot in in each bus that could be considered a bus master until Trag taught me the error of my ways. The thing is, there are contradictory resources in the TIL and Dev Notes that go both ways, and suggest both possibilities. The bottomline is, I lack the time and motivation to delve deep into the source docs and resolve it definitively. I leave it to someone else.

I will say this: vintage machines seem to work best if you proceed under the assumption that slot one IS a true bus master. New World machines really don't seem to care. Gossamer and Artemis beige G3....well, they are Grackle machines, and a little weird, like the Yosemite. I consider these transitional, straddling machines, with one foot in the older PCI spec and one in the newer.

I generally treat the bG3 like a vintage machine, and the Yose like a New World machine.

Whether this is strictly correct or merely an odd habit of a gnarled Mac user is open to debate.