PDA

View Full Version : Relevance of ATTO scores



tmxmnr
05-28-2002, 09:20 AM
I've just done some of my first testing with ATTO. What do the ATTO benchmark scores actually measure in terms of real world performance? (I tried to find an answer by searching the archives, but came up with nothing).

For example, with a single drive the only ATTO score that changes with any significance between the internal bus, an ATA/66 and an ATA/133 card on my QS 733 is "peak read". It is about 40 mb/sec on the int. bus, 50 on the ATA/66 card and 70 on the ATA/133 card.

Sustained read and write and peak write are all basically the same, hovering around 40 to 42 MB/sec mark, regardless of bus.

Is a "peak read" score indicative of the first time access of data on a drive? It would seem so, because both peak read and write occur at the beginning of an ATTO test. However, I don't understand then why "peak write" wouldn't improve equally as much as "peak read" from bus to bus.

Thanks,

Anthony

TZ
05-28-2002, 10:11 AM
Anthony,

Rick has been looking into a RAID database and how best to capture and report user scores. If you want real performance benchmarking I'd take a look at www.storagereview.com (http://www.storagereview.com) where they have more robust testing; there is also a background report on what goes into benchmark utilities and what they use.

MacBench 5 offers some other useful benchmarking; Intech HDST comes with QuickBench; Apple has an OS X DiskBasher performance tool.

A stopwatch and real world operations of what you do is still one of the best, and you'll see that in some of the reports on www.xlr8yourmac.com (http://www.xlr8yourmac.com)

Gregory

ricks
05-28-2002, 01:44 PM
Hey Anthony,

I wish I could claim some real expertise in the benchtest field, I can't. For what it's worth, here's my take on it.

Each bench program tests for somewhat differing attributes via differing methods. Sometimes it is hard to know exactly what you're looking at when you see results.

ATTO does a good job of being repeatable, which is why it's useful. ATTO's peak numbers reflect more on the bus and cache capabilities than on actual platter to head performance. With caching getting faster and faster as well as more versatile, it's sometimes hard to tell if you are testing cache or drive performance.

In testing RAID0 arrays you will see the peak reads/writes drop off seemingly predictably in direct relation to how large the composite cache size is. I say seemingly because I keep finding situations where adding up the total cache of all the drives in an array to get a 'composite total' doesn't always directly relate to where you see a drop off from peak to sustained through-put, wish I knew for sure how come. I would guess that how smart the drives are and how efficient the OS and RAID software programs are is a large factor in those differences.

Kaye talks about burst rate as opposed to peak rate. A drives ability to handle small bursts of data is at a much higher performance than sustainable large file transactions. The peak scores you see are a good indicator of how well a drive handles burst access.

Sustainable through-put is the hardest thing to measure. ATTO has a definite shortcoming because it only measures a maximum data transfer of 8MB or less. A large array can handle 8MB transactions from within the cache. Also, being that sustained performance benchmarks are an average of throughput results, a really high burst rate over a long portion of the test will skew the reported sustained data to a much higher level than actual true real life testing.

The other problem with sustained performance and real life performance is the type of data being accessed and the load the rest of the bus is under for said actual performance. Hard to test for the infinite possibilities. Even something as 'steady' as an audio file being recorded has no continuity as far as the drive is concerned. The data is still 'broken' up into pieces as it's sent to the drive. Requests and Acknowledgments fly back and forth, system housekeeping and drive indexing as well as other PCI and cpu tasks all affect the drives actual sustained rate.

Using benchmarks from programs like ATTO, Intech and FWB are only useful to make comparison of snapshots of your drive/bus/host card/software performance. A limited and tiny test of true drive performance. Full performance testing and graphing would give us too much data to compare. Wait till you see how many things that Storage Review is testing for. For each specialty there are separate requirements, some of the terminology might as well be in Martian for the sense they make to me. (read 'arcane black arts')

But the tests are still valuable.

Rick

Damien
05-28-2002, 11:51 PM
I have been repeatedly told that no single ATA drive yet made can max out an ATA66 bus in "sustained" throughput This seems to be backed up by the "higher burst rate on faster bus with no increase in sustained" that you (and everybody else) are seeing

Seems mid 40's is about it for a single drive but should you get a raid card you will want the 133 card (high burst rate and about 80MB a sec sustained!)

------------------
Damien,

Damien's Stuff (http://www.macmeisters.com/Damien/)

tmxmnr
05-30-2002, 07:30 AM
Thanks for the replies.

So ATTO's benchmarks may actually have more "value" in measuring IDE drives because, until WD's special edition drives with 8 MB buffers, no IDE drive has more than 2MB of buffer? Perhaps, in an OS X release, ATTO ought to consider raising the test size to 16 MB?

I looked at storagereview.com... fairly technical, though I did find the harddrive tutorial to be fascinating, mostly understandable and quite thorough.

It appears in terms of "matching" an equivalent test, the stuff to look at on storagereview.com are the "disk read/trasfer rates". Obviously, the scores between it an ATTO aren't comparable, but to get a feel for what drives of equivalent price/specs perform best. Of course, I would bet 99% of SR's tests are done on Winders, and just the data that is presented here on the RAID database I have found to be extremely useful, especially in terms of what kind of performance gain to expect between the internal bus on a given machine v. using a card (and it is all on Macs!). http://macgurus.com/infopop/emoticons/icon_smile.gif

I think what it really comes down to is if you do video and can afford SCSI, ya get SCSI... that's all there is to it. Otherwise, IDE will do the job unless you are a performance hound (and/or not on a budget). I know Rick has said a number of times on the Furums that you can get a pretty nice IDE RAID set-up for 300 bucks. You can't even get in the SCSI door for that.

When I strike out on my own (which may be sooner rather than later now that the Feds are laying off), I'll look into SCSI.

Thanks for the input,

Anthony

[This message has been edited by tmxmnr (edited 30 May 2002).]