PDA

View Full Version : 8-bay Burly, 6x G2 X15s, UL3D, G4-800DP



kaye
11-30-2001, 07:02 PM
I can confirm what MacMikester said here http://www.macgurus.com/ubb/Forum1/HTML/000206.html (I'm nonsensical my wife says).

To see where the limits were, I built up the Burly first with 2x G2 18GB X15s, one on each channel, all Granite external LVD cabling and Burly internal LVD cabling and LVD terminators, then 4x X15s with two per channel, finally 6x X15s with three per channel. All SoftRAID 2.2.2 with ATTO UL3D firmware 1.64, OS9.2.1.

These are new drives and will be faster when they wear in. I did not optimize the directory or the drives or change factory default Mode Page Parameters. Just ran the tests once for each config. ATTO test, 8MB, Sample Size 2, no System Disk Cache. MacBench 5, Disk and Pub Disk tests.

UPDATE 12/01/01 - I forgot to mention yesterday that I left my internal 2x G2 X15s on one channel striped RAID attached to the UL3D and running. So that may have been hogging some bandwidth tho I ran the ATTO and MB5 tests from one of the ATA drives. I will rerun the Burly 6x striped RAID tests later with the internal RAID disconnected.

2x G2 X15s, one per channel, 256SU:
----------------------------------
ATTO test, PR-213.88, SR-111.53, PW-161.21, SW-118.11
MB5, D-5297, PD-4467

4x G2 X15s, two per channel, 256SU:
----------------------------------
ATTO test, PR-204.02, SR-190.31, PW-201.65, SW-196.66
MB5, D-6101, PD-4541

6x G2 X15s, three per channel, 256SU:
------------------------------------
ATTO test, PR-218.54, SR-194.94, PW-204.06, SW-197.93
MB5, D-6337, PD-4676

6x G2 X15s, three per channel, 128SU:
------------------------------------
ATTO test, PR-218.17, SR-193.60, PW-198.72, SW-186.86
MB5, D-6158, PD-4743

Note, the smaller stripe size will increase PD in MB5. The larger stripe size will increase D. I have seen this numerous times before. k


[This message has been edited by kaye (edited 01 December 2001).]

kaye
12-05-2001, 12:45 PM
Further testing, first back to 6x G2 X15s, three per channel, 256SU. Disconnected internal striped RAID. Then optimized the directory and optimized the RAID with Speed Disk.

6x G2 X15s, three per channel, 256SU:
------------------------------------
ATTO test, PR-214.75, SR-196.78, PW-202.85, SW-197.73 (about the same as previous 256SU test)
MB5, D-6463, PD-4728 (up 2% and 1% respectively)

Yesterday I configured for 2x UL3D and three drives each on one channel of each UL3D. Rebuilt the striped RAID and optimized directory and drives.

6x G2 X15s, 2x UL3Ds, three per one channel, 256SU:
-----------------------------------------------------
ATTO test, PR-232.22, SR-203.94, PW-206.01, SW-202.26

This configuration adds about 7MB/s to SR and 4.5MB/s to SW. For the cost of another UL3D, not significant enough for me. k

Taffy.C
12-05-2001, 08:36 PM
Kaye,
Wow - thanks for doing this research - its great that the theoretical and actual results line up. One question - I'm guessing that the difference between the 4 and 6-drive arrays might be more pronounced in a worst-case (ie. near the center of the platter) situation. Can you test that??

Taffy.C

MacMikester
12-06-2001, 12:12 AM
http://macgurus.com/infopop/emoticons/icon_wink.gifMr. theoretical says:

The differences will be the same if you are testing the inner tracks in all configs. Only the actual throughput numbers will be lower, but the differences will be about the same.

[This message has been edited by MacMikester (edited 05 December 2001).]

kaye
12-06-2001, 12:52 AM
Taffy.C,

I agree with Mr. Theoretical. I will test the six drive array off of one UL3D by creating the smallest second partition I can. Is that what you are looking for, or did you want me to try that with both six and four drive arrays on one UL3D? Let me know.

BTW, these six new G2 18GB X15s have firmware 0004 and all even have the same build number. The two G2 X15s I purchased earlier are firmware 0002 as I recall. Anyway, these 0004 drives have slightly higher SW. All the Mode Page Parameters are identical. So the difference appears to be firmware or I suppose it could be something mechanical. k

Taffy.C
12-06-2001, 01:59 PM
Hi Kaye,
I was interested in a comparison of 4-drive vs. 6-drive performance on the inner platters with one UL3D. My thought was that in the best-case scenario both configs are running pretty close to the PCI slot max throughput, so you don't see much difference - i.e. they are limited by the PCi performance. But in a worst case situation where you are on the inner platters, the speed will be determined by array performance. And 6 drives must be faster than four? Although.... Seagate does spec the X15s at 51MB/s worst case, so 4 x 50 is still 200MB.

The reason I am interested in this is that for high-def 10-bit video I need sustained throughput of 170MB/s and I need to understand how many X15s I need to support this in the worst case. Probably 4, but I'd love it if you could do the measurement.

.... Taffy.C

TZ
12-06-2001, 05:43 PM
One nice think Seagate has managed with the 2G X15 is that it does not really seem to degrade as any other drive will as it gets loaded (though I believe in keeping to 60% full or less). The need to stripe outer tracks might not be as necessary. Usually I'd create a 2nd striped volume for slower static files on the inner most tracks. Single drive, keeps right up in the 51MB/s SW (and almost the same for reads).

Gregory

kaye
12-07-2001, 12:36 AM
Taffy.C,

I'll try it tomorrow morning, both 4x and 6x drives on two channels on one UL3D. k

kaye
12-07-2001, 05:35 PM
Taffy.C,

G4-800DP, tested the Burly with 6x G2 X15s on one UL3D, three per channel. And tested with 4x G2 X15s on one UL3D, two per channel. Rebuilt each striped RAID, optimized the directory and defragged then optimized the directory again.

For the 6x, I created two partitions, the first 104409MB, the second 600MB (100MB per drive). This is really the inner tracks of each drive. I don't think anyone would do serious work with only a total of 600MB left. But the test:

6x G2 X15s, three per channel, 256SU, Partition 2 (600MB):
---------------------------------------------------------
ATTO test, PR-220.83, SR-181.90, PW-202.57, SW-195.12
MB5, D-6281, PD-4578

The above ATTO test (8MB, Sample Size 2, no System Disk Cache) was the best of 12 tests. The SR range 176.80-181.90 with most around 179, SW range 186.65-195.12 with almost all in the 193-194 range. This config would more than satisfy your need for a minimum of 170MB/s.

For the 4x, I created two partitions, the first 69606.5MB, the second 400MB (100MB per drive).

4x G2 X15s, two per channel, 256SU, Partition 2 (400MB):
-------------------------------------------------------
ATTO test, PR-214.28, SR-153.68, PW-183.58, SW-167.03
MB5, D-5805, PD-4320

Whoa, not so good. Once again, the above ATTO test (8MB, Sample Size 2, no System Disk Cache) was the best of 12 tests. The SR range 151.21-153.68 with most around 152-153, SW range 163.54-167.03 with most around 166. This config would not satisfy your need for a minimum of 170MB/s.

Not satisfied with that, so I created a new 4x, two partitions, the first 67958.5MB, the second 2048MB (2GB), thinking that perhaps a more reasonable 2nd partition of 2GB would bring the performance back.

4x G2 X15s, two per channel, 256SU, Partition 2 (2048MB):
--------------------------------------------------------
ATTO test, PR-217.03, SR-153.58, PW-183.84, SW-167.84
MB5, D-5874, PD-4413

Again the above ATTO test was the best of twelve. No point in the ranges. Still does not meet your needs. And contrary to what I thought before, the drop with 4x and dual channel is much more significant than the drop with 6x on the inner tracks. My guess is because the 6x dual channel on the outer tracks reaches a PCI slot ceiling with one UL3D before it reaches a striped drives ceiling whereas the tests on the inner tracks is reaching the striped drives ceiling, if that makes sense. So many numbers, hope I copied it all correctly. k

[This message has been edited by kaye (edited 07 December 2001).]

Taffy.C
12-08-2001, 07:58 PM
Kaye,
Thanks a bunch for all this work! What you discovered was exactly what I was concerned about - that the 4 drive array performance would suffer on the inner platters. I will have to upgrade myself to a 6 drive array - does the need for more money never cease!

One further question - I don't understand the Seagate spec of 51MB/s min for the G2 X15. I thought that that would mean that the worst-case (drive limited) performance would give me 204MB/s min for a 4-drive array. Am I missing something?

Thanks again ... Taffy.C

kaye
12-09-2001, 03:39 AM
That spec INTERNAL FORMATTED TRANSFER RATE (MB/sec)_51.8 to 68.1 is an internal transfer rate somewhere between disk and onboard cache, not the external rate, which they only give as:
EXTERNAL TRANSFER RATE (mbyte/sec) _______40 Sync
Low Voltage Differential(LVD) _______80/320 Sync

In addition to the drives, your rate is affected by your SCSI card, PCI bus speed, RAID overhead, stripe size, Mode Page Parameters, and the test utility (and probably others that I can't think of at this time of night). k

dragon_x
12-09-2001, 11:43 AM
All these tests are very interesting, but I think I (and others?) would be interested in what you can do with the same hardware in
Mac OS 10.1

I would also like to see FINDER copies & duplications in Mac OS 9 & 10.
I bet there would be better peformance with a 4 drive stripped array in 10.1 than with classic. Does the RAID software in 10.1 even support more than two drives on a single volume?

There was a simple test done on networking -
100baseT vs. 1000baseT vs. 1394 IEEE on Windows (2K I think) and Mac (system 10.1). 400Mbps Firewire is almost exactly as fast as 1000baseT. What is more - all these 'networking' technologies are faster on a MacOS 10 powered system (867MhzG4 vs. 2Ghz P4 - IIRC).

Hopefully SoftRAID will be out for 10 in 6 months... maybe by next Christmass.

------------------
To boldly go where no english professor has gone before! -)

dragon_x
12-09-2001, 11:47 AM
At 170MB/s you will need about 10GB of space per minute of video. http://www.macgurus.com/ubb/eek.gif For one hour of video you will need 600GB.

Maybe you really need the 4x36GB x15 setup anyway - since it can only handle 43min of video. Maybe Seagate needs to get those 72GB x15s to market? http://macgurus.com/infopop/emoticons/icon_smile.gif

kaye
12-09-2001, 01:10 PM
d_x,

Yup, I have to wait for SoftRAID to handle OSX. If I recall correctly, Taffy.C has 4x 36GB G2 X15s, and is considering adding two more.

You lost me on the math. If "you will need about 10GB of space per minute of video. For one hour of video you will need 600GB. Maybe you really need the 4x36GB x15 setup anyway - since it can only handle 43min of video." How does 43 minutes of video with 4x 36GB compute, if you need 430GB for 43 minutes? I'm not disputing your 10GB per minute, just can't make the math work. k

Taffy.C
12-10-2001, 12:57 AM
Hi Kaye, d_x,
Yep - some 73G X15s would be nice. I am probably going to end up with at least 8 x 36G drives I suspect - just to get some reasonable storage time for High -def.

Here's another thought that I am wondering about - when you output an edited video to tape from the Non-Linear-Editor, the NLE is basically throwing together lots of separate clips (files) from the array into a continuous stream of video.

The array performance tests which we have been talking about here so far are for continuous read/write of a single file. How do we measure the effective sustained read speed when the array is accessing multiple sequential files all over the disk. Any thoughts on that?

.... Taffy.C

kaye
12-11-2001, 05:41 PM
Taffy.C,

You have stumped me there. But the seek and average access time of the drives is very small. One other point, the ATTO test is not one file size, actually a whole variety of file sizes, tho undoubtedly contiguous, from 32KB up to 8MB, and more than the selectable sizes shown in Max Transfer Size (according to the ATTO docs). k

MacMikester
12-12-2001, 11:55 PM
Hey k, QuickBench Random Read/Random Write tests should approximate the result that Taffy is looking for. The choices of smaller file sizes may not be quite the same as as streaming huge video files together but should give him some idea of the worst-case scenario.

kaye
12-13-2001, 02:55 PM
Great idea MM. Only drawback is that it will only test RR and RW up to 1MB file sizes. The 10MB size won't do it, only sustained read/write. Taffy.C, want to try it and post results? k

[This message has been edited by kaye (edited 13 December 2001).]

dragon_x
12-13-2001, 07:54 PM
How does 43 minutes of video with 4x 36GB compute, if you need 430GB for 43 minutes?
You are right... I must have done crazy MATH in my sleep or something.
http://www.macgurus.com/ubb/confused.gif

4x36GB is only about 15 minutes of video at his data throughput requirements. I think I mis read 4x36 AS 436GB - thus the 43 minutes of video. Of course if someone could invent a RAID cluster that could do this - they would get RICH! http://macgurus.com/infopop/emoticons/icon_biggrin.gif

So what kind of video requires 170MB/s data? I gotta know!

I gues it would be higher resolution than MPEG2 with NO Compression at all? Would this be for movies or iMax movies?

Many - you are going to need one hell of a backup system. I think the biggest tapes only do about 80GB compressed (2:1) - but you can get a bank of tape drives - those DLT 4 drives are EXPENSIVE - about $3,500 for a single drive + tapes + cables + card + software.

I think an 8 drive RAID-0 setup is the biggest (drive wise) that runs with SoftRAID.... I've never heard of any body trying anything bigger.

------------------
To boldly go where no english professor has gone before! -)

kaye
12-14-2001, 12:45 PM
I wish I had some of these benchmarks http://www.athlonmb.com/article-display.php?ArticleID=51&PageID=4 k

kaye
12-15-2001, 01:27 PM
With my PTP mobo out and on the way to Trag for his 9600 Kansas ROMs mod, that frees up a couple of Granite LVD terminators. I have an internal Granite LVD GD4700 34" cable, longest I have, so with both the G4 and Burly open, I think I can jury-rig a config of 8x G2 X15s off of two UL3Ds set up with 2x drives per channel, if the GD4700 will reach. Two of the channels will be internal (one tho will go to the Burly via GD4700) and two external. Will see if that works. Probably start that project tomorrow and need to backup some stuff off of the 2x drives in the G4. Will post results. k

Taffy.C
12-16-2001, 12:57 AM
Hi All,
Well K, I would love to run that benchmark and post the results, however it will have to wait a while - some nice thieves broke into my office and stole my G4/733, Cinema display, UL3D, and a pretty expensive video capture card. So, I am without a system until the insurance check comes through. Funny thing though is that they left my 4x X15 36G cheetah array - what fools!

d_x, the 170MB video is high-definition 1080 line interlaced 16:9 aspect ratio sampled at 10 Bits/Byte. Fully uncompressed. It might be closer to 160 actually. You typically create high definition content source fully uncompressed and then run into a real-time MPEG2 compressor. Normally, it is 8 bits/byte which gives a slightly lower data rate, but for motion graphics 10 bits/byte provides a better source environment. That was probably more than you wanted to know! but you got it anyway.

Taffy.C

kaye
12-17-2001, 02:48 PM
TaffyC,

That is terrible. Don't know what to say. I hope your insurance comes thru for you. In the meantime, what are you running? You might want to wait for Macworld SFO 01/07 - 01/11/02, expo runs 08 - 11, to see what is new and, of course, also find bargains on the existing QuickSilvers online.

I do have an update on 2x UL3D, 8x G2 X15s, two per channel. It's a monster. Save for another post. k

kaye
12-17-2001, 04:39 PM
This is a real jury-rig but it works. 8x G2 18GB x15s, 6x in a Burly all Seagate firmware 0004, 2x in G4-800DP with firmware 0002. 2x UL3D with firmware 1.64, two X15s on each of the four channels. Four of the X15s in the Burly, two per channel on one UL3D, are connected via Granite external LVD cabling, two of the remaining X15s in the Burly are connected via Granite internal LVD cabling and the remaining two X15s in the G4 are connected via Granite internal LVD cabling to the other UL3D, both channels. Total eight striped drives, two per each UL3D channel. Four Granite LVD terminators. Two very open boxes, both G4 and Burly. SoftRAID 2.2.2 with two partitions, a very large one and a very small one (800MB).

8x G2 X15s, 2x UL3Ds, two per each channel, 256SU:
-----------------------------------------------------
ATTO test, PR-236.56, SR-218.36, PW-205.06, SW-203.18
MB5, D-6538, PD-4829

8x G2 X15s, 2x UL3Ds, two per each channel, 256SU, Partition 2 (800MB):
---------------------------------------------------------
ATTO test, PR-236.12, SR-206.12, PW-204.16, SW-202.11

I will test this config some more but so far everything is consistently above 200MB/s sustained read/write for both partitions. k

kaye
12-20-2001, 01:47 PM
Still playing with this rig. New drives getting faster. Latest this morning:

8x G2 X15s, 2x UL3Ds, two per each channel, 256SU:
-----------------------------------------------------
MB5, D-6794, PD-4832

ATTO tests, will try tomorrow. Too many rugrats and Christmas stuff to do. k

Santilli
12-21-2001, 06:51 AM
Kaye:
But does it play quake? Awesome. Can I stop by and play with it;-)?
gs

kaye
12-21-2001, 01:49 PM
gs,

Of course you can stop by. I forget, are we near each other? I don't have Quake on this G4. Think it is on the PTP but that is in pieces for a mod by Trag. If you are serious, stop by, install the app for me, and play to your heart's content. I'm not a gamer so I will watch over your shoulder to learn about doing a game. Just bear in mind that this config is scary with both boxes, G4-800DP and Burly, wide open. k

Santilli
12-27-2001, 06:22 AM
I'll take you up on that offer soon, Kaye. Yes, I sub teach in your area, Mt. Diablo School District in particular.

gs