audio track count - what do I need?

[paul dolden]

Recently through a dealer I bought 8 SATA drives which are installed in a Burly enclosure in raid 0. They are connected to a tempo-X esata PCIX card. The dealer says they are getting 110 mono tracks at 24 bits, 96 khz. I was actually expecting more simulatenous tracks. Does anyone have experinece with such systems and know how many tracks this system should be capable of. Or :how many megabytes does one track of 24 bit , 96 khz need? The system is suppose to be capable of 400 megabytes per second and I think this should create a higher number of tracks than I am currently getting.

[M.Brane]

One thing to be mindfull of when pushing the limits of disk I/O with PCI cards is saturating the PCI bus. There's only so much bandwidth to be had, and running on the razor's edge can cause issues.

Usually there is a large gap between the theoretical, and actual capabilities of the cards.

110 simultanious tracks at 24/96 would be beyond overkill for me.

[paul dolden]

thank you for your response. However i still wonder if the difference between theoritical and real world performance can be as great as i am having. the card and hard drive is suppose to be capable of 400 megabytes. If I am calcuating the track correctly: one 24 bit 96 KHz file would be 96,000 X 3 (3-8 bit work)..therfore 1 track= 288,000 bytes per second. to get megabytes you divide 288,000 by 1024 and divide again by 1024 and you get .2746582. Since i can get 110 tracks we go 110 X .2746582 to get 30.27 megabytes-along way from 400 megabytes.However am i figuring this out correctly?
I should say that my music is several hundred tracks of simultaneous sound (each track is individually recorded in my studio). I have been doing this work for 25 years and i am trying to reduce the amount of track bouncing I have to do.

[TZ]

If you looking for speed, 8 SATA ports is not extremely useful on a G4 because 4 drives will nearly saturate its PCI bus. ataMan weighs in on SATA

So, is this in G5 w/ PCI-X 133MHz bus or something else? There is nothing in your message or user .sig to say what systems you are using. Barefeats: Sonnet w/ Raptor vs Seagate

More drives is not always better. And not all SATA drives are native or made the same.

RAID

[paul dolden]

yes all the equipment will be new. It will be a G5 using the PCI-X 133 mHz bus with the tempo-X ESATA 8 PCIX card. Rick Stephens has also been very helpful in the last 24 hours. He told me the Western Digital drives the dealer put in are not native- SATA drives. therefore this morning I have instructued the dealer to replace those drives with the new Hitachi 160 gig SATA II drives (the new ones)...or: the Seagate 250 Barracuda new SATA drives. Sorry if i am lacking in the right information, i am not a hardware guy. (i am a composer and string player)...that is why i appreciate your help and guidance because i am serious when i say i need several hundred tracks of simultaneous tracks....and unlike most musicians i have money burning a hole in my pocket. I thank you in advance for your help

[TZ]

Then I would skip the 160GB and go for Hitachi T7K250 250GB, or Seagate 250-300GB. Very little price difference in 160 and 250.

There are rather a lot of G5 models, some with 100/133MHz PCI (PCI-X) and some without. PCI-X systems usually also offer 8 RAM slots for 8GB - and Rick has some of the best PC3200.

[paul dolden]

thank you ver much i will ask the dealer to track down the T7K250.

[ataMan]

yes all the equipment will be new. It will be a G5 using the PCI-X 133 mHz bus with the tempo-X ESATA 8 PCIX card.

Sorry, but the Sonnet Tempo eSATA card does not fit in the PCI-X 133 MHz bus (=Slot 4) because the card uses double-stack connectors which are in conflict with a fan connector on the motherboard. Due this shortcoming the Sonnet card can be used in 100 MHz bus slots of G5 (the shared ones) only or you have to unplug the fan (and put your machine at risk).

http://www.barefeats.com/hard45.html

Please read under "SLOT 4 CAVEAT".

It maybe more benefitial for you to try our controller, the SeriTek/1VE4.

http://www.barefeats.com/hard46.html

It fits in the Slot 4, and if you need more drives and more speed, you can add a second controller. The price of a pair SeriTek/1VE4 is the same as of Sonnet card. The controllers are fully bootable (unlike Sonnet), have (IMHO) better hot-swap and yes, it's me who wrote the firmware. Our tech support is said to be better than of Sonnet, too, but I am highly biased; sorry for the "marketing".

All "native" SATA Hitachi drives work fine with the SeriTek/1VE4. The 500 GB just became available.

If you compare a pair of SeriTek/1VE4 controllers used in the "slow" (100 MHz) bus with the Sonnet card used in the same bus you will see the SeriTek is significantly faster at the begin of the disk and has a minor advantage probably within the margin of error at the end:

Read: 488 MB/Sec, Write 550 MB/Sec (Raptor, 10% full)
Read: 477 MB/Sec, Write 491 MB/Sec (Seagate, 10% full)

Read: 417 MB/Sec, Write 420 MB/Sec (Raptor, 90% full)
Read: 315 MB/Sec, Write 297 MB/Sec (Seagate, 90% full)

versus on Sonnet:

Read: 448 MB/Sec, Write 525 MB/Sec (Raptor, 10% full)
Read: 444 MB/Sec, Write 474 MB/Sec (Seagate, 10% full)

Read: 411 MB/Sec, Write 421 MB/Sec (Raptor, 90% full)
Read: 311 MB/Sec, Write 299 MB/Sec (Seagate, 90% full)

I also believe, Sonnet's solution to fit eight(!) connectors on the back of a single PCI bracket and squeeze eight shielded cable connectors to such a tight place is a challenge for the user. Sonnet uses the less common "Type-B" SATA connectors and cables which are in no way better shielded or safe than the cables the original burly-box or our box is using. Besides these cables are still too expen$$$$ive. The common shielded "Type-A" SATA cable we use would directly plug in into either "Burly" or our box, no conversion is necessary, you save a lot on cables - and it is safer too. You won't need any "patch cable" between a common box with "Type-A" connector and the PCI card like you need for Sonnet.

[paul dolden]

Thank you very much for your help again. Although i am sure you have a very good product I feel that my problems are much earlier in the system, at the hard drive level. As i mentioned in point #3 above, in this thread, I am only getting 30 megabytes of throughput with the Western digital hard drives. Currently the dealer is tracking down eight Hitachi T7K250 drives. Then they have to be installed and stripped for raid 0 and benchmarked. New question: what "buffer size" should they use when stripping the hard drives. Remember most of my audio files are only one or two minutes long. If I understand correctly, a smaller buffer size will mean that the files tend to be distributed across more drives. Thus using the real power of having a raid. Any suggestions?

[ataMan]

I am only getting 30 megabytes of throughput with the Western digital hard drives. Currently the dealer is tracking down eight Hitachi T7K250 drives.

I recall, there is an issue on Apple's SATA controller using WDC Raptor drives. I was a lazy ass to try it out, but some people did complain, you are not the first one. I will check it out, maybe it's just a driver bug from Apple. Mabe they did fix it in Tiger?

[TZ]

Artificial benchmarks are not reliable. That is why Barefeats and others will test various applications. for every drive in a RAID there is some overhead needed for the drive and controller that has to be taken into account.

I'm not sure that you can calculate tracks etc and end up at the theoretical MB/sec.

Usually someone will build their own basically, and test each drive fully (maybe for days to break it in) before then RAIDing the set, and even then do more testing and try different block sizes.

I just read about Digi ProTools and how they don't support stripping, and I am out of my water when it comes to audio applications.

with 8 drives, there is increased latency to find (seek) the required sector on the disk drive. If it is all sequential, it ought to be able to pre-fetch the next read. Some drives have different optimizations for write or read performance, they may vary. Some are better at random I/O than sequential.

An empty drive may appear to deliver 540MB/sec but drop to 300MB/sec at 90% full which is part of what was in the Barefeats link.

I posted earlier, but realized later I was thinking about another thread than this

[paul dolden]

Most of my files are about 65 MB (2'00 of 24 bit 96 khz)..they are each recorded one at a time but then i want to play several hundred of them at once. Would it be better in general to use a small stripe size or a large stripe size ?(the range is abour 2 kib to 512 kib?) Is there any advantage in performance to using a program for striping the hard drives like "Soft Raid" or is the programs that come with the Mac good enough?

A correction in my previous commentsabove #3)
I must have the wrong formula, if currently i am getting 110 of 24 bit 96 khz tracks, using the Western Digital hard drives, than i am getting 130 MB/s of throughput. However the system is suppose to achieve 400 MB/s. This seems like a big difference to me.

For those just joining this thread I will be using:
-8 Hitachi T7K250 SATA II drives(drives being changed from Western Digital to Hitachi as I write)
-Tempo-X esata 8 card
-new G5 with PCI-X slot

About Digidesign stuffthese comments should bring lots of responses-i was a protools guy for 10 years and flipped to logic audio and native solutions 6 years ago....so i have some experience)
first rule:digidesign wants to sell you as much hardware as possible. currently they sell you another hard drive and bus card etc for every 10 or 16 (or something) tracks you want right now. (i willl not even begin to discuss all the gear for inputs and outputs you have to buy for your rack just to operate a protools system).Their entire design is based on early 90s concept that computer and hard drive speeds are not fast enough and they need to sell you their own boxes that do a lot of the work. That is why many of us have abandoned the company and put together our own system to meet our own needs using just software solutions like logic audio and the dsp of modern computers to do all our calculations etc. Even with the Western Digital hardrives i am way ahead of a protools system at a fraction of the price. I believe at this point in time even digidesign will not promise more than 48 tracks of 24 bit 96 khz. It is like all the developments in computers and hard drives of the last 10 years (raids, controller cards, etc) has not occurred. Remember most people in audio only need or want 24 or 36 tracks. I am unique in my dense multitracking and always face trying to do the impossible every time i buy gear.,My needs are more in harmony with demanding throughput of visual artists(ir whoever would be reading this thread)-however to return to my question the problem is my file size are probably quite different and that i want to access several hundred audio files at the same time would therefore greatly affect how things should be set up.

[jsnwolf]

Assuming the dealer your speaking to has the EXACT same system as you do, you should ask him the following information:

What is the data in/out per second reported by Activity monitor for that system (system like yours)?

and

What is the CPU usage % when running the full 110 mono tracks?

The details presented here are still really vague. The entire system must be evaluated for an overall performance _estimate_.

From my past experience (working for Digidesign) with SCSI RAIDS (depending on RAID Type), the speed and thurough-put is overkill. Sure, this was still when audio was "the best" at 24bit/48K. You had other inherant problems though, such as PCI buss timing, slave/master clock sync issues, clock jitter, all influence by the SCSI controller being present. That was SCSI though... not SATA. I understand your question here...

The information you seek is very expensive to benchmark, in time and research - does EmApple say anything about SATA RAID configurations and offer compatibility informatoin? Not what I can tell from the product information presented free on Apple.com. So, it's not really so true that Digidesign is in it for the money. That's a very easy attitude to take about a company that is on top of the Digital Audio World Food Chain. It's all about testing and supporting a system they don't need to.

I gotta go home now, so I'm signing off. There's a lot more information to cover here, and I'll offer as much as I can - I plan on testing a SATA RAID with my sample library (which will take me about a year to import from DVD's and CD's... but it's on the way).

until next time...

[paul dolden]

What is the CPU usage % when running the full 110 mono tracks?
-the dealer stoppped at 110 tracks because the the overall performance meters were getting full. However remember logic now has off-line (non-real time) bouncing capabilities so even if you cannot run it real time you could still calculate your final mix (just like when i started working on mainframe computers doing music in 1978).


From my past experience (working for Digidesign) with SCSI RAIDS (depending on RAID Type), You had other inherant problems though, such as PCI buss timing, slave/master clock sync issues, clock jitter, all influence by the SCSI controller being present. That was SCSI though... not SATA.
-my current system (6 years old) is also based around SCSI and Chettah hard drives in a raid. and yes the SATA situation is very different. even at 110 tracks the new system (with Western digital hard drives) is 6.75 times more throughput than my old SCSI system.
The information you seek is very expensive to benchmark, in time and research - does EmApple say anything about SATA RAID configurations and offer compatibility informatoin?
-no that is why i am on this forum.I seem to be alone in the world in my lust for a 400 track multitrack studio (and yes my music is first written out with all 400 tracks-the scores are 6 feet high.)

So, it's not really so true that Digidesign is in it for the money. That's a very easy attitude to take about a company that is on top of the Digital Audio World Food Chain. It's all about testing and supporting a system they don't need to.
-all companies are in for the profit margin -it is called free market.....remember digidesign is on top of the food chain because they were the first out of the block in 1985 and all pro studios around the world have their people trained in these systems. This does not mean that they are the best
at incorporating new design ideas both from the musical and computer world. I will argue they have the best audio editing (sample based rather than tick based-therefore more accurate for lining up phase relationships between tracks). However everyone knows they are very propietery.I know too many people like me who tell clients they are working in Pro-tools but in fact do all the work in a program like Logic using the DSp power of the computer. Also Avid (digdesign) liscencing fees must be huge: For example my mastering plug-ins (Waves Platinuum) cost $2,500 for VST version (i.e. Logic, digital performer etc) The same bundle for TDM (digidesign) cost $4,000 (remember i live in Canada). All plug ins that cover both formats have the same type of difference in price. And all the young producers (hip hop onwards) tend to use Logic etc because a lot of sound processing plug in developers have not been licensed to the propietery format of TDM- i presume because of licensing fee that the Avid corporation demands...
-anyways we should move this discussion to an audio forum...does anyone out there know what stripe size i should use for my type of audio needs (please re-read or read my previous message).

-but overall you are right-what i am trying to measure is hard...however i want to insure i am getting the best performance. remember i am a composer/musician/audio engineer-not a hardware guy so i am trying to learn this world but i am struggling at the beginers level. I thank you for you comments anyways but back to theories or ideas for stip size please or other tips to maximize my system. My other question is there any real advantage to using "Soft Raid" to do the job rather than the native mac stripping program...Remember money is not an issue for me.
I

[TZ]

Disk drive performance. I have 'last generation' SCSI drives that are admirable and deliver 65MB/sec (10K) and 72-75MB/sec (15K) that are two years old. I've been contemplating upgrading now that 10K is up to 85-89MB/sec and 15K is 96MB/sec (nearly 400MB/sec with four, we have a review of 4 x 15K II drives elsewhere). I was disappointed with Cheetah 10K.4's 33-36MB/sec four years ago, but it has come quite far.

At the same time, 7.2K ATA-6 drives are now maxing out what the interface can handle (60MB/sec) and SATA versions clock in at 65MB/sec. There was a time when it seemed to be taking a long time to hit UW 40MB/sec level or beyond.

[paul dolden]

my research shows that the 100 MHz slot PCI-X on a new G5 should be able to handle 800 MB per second.
-the tempo SATA X card can handle 400 MB
-however with the 8 western Digital drives i am only getting about 120 MB
-therefore I think i still need to focus on the hard drive issue. I do not see problems in bottlenecking at the card or slot anymore. It is still about getting that information off the hard drive in the most fast and efficient manner possible.

[TZ]

8 Atlas 15K II won't deliver 800MB/sec. (96MB/sec w/ the best drive).

Subtract bus commands between drives and controllers, too, which can take 10% or more off theoretical.

But yes, it sounds like a problem. PATA <=> SATA bridge. Some drives are not tested or supported (qualified) for a controller. FirmTek usually does more on that level than Sonnet has.

Eight Raptor drives won't deliver more than 560MB/sec, other SATA drives will of course be less so.

A single bad cable can also be a problem. Sometimes one drive is not acting properly, which is why initial tests are done one drive at a time, just to insure. And most of use zero a drive before using, just for insurance.

[jsnwolf]

Paul, first of all, let me congratulate you on the success you have found as a composer and musician. I have just finished reading an online bio of you and I applaud your great accomplishments.

You probably have heard of some of the companies I have worked for (all in my young adult age): Seer Systems, Opcode, Digidesign, Koblo, Apple, and now Giga Designs. The focus of my professional career has always been music technology. I've not only recieved a paycheck from these companies, I own stock most of them and participate in un-public projects that haven't even hit the market yet. I guess I got my finger on the pulse when it comes to Audio Technology - I've been a member of AES ever since I went to college and have been to dozen music technology conferences around the world. I almost became a nephew to the great Dave Smith (Sequential Circuits fame) and I've went on tour with world wide acts like Limp Bizkit (having built the onstage ProTools rig used for live performances). I have been behind the scenes of music technology since I was able to drive a car,but I don't have nearly the celebrity status of someone like yourself. I have worked closely and made friends with great artists such as Herbie Hancock, Chick Korea, Peter Gabriel, and many other "no-names". I'm not here to drop names, or anything, I just want you to understand where I'm coming from as a contributer to this forum - just like I took the time to understand where you're coming from.

After reading your posts, and your bio, I understand "what" you are trying to achieve. What I don't know is everything about "how" you plan on achieving that. If money is no issue to you, why then are you really that concerned about paying Digidesign for their proprietary hardware? Everyone else does (Rhett Lawrence, Walter (Wendy) Carlos, THE PLANT in Salsalito) If you want, I could have a ProTools rig capable of playing AND RECORDING 400+ tracks - and you'd never have to worry about how it was doing it - you would just hit the space bar, and could record 400 sources at the same time - my buddy works at SkyWalker Ranch, and I have visited the Lucas Arts sound stage many times - this is the place where they record the soundtrack for Starwars. I would tell you how they do it and with what equipment they use (because yes, they achieve 400+ tracks and have a mixing console that requires 6 operators and have been doing this since 1978), but I'd violate my NDA. I'll give you a hint though, they're using Fiber Channel now.

So,it is possible to get what you want in a system, but if you're hell bent on using a SATA RAID to achieve it, then you'll have to do the research on it. Most folks like myself in these forums don't have 00's of 000's of dollars (US or Canadian) to be used for determining what "the best" is. But if you do, great! We all want to know...

I'm sorry this post has nothing to do with RAID configurations or stripping, but if you truely want a 400+ track capable machine, and money's not an issue, then you can have it by next week and I can make it happen. If you want to know untested, undocumented information regarding expensive hardware, then you'll have to put the time into it (and money) to find out... and it won't be as easy as finding that guy in a forum to tell you what it is. This is a community and we're all here to offer what we know and learn what we don't. Welcome to the forums and good luck with your project.

P.s. - did you ever find out from the dealer what the disk i/o was? You didn't say in your last post. If it was anything less than the output of the SATA RAID being used, then the track limitation is influenced by the system performance and the software it's running, not the actual RAID itself. This is a very important piece of information, and until we know what that number was, I won't believe that the SATA RAID can only output 110 mono tracks.

[jsnwolf]

The Sonnet Tempo-X eSATA 8 card can achieve 150MB/s per port and also uses Bus Mastering (PCI bus locking) which creates PCI traffic right away on the bus. Any audio program is going to be constantly monitoring the PCI bus, and so Bus Mastering will definatly affect bandwidth.

The benchmarks Sonnet has on their site (BlackMagic Disk Speed Test) was "obtained using Power Mac G5 Dual 2.0 GHz running Mac OS X Version 10.3.6, BlackMagic DeckLink HD Pro card in slot 2, Tempo-X SATA 4+4 card in slot 4 with Maxtor MaXLine III 300 GB Serial ATA disk drives formatted in RAID 0 arrays using SoftRAID 3.1".

I bet that DeckLink HD card is taking up some bandwidth on the PCI buss too, which would explain why even though they have 8 drives capable of 150MB/s output _each_, they're only getting approximately 485MB/s read/write. And who knows if that's sustained read/write. Sustained (not burst) thorough put might be even less.

Going by the bandwidth requirements of PCM 24-bit/96K mono audio track and the given benchmark above, theoretically (without any “other” PCI interference) you’ll get 176 tracks. But that’s with no other PCI peripherals on the bus.

Also Paul, if you're running Tiger Mac OS 10.4.2, make sure you have Spotlight disabled and that you are not connected to the internet (or at least have automatic software update disabled). These two things also affect drive performance drastically when they are indexing. A Norton Utilities box or install CD should not even be in the same room as the G5.

The Maxtor MaXLine III 300's have a 16 MB cache. The Hitachi T7K250 SATA II drives you’re using only has an 8 cache buffer, so I bet that influences the i/o as well… even though, you should be obtaining a rate above the 120MB/s you’re getting. What other hardware is on the PCI buss and what program are you using to monitor – Activity Monitor? Are you actually playing tracks from Logic? I am really curious to know now since there is something obviously affecting your bandwidth… but where?

[unclemac]

A Norton Utilities box or install CD should not even be in the same room as the G5.

HAHAHAHAHAHA!