Results 1 to 20 of 33

Thread: audio track count - what do I need?

Hybrid View

Previous Post Previous Post   Next Post Next Post
  1. #1
    Join Date
    Jul 2005
    Posts
    9

    Default audio track count with 8 sata drives

    Recently through a dealer I bought 8 SATA drives which are installed in a Burly enclosure in raid 0. They are connected to a tempo-X esata PCIX card. The dealer says they are getting 110 mono tracks at 24 bits, 96 khz. I was actually expecting more simulatenous tracks. Does anyone have experinece with such systems and know how many tracks this system should be capable of. Or :how many megabytes does one track of 24 bit , 96 khz need? The system is suppose to be capable of 400 megabytes per second and I think this should create a higher number of tracks than I am currently getting.

  2. #2
    Join Date
    May 2001
    Location
    1hr N/W of LA LA Land
    Posts
    3,315

    Default

    One thing to be mindfull of when pushing the limits of disk I/O with PCI cards is saturating the PCI bus. There's only so much bandwidth to be had, and running on the razor's edge can cause issues.

    Usually there is a large gap between the theoretical, and actual capabilities of the cards.

    110 simultanious tracks at 24/96 would be beyond overkill for me.

  3. #3
    Join Date
    Jul 2005
    Posts
    9

    Default more information

    thank you for your response. However i still wonder if the difference between theoritical and real world performance can be as great as i am having. the card and hard drive is suppose to be capable of 400 megabytes. If I am calcuating the track correctly: one 24 bit 96 KHz file would be 96,000 X 3 (3-8 bit work)..therfore 1 track= 288,000 bytes per second. to get megabytes you divide 288,000 by 1024 and divide again by 1024 and you get .2746582. Since i can get 110 tracks we go 110 X .2746582 to get 30.27 megabytes-along way from 400 megabytes.However am i figuring this out correctly?
    I should say that my music is several hundred tracks of simultaneous sound (each track is individually recorded in my studio). I have been doing this work for 25 years and i am trying to reduce the amount of track bouncing I have to do.

  4. #4
    Join Date
    Jan 2001
    Location
    Mobius Strip
    Posts
    13,045

    Lightbulb

    If you looking for speed, 8 SATA ports is not extremely useful on a G4 because 4 drives will nearly saturate its PCI bus. ataMan weighs in on SATA
    So, is this in G5 w/ PCI-X 133MHz bus or something else? There is nothing in your message or user .sig to say what systems you are using. Barefeats: Sonnet w/ Raptor vs Seagate

    More drives is not always better. And not all SATA drives are native or made the same.

    RAID

  5. #5
    Join Date
    Jul 2005
    Posts
    9

    Default more information

    yes all the equipment will be new. It will be a G5 using the PCI-X 133 mHz bus with the tempo-X ESATA 8 PCIX card. Rick Stephens has also been very helpful in the last 24 hours. He told me the Western Digital drives the dealer put in are not native- SATA drives. therefore this morning I have instructued the dealer to replace those drives with the new Hitachi 160 gig SATA II drives (the new ones)...or: the Seagate 250 Barracuda new SATA drives. Sorry if i am lacking in the right information, i am not a hardware guy. (i am a composer and string player)...that is why i appreciate your help and guidance because i am serious when i say i need several hundred tracks of simultaneous tracks....and unlike most musicians i have money burning a hole in my pocket. I thank you in advance for your help

  6. #6
    Join Date
    Jan 2001
    Location
    Mobius Strip
    Posts
    13,045

    Lightbulb

    Then I would skip the 160GB and go for Hitachi T7K250 250GB, or Seagate 250-300GB. Very little price difference in 160 and 250.

    There are rather a lot of G5 models, some with 100/133MHz PCI (PCI-X) and some without. PCI-X systems usually also offer 8 RAM slots for 8GB - and Rick has some of the best PC3200.

  7. #7
    Join Date
    Feb 2003
    Location
    ex-Cupertino, CA, USA. Now HU+SRB (Europe).
    Posts
    163

    Default

    Quote Originally Posted by paul dolden
    yes all the equipment will be new. It will be a G5 using the PCI-X 133 mHz bus with the tempo-X ESATA 8 PCIX card.
    Sorry, but the Sonnet Tempo eSATA card does not fit in the PCI-X 133 MHz bus (=Slot 4) because the card uses double-stack connectors which are in conflict with a fan connector on the motherboard. Due this shortcoming the Sonnet card can be used in 100 MHz bus slots of G5 (the shared ones) only or you have to unplug the fan (and put your machine at risk).

    http://www.barefeats.com/hard45.html

    Please read under "SLOT 4 CAVEAT".

    It maybe more benefitial for you to try our controller, the SeriTek/1VE4.

    http://www.barefeats.com/hard46.html

    It fits in the Slot 4, and if you need more drives and more speed, you can add a second controller. The price of a pair SeriTek/1VE4 is the same as of Sonnet card. The controllers are fully bootable (unlike Sonnet), have (IMHO) better hot-swap and yes, it's me who wrote the firmware. Our tech support is said to be better than of Sonnet, too, but I am highly biased; sorry for the "marketing".

    All "native" SATA Hitachi drives work fine with the SeriTek/1VE4. The 500 GB just became available.

    If you compare a pair of SeriTek/1VE4 controllers used in the "slow" (100 MHz) bus with the Sonnet card used in the same bus you will see the SeriTek is significantly faster at the begin of the disk and has a minor advantage probably within the margin of error at the end:

    Read: 488 MB/Sec, Write 550 MB/Sec (Raptor, 10% full)
    Read: 477 MB/Sec, Write 491 MB/Sec (Seagate, 10% full)

    Read: 417 MB/Sec, Write 420 MB/Sec (Raptor, 90% full)
    Read: 315 MB/Sec, Write 297 MB/Sec (Seagate, 90% full)

    versus on Sonnet:

    Read: 448 MB/Sec, Write 525 MB/Sec (Raptor, 10% full)
    Read: 444 MB/Sec, Write 474 MB/Sec (Seagate, 10% full)

    Read: 411 MB/Sec, Write 421 MB/Sec (Raptor, 90% full)
    Read: 311 MB/Sec, Write 299 MB/Sec (Seagate, 90% full)

    I also believe, Sonnet's solution to fit eight(!) connectors on the back of a single PCI bracket and squeeze eight shielded cable connectors to such a tight place is a challenge for the user. Sonnet uses the less common "Type-B" SATA connectors and cables which are in no way better shielded or safe than the cables the original burly-box or our box is using. Besides these cables are still too expen$$$$ive. The common shielded "Type-A" SATA cable we use would directly plug in into either "Burly" or our box, no conversion is necessary, you save a lot on cables - and it is safer too. You won't need any "patch cable" between a common box with "Type-A" connector and the PCI card like you need for Sonnet.
    Last edited by ataMan; 07-13-2005 at 05:15 AM.

  8. #8
    Join Date
    Jan 2001
    Location
    Mobius Strip
    Posts
    13,045

    Lightbulb

    Artificial benchmarks are not reliable. That is why Barefeats and others will test various applications. for every drive in a RAID there is some overhead needed for the drive and controller that has to be taken into account.

    I'm not sure that you can calculate tracks etc and end up at the theoretical MB/sec.

    Usually someone will build their own basically, and test each drive fully (maybe for days to break it in) before then RAIDing the set, and even then do more testing and try different block sizes.

    I just read about Digi ProTools and how they don't support stripping, and I am out of my water when it comes to audio applications.

    with 8 drives, there is increased latency to find (seek) the required sector on the disk drive. If it is all sequential, it ought to be able to pre-fetch the next read. Some drives have different optimizations for write or read performance, they may vary. Some are better at random I/O than sequential.

    An empty drive may appear to deliver 540MB/sec but drop to 300MB/sec at 90% full which is part of what was in the Barefeats link.

    I posted earlier, but realized later I was thinking about another thread than this

  9. #9
    Join Date
    Jul 2005
    Posts
    9

    Default stripe size

    Most of my files are about 65 MB (2'00 of 24 bit 96 khz)..they are each recorded one at a time but then i want to play several hundred of them at once. Would it be better in general to use a small stripe size or a large stripe size ?(the range is abour 2 kib to 512 kib?) Is there any advantage in performance to using a program for striping the hard drives like "Soft Raid" or is the programs that come with the Mac good enough?

    A correction in my previous commentsabove #3)
    I must have the wrong formula, if currently i am getting 110 of 24 bit 96 khz tracks, using the Western Digital hard drives, than i am getting 130 MB/s of throughput. However the system is suppose to achieve 400 MB/s. This seems like a big difference to me.

    For those just joining this thread I will be using:
    -8 Hitachi T7K250 SATA II drives(drives being changed from Western Digital to Hitachi as I write)
    -Tempo-X esata 8 card
    -new G5 with PCI-X slot

    About Digidesign stuffthese comments should bring lots of responses-i was a protools guy for 10 years and flipped to logic audio and native solutions 6 years ago....so i have some experience)
    first rule:digidesign wants to sell you as much hardware as possible. currently they sell you another hard drive and bus card etc for every 10 or 16 (or something) tracks you want right now. (i willl not even begin to discuss all the gear for inputs and outputs you have to buy for your rack just to operate a protools system).Their entire design is based on early 90s concept that computer and hard drive speeds are not fast enough and they need to sell you their own boxes that do a lot of the work. That is why many of us have abandoned the company and put together our own system to meet our own needs using just software solutions like logic audio and the dsp of modern computers to do all our calculations etc. Even with the Western Digital hardrives i am way ahead of a protools system at a fraction of the price. I believe at this point in time even digidesign will not promise more than 48 tracks of 24 bit 96 khz. It is like all the developments in computers and hard drives of the last 10 years (raids, controller cards, etc) has not occurred. Remember most people in audio only need or want 24 or 36 tracks. I am unique in my dense multitracking and always face trying to do the impossible every time i buy gear.,My needs are more in harmony with demanding throughput of visual artists(ir whoever would be reading this thread)-however to return to my question the problem is my file size are probably quite different and that i want to access several hundred audio files at the same time would therefore greatly affect how things should be set up.

  10. #10
    Join Date
    Jun 2004
    Location
    Giga's Orchard
    Posts
    56

    Default Activity Monitor

    Assuming the dealer your speaking to has the EXACT same system as you do, you should ask him the following information:

    What is the data in/out per second reported by Activity monitor for that system (system like yours)?

    and

    What is the CPU usage % when running the full 110 mono tracks?

    The details presented here are still really vague. The entire system must be evaluated for an overall performance _estimate_.

    From my past experience (working for Digidesign) with SCSI RAIDS (depending on RAID Type), the speed and thurough-put is overkill. Sure, this was still when audio was "the best" at 24bit/48K. You had other inherant problems though, such as PCI buss timing, slave/master clock sync issues, clock jitter, all influence by the SCSI controller being present. That was SCSI though... not SATA. I understand your question here...

    The information you seek is very expensive to benchmark, in time and research - does EmApple say anything about SATA RAID configurations and offer compatibility informatoin? Not what I can tell from the product information presented free on Apple.com. So, it's not really so true that Digidesign is in it for the money. That's a very easy attitude to take about a company that is on top of the Digital Audio World Food Chain. It's all about testing and supporting a system they don't need to.

    I gotta go home now, so I'm signing off. There's a lot more information to cover here, and I'll offer as much as I can - I plan on testing a SATA RAID with my sample library (which will take me about a year to import from DVD's and CD's... but it's on the way).

    until next time...
    Jason Wolf
    Mac Technologist
    Giga Designs / Waypoint Dist.
    _________ _____ ____ ___ __ _

  11. #11
    Join Date
    Jul 2005
    Posts
    9

    Default please strip size and Soft Raid?

    What is the CPU usage % when running the full 110 mono tracks?
    -the dealer stoppped at 110 tracks because the the overall performance meters were getting full. However remember logic now has off-line (non-real time) bouncing capabilities so even if you cannot run it real time you could still calculate your final mix (just like when i started working on mainframe computers doing music in 1978).


    From my past experience (working for Digidesign) with SCSI RAIDS (depending on RAID Type), You had other inherant problems though, such as PCI buss timing, slave/master clock sync issues, clock jitter, all influence by the SCSI controller being present. That was SCSI though... not SATA.
    -my current system (6 years old) is also based around SCSI and Chettah hard drives in a raid. and yes the SATA situation is very different. even at 110 tracks the new system (with Western digital hard drives) is 6.75 times more throughput than my old SCSI system.
    The information you seek is very expensive to benchmark, in time and research - does EmApple say anything about SATA RAID configurations and offer compatibility informatoin?
    -no that is why i am on this forum.I seem to be alone in the world in my lust for a 400 track multitrack studio (and yes my music is first written out with all 400 tracks-the scores are 6 feet high.)

    So, it's not really so true that Digidesign is in it for the money. That's a very easy attitude to take about a company that is on top of the Digital Audio World Food Chain. It's all about testing and supporting a system they don't need to.
    -all companies are in for the profit margin -it is called free market.....remember digidesign is on top of the food chain because they were the first out of the block in 1985 and all pro studios around the world have their people trained in these systems. This does not mean that they are the best
    at incorporating new design ideas both from the musical and computer world. I will argue they have the best audio editing (sample based rather than tick based-therefore more accurate for lining up phase relationships between tracks). However everyone knows they are very propietery.I know too many people like me who tell clients they are working in Pro-tools but in fact do all the work in a program like Logic using the DSp power of the computer. Also Avid (digdesign) liscencing fees must be huge: For example my mastering plug-ins (Waves Platinuum) cost $2,500 for VST version (i.e. Logic, digital performer etc) The same bundle for TDM (digidesign) cost $4,000 (remember i live in Canada). All plug ins that cover both formats have the same type of difference in price. And all the young producers (hip hop onwards) tend to use Logic etc because a lot of sound processing plug in developers have not been licensed to the propietery format of TDM- i presume because of licensing fee that the Avid corporation demands...
    -anyways we should move this discussion to an audio forum...does anyone out there know what stripe size i should use for my type of audio needs (please re-read or read my previous message).

    -but overall you are right-what i am trying to measure is hard...however i want to insure i am getting the best performance. remember i am a composer/musician/audio engineer-not a hardware guy so i am trying to learn this world but i am struggling at the beginers level. I thank you for you comments anyways but back to theories or ideas for stip size please or other tips to maximize my system. My other question is there any real advantage to using "Soft Raid" to do the job rather than the native mac stripping program...Remember money is not an issue for me.
    I

  12. #12
    Join Date
    Jan 2001
    Location
    Mobius Strip
    Posts
    13,045

    Lightbulb

    Disk drive performance. I have 'last generation' SCSI drives that are admirable and deliver 65MB/sec (10K) and 72-75MB/sec (15K) that are two years old. I've been contemplating upgrading now that 10K is up to 85-89MB/sec and 15K is 96MB/sec (nearly 400MB/sec with four, we have a review of 4 x 15K II drives elsewhere). I was disappointed with Cheetah 10K.4's 33-36MB/sec four years ago, but it has come quite far.

    At the same time, 7.2K ATA-6 drives are now maxing out what the interface can handle (60MB/sec) and SATA versions clock in at 65MB/sec. There was a time when it seemed to be taking a long time to hit UW 40MB/sec level or beyond.

  13. #13
    Join Date
    Jul 2005
    Location
    Tokyo, Japan
    Posts
    1

    Arrow Seek Performance is the Problem?

    Just a quick intro, since it's my first post here: I've been a BSD user for a couple of decades, and a NetBSD developer since '96; some of my code is in MacOS X. I know FFS pretty darn well, and I've got a reasonable amount of experience building and performance-testing disk and RAID systems for database applications.

    Now looking back through this thread, I see a couple of things that really have the alarm bells ringing. Paul said
    Most of my files are about 65 MB (2'00 of 24 bit 96 khz)..they are each recorded one at a time but then i want to play several hundred of them at once.
    And then Ferenc said
    I realize that the most taxing thing likely is the access time of the disk heads since we're trying to do lots of simultaneous files that are relatively small where each file doesn't need to stream very quickly, as opposed to what video people want: a few humongous files that need to stream very quickly.
    So, if I'm not out to lunch here, what we have is an FFS (Unix) or similar filesystem from which you want to do steady reads of smallish chunks of more than a hundred different files, right?

    Well, no wonder you're having problems. If this is really the case, the first thing you want to do is get rid of all of that striping and unlink the disks, going for JBOD (Just a Bunch Of Disks), giving you essentially eight separate disks. Put a separate filesystem on each one, and mount them all separately, and spread your files evenly across all eight filesystems.

    Here's my guess at what might be going on here. I don't have enough information to know that I'm not completely out to lunch here, but if my assumptions are correct, my conclusions probably are.

    You have 200 files, each containing a stereo pair recorded at 24 bit 96 KHz, or 576000 samples per second. I bet they're stored aligned in memory, giving memory consumption a bit over the disk bandwidth needed, but let's for simplicity set them the same and say you need 750 K of memory and bandwith per second per track, for a total of something close to 150 MB/sec.

    Well, for eight disks, pulling 150 MB/sec off them, reading sequentially, is no problem. Heck four standard desktop drives (reading 40 MB/sec) on no special controllers should give you that. But let's look at what we have to read here:

    How much data are we going to buffer here?

    Let's say I'm fairly conservative and, even with all that RAM, I don't count on being able to buffer more than a second of audio per track. (That alone is keeping 200 seconds, or over 3 minutes, of stereo audio in memory--not an insignifcant amount.)

    That means that every second I have to read 600 KB of data from each of 200 different files, and probably write out another 600 KB to my output file. Likely you're going to have to seek for every file read, so that's 200 seeks per second. But it could be even much worse; a Unix inode has a "last read time" that's updated every time you read a file; these may all be updated on a regular basis, too, adding up to another 200 seeks per second for those writes.

    Well, a standard desktop drive these days, last I checked, was good for perhaps 130 transactions per second if there's a fair amount of seeking involved: i.e., it's not something close to a pure sequential read.

    Now, if you're reading from a striped array, each file, especially if you're reading in 600 KB chunks, is going to end up striped across all eight disks. Which means that for each 600 KB read, you're going to have to move all eight disk arms to the appropriate position, do you read, move all eight on to the next, and so on; you've essentially locked them all together. Sure, you'll read that 750 KB a lot faster than you would off one drive, but you need to be doing 200-400 seeks per second on each drive, more than the drive is probably capable of.

    If you instead put 25 files on each of the eight disks, and use them independently, you now have 25 600 KB reads per second from each disk, requiring the same bulk throughput per disk overall, but each disk now has to serve only 25-50 seeks per second, rather than 200-400.

    You're still asking for 200+ transactions per second from your controller, though; you'd better make sure that it can handle that. A lot of the cheaper ones can't.

    You might be better off splitting the load between two 4-disk controllers rather than one 8-disk controller, if that's a problem.

    If you are using FFS, and you mount it with the "noatime" option, you won't update the access times of files and save yourself a whole bunch of seeks and tiny writes right there.

    I don't know if MacOS X has this, but if it does, run "systat vmstat" and it will print out, among other things, a list of the disks in the system and the number of KB transferered and number of transactions per second on each. If, as you crank up the number of tracks, you see this figure go up correspondingly, and it maxes out when you can't add any more tracks, that could be an indication that transaction and seek throughput, rather than raw data throughput, is your problem.

    Just a theory, but worth trying.
    Last edited by TZ; 07-28-2006 at 02:35 AM.

  14. #14
    Join Date
    Aug 2018
    Location
    Virginia Beach; Virginia
    Posts
    2

    Default

    Sonnet's solution to fit eight connectors on the back of a single PCI bracket and squeeze eight shielded cable connectors to such a tight place is a challenge for the user.
    Read: 411 MB/Sec, Write 421 MB/Sec (Raptor, 90% full)
    Read: 311 MB/Sec, Write 299 MB/Sec (Seagate, 90% full)

  15. #15
    Join Date
    Aug 2001
    Location
    Grangeville, ID USA
    Posts
    9,099

    Default

    Quote Originally Posted by lerner View Post
    Sonnet's solution to fit eight connectors on the back of a single PCI bracket and squeeze eight shielded cable connectors to such a tight place is a challenge for the user.
    Read: 411 MB/Sec, Write 421 MB/Sec (Raptor, 90% full)
    Read: 311 MB/Sec, Write 299 MB/Sec (Seagate, 90% full)

    Why are you posting on an 11 year old thread?
    molṑn labe'
    "I am a mortal enemy to arbitrary government and unlimited power. I am naturally very jealous for the rights and liberties of my country, and the least encroachment of those invaluable privileges is apt to make my blood boil."
--Ben Franklin

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •