Slow RAID drives...

[cspurrell]

I just moved my setup down to a new location.

Having big problems with the 5 bay tower.
3x 2TB drives in one Disk, 2x 2TB drives to make up another drive

Even in Finder, it is taking 30 seconds to just show a list of folders on the drive. Then if I dbl click on a folder to see what is inside it takes another 20 - 20 seconds to open up.

Sometimes Lightroom just hangs - even with a new catalog, on the fast(supposedly) eSATA drive I use.

I ran AJA - I got 1.8MB/s

Any ideas of things to try here? I didnt change anything other than to put in a wireless card because I no longer sit right next to the cable modem.

Would like to get some work done before thanksgiving...

Chris

List your computer model,OS,hardware specifics
2006 Mac Pro 1,1 8Gb RAM,ATI Radeon HD 5870 1024Mb, OSX 10.7.3

[unclemac]

Tell us more about the RAID. What version......RAID 1, RAID 0, etc. How does it connect? Firewire, USB, eSATA?

Any other changes, software updates?

What does Disk Utility think of the RAID volume? Any errors, or clues?

[Damien]

Reseat you card and ALL your cables. after a move you may not have gotten one all the way in

[subastein]

Can we have SSD in RAID. Although I haven't tried it but think that it would give a better experience.

Of course and expensive suggestion but just an Idea.

[Damien]

yes you can raid an SSD, never tried it tho

[unclemac]

Sure. Likely will hit a thoughput bottleneck though depending on bus and protocols, at least with many of the current SSDs having read/write speeds over 500.

Much less need of RAID for speed.....more for space (for fast, multi-GB volumes for say HD video rendering).

[subastein]

Me too think that SSD will help a lot in issues related to speed. SSDs are much reliable and faster than usual HDDs.

Else one can also think about disk cleaners to gain free space. What do you think about it?

[rwm]

Else one can also think about disk cleaners to gain free space. What do you think about it?

Your Apple computer OS should take care of this. Fragmentation is not a problem like it was in older days.

[ricks]

SSDs are fine for a computer with a single drive that everything. My personal opinion, based on testing using virtually unlimited number of SSDs, is that for the most part putting your operating system and applications on an SSD is a total waste of money. Yeah, it boots faster. But I am not one to boot a computer very often. And once booted, the OS and applications are in memory and the SSD doesn't really do much if anything for performance.

Depending on what you use your computer for, storing the data on SSDs would make a big improvement in performance. That is if you can afford to get enough SSD capacity to handle the needs.

We absolutely can build a RAID using regular old spinning disks that is every bit as fast as SSDs for move large quantities of data. The key here is that RAIDs do big file transfers better than anything. Itsy bitsy little transfers are best on SSDs. But for a big database nothing beats a big RAID with spinning drives.

Rick

[subastein]

Your Apple computer OS should take care of this. Fragmentation is not a problem like it was in older days.

Yes rwm , you are absolutely correct. The HFS+ file system is smart enough to handle fragmentation up to 20MB of file. But, after that files get fragmented.

Well, that is another case what I want to discuss here is the free memory which OS uses temporarily. If the job can be done in cheaper way then I won't invest in SSDs.

[cspurrell]

First thanks to all that made suggestions... second thanks for taking so long to reply, I have been waiting for my backups to finish...

Actually the most helpful suggestion is/was reseating the eSATA controller inside the box.

My drives are basically 1 striped array of 6Tb (3x2Tb) for speed and 1 mirrored array of 2TB(2 x 2TB) for resiliency. Backed up to a near identical configuration that is usually 120 miles away, and a variety of old hard drives that get repaved and filled up and stored on the shelf, plus a 2TB WD Passport that is just a godsend for keeping Work in Progress stuff on.

Basically my problems persist. I wrote a note to someone that contacted me offline explaining what I do, but basically the problems are these:

1. constant risk of hangup/freeze usually triggered by any kind of multi-threading to the Burly - forget working on photos while rendering a video...

2. General speed problems despite acceptable AJA results.

I do wonder about my Lycom card. I am using the 2 port version which does not really support OSX 10.7.5 and limits out at 5 drives. I am maxxed out at 5 drives and starting to get fullenough to think about upgrading and increasing my storage...

I would rather throw what few brain cells I have at the problem than money, but it does seem that upgrading the eSATA card to the 4 Port Lycom card that supports OSX all the way up to 10.8 makes a lot of sense...

Thoughts?

Meanwhile here is the text of the email I wrote to the friend who contacted me off line for your edification... names changed etc..


<<
Hi

Sorry it took so long to see you message. Are you still having problems?

I didn't really resolve my issue, and had a major crisis on Thursday when no matter what I could not write 1500 images from a major shoot from my card onto my main eSATA hard drive.

Luckily I also have a relatively new MBPro and a Western Digital 2TB Passport Drive so I do have the images somewhere.



Here is where I am

I have moved this computer from a prior location about 120 miles away by putting it carefully in the cab of my truck, wrapped in a blanket. There is a suggestion that this can cause cards to move. So I basically opened it up removed the eSATA card and put it back, making sure it was seated properly.

Initially no big impact, but then, inexplicably it started working, and working much, much faster.

So instincts kicked in and I started the process of refreshing my backups. Which is where I am now...


I do plan on looking at the bigger $184 eSATA controller. I notice I am at the limit on the one I am using Lycom eSATA 2 Port PCIe Host Card and the support for OSX10.7.5 just is not there officially. This is a major problem for me. I know there are people that just stay on the same OS for ever, but I am not one of those. I started doing video, and using FCX and that requires that you upgrade... so I am damned if I do, damned if I don't.

The Lycom 4 Port sSata card does support OSX 10.7.5 and up to 10.8 so I am tempted to switch to that. It also supports more drives than the 2 Port Card and I feel that is part of the problem too.

I basically stopped using my MacPro for video, except to archive work and use the DVD drive to burn projects for clients. I had the fantasy that while I was rendering in FCX I could be working with images in LR and Photoshop, but I basically have to be incredible careful not to do anything other than single thread my hard drive. This can happen with something as trivial as a big metadata update - and everything hangs... because LR does the metadata update in the background and then if you go and work on a new image while that happens it is writing to the same place and bingo...

I am also waiting for Apple to come out with an upgrade to the MacPro that supports thunderbolt and USB 3.0 basically I would prefer spend as little money as I have to to keep things going before I make that investment...

And there is no thunderbolt eSATA inteface to hook my MBPro up to my eSATA drives so I am stuck with my old MacPro Tower if only to manage data...


Chris

[ricks]

I do wonder about my Lycom card. I am using the 2 port version which does not really support OSX 10.7.5 and limits out at 5 drives. I am maxxed out at 5 drives and starting to get fullenough to think about upgrading and increasing my storage...

No problem running 10.7 and 10.8. Just use this driver.

And why can't you attach 10 drives to this card? What is limiting it to 5?


Additionally, adding USB 3 and Thunderbolt to a MacPro is going to add some convenience, but it won't add a bunch of speed. A single PCIe slot is more bandwidth/speed than a Thunderbolt bus. If you need more speed use a faster eSATA card.

Things that add more speed:

Fast bus (the 2 port cards are limited to 150 MB/sec by their chipset. The 4 port card is good for near 1000 MB/sec)

Drives that are not over 1/2 full. (full drives are SLOW)

Newer drives are faster (if your drives are older they are definitely slower. Current model Seagate drives are good for up around 180 MB/sec, that is 30 MB/sec faster than the chipset on a 2 port card)

Set up a RAID (as long as the bus is capable of the added speed, the sky is the limit)

------------------------------------------

I don't consider USB 3 to be all that ready for prime time. Apple seriously screwed the pooch on that with unpredictable USB connections on new Macs.

Thunderbolt is nice for stuff you just gotta have on Thunderbolt. But I would much much rather keep it simpler and have a few PCIe slots. Thunderbolt is definitely more limited than Apple's marketing would leave you to believe.

Rick

[cspurrell]

I updated the driver. I could not see on your product page where it said 10.7.5 was supported, whereas the 4 port card clearly states that 10.4 through 10.8 is supported... anyhow you are the guy and sounds like I should upgrade the card anyway.

My drives, although RAID are now well over 1/2 full, so will order up some bigger drives. I need them anyway to handle the video.

-----

The point about TB and USB 3 is prompted by the fact that I am at a place where my MacBook pro is TB and USB3, whereas my MacPro is eSATA, USB 2.0 and Firewire... I just want the same interfaces on both devices, as long as they are fast and reliable I don't care what they are, if they kept the PCIe slot on the new MacBooks I would be a happy camper.