PDA

View Full Version : HDST for Fibre Channel?



DavidCJones
01-04-2001, 06:48 PM
Hail MacGurus:

I just bought and installed HDST, it *appeared* to be a lifesaver. Natcherly, it doesn't support Fibre Channel SCSI, from what I can tell. I'm still waiting for a response from Intech, but it appears to be true, no Fibre Channel support.

Our company makes Fibre Channel RAIDs. We are a true hardware raid, all drives are striped, plus a redundant. We also support CTQ, Command Tag Queuing, also called Tag Command Queing, AKA asynchronous command queuing.

The HDST package *would* solve all of our problems (if it supported FC):

1. Blocksizes other than 512 bytes. We suffer performance issues writing chunks less than 2048 bytes, because the controller has to do a Read2K, Modify, Write2K to write less than 2K bytes. (4 512-byte drives!) If the device driver understood that (xfers < 2048)= bad, this would easily 3X our performance.

2. Command Tag Queueing. We can support 256 simultaneous commands. Our machines, all G3's, have the SCSI Manager 4.3, but the device drivers we have seen for our ATTO and JNI FC PCI boards do not support CTQ.

3. The Big Question: Is there an existing package, similar to HDST, that supports Fibre Channel SCSI? Many cyber-beers for a solution!

Thank You,
David C. Jones
Storage Concepts Inc.
(949)852-8511 x.344
djones@storageconcepts.com

Ton
01-04-2001, 06:57 PM
Stumbled into this one: http://www.optimatech.com/Diskarray.html
Complete insane pricetag of $240.- though.

kaye
01-04-2001, 09:00 PM
ATTO seems to have what you need here http://www.attotech.com/software/app1.php3 with a description:

ATTO ExpressPro-Tools
SCSI and Fibre Channel Mac Version 2.50 - (File Size 819KB) This download includes the Read me first file, ExpressPro-Tools, ExpressPro-Tools Extension. It can be used to configure the host adapter, create partitions, format partitions, and modify mode pages. A benchmark utility is also included. k

magician
01-04-2001, 10:34 PM
david

we would love to assist you with bringing a good hardware RAID solution to the Mac market. You may want to talk to both SoftRAID and Intech. We can assist with contact info if you want to email us offline.

Hammer used to use a version of SoftRAID called HammerRAID for their FC arrays. Do you guys exclusively build FC RAID? Or do you also do SCSI LVD, and IDE?

DavidCJones
01-05-2001, 01:11 PM
Hi Kaye and Magician:

Thank you both for your responses. We're a bit new to the Mac world, most of our stuff goes into the NT and SGI worlds. I'm still a rookie in the Mac universe.

Umm, Kaye, I *am* using ATTO's pro tools! But unfortunately it avoids using the SCSI manager's asynchronous abilities, no CTQ. I contacted them, they said they have no plans to change this. Also, ProTools will not partition or mount any storage device with > 512-byte blocksize. It recognizes it and says hello, but it stops there.

In the NT world, a storage device can present itself as a > 512-byte blocksize. This is greatly helpful to a true striped RAID, it can read and write all 4 (or 8) drives without the headaches of Read/Modify/Write. If I can find a SCSI driver that can handle a 2048-byte storage device, my problems are solved, and the villagers rejoice.

Magician: Thank You for your assistance, it is greatly appreciated. I'm still waiting for a response from Intech, to see if they handle Fibre Channel. Yes, every product we make is FC. The product we're just now shipping uses 5 IDE drives, one redundant. It is RAID 3, and screams at 96 MB/s. Caveat: only when it's running in 2048-byte mode. In 512-byte mode, it has to slow down to do RMW. (darn).

I *did* find a product similar to HDST that claims to support CTQ, FC, and 2048-byte blocksizes, it's Charismac's Anubis 3. Any experience with this? I'm about to plunk down the $129 for it, what the heck.

We're using ATTO Fibre-Channel SCSI adapters, and also JNI's. My guess is that in order for me to use HDST, or Anubis, or any other flavor of SCSI interceptor, the drive will have to have the JNI or ATTO driver loaded on the RAID, is this true? I'm a little confoozed about this area. If you can read blocks 0 and 1 to locate the adapter-specific driver, then half the battle is won: you are already reading through the adapter, right? (??????) <scratches head>.

I will look into Hammer, SoftRaid, HammerRaid, etc., and also Charismac. Meanwhile, if you'd like to see what neat stuff we're doing here at Storage Concepts (shameless plug coming), look at our website: www.storageconcepts.com (http://www.storageconcepts.com)
I think we're the only kid on the block using IDE drives in a hardware RAID, but I could be wrong, I'm only a programmer.

Thank You for your help,
David C. Jones
djones@storageconcepts.com

kaye
01-05-2001, 02:03 PM
I checked the CharisMac site and Anubis v3.18 does support Fibre Channel. I used to use Anubis on SCSI drives and it was rock solid, but slow compared to other drivers. But there have been several version updates since then. k

kaye
01-05-2001, 02:08 PM
Whoa!! SoftRAID supports FC and you do NOT need to be running a software RAID to utilize their driver. Go here http://www.softraid.com/core.lasso for more info. The Gurus sells SoftRAID and it is the best driver, IMO. k

softraid
01-05-2001, 02:13 PM
A couple suggestions for you:

JNI makes the best card for the Mac. ATTO's can also be good, but I have been told by those who know more than I that there are technical reasons JNI is preferred.

SoftRAID is the only Mac OS driver that supports tagged commmand queuing.
-Intereing that in your " NT world" that it is CTQ - NT always did do things backwards! ;-)

SoftRAID does not support 2K blocks at present.I think we did add 1K block support, but 2K created architectural problems both for us and for Mac OS back when we looked into it. I believe that our next major version will support larger block sizes, assuming we can find an elegant way to add it to our interface.

SoftRAID is also the fastest driver on the Mac market, and perhaps with it, and CTQ (as you put it) that SoftRAID will eliminate those latency issues. No product on the Mac does software RAID 5, but we are adding it soon.

Also, Can you be sure that your controller is CTQ compliant? We have found that MOST hardware RAID controller manufacturers do not follow the spec well, and users have to disable CTQ in SoftRAID. What happens is that there is a hole where the RAID controller has high latency, and simple I/O's can be executed out of order. We implemented all ordered I/O in SoftRAID 2.2 as a test for this, but then it turns out Quantum drives had a bug where there was a 50% performance drip on Sustained writes, making for bad benchemark performance. We had to make a tough choice, and we chose to go with forcing hardware RAID to disable TCQ, and improving performance on Quantum drives.

There are some other issues that hardware RAID has with SoftRAID's driver, it is often too fast for them. If you test it, see if when you create a series of 1 drive/1 LUN's, and then use SoftRAID to stripe them into a 4 or 5 drive stripe. See if your system boots with that configuration. If so, the hardware RAID should work fine on Mac OS. If there are problems, then there are some issues that you will have to work out, or be forced to disabled CTQ on the Mac.

Good luck!

magician
01-05-2001, 02:25 PM
I feel much more comfortable recommending SoftRAID to you, David. I invited Mark James to post, and he did below. Hope that helps. To contact him directly, just shoot an email to support@softraid.com. Most of that email, particularly if addressed to him, will get routed to him. He will surely give you a direct address once you make direct contact.

http://macgurus.com/infopop/emoticons/icon_biggrin.gif

DavidCJones
01-05-2001, 03:56 PM
Hi Magician, Kaye, and Mark:

Gosh, you peeps are soooo kind to help us out!

Darn, if only SoftRaid supported 2048-byte devices, my problems would be over. Avoiding RMW is much more important than TCQ. I have seen our FibreBlock benchmark at 96 MB/s without TCQ, but it was also NOT doing RMW, a mystery we're still trying to figure out. Quite by accident, we discovered that if we partition the RAID in just a certain way, all xfers happen on 2k boundaries, and voila, the thing screams. I have a theory on this: IF the Allocation Block size ends up, by accident or design, being on a 2K boundary, our RMW problem magically disappears. We can only marginally control this behavior, though, by using a partition size of *exactly* 305.34GB, the capacity of the RAID. We haven't found any other partition size that duplicates this wonderful behavior, so it seems to be plain dumb luck that we stumbled on this.

Yes, the 2K blocksize is way more important than </Mac mode ON> TCQ </Mac mode off>, and for this reason I would give the nod to Charismac over SoftRaid, if it indeed supports > 512 byte blocksizes.

Mark: Thanks for the heads up on JNI, we also kinda noticed that it's more gooder (bester?) than the ATTO. It squeezes out about 10% more horsepower than the ATTO. Give yourself one cyber-beer, sir.

It's also possible that with the combined speed of the SoftRaid driver and CTQ, some of the latency could be eliminated.(I think Mark just said that).

Super-gnarly question: Since the Mac is indigenous to 512-byte chunks, how would a 2K-byte driver handle the sitch where the Mac OS wants to write a single 512-byte sector? Would the driver have to do the RMW? This would seem to be unavoidable, and I have written Netware drivers where we had to do this. Netware also eats only 512-byte stuff, so to interface to a 2048-byte Magneto-optical, we had to take it in the shorts and do the RMW in an interrupt routine.

Still, since this is mostly done in cache, it's better for the Processor to do it instead of our slower i960 microcontroller. As long as our Raid sees only 2K byte (or multiples of) reads and writes, we rule. And chances are that a pre-fetch is unnecessary due to cache, we might see only a write instead of a
Read/Modify/Write.

Kaye: I think I will seriously crawl into the Charismac Anubis thing, I'll letcha know how it comes out.

Thanks you guys,

David Jones

PENDRAGON18
01-05-2001, 04:32 PM
Wow... this is way over my head!

I though the native Mac block size in HFS+ drives was 4KB, right?
With HFS (old) the block size was based on the size/#ofblocks - so it could range from small (1KB?) to 64KB (on 4GB partitions).

I ocassionally see issues on CD rom drives where a 512KB block selector (jumper) is set or not set and problems occur. Of course putting a CD ROM drive on even an Ultra2 bus is a bit crazy - as even the top end drives will not scratch the surface of the bandwidth available.

SoftRAID supports HFS and HFS+.
SoftRAID is also the fastest driver on the Mac market, and perhaps with it, and CTQ (as you put it) that SoftRAID will eliminate those latency issues. No product on the Mac does software RAID 5, but we are adding it soon.
Have any firm dates?

I heard this when I bought my MilesU2W card last year.

Any plans to offer Firewire support (single drive, RAID 0, 1, etc...)?

Any plans to support MacOS X (For SoftRAID or DaveCJones&Co) ?

So what is the NIJ card/company? Got any links?
I know I can not afford it, but I can always drool http://macgurus.com/infopop/emoticons/icon_wink.gif

Shove me in the shallow water before I get too deep!-)


later,



------------------
Have fun storming the castle!

softraid
01-05-2001, 05:48 PM
Yes all this gets very complicated....

The Mac HFS+ allocation block size is totally different than the Logical Unit block size, and that is different than the physical device block size. The driver transparently handles all of the block size issues, for better or worse. And HFS+ typically has 4K blocks by default anyway. It is a messy siubject, which doesnt generally affect the user unless someone doesn't do their side of the job very well. So don't worry. ;-)

As far as the other questions:

No product on the Mac does software RAID 5, but we are adding it soon.
Have any firm dates?

No, but we are woking on it now. Hopefully around the time OS X is released, we willhave a lot new features for SoftRAID.

Any plans to offer Firewire support (single drive, RAID 0, 1, etc...)?
We developed FireRAID for VST. FireWire is awkward, and it was too difficult to get the quality we needed off generic FireWire devices, so we focused on a standard which we could get the reliability SoftRAID needed. We cannot afford to jump into a market with a product that ever hangs/crashes. As fireWire matures, we will do a more generic version, but for now, we are letting VST handle this.

Any plans to support MacOS X (For SoftRAID or DaveCJones&Co) ?

We are well underway. ;-)

So what is the JNI card/company? Got any links?
I do not know their URL, unfortunately, but it cannot be hard to find.

BTW: For Storage Concepts, SoftRAID handles LUN's seamlessly, and was originally designed to handle hardware RAID properly.At one point there were controls for various controllers like the DEC and Mylex, but that has all been removed, as our market was not selling RAID hardware, but software.

good luck all.

magician
01-05-2001, 10:38 PM
thanks for jumping in, Mark!

http://macgurus.com/infopop/emoticons/icon_biggrin.gif