PDA

View Full Version : performance testing



marrand
07-12-2001, 03:59 PM
In the "Upgrade or Buy New?" forum, I found an interesting post by CrisYip. Topic: "Altivec Fractal test routine".

So, I am wondering: is performance testing in general of interest to the macgurus' members? are the results meaningful? can they be used to gauge the value of their upgrades? And thus I started a new topic.

I downloaded the little program from the site supplied by ChrisYip, and ran the test in my office computer. Oh, this is a bare bones machine, G3 266 desktop rev. 1, 192 RAM, OS8.1 and I didn't expect much out of it. My home G3 desktop is upgraded and I will run the test on it later.

Altivec Test conditions: default, color speed 10, max count 4896, zoom factor 2.

Test 1: several apps running - 89.2 sec. 159.8 MegaFlops/sec

Test 2: after reboot and no apps running - 87.2 sec. 163.5 MegaFlops/sec.

Well, now I know: my office machine is truly a slowpoke http://macgurus.com/infopop/emoticons/icon_smile.gif

My home G3 was upgraded, and I ran various benchmark tests, as well as real time tasks, before and after installing the G3/500 processor. I will be happy to share the results in case anyone is interested.

ChrisYip
07-12-2001, 04:42 PM
I think it would be an interesting comparison but not to be taken too literally.. I just clocked in 300 mega flops / sec on my G3/500 from my PTP 225 upgrade project - but this was running IE, Excel and BBEdit simultaneously.

I think that where it would be a cool comparison is a before / after same set up run on a machine..

For instance - it might be intriguing to flip the G3/500 I have with the G4/500 I have in my Yikes and rerun and see how it clocks on exactly the same machine

I could do a real experimental design

PTP225 -> G3/500 and then G4/500
Yikes -> G3/500 and then G4/500

that would provide a comparison in terms of the machine hardware and also the processor..

Chris

kaye
07-13-2001, 10:34 AM
From Barefeats:

"6/21/2001 -- I achieved 7 gigaflops running AltiVec Fractal by clustering two Dual G4/533's with a crossover cable, allowing all four CPU's to crunch the same job. A cluster of four G4's (733, 466, 466, 533) produced only 6 gigaflops. I believe clustering (or network rendering) is getting more and more popular. Did you know that LightWave, Bryce, After Effects and Cinema 4D all support network rendering? Visit the AppleSeed site for more details on clustering and instructions on how to do it yourself." k

marrand
07-13-2001, 01:44 PM
Kaye, Chris: my intention was to solicit inputs from "simple" Mac users, particularly those having old machines (G3 and older) who availed themselves to the advice and upgrades suggested by the macgurus. An older Mac is ugraded and the user may want to know: did this slow me down? did it speed me up? Did I get my money's worth? Given that any measurement will depend the software paramenters chosen, the user may want to know what do I have to keep constant in order to produce a meaningful result.

The quote from "Barefeats" is obviously geared towards the speed demons. Dual G4 processors? Wow, that's raw power! The older upgraded Macs don't come near it, and the results on the "supermachines" wouldn't mean much to a "regular" user.

When I read ChrisYip's post on the "Altivec Fractal test routine", I was both curious and spectical. Let me see if I got it straight. One of the main differences between G3 and G4 processors is the built-in velocity engine made by Altivec. G4 has it, G3 doesn't. So, the Altivec Fractal test would naturally be designed to show off the merits of their engine. Shouldn't a G3/500 be always slower than a G4/500 in this particular test?

A more practical question would be: what improvement did I get in going from a G3/266 to G3/500? Just for fun I ran the Altivec Fractal test on my office machine (G3 266) and the upgraded one at home (G3/500). The megaflops went from 160 (88sec) to 310 (45 sec). True, the two Macs weren't identical software-wise, but the rough doubling of speed was there.

Before upgrading to G3/500, I ran a series of careful tests with Norton's System Info, Macbench5.0, and Cinebench 2000. Then repeated the same tests after the processor was installed. Sure, the CPU and FPU almost doubled, but I also learned that my "weakest component" is the videocard, and the "strongest" one is my HD.

The "score" depends strongly on the set of extensions chosen for the test. If one pays no attention to the built-in benchmarks, and reads only the score for his/her computer, these simple routines appear useful for optimizing your software as well.


[This message has been edited by marrand (edited 13 July 2001).]

kaye
07-15-2001, 10:40 AM
Cranked up my PTP with xlr8 G3 at 540/270/1MB and ran that AltiVec Fractal Carbon, though in my mind I couldn't remember Carbon in the app's name. Sure enough, I had AltiVec Fractal Demo IP on the PTP. So I compared the results of the two apps with OS9.1. BTW, older version here http://exodus.physics.ucla.edu/appleseed/HighPerformanceMacMPI.html which led me to older versions of the AppleSeed versions here http://daugerresearch.com/archive/

Troubling to me is that the Carbon version gives much slower results, by about 25% slower. Carbon on OS9.1 sucks, at least on my vintage G3 Mac. Anyway, results with G3-540, OS9.1:

AltiVec Fractal Carbon..41.6sec/342.5MegaFlops
AltiVec Fractal Demo IP.31.4sec/453.2MegaFlops
AltiVec Fractal Demo IP.34.0sec/419.6MegaFlops (G3 set at 500/250 just for this test)

This is with exactly the same setup. I also tried OS9.0.4 and 8.6 with the Demo IP version and, even though OS9.1 has many more inits than my 9.0.4/8.6 folders, 9.1 was still slightly faster.

I also experimented with the G3, reducing the backside cache from 270 to 216, no change in scores to either version of AltiVec Fractal. I then went back to 270 and enabled Write Through. Again, no change in scores to either version of AltiVec Fractal. So the apps appear to be going directly to the processor, not thru backside cache nor thru main memory, or else thru main memory regardless of the xlr8 setting for Write Through. Anyway, backside cache setting and Write Through setting have no effect on the scores for either version of AltiVec Fractal. k

marrand
07-15-2001, 03:33 PM
kaye: your approach was very sophisticated for a beginner like me. http://macgurus.com/infopop/emoticons/icon_smile.gif

I thought that "flops" stands for floating point operations. Do inits and apps use this? If not, that would explain roughly constant scores.

More questions arise:
1. How do you change backside cache? Jumpers?

2. What is Write Through?

3. Doesn't this Fractal test look at only one aspect of your system - the processor? Wouldn't a "coctail" of tests, such as presented in Norton Sytem Info or Macbench be a better assessment of one's overall configuration?

4. Don't your results (and mine) prove that as long as you stick with G3 type processor, you will always get something less than 500 MegaFlops? To get over one gig one needs G4?

5. And how does improvement in this test from 300 to 500 show up in real use? Doesn't one need at least 2X improvement before "seeing" the extra speed in everyday work?

Sorry for the naive questions; I guess it will take me years, if not decades, before I become a guru. http://macgurus.com/infopop/emoticons/icon_smile.gif

dragon_x
07-15-2001, 09:37 PM
I'm sure kaye can answer this, but here my take on it

0. You are mostly correct - It stands for floating point operations per second. Normally its rated in MFLOPS. Apps can use floating point operations or integer operations (I think there are more - 'vector operations' could be associated with Velocity Engine or Atli-Vec).


1. You can set this via the software - like XLR8 Speed Control. When you set it too high the system will become unstable. It may crash after a few seconds or even after many hours. Normally all PPC CPUs are rated to 1:2 - since you dont gain much by going to the higher speeds and it cost a lot more.

2. Write through is a feature that 'slows the cache down'. It can help with stability issues, but is normally only needed if you overclock your system. I think that when this is enabled the CPU only reads from the cache and then writes back to the main (SLOW) memory. I think Mag has tested this more than anybody... I would bet your performace would be better if the cache were clocked at 1:2 (or lower) and write through was off.

3. Well the CPU is a big part of the equation. Fractals throws some fairly intense computations at the system - like PowerControl does for RAM. PowerControl will turn up bad RAM when TTPro and GaugePro will not. MacBench 5 is very old. Its good to a point, but it was never designed with a G3 processor in mind. Norton would probably be better... and Intech has some HD benchmarks. Of course you can alway run REAL WORLD tests. Tests on apps you use it always the best.

4. I would agree here. The whole deal with the G4/500 being a SUPERCOMPUTER is that it can sustaing 1 billion floating point operations per second or 1GFLOP! Its theoretical peak is actually about 4GFlops though. The 7450 should have more FLOPS for CPU cycle because it can do more instructions per cycle... at least that is the way I understand it.

5. I would say I can see improvements of about 20% maybe a bit less. Its not a jaw dropping differences - obviously but it is noticable. It also depends on what you do. I would say any improvement of less than 5% (maybe 10%) is not worth it or noticable. I often read about hardware hackers overclocking stuff until it melts to get a few extra FPS in some game. http://www.macgurus.com/ubb/eek.gif

If you do something every day - for 6 hours a day - even a 10% improvement will be noticable. It would save you and extra 36min per day!

Also while some aspects of your system may improve over 25% with an upgrade, other parts of your system may make the system to appear not as fast. You could get a 15K Cheetah, but not running it on a Ultra2 (or faster) PCI card and/or not running it with a G3 or faster could result in dissappointment at the speed. My 500Mhz B&W G3 is a bit faster at everything when compaired to my 500Mhz PTProG3. This is because the bus is 100Mhz and so is the RAM. While the PTPro has a bus of 50Mhz and RAM is only about 17Mhz.

------------------
Life in the fast lane leads to:
The Resteraunt At The End Of The Universe

marrand
07-15-2001, 11:11 PM
dragon_x: thanks for the thorough reply. wow, quite a treatise!!

XLR8 Speed Control - can I use it even if don't have an XLR8 processor? Mine is a recently installed Sonnet G3/500 in G3 desktop, rev. 1.

After you explained, I see no reason to overclock or fool around with write through. I don't play any video intensive games.

MacBench 5 wasn't designed with G3 processor in mind? Then how come it uses G3 300 as reference benchmark? Using it and Norton System Info I learned that rating of my computer (disregarding the benchmarks) depends strongly on the particular set of extensions I choose. So, these tests seem like valid¬?metersticks to tune my system. Does it makes sense to you?

Well dragon, I don't do the same task 6 hours a day, every day. http://macgurus.com/infopop/emoticons/icon_smile.gif

Every application I launch seems to take about a second or less, except for Photoshop6 which takes 9 sec. Overall I am reasonably satisfied with the speed I have - 90% of the time computer has to wait for me, not I for the computer (after full boot, of course).

dragon_x
07-16-2001, 05:44 PM
XLR8 Speed Control - can I use it even if don't have an XLR8 processor?

Yes. While there is no guarantee that it will work the chips are very similar, if not identical. I have run XLR8 software with PLogix and MacTell. It works great, better than the vendor software.

If your sonnet software is working fine, I would stick with that. Sonnets were not really ment for overclocking or tweaking.

I think I read somewhere that MB5 was not designed for the iMac. Maybe this came out after the bG3/300Mhz?

It does not do Alti-Vec at all. It is also optimzed for 512KB cache or bigger -so some G3's with the 256KB cache running AT CPU speed (2x faster than standard cache) benches poorly. So its probably a good gauge for your system. Again the best gauge is to test the speed of the apps your normally use. Gamers often insist that Quake 3 is the BEST benchmark to see how fast CPU, bus and video are. This is not true if you only play Unreal Tournament like me. http://macgurus.com/infopop/emoticons/icon_smile.gif It would kind of be like have someone bench your computer with InDesign, but you use Quark Express. DOH!-)

marrand
07-17-2001, 10:09 PM
dragon: thanks for the useful tips.

I read in an older Mike Breeden's article that non-PPC native Kensington mouseworks one of the "most cpu cycle consuming addons". He was talking about various extensions and control panels which do nothing for you and use cpu's resources needlessly. That's the kind of stuff I was thinking about when I thought MacBench5 could be used to tweak my system.

A question: what is a "non-PPC native"? I have OS9.0.4; it has PPC. Does that mean anything?

But then I smiled. http://macgurus.com/infopop/emoticons/icon_smile.gif

So what if the cpu has to sweat servicing my Kensington Orbit? It's there for my convenience and pleasure, and that's the first priority, right?? http://macgurus.com/infopop/emoticons/icon_smile.gif

What is the Unreal Tournament?