I found the Computer Language Shootout on debian.org via Jim Rutherford and thought it was just too tempting.. just where does ActionScript fit in there? Hmm.. that's a question I've been wondering since AS3 was released to developers. I did a few tests comparing ActionScript3 and JavaScript and found that AS3 had closed the performance gap finally... but just how does AS3 compare to Fortran?
So I picked the "binary-trees" test to find out.. I've only tinkered with AS3 (I won't be able to use it for production for probably a year) so I've still got a couple of warnings showing up when it compiles, but the results are very promising.
Here are the results for Flash Player 9/ActionScript 3/Flex2 I saw on my old AMD 2Ghz Athlon Win XP system.. It looks like the system they're using, "a single-processor 2.2Ghz AMD™ Sempron™ machine with 512 MB of RAM", is fairly close to what I'm using.. so we're in the ballpark.
And here's where ActionScript3 would fit into the benchmark table for this test..
Now.. as you can see it takes roughly 39 seconds on my system to run this test.. that's not blazing at all when compared to Clean at 1.73 seconds (on their system). ... but.. Ruby came in at 352 seconds.
Here's the SWF that I made if anyone cares to try it.. (requires Flash Player 9). And the code.. in case someone wants to fix my errors (yep.. I'm a AS3 rookie..). Ideally I'd run their tests on my system or have them run my test SWF on their system.
19 May 2006 at 04:35 am | #
For the official test, I would think a stand-alone SWF rather than running in a browser would be a more appropriate test of AS3. Also, you're probably using the debug version of the player, since that's what's required for doing development. However, the public player (not the one from the Adobe Labs site, available only as a "public beta" from the normal Adobe site) is significantly faster than the debug player... often 2x or more. Good luck.
19 May 2006 at 04:37 am | #
BTW, I believe this is the link for the non-debug beta:
http://www.adobe.com/products/flashplayer/public_beta/
19 May 2006 at 07:07 am | #
well.. I'd rather see the tests run in the browser seeing as how that's where the vast majority of Flash code is executed. It's kind of unfair to give numbers for the standalone player (which has been done before) and raise expectations for the Flash that we normally encounter. But it might be interesting to test in both environments and give data for both.. and ideally testing would need to include both IE and FF.. as there's a difference there as well.
I tested with the public beta version of the Flash Player 9 in Firefox a few times just now.. it was a second slower. :(
here's how it broke down:
FP9 debug in Firefox - 39s
FP9 public beta in Firefox - 40s
FP9 public beta in IE - 43s
FP9 standalone - 52s
FP9 debug in IE - 53s
19 May 2006 at 09:28 am | #
Cool idea! Btw, the ZIP does not contain all the necessary source files. In the tree.mxml you're calling main.dostuff(stuff) but main is not defined.
Dirk.
19 May 2006 at 10:41 am | #
oops! fixed now. main.as should be in there..
22 May 2006 at 06:20 pm | #
You might also want to try some more number intensive tests like the mandelbrot one, I would be interested how your machine can handle that one. Here is an quick and dirty version (I haven't checked the results for correctness though, so this might be bad):
http://www.kaourantin.net/source/mandelTest.as
24 May 2006 at 02:23 pm | #
"And here's where ActionScript3 would fit into the benchmark table for this test..."
Please don't show something so bogus - the timings you have made don't fit into the benchmark table in any way at all!
They have been made on different hardware, on different OS, using a different timing method...
If you really want to know how ActionScript would perform on The Computer Language Shootout, then explain how we can run ActionScript to read and write from stdio on Linux, request that we add ActionScript, and provide programs
http://shootout.alioth.debian.org/gp4sandbox/faq.php#report
24 May 2006 at 04:33 pm | #
Isaac.. yeah... sorry, didn't mean to step on toes there.
I'll submit ActionScript code for official testing in a bit once I get some more of the tests ready, and allow plenty of time for other Flash developers to look over my code and make sure the code is worthy of being submitted. In the meantime, if you're curious you can click the "test the SWF" link above ( http://oddhammer.com/actionscriptperformance/set5/binarytree/tree.html ) and if you have the Flash Player 8 installed then you'll be able to run the test on your system.
25 May 2006 at 12:28 pm | #
Mike "I'll submit ActionScript code for official testing..."
Remember, the programs must be able to read and write from stdio in a linux shell - otherwise they won't fit into the measurement framework. So it's usually a good idea to get hello world (startup) and sum-file working.
http://shootout.alioth.debian.org/gp4sandbox/benchmark.php?test=hello&lang=all
http://shootout.alioth.debian.org/gp4sandbox/benchmark.php?test=sumcol&lang=all