oddhammer

1. Ted Patrick says:
   29 Jul 2006  at  09:59 am | #
   I took a look at the MXML and AS3 and there is not much optimization of these tests. I would question the usefulness of these tests as a guage of AS3 performance.
   
   Here are a couple of points:
   
   1. No strong typing - Typing coorolates directly to AS3 JIT compilation. If Types can be determines in advance, blocks of code are JIT compiled into native machine code during first execution.
   
   2. Looping with Number vs casting unit - Using the new uint type for loop iterators speeds loop code by about 20%. uint is smaller in memory and is the native type for looping operations. If you use number, the numbers are interpreted to uint forcing a type conversion on every loop pass.
   
   3. Using temporary variables forces a function call to create an Activation Object with each execution. Typically use of a Class method would be to process data within a class as properties or passed data. The player is optimized to handle processing of strongly types class members.
   
   We should take some time and define a more realistic bench test for AS3. In the light that these tests are under, AS3 seems far slower than it is in practice.
   
   I will take some time work on optimized versions of these tests for Adobe MAX presentation on AS3 Perfomance Tuning. AS3 as a language is far deeper than most people realize or have explored. There is lots of room for performance tuning and defining best practices that align with this performance.
   
   Cheers,
   
   Ted Patrick
   Flex Evangelist
   Adobe Systems
2. mike says:
   29 Jul 2006  at  12:55 pm | #
   Ted - did you look at the MXML files from Brian's site or mine? Mine are not optimized at all because.. well.. I didn't know strong typing made a performance difference when I first did these last year. Brian added the strong typing and there was clearly a performance gain.
   
   As far as creating more realistic bench marks.. well.. the goal I had in mind was to take existing JavaScript benchmarks and convert them as closely as possible into ActionScript to compare apples to apples on JavaScript turf. That, and well, I haven't worked with AS3 enough yet to find performance tricks to write benchmark tests around. I thought the tests did a decent job of showing that AS3 is as fast or faster than JavaScript.
   
   Feel free to use my code as a base, and I can run your optimized SWFs on my system for comparison with the old tests. I'd love to see how far this can be pushed.
3. Brian Deitte says:
   30 Jul 2006  at  12:49 pm | #
   Hi Ted,
   
   I'd check out the updated tests I worked on here: http://kuwamoto.org/2006/06/15/avoid-ints-in-actionscript . I did not know that uint would be better for loops. I used Number for everything, based on Sho's post on Number: http://kuwamoto.org/2006/06/15/avoid-ints-in-actionscript .
   
   I don't see the ActionScript numbers as being too bad! Based on my calculations, averaging the IE and Firefox numbers, I see FP9 as 11x faster than Javascript.
4. Brian Deitte says:
   30 Jul 2006  at  12:52 pm | #
   That first link should have been: http://www.deitte.com/archives/2006/07/updated_perform.htm
5. Ted Patrick says:
   30 Jul 2006  at  02:21 pm | #
   Gary Grossman did a presentation at the Adobe Component summit on performance tuning. This is the data on loops and int use directly from his slides:
   
   * Putting in a coerce to int/uint can help performance, if the compiler cannot infer that int/uint is what you want
   * Array access has fast paths for int/uint, so coercion of index can help performance
   
   var i:int;
   // i*2 gets promoted to Number
   for (i=0; i<10000; i++) {
   a[i*2] = 0;
   }
   
   // Goes through fast path
   for (i=0; i<10000; i++) {
   a[int(i*2+1)] = 1;
   }
   
   ----------
   Ted: So in this case casting the math of i*2+1 to int(), causes the compile to optimize the bytecode during compilation. This prevents promotion of the index to a Number and thus results in faster loops and access to index based results.
   
   cheers,
   
   Ted
6. Ted Patrick says:
   30 Jul 2006  at  02:27 pm | #
   Another loop optimization is sub expression elimination but this has side effects given use of getter/setters. A common mistake is doing this:
   
   for (var i:int=0; i<a.length; i++)
   {
   processRecord(a);
   }
   
   Assume a is an Array. Here a.length must be evaluated every pass of the loop becuase it has the potential to change in value within every pass.
   
   This code is far faster:
   
   var n:int = a.length;
   for (var i:int=0; i<n; i++)
   {
   processRecord(a);
   }
   
   Cheers,
   
   Ted
7. Ted Patrick says:
   30 Jul 2006  at  02:53 pm | #
   Another thing to point out with these tests is that the MXML version are running the Flex Framework in addition to just the AS3 test. In the javascript tests they are running just a block of js directly into the JS interpreter, in the MXML comparables they are running some 200 extra classes in player memory combined with graphical rendering components, events, etc. Every rendering cycle there is lots of AS3 Flex Framework code executing. This will adversely affect performance when comparing VMs/interpreters purely. It is similar to using a Java Swing application to test Java performance, the Swing components/framework will affect the memory, gc, and overall performance of the tests.
   
   Let me put all these tests into an AS3 class under an ASProject and see what happens. I will work from Brian's MXML to do this.
   
   Cheers,
   
   Ted
8. Ted Patrick says:
   30 Jul 2006  at  09:46 pm | #
   Mike,
   
   I put together a 'moderately' optimized version of the tests using ASProject this afternoon. They are available here with full source:
   
   http://www.onflex.org/perf/
   http://www.onflex.org/perf/srcview/
   
   In several of the tests, I am seeing 2-10 time improvement in performance atop your existing AS3 results. Please take a look at the posted tests and confirm some of the results that I am seeing on your target test machine.
   
   Regards,
   
   Ted Patrick
9. mike says:
   31 Jul 2006  at  12:01 pm | #
   Here are the results when I run on my system:
   TEST 1: 499 ms
   TEST 2: 412 ms
   TEST 3: 163 ms
   TEST 4: 162 ms
   TEST 5: 6 ms
   TEST 6: 3 ms
   TEST 7: 103 ms
   TEST 8: 8 ms
   TEST 9: 26 ms
   TEST 10: 45 ms
   TEST 11: 95 ms
   TEST 12: 2 ms
   
   So some improvement in a few of them.
   One thing to be careful of though is keeping the JavaScript and AS tests as close to possible as far as what they are doing.. if JS is calculating the length of the string with each iteration through the foor loop and that procedure is removed/optimized in the AS test then the results are skewed. So I think (it's been a while since I looked at this stuff) some of the optimization tricks you mentioned were left out of the AS because they weren't used in the original JS.
   
   Anyway, thanks Ted and Brian for updating the code! I'll get the table updated and cleaned up later this week to show the updated results.
10. Ted Patrick says:
    31 Jul 2006  at  12:43 pm | #
    Mike,
    
    I agree 100%. I was careful to preserve all features of the tests but JS and AS3 are wildly different languages and there is now a wide array of optimization on the AS3/JIT compilation path. The source is available and what I should be obvious. If there is any change in the output behavior of the tests, then it is a bug.
    
    Your results are much slower than mine and my machine is 1.6Mhz w 512MB. Here are the results I am seeing hence my post.
    
    TEST 1: 291 ms
    TEST 2: 281 ms
    TEST 3: 151 ms
    TEST 4: 156 ms
    TEST 5: 1 ms
    TEST 6: 2 ms
    TEST 7: 78 ms
    TEST 8: 8 ms
    TEST 9: 35 ms
    TEST 10: 45 ms
    TEST 11: 80 ms
    TEST 12: 2 ms