Skip to content

Performance Tests

Jump to bottom
ricardoquesada edited this page Dec 3, 2014 · 8 revisions

How to run performance tests

Basic rules

  • Never run the tests on a Simulator and/or Emulator. Always use real devices
  • The older the device the better
  • Never run the tests in DEBUG mode. Always use RELEASE mode
  • Double check that RELEASE mode is using all the possible optimizations for the C++ code

Devices are multitasking, and many tasks are run in the background and those tasks could affect the performance. In order to minimize that do:

  • Turn Airplane mode ON
  • Disable Wifi
  • Disable Bluetooth
  • Kill all running tasks / applications

Logging

  • Performance Tests must be run before releasing an stable version
  • The results must be logged in this spreadsheet
  • In order to know the performance of the new release, a comparison must be done with the previous versions. The comparison must try to use the same testing environment. As an example:
    • Same devices (MUST)
    • Same device operating system (SHOULD)
    • Same compiler and toolchain (DESIRABLE)

Starting from Cocos2d-x v3.3, the "auto run" feature of Sprite Performance Tests logs in the console the output of the tests. This output can be copy & pasted into the spreadsheet

![][2]

Getting the output of the console

As of today, there are 3 ways to get the performance results:

  1. Manually: Run each tests manually and look at the FPS
  2. Automatically: Run the tests from Xcode (or Visual Studio or Instruments) and use Xcode's console to see the cocos2d output
  3. Automatically: Run the tests from the device, and connect to the cocos2d console using telnet

1 should be avoided since it is manual. 3 is preferred over 2, at least for old iOS devices. 2 affected the performance on iOS 6 on and iPod Touch 4.

[2]:

Clone this wiki locally