Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

No Sleeping on the Job #1246

Merged
merged 1 commit into from
Apr 29, 2023
Merged

Conversation

rmartin16
Copy link
Member

@rmartin16 rmartin16 commented Apr 28, 2023

Changes

  • Replace any calls to time.sleep(x) with time.sleep(0)
  • I'm seeing ~37% speedup locally from 29s to 18s
  • (Note: I'm working on getting pytest-xdist working....tests run in <5s then on 8 hyper threaded cores)

Examples of Slow Tests

Top 50 previously slowest tests
0.55s call     tests/integrations/subprocess/test_Subprocess__stream_output.py::test_stuck_streamer
0.35s call     tests/integrations/subprocess/test_Subprocess__stream_output.py::test_keyboard_interrupt
0.33s call     tests/platforms/iOS/xcode/test_run.py::test_run_app_simulator_launch_failure
0.27s call     tests/platforms/iOS/xcode/test_run.py::test_run_app_simulator_booted
0.26s call     tests/platforms/iOS/xcode/test_run.py::test_run_app_simulator_shut_down
0.26s call     tests/platforms/iOS/xcode/test_run.py::test_run_app_with_passthrough
0.26s call     tests/platforms/iOS/xcode/test_run.py::test_run_app_simulator_booted_underscore
0.26s call     tests/platforms/iOS/xcode/test_run.py::test_run_app_simulator_non_integer_pid
0.26s call     tests/platforms/iOS/xcode/test_run.py::test_run_app_simulator_no_pid
0.26s call     tests/platforms/iOS/xcode/test_run.py::test_run_app_test_mode
0.26s call     tests/platforms/iOS/xcode/test_run.py::test_run_app_test_mode_with_passthrough
0.26s call     tests/platforms/macOS/app/test_run.py::test_run_app_find_pid_failed
0.26s call     tests/platforms/macOS/xcode/test_run.py::test_run_app
0.25s call     tests/platforms/macOS/xcode/test_run.py::test_run_app_test_mode_with_passthrough
0.25s call     tests/platforms/macOS/xcode/test_run.py::test_run_app_test_mode
0.25s call     tests/platforms/macOS/app/test_run.py::test_run_app_test_mode
0.25s call     tests/platforms/macOS/app/test_run.py::test_run_app
0.25s call     tests/platforms/macOS/xcode/test_run.py::test_run_app_with_passthrough
0.25s call     tests/platforms/macOS/app/test_run.py::test_run_app_with_passthrough
0.25s call     tests/platforms/macOS/app/test_run.py::test_run_app_failed
0.21s call     tests/integrations/subprocess/test_Subprocess__run__controlled_console.py::test_text_eq_true_default_overriding[in_kwargs3-kwargs3]
0.21s call     tests/integrations/subprocess/test_Subprocess__run__controlled_console.py::test_text_eq_true_default_overriding[in_kwargs2-kwargs2]
0.21s call     tests/integrations/subprocess/test_Subprocess__run__controlled_console.py::test_text_eq_true_default_overriding[in_kwargs4-kwargs4]
0.21s call     tests/integrations/subprocess/test_Subprocess__run__controlled_console.py::test_text_eq_true_default_overriding[in_kwargs5-kwargs5]
0.21s call     tests/integrations/subprocess/test_Subprocess__run__controlled_console.py::test_text_eq_true_default_overriding[in_kwargs0-kwargs0]
0.21s call     tests/integrations/subprocess/test_Subprocess__run__stream_output__True.py::test_text_eq_true_default_overriding[in_kwargs1-kwargs1]
0.21s call     tests/integrations/subprocess/test_Subprocess__run__controlled_console.py::test_text_eq_true_default_overriding[in_kwargs1-kwargs1]
0.21s call     tests/integrations/subprocess/test_Subprocess__run__stream_output__True.py::test_text_eq_true_default_overriding[in_kwargs0-kwargs0]
0.21s call     tests/integrations/subprocess/test_Subprocess__run__stream_output__True.py::test_text_eq_true_default_overriding[in_kwargs2-kwargs2]
0.21s call     tests/integrations/subprocess/test_Subprocess__run__stream_output__True.py::test_text_eq_true_default_overriding[in_kwargs4-kwargs4]
0.21s call     tests/integrations/subprocess/test_Subprocess__run__stream_output__True.py::test_text_eq_true_default_overriding[in_kwargs3-kwargs3]
0.19s call     tests/test_mainline.py::test_interrupted_command_with_log
0.19s call     tests/test_mainline.py::test_unknown_command_error
0.18s call     tests/integrations/subprocess/test_Subprocess__run__stream_output__True.py::test_call_with_start_new_session[Linux-None-run_kwargs0]
0.13s teardown tests/platforms/windows/visualstudio/test_run.py::test_run_app_test_mode_with_args
0.11s call     tests/integrations/subprocess/test_Subprocess__run__controlled_console.py::test_debug_call_with_env
0.11s call     tests/integrations/subprocess/test_Subprocess__run__stream_output__True.py::test_calledprocesserror_exception_logging
0.11s call     tests/integrations/subprocess/test_Subprocess__run__controlled_console.py::test_debug_call
0.11s call     tests/integrations/subprocess/test_Subprocess__run__stream_output__True.py::test_debug_call_with_env
0.11s call     tests/integrations/subprocess/test_Subprocess__run__stream_output__True.py::test_call[Darwin]
0.11s call     tests/integrations/subprocess/test_Subprocess__run__stream_output__True.py::test_call_with_start_new_session[Darwin-True-run_kwargs4]
0.11s call     tests/integrations/subprocess/test_Subprocess__run__stream_output__True.py::test_call_with_start_new_session[Windows-False-run_kwargs8]
0.11s call     tests/integrations/subprocess/test_Subprocess__run__controlled_console.py::test_calledprocesserror
0.11s call     tests/integrations/subprocess/test_Subprocess__run__stream_output__True.py::test_debug_call
0.11s call     tests/integrations/subprocess/test_Subprocess__run__controlled_console.py::test_call
0.11s call     tests/integrations/subprocess/test_Subprocess__run__stream_output__True.py::test_call_with_arg
0.11s call     tests/integrations/subprocess/test_Subprocess__run__stream_output__True.py::test_call_with_start_new_session[Darwin-None-run_kwargs3]
0.11s call     tests/integrations/subprocess/test_Subprocess__run__controlled_console.py::test_call_with_arg
0.11s call     tests/integrations/subprocess/test_Subprocess__run__stream_output__True.py::test_call_with_start_new_session[Windows-True-run_kwargs7]
0.11s call     tests/integrations/subprocess/test_Subprocess__run__controlled_console.py::test_stderr_is_redirected
Top 50 currently slowest tests
0.35s call     tests/integrations/subprocess/test_Subprocess__stream_output.py::test_keyboard_interrupt
0.20s call     tests/test_mainline.py::test_unknown_command_error
0.19s call     tests/test_mainline.py::test_interrupted_command_with_log
0.13s teardown tests/platforms/windows/visualstudio/test_run.py::test_run_app_test_mode_with_args
0.09s call     tests/integrations/subprocess/test_Subprocess__run__stream_output__True.py::test_call_with_start_new_session[Linux-None-run_kwargs0]
0.08s call     tests/platforms/iOS/xcode/test_run.py::test_run_app_simulator_uninstall_failure
0.08s call     tests/platforms/web/static/test_run.py::test_cleanup_server_error[localhost-8080-exception3-localhost:8080 is already in use.]
0.07s setup    tests/integrations/linuxdeploy/test_LinuxDeploy__is_elf_file.py::test_is_elf_header_positive_detection
0.07s call     tests/console/test_Log.py::test_save_log_to_file_with_multiple_exceptions
0.06s call     tests/integrations/cookiecutter/test_RGBExtension.py::test_py_tag[#000000-0.0-0.0-0.0]
0.06s setup    tests/commands/upgrade/test_call.py::test_list_specific_tools
0.04s call     tests/platforms/linux/system/test_run.py::test_supported_host_os
0.04s call     tests/test_mainline.py::test_command
0.04s call     tests/test_mainline.py::test_interrupted_command
0.04s call     tests/platforms/linux/appimage/test_build.py::test_build_appimage_with_plugins_in_docker
0.03s call     tests/platforms/linux/flatpak/test_mixin.py::test_verify_linux
0.03s call     tests/test_mainline.py::test_command_warning
0.03s call     tests/platforms/linux/flatpak/test_open.py::test_open
0.03s call     tests/platforms/linux/system/test_build.py::test_build_app
0.03s setup    tests/platforms/windows/app/test_package.py::test_package_formats
0.03s call     tests/platforms/linux/appimage/test_build.py::test_build_appimage_in_docker
0.03s call     tests/integrations/android_sdk/AndroidSDK/test_verify_avd.py::test_unrecognized_emulator_skin
0.02s call     tests/platforms/linux/appimage/test_build.py::test_build_appimage_with_plugin
0.02s setup    tests/integrations/docker/test_DockerAppContext__run.py::test_cwd
0.02s call     tests/test_mainline.py::test_help
0.02s call     tests/test_mainline.py::test_test_failure
0.02s call     tests/config/test_parse_config.py::test_requires
0.02s setup    tests/integrations/docker/test_DockerAppContext__run.py::test_call_with_arg_and_env
0.02s setup    tests/integrations/java/test_JDK__verify.py::test_no_javac[Windows-java_home3-NotADirectoryError]
0.02s call     tests/commands/create/test_call.py::test_create
0.02s call     tests/integrations/linuxdeploy/test_LinuxDeploy__verify_plugins.py::test_complex_plugin_config
0.02s call     tests/test_mainline.py::test_command_error
0.02s call     tests/integrations/xcode/test_confirm_xcode_license_accepted.py::test_license_status_unknown
0.02s setup    tests/integrations/docker/test_DockerAppContext__run.py::test_call_with_path_arg_and_env
0.02s call     tests/test_cmdline.py::test_run_command[run --test -r-expected_options4]
0.02s setup    tests/integrations/docker/test_DockerAppContext__dockerize_path.py::test_dockerize_path[value-value]
0.02s setup    tests/integrations/docker/test_DockerAppContext__dockerize_path.py::test_dockerize_path[/unmodified/path:{tmp_path}/bundle/path/to/file:{tmp_path}/briefcase/path/to/other/file-/unmodified/path:/app/path/to/file:/home/brutus/.cache/briefcase/path/to/other/file]
0.02s call     tests/test_cmdline.py::test_run_command[run -- --test-expected_options7]
0.02s setup    tests/integrations/docker/test_DockerAppContext__run.py::test_simple_verbose_call
0.02s setup    tests/integrations/docker/test_DockerAppContext__check_output.py::test_simple_call
0.02s call     tests/console/test_Log.py::test_save_log_to_file_with_exception
0.02s call     tests/integrations/android_sdk/AndroidSDK/test_verify.py::test_download_sdk_legacy_install
0.02s setup    tests/integrations/docker/test_DockerAppContext__check_output.py::test_extra_mounts
0.02s setup    tests/integrations/docker/test_DockerAppContext__check_output.py::test_call_with_path_arg_and_env
0.02s setup    tests/integrations/docker/test_DockerAppContext__check_output.py::test_simple_verbose_call
0.02s setup    tests/integrations/docker/test_DockerAppContext__check_output.py::test_call_with_arg_and_env
0.02s call     tests/test_cmdline.py::test_bare_command_help
0.02s setup    tests/integrations/docker/test_DockerAppContext__check_output.py::test_cwd
0.02s setup    tests/integrations/docker/test_DockerAppContext__run.py::test_extra_mounts
0.02s call     tests/test_cmdline.py::test_run_command[run --test -- --test-expected_options8]

PR Checklist:

  • All new features have been tested
  • All new features have been documented
  • I have read the CONTRIBUTING.md file
  • I will abide by the code of conduct

@rmartin16 rmartin16 force-pushed the no-sleep-till-brooklyn branch from 8ee021c to 8561dec Compare April 28, 2023 21:50
@rmartin16 rmartin16 marked this pull request as ready for review April 28, 2023 23:38
Copy link
Member

@freakboy3742 freakboy3742 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was going to comment that my only reservation about this PR was that it had an insufficient number of Beastie Boys references... and then I saw the branch name. :-P

Looks like a good set of cleanups to me, and I'm seeing a 100% speedup on test runs, which is a very nice win.

@freakboy3742 freakboy3742 merged commit 03bd22f into beeware:main Apr 29, 2023
@rmartin16 rmartin16 deleted the no-sleep-till-brooklyn branch April 29, 2023 01:36
# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants