Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

#

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Jump to bottom

Fix --rerun-big-tests not working with cth_validate_nodes #4431

Merged
merged 2 commits into from
Dec 12, 2024
Merged

Fix --rerun-big-tests not working with cth_validate_nodes #4431

merged 2 commits into from
Dec 12, 2024

Conversation

arcusfelis
Copy link
Contributor

@arcusfelis arcusfelis commented Dec 12, 2024 •
edited
Loading

This PR addresses "fixes after MIM-2329".

Proposed changes include:

  • Fix in examples (wrong order)
  • Properly parse TEST_HOSTS="", used in --rerun-big-tests.

Fixes:

*** CT 2024-12-12 12:53:32.459 *** Suite Hook
[🔗](file:///Users/mikhailuvarov/erlang/esl/MongooseIM/big_tests/ct_report/ct_run.test@MikhailU.2024-12-12_12.53.31/big_tests.tests.private_SUITE.logs/run.2024-12-12_12.53.32/private_suite.init_per_suite.html#e-1)

Call to CTH failed: error:{{undefined,{hosts,'',node}},
                           [{distributed_helper,get_or_fail,1,
                                [{file,
                                     "/Users/mikhailuvarov/erlang/esl/MongooseIM/big_tests/../test/common/distributed_helper.erl"},
                                 {line,189}]},
                            {distributed_helper,rpc_spec,1,
                                [{file,
                                     "/Users/mikhailuvarov/erlang/esl/MongooseIM/big_tests/../test/common/distributed_helper.erl"},
                                 {line,185}]},
                            {distributed_helper,validate_node,1,
                                [{file,
                                     "/Users/mikhailuvarov/erlang/esl/MongooseIM/big_tests/../test/common/distributed_helper.erl"},
                                 {line,238}]},
                            {distributed_helper,
                                '-validate_nodes/1-lc$^0/1-0-',1,
                                [{file,
                                     "/Users/mikhailuvarov/erlang/esl/MongooseIM/big_tests/../test/common/distributed_helper.erl"},
                                 {line,214}]},
                            {distributed_helper,validate_nodes,1,
                                [{file,
                                     "/Users/mikhailuvarov/erlang/esl/MongooseIM/big_tests/../test/common/distributed_helper.erl"},
                                 {line,214}]},
                            {cth_validate_nodes,pre_init_per_suite,3,
                                [{file,
                                     "/Users/mikhailuvarov/erlang/esl/MongooseIM/big_tests/src/cth_validate_nodes.erl"},
                                 {line,22}]},
                            {ct_hooks,catch_apply,3,
                                [{file,"ct_hooks.erl"},{line,491}]},
                            {ct_hooks,do_call_generic,4,
                                [{file,"ct_hooks.erl"},{line,247}]}]}

@mongoose-im
Copy link
Collaborator

mongoose-im commented Dec 12, 2024 •
edited
Loading

elasticsearch_and_cassandra_27 / elasticsearch_and_cassandra_mnesia / 6764754
Reports root/ big
OK: 472 / Failed: 0 / User-skipped: 49 / Auto-skipped: 0

small_tests_26 / small_tests / 6764754
Reports root / small

small_tests_27 / small_tests / 6764754
Reports root / small

small_tests_27_arm64 / small_tests / 6764754
Reports root / small

dynamic_domains_mssql_mnesia_27 / odbc_mssql_mnesia / 6764754
Reports root/ big
OK: 1675 / Failed: 2 / User-skipped: 89 / Auto-skipped: 3114

graphql_server_SUITE:admin_cli:clustering_tests:remove_alive_from_cluster 
{failed,
  {graphql_server_SUITE,end_per_testcase,
    {'EXIT',
      {{badrpc,nodedown},
       [{distributed_helper,rpc,
          [#{node => mongooseim3@localhost},
           mnesia,system_info,
           [is_running]],
          [{file,
             "/home/circleci/project/big_tests/../test/common/distributed_helper.erl"},
           {line,140}]},
        {distributed_helper,has_mnesia,1,
          [{file,
             "/home/circleci/project/big_tests/../test/common/distributed_helper.erl"},
           {line,31}]},
        {distributed_helper,remove_node_from_cluster,2,
          [{file,
             "/home/circleci/project/big_tests/../test/common/distributed_helper.erl"},
           {line,53}]},
        {graphql_server_SUITE,end_per_testcase,2,
          [{file,
             "/home/circleci/project/big_tests/tests/graphql_server_SUITE.erl"},
           {line,121}]},
        {test_server,do_end_per_testcase,4,
          [{file,"test_server.erl"},{line,1638}]},
        {test_server,run_test_case_eval1,6,
          [{file,"test_server.erl"},{line,1346}]},
        {test_server,run_test_case_eval,9,
          [{file,"test_server.erl"},{line,1235}]}]}}}}

Report log

graphql_server_SUITE:admin_cli:clustering_tests:remove_node_test 
{error,{#{what => invalid_response_code,expected_type => ok,
      response_code => {exit_status,3}},
    [{graphql_helper,assert_response_code,2,
             [{file,"/home/circleci/project/big_tests/tests/graphql_helper.erl"},
              {line,258}]},
     {graphql_helper,get_ok_value,2,
             [{file,"/home/circleci/project/big_tests/tests/graphql_helper.erl"},
              {line,241}]},
     {graphql_server_SUITE,remove_node_test,1,
                 [{file,"/home/circleci/project/big_tests/tests/graphql_server_SUITE.erl"},
                {line,225}]},
     {test_server,ts_tc,3,[{file,"test_server.erl"},{line,1794}]},
     {test_server,run_test_case_eval1,6,
            [{file,"test_server.erl"},{line,1303}]},
     {test_server,run_test_case_eval,9,
            [{file,"test_server.erl"},{line,1235}]}]}}

Report log

graphql_server_SUITE:admin_cli:clustering_tests:stop_node_test 
{error,{#{what => invalid_response_code,expected_type => ok,
      response_code => {exit_status,3}},
    [{graphql_helper,assert_response_code,2,
             [{file,"/home/circleci/project/big_tests/tests/graphql_helper.erl"},
              {line,258}]},
     {graphql_helper,get_ok_value,2,
             [{file,"/home/circleci/project/big_tests/tests/graphql_helper.erl"},
              {line,241}]},
     {graphql_server_SUITE,stop_node_test,1,
                 [{file,"/home/circleci/project/big_tests/tests/graphql_server_SUITE.erl"},
                {line,230}]},
     {test_server,ts_tc,3,[{file,"test_server.erl"},{line,1794}]},
     {test_server,run_test_case_eval1,6,
            [{file,"test_server.erl"},{line,1303}]},
     {test_server,run_test_case_eval,9,
            [{file,"test_server.erl"},{line,1235}]}]}}

Report log

graphql_metric_SUITE:init_per_suite 
{fail,[{validate_node_failed,{badrpc,nodedown},mongooseim3@localhost}]}

Report log

inbox_SUITE:init_per_suite 
{fail,[{validate_node_failed,{badrpc,nodedown},mongooseim3@localhost}]}

Report log

inbox_extensions_SUITE:init_per_suite 
{fail,[{validate_node_failed,{badrpc,nodedown},mongooseim3@localhost}]}

Report log

last_SUITE:init_per_suite 
{fail,[{validate_node_failed,{badrpc,nodedown},mongooseim3@localhost}]}

Report log

login_SUITE:init_per_suite 
{fail,[{validate_node_failed,{badrpc,nodedown},mongooseim3@localhost}]}

Report log

mam_SUITE:init_per_suite 
{fail,[{validate_node_failed,{badrpc,nodedown},mongooseim3@localhost}]}

Report log

mam_proper_SUITE:init_per_suite 
{fail,[{validate_node_failed,{badrpc,nodedown},mongooseim3@localhost}]}

Report log

mam_send_message_SUITE:init_per_suite 
{fail,[{validate_node_failed,{badrpc,nodedown},mongooseim3@localhost}]}

Report log

metrics_c2s_SUITE:init_per_suite 
{fail,[{validate_node_failed,{badrpc,nodedown},mongooseim3@localhost}]}

Report log

metrics_roster_SUITE:init_per_suite 
{fail,[{validate_node_failed,{badrpc,nodedown},mongooseim3@localhost}]}

Report log

metrics_session_SUITE:init_per_suite 
{fail,[{validate_node_failed,{badrpc,nodedown},mongooseim3@localhost}]}

Report log

metrics_api_SUITE:init_per_suite 
{fail,[{validate_node_failed,{badrpc,nodedown},mongooseim3@localhost}]}

Report log

mod_blocking_SUITE:init_per_suite 
{fail,[{validate_node_failed,{badrpc,nodedown},mongooseim3@localhost}]}

Report log

mod_http_upload_SUITE:init_per_suite 
{fail,[{validate_node_failed,{badrpc,nodedown},mongooseim3@localhost}]}

Report log

mod_ping_SUITE:init_per_suite 
{fail,[{validate_node_failed,{badrpc,nodedown},mongooseim3@localhost}]}

Report log

mod_time_SUITE:init_per_suite 
{fail,[{validate_node_failed,{badrpc,nodedown},mongooseim3@localhost}]}

Report log

mod_version_SUITE:init_per_suite 
{fail,[{validate_node_failed,{badrpc,nodedown},mongooseim3@localhost}]}

Report log

mongooseimctl_SUITE:init_per_suite 
{fail,[{validate_node_failed,{badrpc,nodedown},mongooseim3@localhost}]}

Report log

muc_SUITE:init_per_suite 
{fail,[{validate_node_failed,{badrpc,nodedown},mongooseim3@localhost}]}

Report log

muc_http_api_SUITE:init_per_suite 
{fail,[{validate_node_failed,{badrpc,nodedown},mongooseim3@localhost}]}

Report log

muc_light_SUITE:init_per_suite 
{fail,[{validate_node_failed,{badrpc,nodedown},mongooseim3@localhost}]}

Report log

muc_light_legacy_SUITE:init_per_suite 
{fail,[{validate_node_failed,{badrpc,nodedown},mongooseim3@localhost}]}

Report log

32 errors were truncated

ldap_mnesia_27 / ldap_mnesia / 6764754
Reports root/ big
OK: 2353 / Failed: 0 / User-skipped: 912 / Auto-skipped: 0

ldap_mnesia_26 / ldap_mnesia / 6764754
Reports root/ big
OK: 2353 / Failed: 0 / User-skipped: 912 / Auto-skipped: 0

dynamic_domains_pgsql_mnesia_27 / pgsql_mnesia / 6764754
Reports root/ big
OK: 4761 / Failed: 0 / User-skipped: 119 / Auto-skipped: 0

dynamic_domains_mysql_redis_27 / mysql_redis / 6764754
Reports root/ big
OK: 4726 / Failed: 0 / User-skipped: 154 / Auto-skipped: 0

dynamic_domains_pgsql_mnesia_26 / pgsql_mnesia / 6764754
Reports root/ big
OK: 4761 / Failed: 0 / User-skipped: 119 / Auto-skipped: 0

internal_mnesia_27 / internal_mnesia / 6764754
Reports root/ big
OK: 2495 / Failed: 0 / User-skipped: 770 / Auto-skipped: 0

pgsql_cets_27 / pgsql_cets / 6764754
Reports root/ big
OK: 4850 / Failed: 0 / User-skipped: 188 / Auto-skipped: 0

pgsql_mnesia_27 / pgsql_mnesia / 6764754
Reports root/ big
OK: 5151 / Failed: 0 / User-skipped: 128 / Auto-skipped: 0

mysql_redis_27 / mysql_redis / 6764754
Reports root/ big
OK: 5130 / Failed: 0 / User-skipped: 149 / Auto-skipped: 0

pgsql_mnesia_26 / pgsql_mnesia / 6764754
Reports root/ big
OK: 5151 / Failed: 0 / User-skipped: 128 / Auto-skipped: 0

cockroachdb_cets_27 / cockroachdb_cets / 6764754
Reports root/ big
OK: 4849 / Failed: 1 / User-skipped: 188 / Auto-skipped: 0

sm_SUITE:ws_tests:ping_timeout 
{error,
  {{assertion_failed,assert,is_presence,
     {xmlel,<<"iq">>,
       [{<<"from">>,<<"localhost">>},
        {<<"to">>,
         <<"geralt_ping_timeout_3530@localhost/escalus-default-resource">>},
        {<<"type">>,<<"get">>},
        {<<"id">>,<<"ce30886f350fb6ae">>},
        {<<"xmlns">>,<<"jabber:client">>}],
       [{xmlel,<<"ping">>,[{<<"xmlns">>,<<"urn:xmpp:ping">>}],[]}]},
     "<iq from='localhost' to='geralt_ping_timeout_3530@localhost/escalus-default-resource' type='get' id='ce30886f350fb6ae' xmlns='jabber:client'><ping xmlns='urn:xmpp:ping'/></iq>"},
   [{escalus_new_assert,assert_true,2,
      [{file,
         "/home/circleci/project/big_tests/_build/default/lib/escalus/src/escalus_new_assert.erl"},
       {line,84}]},
    {sm_helper,initial_presence_step,2,
      [{file,"/home/circleci/project/big_tests/tests/sm_helper.erl"},
       {line,138}]},
    {escalus_connection,connection_step,2,
      [{file,
         "/home/circleci/project/big_tests/_build/default/lib/escalus/src/escalus_connection.erl"},
       {line,163}]},
    {lists,foldl_1,3,[{file,"lists.erl"},{line,2151}]},
    {escalus_connection,start,2,
      [{file,
         "/home/circleci/project/big_tests/_build/default/lib/escalus/src/escalus_connection.erl"},
       {line,145}]},
    {sm_helper,connect_spec,3,
      [{file,"/home/circleci/project/big_tests/tests/sm_helper.erl"},
       {line,156}]},
    {sm_SUITE,ping_timeout,1,
      [{file,"/home/circleci/project/big_tests/tests/sm_SUITE.er...

Report log

mssql_mnesia_27 / odbc_mssql_mnesia / 6764754
Reports root/ big
OK: 5146 / Failed: 0 / User-skipped: 133 / Auto-skipped: 0

cockroachdb_cets_27 / cockroachdb_cets / 6764754
Reports root/ big
OK: 98 / Failed: 0 / User-skipped: 0 / Auto-skipped: 0

dynamic_domains_mssql_mnesia_27 / odbc_mssql_mnesia / 6764754
Reports root/ big
OK: 225 / Failed: 3 / User-skipped: 36 / Auto-skipped: 0

@codecov Codecov
Copy link

codecov bot commented Dec 12, 2024 •
edited
Loading

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 85.34%. Comparing base (3b5713a) to head (6764754).
Report is 3 commits behind head on master.

Additional details and impacted files 
@@            Coverage Diff             @@
##           master    #4431      +/-   ##
==========================================
- Coverage   85.38%   85.34%   -0.04%     
==========================================
  Files         549      549              
  Lines       33861    33861              
==========================================
- Hits        28913    28900      -13     
- Misses       4948     4961      +13

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

chrzaszcz
chrzaszcz approved these changes Dec 12, 2024
View reviewed changes
Copy link
Member

@chrzaszcz chrzaszcz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👌

@arcusfelis arcusfelis marked this pull request as ready for review December 12, 2024 13:09
@chrzaszcz chrzaszcz merged commit 15a7e84 into master Dec 12, 2024
4 checks passed
@chrzaszcz chrzaszcz deleted the fix-rerun-big-tests branch December 12, 2024 13:11
@jacekwegr jacekwegr added this to the 6.3.1 milestone Dec 23, 2024
# for free to join this conversation on GitHub. Already have an account? # to comment
Reviewers

@chrzaszcz chrzaszcz chrzaszcz approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
6.3.1
Development

Successfully merging this pull request may close these issues.
4 participants
@arcusfelis @mongoose-im @chrzaszcz @jacekwegr