Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Implement broadcasts in async pools #3755

Merged
merged 1 commit into from
Sep 9, 2022
Merged

Implement broadcasts in async pools #3755

merged 1 commit into from
Sep 9, 2022

Conversation

NelsonVides
Copy link
Collaborator

Might be useful for telling all workers about a certain event.

@mongoose-im

This comment was marked as outdated.

@codecov
Copy link

codecov bot commented Sep 9, 2022

Codecov Report

Merging #3755 (8130597) into master (77951fc) will increase coverage by 0.05%.
The diff coverage is 69.23%.

@@            Coverage Diff             @@
##           master    #3755      +/-   ##
==========================================
+ Coverage   82.66%   82.71%   +0.05%     
==========================================
  Files         529      529              
  Lines       33957    33970      +13     
==========================================
+ Hits        28069    28097      +28     
+ Misses       5888     5873      -15     
Impacted Files Coverage Δ
src/async_pools/mongoose_batch_worker.erl 66.03% <0.00%> (-1.27%) ⬇️
src/async_pools/mongoose_aggregator_worker.erl 61.81% <62.50%> (+0.11%) ⬆️
src/async_pools/mongoose_async_pools.erl 92.30% <100.00%> (+0.64%) ⬆️
src/inbox/mod_inbox_rdbms_async.erl 70.58% <0.00%> (-1.48%) ⬇️
src/mod_muc_room.erl 78.39% <0.00%> (-0.12%) ⬇️
src/pubsub/mod_pubsub.erl 73.18% <0.00%> (+0.18%) ⬆️
src/ejabberd_ctl.erl 54.14% <0.00%> (+0.29%) ⬆️
src/mod_roster.erl 79.18% <0.00%> (+0.47%) ⬆️
src/logger/mongoose_log_filter.erl 79.45% <0.00%> (+1.36%) ⬆️
src/mod_roster_riak.erl 98.46% <0.00%> (+1.53%) ⬆️
... and 5 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

@NelsonVides NelsonVides marked this pull request as draft September 9, 2022 12:34
There is two new interfaces: one simply broadcasts a new value given a
key to all workers. The other, broadcasts new important data that should
be aggregated to all available keys. This is useful if we need to modify
whatever tasks are pending.
@mongoose-im
Copy link
Collaborator

mongoose-im commented Sep 9, 2022

small_tests_24 / small_tests / 8130597
Reports root / small


small_tests_25 / small_tests / 8130597
Reports root / small


dynamic_domains_pgsql_mnesia_24 / pgsql_mnesia / 8130597
Reports root/ big
OK: 3583 / Failed: 0 / User-skipped: 88 / Auto-skipped: 0


ldap_mnesia_24 / ldap_mnesia / 8130597
Reports root/ big
OK: 1948 / Failed: 0 / User-skipped: 551 / Auto-skipped: 0


dynamic_domains_mysql_redis_25 / mysql_redis / 8130597
Reports root/ big
OK: 3566 / Failed: 0 / User-skipped: 105 / Auto-skipped: 0


ldap_mnesia_25 / ldap_mnesia / 8130597
Reports root/ big
OK: 1948 / Failed: 0 / User-skipped: 551 / Auto-skipped: 0


dynamic_domains_pgsql_mnesia_25 / pgsql_mnesia / 8130597
Reports root/ big
OK: 3583 / Failed: 0 / User-skipped: 88 / Auto-skipped: 0


dynamic_domains_mssql_mnesia_25 / odbc_mssql_mnesia / 8130597
Reports root/ big
OK: 3582 / Failed: 1 / User-skipped: 88 / Auto-skipped: 0

rest_client_SUITE:roster:add_and_remove_some_contacts_properly
{error,
  {timeout_when_waiting_for_stanza,
    [{escalus_client,wait_for_stanza,
       [{client,
          <<"bob_add_and_remove_some_contacts_properly_2285@domain.example.com/res1">>,
          escalus_tcp,<0.9321.2>,
          [{event_manager,<0.9281.2>},
           {server,<<"domain.example.com">>},
           {username,
             <<"bOb_add_and_remove_some_contacts_properly_2285">>},
           {resource,<<"res1">>}],
          [{event_client,
             [{event_manager,<0.9281.2>},
            {server,<<"domain.example.com">>},
            {username,
              <<"bOb_add_and_remove_some_contacts_properly_2285">>},
            {resource,<<"res1">>}]},
           {resource,<<"res1">>},
           {username,
             <<"bob_add_and_remove_some_contacts_properly_2285">>},
           {server,<<"domain.example.com">>},
           {host,<<"localhost">>},
           {port,5222},
           {auth,{escalus_auth,auth_plain}},
           {wspath,undefined},
           {username,
             <<"bOb_add_and_remove_some_contacts_properly_2285">>},
           {server,<<"domain.example.com">>},
           {host,<<"localhost">>},
           {password,<<"makrolika">>},
           {stream_id,<<"7ac04e402fd57291">>}]},
        5000],
       [{file,
          "/home/circleci/project/big_tests/_build/default/lib/escalus/src/escalus_client.erl"},
        {line,136}]},
     {rest_client_SUITE,add_contact_check_roster_push,2,
       [{file,
          "/home/circleci/project/big_te...

Report log


pgsql_mnesia_24 / pgsql_mnesia / 8130597
Reports root/ big
OK: 3957 / Failed: 0 / User-skipped: 97 / Auto-skipped: 0


internal_mnesia_25 / internal_mnesia / 8130597
Reports root/ big
OK: 2069 / Failed: 0 / User-skipped: 430 / Auto-skipped: 0


elasticsearch_and_cassandra_25 / elasticsearch_and_cassandra_mnesia / 8130597
Reports root/ big
OK: 2409 / Failed: 0 / User-skipped: 425 / Auto-skipped: 0


pgsql_mnesia_25 / pgsql_mnesia / 8130597
Reports root/ big
OK: 3957 / Failed: 0 / User-skipped: 97 / Auto-skipped: 0


mysql_redis_25 / mysql_redis / 8130597
Reports root/ big
OK: 3963 / Failed: 1 / User-skipped: 102 / Auto-skipped: 0

pep_SUITE:pep_tests:unsubscribe_after_presence_unsubscription
{error,
  {{badmatch,
     [{xmlel,<<"message">>,
        [{<<"from">>,
        <<"alice_unsubscribe_after_presence_unsubscription_2232@localhost">>},
         {<<"to">>,
        <<"bob_unsubscribe_after_presence_unsubscription_2232@localhost/res1">>},
         {<<"type">>,<<"headline">>}],
        [{xmlel,<<"event">>,
           [{<<"xmlns">>,
           <<"http://jabber.org/protocol/pubsub#event">>}],
           [{xmlel,<<"items">>,
            [{<<"node">>,<<"ZlATXmdu6WuCowBp6Bn+fg==">>}],
            [{xmlel,<<"item">>,
               [{<<"id">>,<<"salmon">>}],
               [{xmlel,<<"entry">>,
                  [{<<"xmlns">>,
                  <<"http://www.w3.org/2005/Atom">>}],
                  []}]}]}]},
         {xmlel,<<"headers">>,
           [{<<"xmlns">>,<<"http://jabber.org/protocol/shim">>}],
           []}]}]},
   [{pep_SUITE,'-unsubscribe_after_presence_unsubscription/1-fun-0-',2,
      [{file,"/home/circleci/project/big_tests/tests/pep_SUITE.erl"},
       {line,384}]},
    {escalus_story,story,4,
      [{file,
         "/home/circleci/project/big_tests/_build/default/lib/escalus/src/escalus_story.erl"},
       {line,72}]},
    {test_server,ts_tc,3,[{file,"test_server.erl"},{line,1782}]},
    {test_server,run_test_case_eval1,6,
      [{file,"test_server.erl"},{line,1291}]},
    {test_server,run_test_case_eval,9,
      [{file,"test_server.erl"},{line,1223}]}]}}

Report log


mssql_mnesia_25 / odbc_mssql_mnesia / 8130597
Reports root/ big
OK: 3955 / Failed: 2 / User-skipped: 97 / Auto-skipped: 0

muc_SUITE:hibernation:hibernated_room_can_be_queried_for_archive
{error,{{assertion_failed,assert,is_groupchat_message,
              [<<"Restorable message">>],
              undefined,"undefined"},
    [{escalus_new_assert,assert_true,2,
               [{file,"/home/circleci/project/big_tests/_build/default/lib/escalus/src/escalus_new_assert.erl"},
                {line,84}]},
     {muc_SUITE,wait_for_mam_result,3,
          [{file,"/home/circleci/project/big_tests/tests/muc_SUITE.erl"},
           {line,4394}]},
     {muc_SUITE,'-hibernated_room_can_be_queried_for_archive/1-fun-0-',3,
          [{file,"/home/circleci/project/big_tests/tests/muc_SUITE.erl"},
           {line,4130}]},
     {escalus_story,story,4,
            [{file,"/home/circleci/project/big_tests/_build/default/lib/escalus/src/escalus_story.erl"},
             {line,72}]},
     {muc_SUITE,hibernated_room_can_be_queried_for_archive,1,
          [{file,"/home/circleci/project/big_tests/tests/muc_SUITE.erl"},
           {line,4126}]},
     {test_server,ts_tc,3,[{file,"test_server.erl"},{line,1782}]},
     {test_server,run_test_case_eval1,6,
            [{file,"test_server.erl"},{line,1291}]},
     {test_server,run_test_case_eval,9,
            [{file,"test_server.erl"},{line,1223}]}]}}

Report log

push_integration_SUITE:pubsub_ful:pm_notifications_with_inbox:inbox_msg_unread_count_fcm
{error,
  {{assertMatch,
     [{module,push_integration_SUITE},
      {line,662},
      {expression,"Data"},
      {pattern,"# { << \"message-count\" >> := ExpectedCount }"},
      {value,
        #{<<"last-message-body">> => <<"Private message">>,
        <<"last-message-sender">> =>
          <<"alice_inbox_msg_unread_count_fcm_2587@localhost">>,
        <<"message-count">> => 1}}]},
   [{push_integration_SUITE,check_notification,2,
      [{file,
         "/home/circleci/project/big_tests/tests/push_integration_SUITE.erl"},
       {line,662}]},
    {push_integration_SUITE,'-inbox_msg_unread_count/3-fun-0-',6,
      [{file,
         "/home/circleci/project/big_tests/tests/push_integration_SUITE.erl"},
       {line,578}]},
    {escalus_story,story,4,
      [{file,
         "/home/circleci/project/big_tests/_build/default/lib/escalus/src/escalus_story.erl"},
       {line,72}]},
    {test_server,ts_tc,3,[{file,"test_server.erl"},{line,1782}]},
    {test_server,run_test_case_eval1,6,
      [{file,"test_server.erl"},{line,1291}]},
    {test_server,run_test_case_eval,9,
      [{file,"test_server.erl"},{line,1223}]}]}}

Report log


riak_mnesia_24 / riak_mnesia / 8130597
Reports root/ big
OK: 2248 / Failed: 0 / User-skipped: 418 / Auto-skipped: 0


dynamic_domains_mssql_mnesia_25 / odbc_mssql_mnesia / 8130597
Reports root/ big
OK: 3583 / Failed: 0 / User-skipped: 88 / Auto-skipped: 0


mssql_mnesia_25 / odbc_mssql_mnesia / 8130597
Reports root/ big
OK: 3957 / Failed: 0 / User-skipped: 97 / Auto-skipped: 0

@NelsonVides NelsonVides marked this pull request as ready for review September 9, 2022 13:02
@arcusfelis arcusfelis merged commit 7e19c4f into master Sep 9, 2022
@arcusfelis arcusfelis deleted the async/broadcasts branch September 9, 2022 15:09
@chrzaszcz chrzaszcz added this to the 6.0.0 milestone Dec 12, 2022
# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants