Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Allow registering search attributes without Advance Visibility enabled #5185

Conversation

lancezhao-ins
Copy link
Contributor

What changed?

  • Remove validation for Advance Visibility Store
  • Add Advance Visibility Config check before updating ElasticSearch/OpenSearch mapping
  • Remove co-related test of validate 'no advanced config' for AddSearchAttribute
  • Update CHANGELOG.md

Why?
It's convenient to test the search attribute API in the workflow code before using advanced visibility.

How did you test it?
Tested locally with AdvancedVisibility Enabled and AdvancedVisibility Disabled

Potential risks

Release notes
Allow registering search attributes when Advanced Visibility is not enabled

Documentation Changes

…nfig

- Remove validation for Advance Visibility Store
- Add Advance Visibility Config check before update ElasticSearch/OpenSearch mapping
- Remove co-related test for 'no advanced config'
Update CHANGELOG.md
@CLAassistant
Copy link

CLAassistant commented Apr 3, 2023

CLA assistant check
All committers have signed the CLA.

@lancezhao-ins lancezhao-ins changed the title [Draft] Allow registering search attributes without Advance Visibility enabled Allow registering search attributes without Advance Visibility enabled Apr 3, 2023
@lancezhao-ins lancezhao-ins marked this pull request as draft April 3, 2023 04:46
@lancezhao-ins lancezhao-ins marked this pull request as ready for review April 3, 2023 04:52
@longquanzheng
Copy link
Contributor

if adh.esClient.IsNotFoundError(err) {
err = adh.params.ESClient.CreateIndex(ctx, index)
// when have valid advance visibility config, update elasticsearch mapping, new added field will not be able to remove or update
if err := adh.validateConfigForAdvanceVisibility(); err == nil {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The only downside of this approach is that NOOP will never be logged/reported, maybe add warn level message when validation is failing?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added warning "Skip updating OpenSearch/ElasticSearch mapping since Advance Visibility hasn't been enabled." when Advance Visibility is not enabled.

@lancezhao-ins
Copy link
Contributor Author

lgtm. Can you also test it with https://github.com/uber-common/cadence-samples/tree/master/cmd/samples/recipes/searchattributes ?

Passed using ./docker/dev/cassandra-esv7-kafka.yml setup.
cadence-samples-searchattributes-test

@coveralls
Copy link

coveralls commented Apr 4, 2023

Pull Request Test Coverage Report for Build 01876c51-aad7-40a4-82c5-60702c96798c

  • 11 of 17 (64.71%) changed or added relevant lines in 1 file are covered.
  • 115 unchanged lines in 23 files lost coverage.
  • Overall coverage increased (+0.02%) to 57.115%

Changes Missing Coverage Covered Lines Changed/Added Lines %
service/frontend/adminHandler.go 11 17 64.71%
Files with Coverage Reduction New Missed Lines %
service/history/queue/transfer_queue_processor_base.go 1 77.62%
common/membership/hashring.go 2 83.54%
common/persistence/executionManager.go 2 78.89%
common/persistence/nosql/nosqlplugin/cassandra/tasks.go 2 75.58%
common/persistence/statsComputer.go 2 94.64%
common/task/parallelTaskProcessor.go 2 92.75%
common/task/weightedRoundRobinTaskScheduler.go 2 89.12%
service/frontend/adminHandler.go 2 47.88%
service/frontend/workflowHandler.go 2 59.97%
service/history/task/transfer_standby_task_executor.go 2 87.24%
Totals Coverage Status
Change from base Build 01876c46-6dd5-40a9-9fb8-59e73393619c: 0.02%
Covered Lines: 85302
Relevant Lines: 149351

💛 - Coveralls

@longquanzheng longquanzheng enabled auto-merge (squash) April 10, 2023 17:58
@longquanzheng longquanzheng merged commit 9fc4485 into cadence-workflow:master Apr 10, 2023
davidporter-id-au added a commit that referenced this pull request Apr 12, 2023
commit eade936
Author: David Porter <david.porter@uber.com>
Date:   Wed Apr 12 13:54:13 2023 -0700

    Corrects the config-store handling for not-found errors (#5203)

commit 9fc4485
Author: lancezhao-ins <99238165+lancezhao-ins@users.noreply.github.com>
Date:   Tue Apr 11 04:21:15 2023 +1000

    Allow registering search attributes without Advance Visibility enabled (#5185)

    * remove validation & test for add search attribute with no advanced config

    - Remove validation for Advance Visibility Store
    - Add Advance Visibility Config check before update ElasticSearch/OpenSearch mapping
    - Remove co-related test for 'no advanced config'

    * Update CHANGELOG.md

    Update CHANGELOG.md

    * Add warn level message if skip updating OpenSearch/ElasticSearch mapping

    * Add warn level message and add validSearchAttributes in development.yaml

    ---------

    Co-authored-by: Quanzheng Long <prclqz@gmail.com>

commit d165c7b
Author: neil-xie <104041627+neil-xie@users.noreply.github.com>
Date:   Mon Apr 10 10:46:04 2023 -0700

    Update idls version (#5200)

commit b2bc8bf
Author: Mantas Šidlauskas <mantass@netapp.com>
Date:   Mon Apr 10 20:12:53 2023 +0300

    Add thin ES clients (#5162)

    * Add thin ES clients
davidporter-id-au added a commit that referenced this pull request Apr 13, 2023
commit eade936
Author: David Porter <david.porter@uber.com>
Date:   Wed Apr 12 13:54:13 2023 -0700

    Corrects the config-store handling for not-found errors (#5203)

commit 9fc4485
Author: lancezhao-ins <99238165+lancezhao-ins@users.noreply.github.com>
Date:   Tue Apr 11 04:21:15 2023 +1000

    Allow registering search attributes without Advance Visibility enabled (#5185)

    * remove validation & test for add search attribute with no advanced config

    - Remove validation for Advance Visibility Store
    - Add Advance Visibility Config check before update ElasticSearch/OpenSearch mapping
    - Remove co-related test for 'no advanced config'

    * Update CHANGELOG.md

    Update CHANGELOG.md

    * Add warn level message if skip updating OpenSearch/ElasticSearch mapping

    * Add warn level message and add validSearchAttributes in development.yaml

    ---------

    Co-authored-by: Quanzheng Long <prclqz@gmail.com>

commit d165c7b
Author: neil-xie <104041627+neil-xie@users.noreply.github.com>
Date:   Mon Apr 10 10:46:04 2023 -0700

    Update idls version (#5200)

commit b2bc8bf
Author: Mantas Šidlauskas <mantass@netapp.com>
Date:   Mon Apr 10 20:12:53 2023 +0300

    Add thin ES clients (#5162)

    * Add thin ES clients

Merge issue
davidporter-id-au added a commit that referenced this pull request Apr 19, 2023
commit b18be27
Author: Mantas Šidlauskas <mantass@netapp.com>
Date:   Tue Apr 18 14:12:09 2023 +0300

    Add generic ES query building utilities (#5168)

commit 824f0ac
Author: agautam478 <72432016+agautam478@users.noreply.github.com>
Date:   Fri Apr 14 13:37:29 2023 -0700

    Fixed the nil pointer issues in the InactiveDomain Invariant (#5213)

commit c5678dd
Author: Ketsia <115650494+ketsiambaku@users.noreply.github.com>
Date:   Wed Apr 12 15:21:10 2023 -0700

    Fix consistent query metric (#5170)

    * add shardid tag to log

    * remove counter for overall scope

    * fix lint

commit eade936
Author: David Porter <david.porter@uber.com>
Date:   Wed Apr 12 13:54:13 2023 -0700

    Corrects the config-store handling for not-found errors (#5203)

commit 9fc4485
Author: lancezhao-ins <99238165+lancezhao-ins@users.noreply.github.com>
Date:   Tue Apr 11 04:21:15 2023 +1000

    Allow registering search attributes without Advance Visibility enabled (#5185)

    * remove validation & test for add search attribute with no advanced config

    - Remove validation for Advance Visibility Store
    - Add Advance Visibility Config check before update ElasticSearch/OpenSearch mapping
    - Remove co-related test for 'no advanced config'

    * Update CHANGELOG.md

    Update CHANGELOG.md

    * Add warn level message if skip updating OpenSearch/ElasticSearch mapping

    * Add warn level message and add validSearchAttributes in development.yaml

    ---------

    Co-authored-by: Quanzheng Long <prclqz@gmail.com>

commit d165c7b
Author: neil-xie <104041627+neil-xie@users.noreply.github.com>
Date:   Mon Apr 10 10:46:04 2023 -0700

    Update idls version (#5200)

commit b2bc8bf
Author: Mantas Šidlauskas <mantass@netapp.com>
Date:   Mon Apr 10 20:12:53 2023 +0300

    Add thin ES clients (#5162)

    * Add thin ES clients
# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants