Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Batch Couch DB create calls #6779

Merged
merged 4 commits into from
Jul 12, 2023
Merged

Conversation

akhenry
Copy link
Contributor

@akhenry akhenry commented Jul 7, 2023

Closes VIPEROMCT-345

Describe your changes:

Implements persistence batching via debouncing for the Couch create action which in the case of Import From JSON can result in a very large number of POST requests to the Couch server.

All Submissions:

  • Have you followed the guidelines in our Contributing document?
  • Have you checked to ensure there aren't other open Pull Requests for the same update/change?
  • Is this change backwards compatible? For example, developers won't need to change how they are calling the API or how they've extended core plugins such as Tables or Plots.

Author Checklist

  • Changes address original issue?
  • Tests included and/or updated with changes?
  • Command line build passes?
  • Has this been smoke tested?
  • Testing instructions included in associated issue OR is this a dependency/testcase change?

Reviewer Checklist

  • Changes appear to address issue?
  • Reviewer has tested changes by following the provided instructions?
  • Changes appear not to be breaking changes?
  • Appropriate automated tests included?
  • Code style and in-line documentation are appropriate?
  • Has associated issue been labelled unverified? (only applicable if this PR closes the issue)
  • Has associated issue been labelled bug? (only applicable if this PR is for a bug fix)

@akhenry akhenry requested a review from shefalijoshi July 7, 2023 02:00
@deploysentinel
Copy link

deploysentinel bot commented Jul 7, 2023

Current Playwright Test Results Summary

✅ 13 Passing - ⚠️ 1 Flaky

Run may still be in progress, this comment will be updated as current testing workflow or job completes...

(Last updated on 07/12/2023 04:35:45am UTC)

Run Details

Running Workflow e2e-couchdb on Github Actions

Commit: d3e8669

Started: 07/12/2023 04:31:48am UTC

⚠️ Flakes

📄   functional/plugins/notebook/notebookWithCouchDB.e2e.spec.js • 1 Flake

Test Case Results

Test Case Last 7 days Failures Last 7 days Flakes
Notebook Tests with CouchDB @couchdb Inspect Notebook Entry Network Requests
Retry 1Initial Attempt
14.29% (4) 4 / 28 runs
failed over last 7 days
67.86% (19) 19 / 28 runs
flaked over last 7 days

View Detailed Build Results


Current Playwright Test Results Summary

✅ 123 Passing - ⚠️ 2 Flaky

Run may still be in progress, this comment will be updated as current testing workflow or job completes...

(Last updated on 07/12/2023 04:35:45am UTC)

Run Details

Running Job e2e-stable on CircleCI

Commit: d3e8669

Started: 07/12/2023 04:30:01am UTC

⚠️ Flakes

📄   functional/plugins/plot/autoscale.e2e.spec.js • 1 Flake

Test Case Results

Test Case Last 7 days Failures Last 7 days Flakes
Autoscale User can set autoscale with a valid range @snapshot
Retry 1Initial Attempt
32.10% (26) 26 / 81 runs
failed over last 7 days
3.70% (3) 3 / 81 runs
flaked over last 7 days

📄   functional/plugins/imagery/exampleImagery.e2e.spec.js • 1 Flake

Test Case Results

Test Case Last 7 days Failures Last 7 days Flakes
Example Imagery Object Can use Mouse Wheel to zoom in and out of latest image
Retry 1Initial Attempt
4.71% (4) 4 / 85 runs
failed over last 7 days
76.47% (65) 65 / 85 runs
flaked over last 7 days

View Detailed Build Results


@akhenry akhenry changed the title Batch persistence for fun and profit Batch Couch DB create calls Jul 7, 2023
@unlikelyzero unlikelyzero added the pr:e2e:couchdb npm run test:e2e:couchdb label Jul 7, 2023
@github-actions github-actions bot removed the pr:e2e:couchdb npm run test:e2e:couchdb label Jul 7, 2023
Copy link
Contributor

@shefalijoshi shefalijoshi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good!
However, this doesn't seem to have fixed the issue VIPEROMCT-345 completely. (Admittedly I tested this on Firefox)
Open MCT froze on and off while trying to complete the large import. That said, the browser did not crash.
If this is the desired outcome, then we should merge this.

@shefalijoshi
Copy link
Contributor

Looks good! However, this doesn't seem to have fixed the issue VIPEROMCT-345 completely. (Admittedly I tested this on Firefox) Open MCT froze on and off while trying to complete the large import. That said, the browser did not crash. If this is the desired outcome, then we should merge this.

Apparently the freezing is a different issue that's being fixed elsewhere. The error mentioned in the linked issue is no longer visible, so I'm merging this.

@shefalijoshi shefalijoshi enabled auto-merge (squash) July 10, 2023 20:57
@codecov
Copy link

codecov bot commented Jul 10, 2023

Codecov Report

Merging #6779 (ec9fd7c) into master (d08ea62) will increase coverage by 0.09%.
The diff coverage is 100.00%.

❗ Current head ec9fd7c differs from pull request most recent head d3e8669. Consider uploading reports for the commit d3e8669 to get more accurate results

@@            Coverage Diff             @@
##           master    #6779      +/-   ##
==========================================
+ Coverage   53.74%   53.83%   +0.09%     
==========================================
  Files         624      624              
  Lines       24879    24867      -12     
  Branches     2498     2493       -5     
==========================================
+ Hits        13370    13387      +17     
+ Misses      10841    10812      -29     
  Partials      668      668              
Flag Coverage Δ *Carryforward flag
e2e-full 42.25% <ø> (+0.08%) ⬆️ Carriedforward from 447d9fa
e2e-stable 55.22% <0.00%> (-0.31%) ⬇️
unit 48.56% <100.00%> (+0.11%) ⬆️

*This pull request uses carry forward flags. Click here to find out more.

Impacted Files Coverage Δ
...c/plugins/persistence/couch/CouchObjectProvider.js 91.58% <100.00%> (+0.86%) ⬆️

... and 15 files with indirect coverage changes


Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d08ea62...d3e8669. Read the comment docs.

@unlikelyzero unlikelyzero added the pr:e2e:couchdb npm run test:e2e:couchdb label Jul 12, 2023
@github-actions github-actions bot removed the pr:e2e:couchdb npm run test:e2e:couchdb label Jul 12, 2023
@shefalijoshi shefalijoshi merged commit ac22beb into master Jul 12, 2023
7 checks passed
@shefalijoshi shefalijoshi deleted the batch-persistence-for-fun-and-profit branch July 12, 2023 04:36
# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants