Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

fix: Output JSONPath expression with match count message #2118

Merged
merged 1 commit into from
Dec 19, 2023

Conversation

mjsqu
Copy link
Contributor

@mjsqu mjsqu commented Dec 19, 2023

Adding the expression to the jsonpath logging message can be helpful in determining whether the message refers to a "next page" jsonpath parsing call, or a "records" parsing call


📚 Documentation preview 📚: https://meltano-sdk--2118.org.readthedocs.build/en/2118/

@mjsqu mjsqu changed the title feat(logs): Output jsonpath expression with match count message feat: Output jsonpath expression with match count message Dec 19, 2023
@mjsqu
Copy link
Contributor Author

mjsqu commented Dec 19, 2023

Sample output with:

poetry run pytest tests/core/rest/test_pagination.py --log-cli-level=INFO
====================================================================================================== test session starts =======================================================================================================
platform linux -- Python 3.8.16, pytest-7.4.3, pluggy-1.2.0
codspeed: 2.2.0 (callgraph: not supported)
benchmark: 4.0.0 (defaults: timer=time.perf_counter disable_gc=False min_rounds=5 min_time=0.000005 max_time=1.0 calibration_precision=10 warmup=False warmup_iterations=100000)
sqlalchemy: 2.0.23
rootdir: /home/mjohnston/repos/sdk
configfile: pyproject.toml
plugins: time-machine-2.13.0, benchmark-4.0.0, codspeed-2.2.0, requests-mock-1.11.0, xdoctest-1.1.2, snapshot-0.9.0
collected 13 items                                                                                                                                                                                                               

tests/core/rest/test_pagination.py::test_paginator_base_missing_implementation PASSED                                                                                                                                      [  7%]
tests/core/rest/test_pagination.py::test_single_page_paginator PASSED                                                                                                                                                      [ 15%]
tests/core/rest/test_pagination.py::test_paginator_page_number_missing_implementation PASSED                                                                                                                               [ 23%]
tests/core/rest/test_pagination.py::test_paginator_offset_missing_implementation PASSED                                                                                                                                    [ 30%]
tests/core/rest/test_pagination.py::test_paginator_hateoas_missing_implementation PASSED                                                                                                                                   [ 38%]
tests/core/rest/test_pagination.py::test_paginator_attributes 
--------------------------------------------------------------------------------------------------------- live log call ----------------------------------------------------------------------------------------------------------
INFO     singer_sdk.helpers.jsonpath:jsonpath.py:36 JSONPath $.nextPageToken match count: 1
PASSED                                                                                                                                                                                                                     [ 46%]
tests/core/rest/test_pagination.py::test_paginator_loop 
--------------------------------------------------------------------------------------------------------- live log call ----------------------------------------------------------------------------------------------------------
INFO     singer_sdk.helpers.jsonpath:jsonpath.py:36 JSONPath $.nextPageToken match count: 1
INFO     singer_sdk.helpers.jsonpath:jsonpath.py:36 JSONPath $.nextPageToken match count: 1
PASSED                                                                                                                                                                                                                     [ 53%]
tests/core/rest/test_pagination.py::test_paginator_page_number PASSED                                                                                                                                                      [ 61%]
tests/core/rest/test_pagination.py::test_paginator_offset 
--------------------------------------------------------------------------------------------------------- live log call ----------------------------------------------------------------------------------------------------------
INFO     singer_sdk.helpers.jsonpath:jsonpath.py:36 JSONPath $[*] match count: 2
INFO     singer_sdk.helpers.jsonpath:jsonpath.py:36 JSONPath $[*] match count: 1
INFO     singer_sdk.helpers.jsonpath:jsonpath.py:36 JSONPath $[*] match count: 0

Open to suggestions regarding the wording of the output message

Copy link

codspeed-hq bot commented Dec 19, 2023

CodSpeed Performance Report

Merging #2118 will not alter performance

Comparing mjsqu:feat/improved_jsonpath_logs (e34c62d) with main (40b6333)

Summary

✅ 6 untouched benchmarks

Copy link

codecov bot commented Dec 19, 2023

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (40b6333) 87.76% compared to head (e34c62d) 87.76%.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #2118   +/-   ##
=======================================
  Coverage   87.76%   87.76%           
=======================================
  Files          60       60           
  Lines        4946     4946           
  Branches     1007     1007           
=======================================
  Hits         4341     4341           
  Misses        424      424           
  Partials      181      181           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Collaborator

@edgarrmondragon edgarrmondragon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice, thanks @mjsqu!

@edgarrmondragon edgarrmondragon changed the title feat: Output jsonpath expression with match count message fix: Output JSONPath expression with match count message Dec 19, 2023
@edgarrmondragon edgarrmondragon added this pull request to the merge queue Dec 19, 2023
@edgarrmondragon edgarrmondragon removed this pull request from the merge queue due to a manual request Dec 19, 2023
@edgarrmondragon edgarrmondragon added this pull request to the merge queue Dec 19, 2023
Merged via the queue into meltano:main with commit 49c2a9c Dec 19, 2023
34 checks passed
# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants