Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Add Rounding Effect Analysis Feature #3

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

even-wei
Copy link
Collaborator

Overview:
This PR introduces a new Rounding Effect Analysis feature, aimed at analyzing and reporting the impacts of rounding in our data processing.

Changes:

  • Implemented a new analysis module for rounding effects.
  • Integrated this module with existing data processing pipelines.
  • Added relevant documentation and examples for usage.

Rationale:
To better understand and mitigate any potential distortions caused by data rounding.

@kentwelcome kentwelcome force-pushed the feature/add-rounding-effect-analysis branch from 3422476 to ea1949e Compare March 28, 2024 08:58
@DataRecce DataRecce deleted a comment from github-actions bot Apr 22, 2024
@kentwelcome kentwelcome force-pushed the feature/add-rounding-effect-analysis branch 2 times, most recently from f77c882 to dd81eee Compare April 26, 2024 08:47
@DataRecce DataRecce deleted a comment from github-actions bot Apr 26, 2024
Copy link

Recce Summary

Lineage Graph

graph LR
model.jaffle_shop.stg_payments["stg_payments [MODIFIED]"]
style model.jaffle_shop.stg_payments stroke:#ffa502
model.jaffle_shop.stg_payments---->model.jaffle_shop.customers
model.jaffle_shop.stg_payments---->model.jaffle_shop.orders
model.jaffle_shop.stg_payments-...->model.jaffle_shop.rounding_effect
model.jaffle_shop.customers["customers"]
model.jaffle_shop.customers---->model.jaffle_shop.customer_segments
model.jaffle_shop.customers---->model.jaffle_shop.customer_order_pattern
model.jaffle_shop.orders["orders"]
model.jaffle_shop.rounding_effect["rounding_effect [ADDED]"]
style model.jaffle_shop.rounding_effect stroke:#1dce00
model.jaffle_shop.customer_segments["customer_segments"]
model.jaffle_shop.customer_order_pattern["customer_order_pattern"]

Loading

Next Steps

If you want to check more detail inforamtion about the recce result,
please download the artifact file and open it by Recce CLI.

  # Unzip the downloaded artifact file
  tar -xf recce-state-file.zip

  # Launch the recce server based on the state file
  recce server --review recce_state.json

@popcornylu popcornylu force-pushed the feature/add-rounding-effect-analysis branch from dd81eee to c3c7cbd Compare May 29, 2024 06:03
Copy link

Recce Summary

Lineage Graph

graph LR
model.jaffle_shop.stg_payments["stg_payments

[What's Changed]
Code, Schema"]
style model.jaffle_shop.stg_payments stroke:#ffa502
model.jaffle_shop.stg_payments---->model.jaffle_shop.orders
model.jaffle_shop.stg_payments---->model.jaffle_shop.customers
model.jaffle_shop.stg_payments-...->model.jaffle_shop.rounding_effect
model.jaffle_shop.orders["orders"]
model.jaffle_shop.customers["customers"]
model.jaffle_shop.customers---->model.jaffle_shop.customer_segments
model.jaffle_shop.customers---->model.jaffle_shop.customer_order_pattern
model.jaffle_shop.rounding_effect["rounding_effect

[What's Changed]
Added Node"]
style model.jaffle_shop.rounding_effect stroke:#1dce00
model.jaffle_shop.customer_segments["customer_segments"]
model.jaffle_shop.customer_order_pattern["customer_order_pattern"]

Loading

Checks Summary

Checks Run Data Mismatch Detected
5 0

Next Steps

If you want to check more detail inforamtion about the recce result, please download the artifact file and open it by Recce CLI.

How to check the recce result

# Unzip the downloaded artifact file
tar -xf recce-state-file.zip

# Launch the recce server based on the state file
recce server --review recce_state.json

# Open the recce server http://localhost:8000 by your browser

@popcornylu popcornylu force-pushed the feature/add-rounding-effect-analysis branch from c3c7cbd to bfa3d74 Compare May 30, 2024 09:52
Copy link

Recce Summary

Lineage Graph

graph LR
model.jaffle_shop.stg_payments["stg_payments

[What's Changed]
Code, Schema"]
style model.jaffle_shop.stg_payments stroke:#ffa502
model.jaffle_shop.stg_payments---->model.jaffle_shop.orders
model.jaffle_shop.stg_payments---->model.jaffle_shop.customers
model.jaffle_shop.stg_payments-...->model.jaffle_shop.rounding_effect
model.jaffle_shop.orders["orders"]
model.jaffle_shop.customers["customers"]
model.jaffle_shop.customers---->model.jaffle_shop.customer_segments
model.jaffle_shop.customers---->model.jaffle_shop.customer_order_pattern
model.jaffle_shop.rounding_effect["rounding_effect

[What's Changed]
Added Node"]
style model.jaffle_shop.rounding_effect stroke:#1dce00
model.jaffle_shop.customer_segments["customer_segments"]
model.jaffle_shop.customer_order_pattern["customer_order_pattern"]

Loading

Checks Summary

Checks Run Data Mismatch Detected
5 0

Next Steps

If you want to check more detail inforamtion about the recce result, please download the artifact file and open it by Recce CLI.

How to check the recce result

# Unzip the downloaded artifact file
tar -xf recce-state-file.zip

# Launch the recce server based on the state file
recce server --review recce_state.json

# Open the recce server http://localhost:8000 by your browser

@popcornylu popcornylu force-pushed the feature/add-rounding-effect-analysis branch from bfa3d74 to 5eb597b Compare June 5, 2024 08:23
Copy link

github-actions bot commented Jun 5, 2024

Recce Summary

Lineage Graph

graph LR
model.jaffle_shop.stg_payments["stg_payments

[What's Changed]
Code, Schema"]
style model.jaffle_shop.stg_payments stroke:#ffa502
model.jaffle_shop.stg_payments---->model.jaffle_shop.orders
model.jaffle_shop.stg_payments---->model.jaffle_shop.customers
model.jaffle_shop.stg_payments-...->model.jaffle_shop.rounding_effect
model.jaffle_shop.orders["orders

[What's Changed]
"]
model.jaffle_shop.customers["customers

[What's Changed]
"]
model.jaffle_shop.customers---->model.jaffle_shop.customer_order_pattern
model.jaffle_shop.customers---->model.jaffle_shop.customer_segments
model.jaffle_shop.rounding_effect["rounding_effect

[What's Changed]
Added Node"]
style model.jaffle_shop.rounding_effect stroke:#1dce00
model.jaffle_shop.customer_order_pattern["customer_order_pattern"]
model.jaffle_shop.customer_segments["customer_segments"]

Loading

Checks Summary

Checks Run Data Mismatch Detected
4 2

Checks of Data Mismatch Detected

Name Type Mismatched Nodes
Model schema of customers, orders and modified nodes Schema Diff stg_payments, rounding_effect, and 2 more nodes
Row count of customers, orders and modified table model Row Count Diff rounding_effect, orders, customers

How to check the recce result

# Launch the recce server based on the state file
recce server --review --cloud --password yummy_jaffle

# Open the recce server http://localhost:8000 by your browser

@kentwelcome kentwelcome force-pushed the feature/add-rounding-effect-analysis branch 2 times, most recently from 13b9eea to 48d079b Compare June 6, 2024 18:31
@wcchang1115 wcchang1115 force-pushed the feature/add-rounding-effect-analysis branch 2 times, most recently from cbb7071 to 876790a Compare June 20, 2024 03:44
@wcchang1115 wcchang1115 force-pushed the feature/add-rounding-effect-analysis branch from 876790a to 675f3f4 Compare July 4, 2024 10:32
@kentwelcome kentwelcome force-pushed the feature/add-rounding-effect-analysis branch from 675f3f4 to 2f94dcb Compare July 9, 2024 07:31
@wcchang1115 wcchang1115 force-pushed the feature/add-rounding-effect-analysis branch 2 times, most recently from 675f3f4 to 2783f65 Compare August 14, 2024 02:10
Signed-off-by: Even Wei <shinycockorach@gmail.com>
@kentwelcome kentwelcome force-pushed the feature/add-rounding-effect-analysis branch from 2783f65 to ddba084 Compare September 20, 2024 09:05
# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant