Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Fix causal impact calculation error in synthetic control and interrupted time series #296

Merged
merged 3 commits into from
Mar 4, 2024

Conversation

drbenvincent
Copy link
Collaborator

@drbenvincent drbenvincent commented Mar 1, 2024

  • There was an error in the causal impact calculations which affected Interrupted Time Series and Synthetic control. The causal impact was being calculated using the expectation of the posterior predictive distribution. But it should have been based on the actual posterior predictive values which have additional likelihood-related observation noise. Therefore, previous estimates of causal impact would have slightly underestimated the uncertainty.
  • I also re-ran the affected notebooks
  • Closes Ensure causal impact is calculated from posterior predicted observations, not expected values #294

Copy link

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.


Powered by ReviewNB

@drbenvincent drbenvincent requested a review from juanitorduz March 1, 2024 16:48
@drbenvincent drbenvincent assigned NathanielF and unassigned NathanielF Mar 1, 2024
@drbenvincent drbenvincent requested a review from NathanielF March 1, 2024 16:49
@drbenvincent drbenvincent added the bug Something isn't working label Mar 1, 2024
@drbenvincent drbenvincent requested a review from jpreszler March 1, 2024 16:49
Copy link

codecov bot commented Mar 1, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 77.06%. Comparing base (d5e493c) to head (7541516).

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #296   +/-   ##
=======================================
  Coverage   77.06%   77.06%           
=======================================
  Files          21       21           
  Lines        1378     1378           
=======================================
  Hits         1062     1062           
  Misses        316      316           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@drbenvincent
Copy link
Collaborator Author

I'll cut a point release when this is merged.

Copy link
Collaborator

@juanitorduz juanitorduz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch!

@drbenvincent drbenvincent merged commit 7c621cc into main Mar 4, 2024
10 checks passed
# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Ensure causal impact is calculated from posterior predicted observations, not expected values
3 participants