Swap out patsy for formulae

[ksolarski]

Solving issue #386

Starting with DiD, will continue with other methods if you with general design @drbenvincent

Seems like the key practical difference between formulae and patsy is lack of build_design_matrices method in formulae. User has to then provide formula again.

---

📚 Documentation preview 📚: https://causalpy--463.org.readthedocs.build/en/463/

[drbenvincent]

Cool. Thanks @ksolarski, just a quick reply from my phone...

Don't do this for the synthetic control because I have an in progress PR that will change it. It won't have a formula input.

But can I just get some clarification... does this change the API? Can we get the exact same functionality? If not, let's think again.

Will try to look at the code properly when I can 👍🏻

[drbenvincent]

I can't find where I saw it in the patsy docs at this point. But I think one of the things that build_design_matrices did was to ensure that predictions on new/out of sample data are correct. For example, you could get a situation where you don't have all levels of a categorical variable in one predictor for out of sample data. So I think if you to it naively, you can get silent errors.

I'm not 100% sure that this is a problem, and apologies I can't find the relevant part in the docs. But does my concern make sense?

[ksolarski]

You're right, Patsy has the power of preserving the transformation / encoding of variables through build_design_matrices method. There's no equivalent way in formulae so it's certainly not straightforward to copy paste the current behaviour with formulae.

However, Patsy repo suggests migration to https://github.com/matthewwardrop/formulaic instead, which is capable of "reusing the encoding choices made during conversion of one data-set on other datasets." (see https://matthewwardrop.github.io/formulaic/latest/). There's also a migration guide from Patsy to Formulaic to switch would be easy. It also supports many operators: https://matthewwardrop.github.io/formulaic/latest/guides/grammar/

Did you check out this library before? What do you think about using this instead of formulae?

[ksolarski]

@drbenvincent any strong opinions about using formulaic instead of formulae package?

[drbenvincent]

Sorry for the delayed response @ksolarski. So as far as I understand, formulaic is considered the successor to patsy. Though it is only formulae which allows hierarchical modeling?

Right now there are no use-cases for hierarchical modelling. That might change in the future, though I don't have any specific use cases in mind.

So I guess the only choice at the moment is formulaic. But I'll just ask @tomicapretto if there's any plan on how it handles out of sample prediction when not all levels of categorical variables are in the out of sample data (see short discussion above).

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 94.66%. Comparing base (a39e015) to head (41a0236).
Report is 23 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #463      +/-   ##
==========================================
- Coverage   94.66%   94.66%   -0.01%     
==========================================
  Files          32       32              
  Lines        2195     2194       -1     
==========================================
- Hits         2078     2077       -1     
  Misses        117      117              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[ksolarski]

@drbenvincent Yes, from the docs it seems that no hierarchical models are allowed in the formulaic. I think it properly handles the transformation of categorical variables in out of sample data. Here's some small example that I generated:

import pandas as pd
from formulaic import model_matrix
import formulaic

# Create a training dataset
train_data = pd.DataFrame(
    {
        "feature1": ["A", "B", "C", "D"],
        "target": [0, 1, 0, 1],
    }
)

# Create a test dataset
test_data = pd.DataFrame(
    {
        "feature1": [
            "A",  # In training
            "D",  # In training
            "E",  # Not in training
        ],
        "target": [0, 1, 0],
    }
)

# Generate the model matrix for the training data
train_matrix = model_matrix("target ~ 0 + feature1", train_data)

# Print the training matrix and spec
print("Training Matrix:")
print(train_matrix)

# Use the same spec to transform the test data
test_matrix = model_matrix(spec=train_matrix.model_spec, data=test_data)

# Print the test matrix - see that columns are properly aligned from the training data transformation
print("\nTest Matrix:")
print(test_matrix)

Is that the problem you had in mind or something else?