discrete choice modeling blogpost

[drbenvincent]

This PR adds a notebooks which will form the second Colgate client write-up blog posts.

The first post was Causal sales analytics: Are my sales incremental or cannibalistic?

NOTE: I'll be pretty aggressive about hiding most of the code cells in the final blogpost in order to maximise readability.

Current state: At this point (2024/10/31) I've basically written the first half of the blog post. It outlines the basic discrete choice model and sets up the core limitation of producing uninteresting cannibalization effects.

TODO

• We might want to play with the random seed to get the synthetic data nice
• We might also want to tweak the synthetic price data to allow for better parameter identiability
• Potentially add a manufacturer (or benefit) effect to really show the lack of interesting cannibalization effects.
• I'm hoping that either @ricardoV94 or @lucianopaz or @cluhmann will take over the reigns and continue the blog post to talk about the core innovations of what we did. We are allowed to talk about the maths of the nested logit, but we're not allowed to present code to implement it.
• Hoping someone can write a nice overview of the cool new stuff that was done. I'll then come back in and wrap it up with the executive summary at the start and a conclusion summary at the end.

[review-notebook-app]

Check out this pull request on 

See visual diffs & provide feedback on Jupyter Notebooks.

---

Powered by ReviewNB

[ricardoV94]

@drbenvincent I'm leaving some comments from your part of the blogpost, before I start on the second part:

1. A single intercept doesn't make sense. I assume you meant B_0_^i (one intercept per item), as you have in the model description below?
2. In the model description the line u_{i,t} = a = b, dosen't make math sense. You introduce a log of the price, which is clearly not equivalent to the original expression on price. Also the brackets are strange. You are not multiplying the intercept by the price, I assume?
3. Multinomial model is very confident because total_sales is high. We obviously had more noise in our data, and that's why we had to switch the the DirichletMultinomial model instead, as the Multinomial cannot expalin so large errors with these high total sales. May be worth mentioning?
4. "So this is all great, but it's the kind of output that data scientists would enjoy." Is there irony in this sentence or missing a "not"?
5. What-if scenario. Needs a bit more text explaining what results we can see from the 5 plots?
6. I don't like the plot showing the market share before and after as the distance from the x=y line. This will never show anything interesting unless you remove an item that has a sizeable portion of the market-share (which you would never do anyway). It's also mostly wasted white space on the plot. I would rather show the ratio of market share before or after as a plot-bar, which will clearly show everything going up by the same %. Conversely: imagine you remove an item with a 1% market share and another item takes all this market share (very interesting perfect cannibalization), it would go from x to x+1%, which would still look super boring on the plot you defined. The plot is not good to show what you want.
7. Prior for intercept should be zerosum, otherwise there's one too many parameters.

Obligatory message: I think overall the blog is in a pretty nice shape!

[ricardoV94]

I'm going to push a second NB that uses pre-generated data according to the NLM. I think this will streamline the blogpost, showing where it fails and why the NLM can address it. Not changing the original NB so we can compare, because git changes suck for NBs

[ricardoV94]

@drbenvincent I pushed an updated nb. I did the following changes:

1. Replace data generation on the NB by a pre-designed dataset (also uploaded), according to a NLM with two levels (segment -> manufacturer). I tried to use red/blue tones to distinguish the segment. We may want to change it from adults vs kids to something more "competitive" like you had before, if it seems unreasonable that cross-segment cannibalization would be observed. I chose kids for the motivation part for the NLM I wrote. It felt easier with that as an example.
2. I then define a one-level NLM with segment only, and write a naive code implementation, hard-codde for this case. The idea here is to be just a teaser into the full fledged code that we'll put on pymc-marketing.
3. The simulated data with two levels can't be fit perfectly with the one level NLM. I didn't want the model to look too good, and this can then be suggested as a next step we would take if we wanted to improve the model.

Other stuff:

1. ZeroSumNormal on the intercepts
2. Use standardized log-price to avoid the intercept-slop indeterminacy (sampling is much faster)
3. Remove the diagonal plot and show the relative change in market share at the last day. This shows it is a single degree of change for the simple model, but two degrees of change for the nested model. Adding a second level would make 3 degrees of change, but we actually don't have enough items on the market by that time to show it, so the plot would look the same.

Questions:

1. Not sure about the ellipsis plot now that we don't have "true" data to reference. They also don't look pretty, but maybe I messed them up.

TODO:

1. Write something about the math of the model? How deep do we want to go? Perhaps (perhaps @lucianopaz is the best person for it?)
2. Cleanup the relative change plot to have items on the xticks
3. Rerun the NB from top to bottom
4. Add more text on the fittings/predictions/counterfactuals of the NLM.

To run the NB you will need to comment out compile_kwargs=dict(mode="NUMBA")) from pm.sample. That will only be available in the next release of PyMC. I needed it for the model to actual sample at a reasonable pace on my machine.