A/B testing, feature usage statistics

[nicolas-raoul]

Today I learnt that the Wikipedia app does A/B testing:
https://www.mediawiki.org/wiki/Wikimedia_Apps/Team/Android/Recommended_Content_in_Search

Until now I thought this was too privacy-invasive for WMF.

We should check how they are doing it, and see whether our app could benefit from it.

Even getting the number of users who use custom picker vs. system picker, for instance, would be super useful.

[whym]

This ABTest.kt seems relevant. (And more generally, files under app/src/main/java/org/wikipedia/analytics in the Wikipedia app repository.)

It seems as though they measure granular data like clicks of buttons in the UI. wikimedia/apps-android-wikipedia#3234

I would assume the data collection/management/visualization side is using Wikimedia's general analytics infrastructure like https://wikitech.wikimedia.org/wiki/Event_Platform .

[nicolas-raoul]

Very insightful!

As a first A/B test to understand the mechanism, I would suggest showing either Contributions or Nearby as the home activity, and measuring which one usually leads to the highest number of uploads used in articles. Anything easier to implement is fine though.