Split "Experiment Details" out from feature gate info, add more experiment metadata #234
Conversation
| @@ -1,16 +1,27 @@ | |||
| class Decision | |||
| include ActiveModel::Model | |||
|
|
|||
| attr_accessor :split, :variant | |||
| attr_accessor :split, :variant, :takeaways | |||
There was a problem hiding this comment.
ah I totally forgot to include implemented as a required input for completed splits (being the variant name that was chose; typically just control or treatement, but sometimes something else for multivariant tests). Can we add that to the view?
There was a problem hiding this comment.
There is existing logic for determining an implemented variant by looking at the split's variant registry -- if decided_at is true, then whichever is at 100% was the implemented variant (and this shows up in the UI too).
I can find a way to surface that variant through to our internal systems too, using identical logic.
| super || split.registry.keys.find { |v| v == 'control' } | ||
| end | ||
|
|
||
| def self.available_tests |
There was a problem hiding this comment.
where will we actually enumerate these? (same with available_segments)
There was a problem hiding this comment.
Over in our internal release there's a Rails initializer that sets up all of the Betterment-specific tests & segments.
smudge
force-pushed
the
smudge/experiment-details
branch
from
71b923b to
cac85a7
Compare
Summary
This is the foundation for what may become a set of PRs that improve the experience for managing metadata for split experiments (& better distinguishing them from feature gates by way of a new UI section):
This section only appears for non-feature-gate splits, and enables users to add & manage metadata about the experiment.
None of this data does anything! It's merely intended as a convenience for users of TestTrack who wish to perform downstream analysis of splits, and want to have metadata colocated with the rest of their split data:
Bonus
This also adds takeaways to the split decision view, so that takeaways can be captured during the decision event:
Other Information
There are other changes I'd like to make to this (like, more thoughtful validations, and potentially more configurability) but this PR is meant to set the foundation.
This also ships with completely empty configs for
ExperimentDetail.available_testsExperimentDetail.available_segments-- these can be configured at boot time by dropping a file inconfig/initializersthat populates these hashes (name => description). Description is not currently used but I'd like to put it in a tooltip (future PR)./domain @Betterment/test_track_core
/platform @Betterment/test_track_core