Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

#

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Update rollouts-and-experiments.mdx #624

Merged
merged 1 commit into from
Aug 13, 2024
Merged

Update rollouts-and-experiments.mdx #624

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 12 additions & 10 deletions docs/faq/rollouts/rollouts-and-experiments.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,24 +18,26 @@ You can run a 100% rollout and a 5% experiment and both will fill up, but everyo

----

### Question
If there is a concurrent experiment and rollout on the same featureId, once the experiment ends, assuming the rollout is at 100%, a given client will immediately be put into the rollout. Correct?
### Question: What happens when an experiment and rollout are on the same feature, and the experiment ends?

### Answer
The short answer is **“yes”**, in most situations.

The long answer is: while both are running, an experiment client with actually be enrolled in both and the treatment they’ll receive will be the combination of both. This enables clients in subsequent experiments to easily get rolled out winners of prior experiments.
Typically users will see the experiment applied, and then get the rollout treatment when that ends with a small delay to re-evaluate.

If both the experiment and the rollout are focused on the same fields of the treatment JSON, **the experiment will take precedence** and in this case the behavior will be as you state: effectively, the client immediately joins the rollout.
If both the experiment and the rollout are focused on the same feature, **the experiment will take precedence** and in this case the behavior will be seeing the experiment, when that ends the client will unenroll, then upon next recipe evaluation will join the rollout.

If they’re focused on different fields of the JSON, the behavior the clients gets will go from “experiment+rollout” to “rollout only”.
UNLESS the feature is mobile messaging, in that case while both are running, an experiment client with actually be enrolled in both and the treatment they’ll receive will be the combination of both. This enables clients in subsequent experiments to easily get rolled out winners of prior experiments. If they’re focused on different fields of the JSON, the behavior the clients gets will go from “experiment+rollout” to “rollout only”.

----

### Question
I have a feature that is now on by default in v115, and I want to wait until it has saturated the user base before shutting off the rollout. Do we have a standard around how long we should keep a rollout going in a situation like this?
### Question - How do I decide when to end a rollout?

ex: I have a feature that is now on by default in v115, and I want to wait until it has saturated the user base before shutting off the rollout. Do we have a standard around how long we should keep a rollout going in a situation like this?

### Answer
We typically wait 1-3 releases for the upgrade tail to catch up. You'll probably have the vast majority, so it just depends on the ramifications if people have it enabled on 114 and then lose it.

If it's **not** a big deal, turn off the rollout in 116 and most people will have it and the stragglers will get it when they upgrade. If it **is** a big deal - turn off in 117 or 118 and you'll have reached a few percent more with each release.
If it's **not** a big deal if the feature goes away, turn off the rollout in the next release. People who haven't yet upgraded will lose the feature until they upgrade.

If it **is** a big deal for the feature to go away (think UX changes or bug fixes) - turn off in 117 or 118 and you'll have reached a few percent more with each release.

If it is a very big deal - we sometimes leave those running for 6 months or so to hit maximum reach. Please remember to set a reminder appointment to people on your team so someone remembers and doesn't leave it on forever.
Loading