Lapses on non-daily habits decrease the score too much

[iSoron]

Suppose that you have a habit that should be performed 3 times per week. In Loop Habit Tracker 1.8, if you consistently perform this habit 3 times per week, your score will eventually converge to 100%, as expected. However, if you only perform it 2 times per week, your score will eventually converge to around 28.6%. Given that you have successfully done 2/3 of the repetitions, the correct score should be 66% instead.

The issue happens because one failed repetition causes Loop not to generate 4 automatic checkmarks. The impact of a single lapse, therefore, is multiplied by 5. The final score becomes 28.6% because Loop considers your success rate to be 2/7. The same issue happens with other non-daily habits.

This issue was first identified by @KristianTashkov in #630 and #633. In these issues, a proposed solution is to allow the user to specify a fixed set of weekdays in which the habit cannot be performed. I unfortunately don't think this proposed solution addresses the root cause of the problem described here. In the example above, it may be the case that the user can perform the habit any day of the week. I am opening this issue for us to discuss possible alternative solutions.

[iSoron]

This has been fixed in 8b847ae by using a rolling sum instead of relying on implicit checkmarks. This solution seems to work well so far, but I will leave this issue open a while longer in case anyone has feedback.

[KristianTashkov]

I will try to answer more in detail when I have more free time but a couple of observations on the recent changes you introduced:

1. You have a bug where skip days can skip decreasing the rolling sum from previous days
2. It's interesting to me that you implemented exactly the behavior I was proposing in another issue while I was under the impression you didn't approve and ignored me. Talking about looking at a 2 week interval for irregular habits and allow the user to have variable repetitions throughout it. Keep in mind that people were against that in a previous thread, which is why I was proposing it as a habit option, not by default.
3. Another thing you fixed with this logic that I was complaining about, was double counting repetitions. With this change one repetition gives its value to a single "denominator" interval only (well technically half for 2 * denominator but still total of 1). Previous logic was allowing a repetition to be counted for previous week and for next week at the same time. I approve in general, but keep in mind this is a change from previous behavior.
4. Right now YES_AUTO is completely disjointed from scores and you have cases where there is a YES_AUTO check but score is decreasing or vice versa.

Overall this logic is a lot more inline with my understanding of how scores should work for irregular habits, but we need to think about YES_AUTO and streaks according to that logic as well.
Another thing to note is that while this is a step in the right direction and fixes the score problem of irregular days, visually we are still lacking. The proposal for active skip days is still relevant in mind, for multiple reasons, but I will try to formulate an opinion there later and write it out.

[iSoron]

@KristianTashkov wrote:

You have a bug where skip days can skip decreasing the rolling sum from previous days

Thanks, fixed in ddea9e7

Right now YES_AUTO is completely disjointed from scores and you have cases where there is a YES_AUTO check but score is decreasing or vice versa

I think reconciling the two will be hard. For example, in 3/7 habits, if the user consistently performs only 2/7, they will never see YES_AUTO checkmarks, regardless of how we compute them.

[KristianTashkov]

Thanks, fixed in ddea9e7

Actually I realized there is another problem with the new logic and skips. If you have to skip a large portion of days (vacation from work or injury making it impossible to work out for example), when you return and start doing the habit the first couple of days your score will decrease because your rolling sum with be 0. Instead of looking back denominator days to determine when rolling sum decrease, should we have a rolling list of denominator size entries. Skip days won't be added to the list so you will always have a rolling sum/list of the last denominator non-skipped entries. This way SKIP will be an actual pause and won't affect calculations.