Preserve observed progression steps in exercise forecasts

docs/forecasting.md

@@ -51,10 +51,16 @@ evidence conservative. A stall means failing to improve the rep best at the
 current load, including drops/recoveries rather than counting every rebound as
 new progress.
 
-The usual pre-increase reps and an actually observed load increment are inferred
+The usual pre-increase reps and most common load increment are inferred
-from the recent 24 sessions. Load increases are allowed at the inferred threshold;
+from the recent 24 sessions. The model preserves the observed mix of load
+increments from its transition window rather than replacing every increase with
+the most common one. Jumps over 15% of their original source load are excluded
+from progression evidence. Load increases are allowed at the inferred threshold;
 their reset reps come from actual transitions. No fixed 8–12 prescription is
 hardcoded. If there are no observed increases, the model cannot invent them.
+Rep-building transitions are excluded at the ceiling, so clipping a rep gain
+cannot turn it into artificial evidence of a stall. Actual repeats and the
+explicit repeat-set prior remain in the model.
 
 The next set is the most likely **joint** outcome. For longer horizons, 512
 deterministically seeded trajectories produce scenario sets at the median load
@@ -87,15 +93,15 @@ results, not independent prospective validation or a guarantee for every exercis
 
 | Horizon | Holdouts | Model | Load MAE (kg) | Rep MAE | E1RM MAE (kg) |
 | --- | ---: | --- | ---: | ---: | ---: |
-| Next session | 1,929 | Learned transitions | 0.312 | 0.692 | 1.893 |
+| Next session | 1,929 | Learned transitions | 0.272 | 0.687 | 1.863 |
 | Next session | 1,929 | Repeat last set | 0.506 | 0.800 | 2.395 |
 | Next session | 1,929 | Damped trend | 0.374 | 0.800 | 2.133 |
 | Next session | 1,929 | Linear trend | 0.422 | 0.800 | 2.118 |
-| 4 sessions | 1,636 | Learned transitions | 1.022 | 1.084 | 3.243 |
+| 4 sessions | 1,636 | Learned transitions | 0.953 | 1.030 | 3.112 |
 | 4 sessions | 1,636 | Repeat last set | 1.884 | 1.145 | 4.159 |
 | 4 sessions | 1,636 | Damped trend | 1.074 | 1.145 | 3.436 |
 | 4 sessions | 1,636 | Linear trend | 1.201 | 1.145 | 3.581 |
-| 8 sessions | 1,308 | Learned transitions | 1.782 | 1.174 | 4.114 |
+| 8 sessions | 1,308 | Learned transitions | 1.716 | 1.173 | 4.151 |
 | 8 sessions | 1,308 | Repeat last set | 3.545 | 1.158 | 5.906 |
 | 8 sessions | 1,308 | Damped trend | 1.903 | 1.158 | 4.518 |
 | 8 sessions | 1,308 | Linear trend | 2.056 | 1.158 | 4.845 |
@@ -104,6 +110,21 @@ The learned model improves aggregate load and E1RM errors at all tested horizons
 At eight sessions it does **not** beat persistence on reps; difficult sessions
 remain unpredictable. The UI includes exercise-specific recent error metrics.
 
+After correcting the collapsed load increments and artificial ceiling repeats,
+next-session and four-session errors improved over the original transition model.
+Eight-session load error improved from 1.782 to 1.716 kg, but E1RM error rose
+slightly from 4.114 to 4.151 kg. This adjustment is not an accuracy gain at every
+horizon. All 23 unit tests pass, including mixed increments, ceiling transitions,
+outlier increases and unchanged plateau behaviour.
+
+Peter's Stiff Leg Deadlift (TNG) eight-week scenario changes from 181 kg × 7 to
+183 kg × 6, and its 26-week scenario from 183.5 kg × 7 to 188 kg × 6. These are
+joint load/rep outcomes: a higher load does not necessarily mean a higher E1RM
+at every horizon. DB Bulgarian Split Squat's 26-week historical scenario changes
+from 39 kg × 10 to 39.5 kg × 11; its history ends on 11 January 2024. The current
+Smith machine Bulgarian split squat has only two sessions and remains withheld
+until four valid sessions are available.
+
 For 30 degrees Smith Press, the latest block has 39 sessions. The current load
 block contains seven sessions at 93.5 kg, progressing 4→5→6→6→6→6→7 reps.
 Three sessions did not improve the rep best. The inferred threshold is nine reps

forecasting.py

@@ -134,11 +134,19 @@ class TransitionModel:
         probabilities = np.exp(-distance) * 0.5 ** (self.age / 8)
         increase = self.dw > 0.01
         decrease = self.dw < -0.01
+        # Preserve the observed mix of increments; the mode alone can turn
+        # alternating small/large plate jumps into permanently tiny steps.
+        ordinary_increase = self.dw <= self.source_w * 0.15
+        probabilities[:, increase & ~ordinary_increase] = 0
         probabilities[:, increase] *= (reps >= self.profile['ceiling'])[:, None]
+        # A rep-building transition is not evidence of stalling at the ceiling.
+        # Exclude it there instead of clipping its gain into an artificial repeat.
+        rep_gain = (abs(self.dw) < 0.01) & (self.dr > 0)
+        probabilities[:, rep_gain] *= (reps < self.profile['ceiling'])[:, None]
         # Calendar horizons beyond ~12 sessions are scenarios. Damp the rate of
         # further load advances rather than extrapolating beginner gains forever.
         probabilities[:, increase] *= 0.98 ** max(0, step - 12)
-        delta_w = np.where(increase, self.profile['increment'], self.dw)
+        delta_w = self.dw
         delta_w = np.maximum(delta_w, -weights[:, None] * 0.15)
         out_w = np.maximum(weights[:, None] + delta_w, 0.5)
         out_r = np.clip(reps[:, None] + self.dr, 1, 20)

templates/partials/exercise_history.html

@@ -45,7 +45,7 @@
             </div>
             <p>At the current load: {{ forecast.profile.block_sessions }} sessions, from {{ forecast.profile.block_start_reps }} to a best of {{ forecast.profile.block_best_reps }} reps. {{ forecast.profile.block_stalls }} sessions did not improve that rep best.</p>
             {% if forecast.profile.increment and forecast.profile.ceiling_evidence >= 2 %}
-            <p>Your recent load increases usually follow {{ forecast.profile.ceiling }} reps, using {{ '%g'|format(forecast.profile.increment) }} kg steps. The model learns repeats, rep gains, difficult sessions and rep resets from this exercise.</p>
+            <p>Your recent load increases usually follow {{ forecast.profile.ceiling }} reps. The most common step is {{ '%g'|format(forecast.profile.increment) }} kg; forecasts use the recorded mix of steps. The model learns repeats, rep gains, difficult sessions and rep resets from this exercise.</p>
             {% else %}<p>There is not enough repeated loading history to infer a reliable rep threshold. Repeated sets remain a possible outcome.</p>{% endif %}
             {% if forecast.validation %}
             {% set validation = forecast.validation.transitions %}

templates/partials/sparkline.html

@@ -110,7 +110,7 @@ M{% for value, position in lower_points %}{{ (((position * vb_width) + margin) |
         </div>
         <div class="forecast-caption forecast-assumption">
             {% if forecast.profile.increment and forecast.profile.ceiling_evidence >= 2 %}
-            Your pattern: build toward {{ forecast.profile.ceiling }} reps, then +{{ '%g'|format(forecast.profile.increment) }} kg.
+            Your pattern: build toward {{ forecast.profile.ceiling }} reps; most common increase +{{ '%g'|format(forecast.profile.increment) }} kg.
             {% else %}
             Learning from your recent repeats, rep changes and load changes.
             {% endif %}

tests/test_forecasting.py

@@ -47,6 +47,27 @@ class ForecastTests(unittest.TestCase):
         f=forecast(rows([(50,8)]*16),52)
         self.assertTrue(all(p['projected_weight']==50 and p['projected_repetitions']==8 for p in f['points']))
 
+    def test_mixed_observed_load_steps_are_preserved(self):
+        m = TransitionModel(rows([(100,7),(100.5,7),(101.5,7),(102,7),(104,7)]))
+        w, r, p = m.outcomes([104], [7], [0])
+        increases = set(np.round(w[(w > 104) & (p > 0)] - 104, 3))
+        self.assertEqual(increases, {0.5, 1.0, 2.0})
+
+    def test_ceiling_does_not_turn_rep_gains_into_stall_evidence(self):
+        m = TransitionModel(rows([(100,6),(100,7),(101,6),(101,7),(102,6),(102,7)]))
+        w, r, p = m.outcomes([102], [7], [0])
+        # At the ceiling only the explicit prior can repeat this set;
+        # the observed rep-building transitions contribute no probability.
+        self.assertAlmostEqual(float(p[(w == 102) & (r == 7)].sum()), float(p[0,0]))
+        self.assertTrue(np.any((w > 102) & (p > 0)))
+        w, r, p = m.outcomes([102], [6], [0])
+        self.assertTrue(np.any((w == 102) & (r == 7) & (p > 0)))
+
+    def test_large_load_jump_is_not_reused_as_routine_progression(self):
+        m = TransitionModel(rows([(50,7),(50.5,7),(51,7),(100,7)]))
+        w, r, p = m.outcomes([100], [7], [0])
+        self.assertEqual(set(w[(w > 100) & (p > 0)]), {100.5})
+
     def test_months_off_excludes_old_performance(self):
         older=rows([(150,10)]*12,start=date(2024,1,1))
         fresh=rows([(50,6),(50,7),(50,7),(50,8)], start=date(2026,1,1))