Add agent rehearsal harness, findings, and per-run evidence for split-loop-with-lambda

Six scripted pi sub-agent runs (3 tool regimes × 2 models) rehearsed the
exercise; deterministic gate = solution test + approved file. Key results:
mandated CLI triples cost on the strong model (4:06 vs 0:39, PASS both);
weak model fails only under the mandate (A+Q), passes when free or optional.
Drift documented with code: scorer.updateSelection left inside the extracted
method (A+Q, C+Q). Harness in tools/rehearse; summary in
rehearsal-findings.md; per-run evidence in rehearsal-runs/; cheat sheet in
rehearse-with-agent.md corrected (lambda-typed parameters unsupported).
This commit is contained in:
Your Name
2026-09-27 17:36:19 +00:00
parent a27e32688a
commit b993f25c53
51 changed files with 2281 additions and 2 deletions
@@ -0,0 +1,35 @@
--- Run A ---
session file: /tmp/rehearsal/runs/A/session/2026-09-27T14-48-26-884Z_01a0e356-ba04-72d0-9e2e-57b0a6b3e25d.jsonl
model: z-ai/glm-5.3-flash
session span: 245.6 s
assistant messages: 27
tool calls: 27 total
bash 19
read 4
edit 4
tokens input: 109575
tokens output: 12572
tokens cacheRead: 288000
tokens cacheWrite: 0
tokens reasoning: 8571
tokens TOTAL (sum): 418718
--- /usr/bin/time -v (wall clock + max RSS) ---
Elapsed (wall clock) time (h:mm:ss or m:ss): 4:05.76
Maximum resident set size (kbytes): 262040
Exit status: 0
--- pi stdout (last 15 lines) ---
+ for (Instant interestingDate : interestingDates) {
+ var forecastDate = Instant.parse("2023-04-26T14:00:00.000-07:00");
+ long daysForward = ChronoUnit.DAYS.between(forecastDate, interestingDate);
+ Boolean lookupWeather = dayOffsetToWeather.apply(daysForward);
+ expectedWeather.add(lookupWeather);
+ }
+ return expectedWeather;
+ }
--- a/src/test/java/codingdojo/LongRangeForecastsTest.java (forced call-site update)
- var actual = new LongRangeForecasts().longRangeForecast(quarter);
+ var actual = new LongRangeForecasts().longRangeForecast();
```
The scorer dependency is now injectable: `longRangeForecast()` delegates with a lambda wrapping the `scorer` field, and tests can pass a stub `LongFunction<Boolean>` instead.