Add agent rehearsal harness, findings, and per-run evidence for split-loop-with-lambda
Six scripted pi sub-agent runs (3 tool regimes × 2 models) rehearsed the exercise; deterministic gate = solution test + approved file. Key results: mandated CLI triples cost on the strong model (4:06 vs 0:39, PASS both); weak model fails only under the mandate (A+Q), passes when free or optional. Drift documented with code: scorer.updateSelection left inside the extracted method (A+Q, C+Q). Harness in tools/rehearse; summary in rehearsal-findings.md; per-run evidence in rehearsal-runs/; cheat sheet in rehearse-with-agent.md corrected (lambda-typed parameters unsupported).
This commit is contained in:
Your Name
@@ -0,0 +1,31 @@
|
||||
# Rehearsal workspace — refactoring task
|
||||
|
||||
You are refactoring `src/main/java/codingdojo/LongRangeForecasts.java` in this
|
||||
Maven project. Run `mvn -q test` after each change (mvn is on your PATH). It may
|
||||
fail — that is expected: the dependency on `Scorer` makes the method untestable —
|
||||
but the code must still *compile*.
|
||||
|
||||
## Goal
|
||||
|
||||
In `LongRangeForecasts.longRangeForecast()`, the loop over `interestingDates`
|
||||
collects weather by calling `scorer` (an unwanted external dependency).
|
||||
|
||||
1. Introduce a lambda for the scorer-dependent code and move its definition to
|
||||
the top of the method, leaving the invocation where it is. (No refactoring
|
||||
tool supports this — do it as a minimal manual edit.)
|
||||
2. Extract the weather-collection loop into a new method `collectExpectedWeather`
|
||||
that takes the lambda as a parameter (suggested type `LongFunction<Boolean>`).
|
||||
3. Add the lambda as a parameter to `longRangeForecast` itself: the signature
|
||||
must become `longRangeForecast(LongFunction<Boolean> dayOffsetToWeather)`,
|
||||
with a public no-arg overload `longRangeForecast()` that delegates and wraps
|
||||
the `scorer` field in a lambda. Do not keep the `String quarter` parameter.
|
||||
(`java-refactor change-method-signature` or `java-refactor introduce-param`
|
||||
may help; fall back to a minimal manual edit if no subcommand can do it.)
|
||||
4. Stop. Report the final diff.
|
||||
|
||||
## Tool rules (variant A)
|
||||
You may only change code by invoking `java-refactor` CLI commands
|
||||
(`java-refactor --help` lists them; it is on your PATH). Never edit `.java`
|
||||
files with text edits — the ONLY exceptions are the introduce-lambda edit
|
||||
(step 1) and, if no `java-refactor` subcommand can accomplish it, a minimal
|
||||
manual edit for step 3.
|
||||
@@ -0,0 +1,92 @@
|
||||
Only in /tmp/rehearsal/runs/A/workspace: AGENTS.md
|
||||
diff -ru '--exclude=target' /tmp/rehearsal/pristine/src/main/java/codingdojo/LongRangeForecasts.java /tmp/rehearsal/runs/A/workspace/src/main/java/codingdojo/LongRangeForecasts.java
|
||||
--- /tmp/rehearsal/pristine/src/main/java/codingdojo/LongRangeForecasts.java 2026-09-27 14:39:01.304467310 +0000
|
||||
+++ /tmp/rehearsal/runs/A/workspace/src/main/java/codingdojo/LongRangeForecasts.java 2026-09-27 14:52:09.626887233 +0000
|
||||
@@ -3,36 +3,25 @@
|
||||
import java.time.Instant;
|
||||
import java.time.temporal.ChronoUnit;
|
||||
import java.util.*;
|
||||
+import java.util.function.LongFunction;
|
||||
|
||||
public class LongRangeForecasts {
|
||||
|
||||
private final Scorer scorer = new Scorer();
|
||||
|
||||
- public Map<IceCream, Integer> longRangeForecast(String quarter) {
|
||||
+ public Map<IceCream, Integer> longRangeForecast() {
|
||||
+ return longRangeForecast(daysForward -> {
|
||||
+ scorer.updateSelection();
|
||||
+ return scorer.lookupWeather(daysForward);
|
||||
+ });
|
||||
+ }
|
||||
|
||||
+ public Map<IceCream, Integer> longRangeForecast(LongFunction<Boolean> dayOffsetToWeather) {
|
||||
var result = new HashMap<IceCream, Integer>();
|
||||
- if (Objects.equals(quarter, "Q1") || Objects.equals(quarter, "Q4")) {
|
||||
- for (IceCream flavour : IceCream.values()) {
|
||||
- result.put(flavour, 5);
|
||||
- }
|
||||
- }
|
||||
for (IceCream flavour : IceCream.values()) {
|
||||
result.put(flavour, 0);
|
||||
}
|
||||
- List<Instant> interestingDates = List.of(
|
||||
- Instant.parse("2023-05-01T14:00:00.000-07:00"),
|
||||
- Instant.parse("2023-05-18T14:00:00.000-07:00"),
|
||||
- Instant.parse("2023-06-05T14:00:00.000-07:00"),
|
||||
- Instant.parse("2023-06-23T14:00:00.000-07:00")
|
||||
- );
|
||||
- List<Boolean> expectedWeather = new ArrayList<>();
|
||||
- for (Instant interestingDate : interestingDates) {
|
||||
- scorer.updateSelection();
|
||||
- var forecastDate = Instant.parse("2023-04-26T14:00:00.000-07:00");
|
||||
- long daysForward = ChronoUnit.DAYS.between(forecastDate, interestingDate);
|
||||
- Boolean lookupWeather = scorer.lookupWeather(daysForward);
|
||||
- expectedWeather.add(lookupWeather);
|
||||
- }
|
||||
+ List<Boolean> expectedWeather = collectExpectedWeather(dayOffsetToWeather);
|
||||
|
||||
for (IceCream flavour : result.keySet()) {
|
||||
result.put(flavour, result.get(flavour) + 10);
|
||||
@@ -46,4 +35,21 @@
|
||||
}
|
||||
return result;
|
||||
}
|
||||
+
|
||||
+ private List<Boolean> collectExpectedWeather(LongFunction<Boolean> dayOffsetToWeather) {
|
||||
+ List<Instant> interestingDates = List.of(
|
||||
+ Instant.parse("2023-05-01T14:00:00.000-07:00"),
|
||||
+ Instant.parse("2023-05-18T14:00:00.000-07:00"),
|
||||
+ Instant.parse("2023-06-05T14:00:00.000-07:00"),
|
||||
+ Instant.parse("2023-06-23T14:00:00.000-07:00")
|
||||
+ );
|
||||
+ List<Boolean> expectedWeather = new ArrayList<>();
|
||||
+ for (Instant interestingDate : interestingDates) {
|
||||
+ var forecastDate = Instant.parse("2023-04-26T14:00:00.000-07:00");
|
||||
+ long daysForward = ChronoUnit.DAYS.between(forecastDate, interestingDate);
|
||||
+ Boolean lookupWeather = dayOffsetToWeather.apply(daysForward);
|
||||
+ expectedWeather.add(lookupWeather);
|
||||
+ }
|
||||
+ return expectedWeather;
|
||||
+ }
|
||||
}
|
||||
diff -ru '--exclude=target' /tmp/rehearsal/pristine/src/test/java/codingdojo/LongRangeForecastsTest.java /tmp/rehearsal/runs/A/workspace/src/test/java/codingdojo/LongRangeForecastsTest.java
|
||||
--- /tmp/rehearsal/pristine/src/test/java/codingdojo/LongRangeForecastsTest.java 2026-09-27 14:39:01.304467310 +0000
|
||||
+++ /tmp/rehearsal/runs/A/workspace/src/test/java/codingdojo/LongRangeForecastsTest.java 2026-09-27 14:51:53.937183804 +0000
|
||||
@@ -14,7 +14,7 @@
|
||||
@Test
|
||||
void longRangeForecast() {
|
||||
var quarter = "Q3";
|
||||
- var actual = new LongRangeForecasts().longRangeForecast(quarter);
|
||||
+ var actual = new LongRangeForecasts().longRangeForecast();
|
||||
StringBuilder printedResult = printIceCreamForecast(actual);
|
||||
|
||||
Approvals.verify(printedResult);
|
||||
diff -ru '--exclude=target' /tmp/rehearsal/pristine/src/test/java/codingdojo/LongRangeForecastsTest.longRangeForecast.received.txt /tmp/rehearsal/runs/A/workspace/src/test/java/codingdojo/LongRangeForecastsTest.longRangeForecast.received.txt
|
||||
--- /tmp/rehearsal/pristine/src/test/java/codingdojo/LongRangeForecastsTest.longRangeForecast.received.txt 2026-09-27 14:39:03.582411308 +0000
|
||||
+++ /tmp/rehearsal/runs/A/workspace/src/test/java/codingdojo/LongRangeForecastsTest.longRangeForecast.received.txt 2026-09-27 14:52:12.646830236 +0000
|
||||
@@ -1 +1 @@
|
||||
-{Strawberry:12, Chocolate:12, Vanilla:12, }
|
||||
\ No newline at end of file
|
||||
+{Strawberry:10, Chocolate:10, Vanilla:10, }
|
||||
\ No newline at end of file
|
||||
@@ -0,0 +1,35 @@
|
||||
--- Run A ---
|
||||
session file: /tmp/rehearsal/runs/A/session/2026-09-27T14-48-26-884Z_01a0e356-ba04-72d0-9e2e-57b0a6b3e25d.jsonl
|
||||
model: z-ai/glm-5.3-flash
|
||||
session span: 245.6 s
|
||||
assistant messages: 27
|
||||
tool calls: 27 total
|
||||
bash 19
|
||||
read 4
|
||||
edit 4
|
||||
tokens input: 109575
|
||||
tokens output: 12572
|
||||
tokens cacheRead: 288000
|
||||
tokens cacheWrite: 0
|
||||
tokens reasoning: 8571
|
||||
tokens TOTAL (sum): 418718
|
||||
--- /usr/bin/time -v (wall clock + max RSS) ---
|
||||
Elapsed (wall clock) time (h:mm:ss or m:ss): 4:05.76
|
||||
Maximum resident set size (kbytes): 262040
|
||||
Exit status: 0
|
||||
--- pi stdout (last 15 lines) ---
|
||||
+ for (Instant interestingDate : interestingDates) {
|
||||
+ var forecastDate = Instant.parse("2023-04-26T14:00:00.000-07:00");
|
||||
+ long daysForward = ChronoUnit.DAYS.between(forecastDate, interestingDate);
|
||||
+ Boolean lookupWeather = dayOffsetToWeather.apply(daysForward);
|
||||
+ expectedWeather.add(lookupWeather);
|
||||
+ }
|
||||
+ return expectedWeather;
|
||||
+ }
|
||||
|
||||
--- a/src/test/java/codingdojo/LongRangeForecastsTest.java (forced call-site update)
|
||||
- var actual = new LongRangeForecasts().longRangeForecast(quarter);
|
||||
+ var actual = new LongRangeForecasts().longRangeForecast();
|
||||
```
|
||||
|
||||
The scorer dependency is now injectable: `longRangeForecast()` delegates with a lambda wrapping the `scorer` field, and tests can pass a stub `LongFunction<Boolean>` instead.
|
||||
@@ -0,0 +1,91 @@
|
||||
Done — stopping here as instructed. Summary of how each step was performed:
|
||||
|
||||
## Steps taken
|
||||
|
||||
1. **Introduce lambda (manual edit, the allowed exception):** Moved the scorer-dependent code (`scorer.updateSelection()` / `scorer.lookupWeather(...)`) into a `LongFunction<Boolean> dayOffsetToWeather` lambda defined at the top of the method; the loop now calls `dayOffsetToWeather.apply(daysForward)` at the original invocation point. ✅ compiles, same 2 pre-existing test failures.
|
||||
|
||||
2. **Extract method (via `java-refactor extract`):** Selected the `interestingDates`/`expectedWeather` declarations + loop; produced exactly `private List<Boolean> collectExpectedWeather(LongFunction<Boolean> dayOffsetToWeather)`. ✅ same test result.
|
||||
|
||||
3. **Signature change (manual fallback):** No `java-refactor` subcommand could do it — I verified with dry-runs:
|
||||
- `decompose-conditional` generated broken code (out-of-scope `quarter`)
|
||||
- `remove-param` refuses: parameter still referenced
|
||||
- `introduce-param` refuses: "Selection is already a simple name"
|
||||
- `extract` generated invalid `var` parameter types; `change-method-signature` can't add/remove parameters
|
||||
|
||||
So per AGENTS.md I made a minimal manual edit: signature → `longRangeForecast(LongFunction<Boolean> dayOffsetToWeather)`, added public no-arg overload wrapping the `scorer` field in a lambda, dropped the `String quarter` parameter (and the dead `if` block that used it — its writes were unconditionally overwritten by the next loop anyway), and updated the test's call site to the no-arg overload (forced by the signature change). ✅ compiles; only the 2 baseline failures remain.
|
||||
|
||||
## Final diff
|
||||
|
||||
```diff
|
||||
--- a/src/main/java/codingdojo/LongRangeForecasts.java
|
||||
+++ b/src/main/java/codingdojo/LongRangeForecasts.java
|
||||
import java.time.Instant;
|
||||
import java.time.temporal.ChronoUnit;
|
||||
import java.util.*;
|
||||
+import java.util.function.LongFunction;
|
||||
|
||||
private final Scorer scorer = new Scorer();
|
||||
|
||||
- public Map<IceCream, Integer> longRangeForecast(String quarter) {
|
||||
+ public Map<IceCream, Integer> longRangeForecast() {
|
||||
+ return longRangeForecast(daysForward -> {
|
||||
+ scorer.updateSelection();
|
||||
+ return scorer.lookupWeather(daysForward);
|
||||
+ });
|
||||
+ }
|
||||
+
|
||||
+ public Map<IceCream, Integer> longRangeForecast(LongFunction<Boolean> dayOffsetToWeather) {
|
||||
var result = new HashMap<IceCream, Integer>();
|
||||
- if (Objects.equals(quarter, "Q1") || Objects.equals(quarter, "Q4")) {
|
||||
- for (IceCream flavour : IceCream.values()) {
|
||||
- result.put(flavour, 5);
|
||||
- }
|
||||
- }
|
||||
for (IceCream flavour : IceCream.values()) {
|
||||
result.put(flavour, 0);
|
||||
}
|
||||
- List<Instant> interestingDates = List.of(
|
||||
- Instant.parse("2023-05-01T14:00:00.000-07:00"),
|
||||
- Instant.parse("2023-05-18T14:00:00.000-07:00"),
|
||||
- Instant.parse("2023-06-05T14:00:00.000-07:00"),
|
||||
- Instant.parse("2023-06-23T14:00:00.000-07:00")
|
||||
- );
|
||||
- List<Boolean> expectedWeather = new ArrayList<>();
|
||||
- for (Instant interestingDate : interestingDates) {
|
||||
- scorer.updateSelection();
|
||||
- var forecastDate = Instant.parse("2023-04-26T14:00:00.000-07:00");
|
||||
- long daysForward = ChronoUnit.DAYS.between(forecastDate, interestingDate);
|
||||
- Boolean lookupWeather = scorer.lookupWeather(daysForward);
|
||||
- expectedWeather.add(lookupWeather);
|
||||
- }
|
||||
+ List<Boolean> expectedWeather = collectExpectedWeather(dayOffsetToWeather);
|
||||
|
||||
for (IceCream flavour : result.keySet()) {
|
||||
...
|
||||
}
|
||||
return result;
|
||||
}
|
||||
+
|
||||
+ private List<Boolean> collectExpectedWeather(LongFunction<Boolean> dayOffsetToWeather) {
|
||||
+ List<Instant> interestingDates = List.of(
|
||||
+ Instant.parse("2023-05-01T14:00:00.000-07:00"),
|
||||
+ Instant.parse("2023-05-18T14:00:00.000-07:00"),
|
||||
+ Instant.parse("2023-06-05T14:00:00.000-07:00"),
|
||||
+ Instant.parse("2023-06-23T14:00:00.000-07:00")
|
||||
+ );
|
||||
+ List<Boolean> expectedWeather = new ArrayList<>();
|
||||
+ for (Instant interestingDate : interestingDates) {
|
||||
+ var forecastDate = Instant.parse("2023-04-26T14:00:00.000-07:00");
|
||||
+ long daysForward = ChronoUnit.DAYS.between(forecastDate, interestingDate);
|
||||
+ Boolean lookupWeather = dayOffsetToWeather.apply(daysForward);
|
||||
+ expectedWeather.add(lookupWeather);
|
||||
+ }
|
||||
+ return expectedWeather;
|
||||
+ }
|
||||
|
||||
--- a/src/test/java/codingdojo/LongRangeForecastsTest.java (forced call-site update)
|
||||
- var actual = new LongRangeForecasts().longRangeForecast(quarter);
|
||||
+ var actual = new LongRangeForecasts().longRangeForecast();
|
||||
```
|
||||
|
||||
The scorer dependency is now injectable: `longRangeForecast()` delegates with a lambda wrapping the `scorer` field, and tests can pass a stub `LongFunction<Boolean>` instead.
|
||||
@@ -0,0 +1 @@
|
||||
Read AGENTS.md in this directory and execute the refactoring task it describes. Run mvn -q test after each change. When done, stop and report the final diff of LongRangeForecasts.java.
|
||||
@@ -0,0 +1,23 @@
|
||||
Command being timed: "timeout -k 15 600 pi -p -a --session-dir /tmp/rehearsal/runs/A/session --name rehearsal-A Read AGENTS.md in this directory and execute the refactoring task it describes. Run mvn -q test after each change. When done, stop and report the final diff of LongRangeForecasts.java."
|
||||
User time (seconds): 43.49
|
||||
System time (seconds): 3.19
|
||||
Percent of CPU this job got: 18%
|
||||
Elapsed (wall clock) time (h:mm:ss or m:ss): 4:05.76
|
||||
Average shared text size (kbytes): 0
|
||||
Average unshared data size (kbytes): 0
|
||||
Average stack size (kbytes): 0
|
||||
Average total size (kbytes): 0
|
||||
Maximum resident set size (kbytes): 262040
|
||||
Average resident set size (kbytes): 0
|
||||
Major (requiring I/O) page faults: 0
|
||||
Minor (reclaiming a frame) page faults: 1012660
|
||||
Voluntary context switches: 94163
|
||||
Involuntary context switches: 2559
|
||||
Swaps: 0
|
||||
File system inputs: 0
|
||||
File system outputs: 3992
|
||||
Socket messages sent: 0
|
||||
Socket messages received: 0
|
||||
Signals delivered: 0
|
||||
Page size (bytes): 4096
|
||||
Exit status: 0
|
||||
@@ -0,0 +1,10 @@
|
||||
--- Run A ---
|
||||
final diff vs pristine: 49 changed lines -> /tmp/rehearsal/runs/A/diff.patch
|
||||
mvn test with solution's test + approved file:
|
||||
RESULT: PASS (deterministic)
|
||||
structural checks:
|
||||
[x] method collectExpectedWeather exists
|
||||
[x] takes LongFunction<Boolean> parameter
|
||||
[i] note: 'scorer' still referenced in LongRangeForecasts.java
|
||||
[x] lambda syntax present
|
||||
[i] main class differs from reference solution (may still be a valid variant)
|
||||
Reference in New Issue
Block a user