draft for pi skill

app/priv/blog/engineering/2026/07-16-at-coding-agent-skill-to-guide-editing-in-place.md

@@ -0,0 +1,225 @@
+%{
+  title: "At coding agent skill to guide editing in-place",
+  author: "Willem van den Ende",
+  tags: ~w(AI coding-agent baby-steps),
+  description: "With an @<agent> instruction in a file, you can use your regular coding agent on small parts of your code, without modifiying your editor / IDE.",
+  published: true
+}
+---
+
+TLDR
+===
+
+I made an agent skill that let's me one of Aider's favourite tricks: naming the agent in the code with `@aider extract the code below into a method <foo>`, so you don't have to go back and forth between a chat window and the code as much.
+
+Aider is a coding agent that precedes claude code. It didn't have tool calls, but did have very well defined mini-flows like this.
+
+I remember Aider picking this up on save, and executing immediately. This skill is a bit cruder, I run the command `skill:at-pi` and then it searches in recently changed files for lines annotated with `@pi` or `@Pi`.
+
+The main benefit of an @<agent> instruction in a file, is that you can use any coding agent and any editor / IDE. I am not a fan of 'spicy autocomplete' that just starts generating text while I am trying to do some wishful thinking.
+
+A skill like this isn't that hard to make. One prompt, and some iteration to make it work for me. Full skill is included at the bottom of this post. Read through it, and make your own. Curisous what you'll come up with.
+
+
+Backstory
+===
+I am slowly making WeReview 2.0 production ready. It was mostly done with [Synthetic TDD](/blog/engineering/synthetic-tdd) so far. I was blown away over a year ago when I managed to do a rewrite of 80% of WeReview in 5 days calendar time, during a school holiday. WeReview is an opinionated session review system that helps focused conferences create programs fast with little fuss. This was one of those 'blown away' moments.
+
+I have in places explicitly driven the design, like making flexible forms with the help of an Allium Spec as a separate system. Answering questions about "where are we" and "how did that flow work again?", "what do we still need to do" is sometimes more work than I like. So I have decided to generate documents showing the flow from one screen to the next as a side effect of running integration tests. This was inspired by a Matteo Vaccari's talk on TDD and AI, in which he demonstrated his customer focused tests that are not necessarily end to end, inspired by Ward Cunningham's Fit. That reminded my of Ward Cunningham's Swim system, that he made for the Eclipse Foundation. There is a paper about it that I will include in a follow up post.
+
+My users use the application through screens, I can model the domain as much as I want, but that is what they interact with. I have cheaply made end to end tests, but they are slow and fiddly, and give me a 'moment in time'. I can render videos, but those are also not easy to follow for the key stakeholders. So a document describing steps in the flow, maybe with side steps as in Ward Cunninghams' work felt like a good idea, and reading his paper backwards it can be done in small steps.
+
+Doing that took extending tests to output HTML in the right places. For this Pi + Qwen worked quite well, in the get it Run sense. Get it right, less so. I had some ideas on extracting a DSL, broke them down into tasks. When I do that, I tend to over-engineer. My coding agent less so, but once it worked, I had 400 lines of code for a test that was just spanning three screens. 
+
+This took 1.6 minutes, which is not particularly fast. But I was doing other things. There is refactoring tooling for Elixir, but it is not that easy to setup, I am in the process of doing it. I have paid attention to the extract setup refactoring more closely and will do the next one by hand...
+
+Also note, that as Pi / Qwen reported success, that not all the tests have been run (there are 998 tests in total, this is an umbrella project with `make test` at the root)
+
+> - ✅ All 739 tests pass
+> - ✅ Clean compilation (no warnings)
+> - ✅ Committed as 2e2086d
+
+About the skill
+====
+
+It is probably overkill to use `git status --porcelain` a search through all files might work as well, or better. `rg` is not working in my agents' sandbox for some reason, I need to figure this out still. It is also specific to elixir in places. Also not sure if prioritization is necesary. I put it in, because when I see multiple things that need work in a piece of code, I want to annotate before I forget (this is analogous to putting test lists in code, but for refactorings too).
+
+The At Pi skill in full
+=====
+
+```
+---
+name: at-pi
+description: Finds @pi comments in staged/unstaged changed files, presents them for prioritization, then executes each via a sub-agent (pi -p) with context from relevant imports. Use when you want to batch-execute developer instructions left as @pi annotations across changed files.
+---
+```
+
+# @pi Skill
+
+Scans changed git files for `@pi` annotated instructions, prioritizes them with the user, then executes each sequentially via a sub-agent with contextual file information.
+
+## Step 1: Discover Changed Files
+
+Run:
+```bash
+git status --porcelain | awk 'NR>0 {print $2}'
+```
+
+This returns staged + unstaged + untracked files (one path per line).
+
+Filter to code files you care about: `.ex`, `.exs`, `.heex`, `.js`, `.ts`, `.py`, `.rb`, etc.
+
+## Step 2: Scan for @pi Comments
+
+For each changed file, scan for `@pi` annotations. Two formats are supported:
+
+### Single-line
+```elixir
+# @pi fix the email validation to also reject nil
+def validate_email(email), do: ...
+```
+Instruction = everything after `@pi ` on that same line.
+
+### Multi-line (block comment)
+```elixir
+# @pi
+# Fix the email validation logic here.
+# It should handle: nil, empty string, and invalid format.
+# See WereviewWeb.EmailValidator for reference patterns.
+def validate_email(email), do: ...
+```
+Instruction = everything after `@pi` on the tag line (may be empty), plus all continuation lines that start with `# ` (or `// ` for JS/TS) until a blank line, end of file, or a non-annotation comment line.
+
+Strip the leading `# `, `// `, or `; ` markers and join into one instruction string.
+
+### Parsing approach
+Use `grep -ni '@pi' <file>` to find lines with `@pi` (case-insensitive — catches `@Pi`, `@PI`, `@pi`, etc.), then read surrounding context to capture multi-line blocks.
+
+### Context resolution (important)
+When a `@pi` comment appears immediately **above** a function definition, struct, module, or code block, the instruction likely refers to **that code**. Read that code to identify its name, arity, and purpose. Infer this relationship and carry it into the sub-agent prompt.
+
+For example:
+```elixir
+# @Pi: move to Swimex.Report
+defp extract_fragment(full_html, selector) do
+```
+The intent is: move `extract_fragment/2` into the Swimex.Report module. The sub-agent must understand **what** is being referred to, not just the bare instruction text.
+
+## Step 3: Build Instruction List
+
+For each `@pi` comment found, build a structured entry:
+
+```
+[N] FILE:PATH:LINE  "instruction text"
+```
+
+Where `[N]` is the discovery order index (1-based).
+
+Example output:
+```
+Found 3 @pi annotations in changed files:
+
+[1] wereview/apps/wereview/test/support/swimex_report.ex:82
+    "move the output_dir logic here to use the WEREREVIEW_TEST_REPORT_DIR env var"
+
+[2] wereview/lib/wereview_web/email_validator.ex:15
+    "add regex validation for international domain names"
+
+[3] wereview/apps/wereview_core/lib/wereview/tenants.ex:44
+    "handle the case where tenant slug already exists — return {:error, :duplicate_slug}"
+```
+
+## Step 4: Prioritize with User
+
+For each instruction, **resolve its intent** before presenting:
+
+1. If the `@pi` comment is immediately above a code block (function/struct/module), identify that code and describe what will happen.
+2. Present a clear action statement, not just the raw instruction text.
+
+Example output:
+```
+Found 3 @pi annotations in changed files:
+
+[1] wereview/apps/wereview/test/support/swimex_report.ex:82
+    "move the output_dir logic here to use the WEREREVIEW_TEST_REPORT_DIR env var"
+
+[2] wereview/lib/wereview_web/email_validator.ex:15
+    "add regex validation for international domain names"
+
+[3] wereview/apps/wereview/test/wereview_web/live/session_submit_live_test.exs:7
+    → The function extract_fragment/2 will be moved to Swimex.Report
+    (raw instruction: "todo move to Swimex.Report")
+```
+
+Ask the user to reorder by specifying a sequence. For example:
+
+> Execute in this order: 3, 1, 2 (skip 2)
+
+Or just press Enter to execute in discovery order. The user can skip any by omitting it.
+
+## Step 5: Discover Relevant Context Files
+
+For each instruction you are about to execute:
+
+1. Look at the file containing the `@pi` comment
+2. Extract import/require/use/include statements from that file to find relevant neighbors
+3. For Elixir (`.ex` / `.exs`):
+   - `import X` — module being imported
+   - `require X` — module being required
+   - `use X` — module being used with callbacks
+4. Resolve module names to actual file paths in the project
+5. Read those files so you know what's available
+
+Example for Elixir:
+```elixir
+# From wereview/lib/wereview_web/email_validator.ex
+import WereviewWeb.EmailHelpers    → wereview/lib/wereview_web/email_helpers.ex
+use WereviewCore.Validation         → wereview/apps/wereview_core/lib/validation.ex
+```
+
+## Step 6: Execute via Sub-Agent (Sequential)
+
+For each instruction in the prioritized order:
+
+1. **Build the sub-agent prompt**: Combine the original `@pi` instruction with context about nearby relevant files. Include:
+   - The **resolved intent** (what code/block the annotation refers to, e.g., "move function `extract_fragment/2`")
+   - The raw instruction text
+   - The file path and line number of the annotation
+   - A brief summary of what the relevant imported/required files provide (if you read them)
+   - Any other contextual info you already know about the codebase
+
+2. **Spawn the sub-agent**:
+   ```bash
+   pi -p "<enriched prompt>"
+   ```
+
+3. The sub-agent will modify files in place. You do NOT re-read or re-apply changes yourself — trust the sub-agent to make its changes.
+
+## Step 7: Show Changes and Ask Confirmation
+
+After all instructions are executed:
+
+1. Run `git diff` to show all uncommitted changes
+2. Present the diff summary to the user:
+   ```
+   Changes after @pi execution:
+   
+   modified: wereview/apps/wereview/test/support/swimex_report.ex
+     - removed unused @output_dir (already handled internally)
+     + added email validation logic
+   
+   modified: wereview/lib/wereview_web/email_validator.ex
+     + international domain regex support
+   
+   Commit these changes? (y/n)
+   ```
+
+3. If user says `y`: run `git add -A && git commit -m "Execute @pi annotations"`
+4. If user says `n`: leave changes uncommitted for manual review
+
+## Error Handling
+
+- If a file cannot be read, skip it and note the error
+- If no `@pi` comments are found in changed files, report: "No @pi annotations found in changed files."
+- If `git status` fails (not in a git repo), stop and inform the user
+- If the user cancels mid-execution, show partial results so far and exit