firehose/app/priv/blog/engineering/2026/07-16-at-coding-agent-skill-to-guide-editing-in-place.md

%{ title: "At coding agent skill to guide editing in-place", author: "Willem van den Ende", tags: ~w(AI coding-agent baby-steps), description: "With an @ instruction in a file, you can use your regular coding agent on small parts of your code, without modifiying your editor / IDE.", published: true }

TLDR

I made an agent skill that let's me one of Aider's favourite tricks: naming the agent in the code with @aider extract the code below into a method <foo>, so you don't have to go back and forth between a chat window and the code as much.

Aider is a coding agent that precedes claude code. It didn't have tool calls, but did have very well defined mini-flows like this.

I remember Aider picking this up on save, and executing immediately. This skill is a bit cruder, I run the command skill:at-pi and then it searches in recently changed files for lines annotated with @pi or @Pi.

The main benefit of an @ instruction in a file, is that you can use any coding agent and any editor / IDE. I am not a fan of 'spicy autocomplete' that just starts generating text while I am trying to do some wishful thinking.

A skill like this isn't that hard to make. One prompt, and some iteration to make it work for me. Full skill is included at the bottom of this post. Read through it, and make your own. Curisous what you'll come up with.

Backstory

I am slowly making WeReview 2.0 production ready. It was mostly done with Synthetic TDD so far. I was blown away over a year ago when I managed to do a rewrite of 80% of WeReview in 5 days calendar time, during a school holiday. WeReview is an opinionated session review system that helps focused conferences create programs fast with little fuss. This was one of those 'blown away' moments.

I have in places explicitly driven the design, like making flexible forms with the help of an Allium Spec as a separate system. Answering questions about "where are we" and "how did that flow work again?", "what do we still need to do" is sometimes more work than I like. So I have decided to generate documents showing the flow from one screen to the next as a side effect of running integration tests. This was inspired by a Matteo Vaccari's talk on TDD and AI, in which he demonstrated his customer focused tests that are not necessarily end to end, inspired by Ward Cunningham's Fit. That reminded my of Ward Cunningham's Swim system, that he made for the Eclipse Foundation. There is a paper about it that I will include in a follow up post.

My users use the application through screens, I can model the domain as much as I want, but that is what they interact with. I have cheaply made end to end tests, but they are slow and fiddly, and give me a 'moment in time'. I can render videos, but those are also not easy to follow for the key stakeholders. So a document describing steps in the flow, maybe with side steps as in Ward Cunninghams' work felt like a good idea, and reading his paper backwards it can be done in small steps.

Doing that took extending tests to output HTML in the right places. For this Pi + Qwen worked quite well, in the get it Run sense. Get it right, less so. I had some ideas on extracting a DSL, broke them down into tasks. When I do that, I tend to over-engineer. My coding agent less so, but once it worked, I had 400 lines of code for a test that was just spanning three screens.

Refactoring by hand and by Agent

This took 1.6 minutes, which is not particularly fast. But I was doing other things. There is refactoring tooling for Elixir, but it is not that easy to setup, I am in the process of doing it. I have paid attention to the extract setup refactoring more closely and will do the next one by hand, probably. I have done some extract functions by hand to keep a feel for Elixir, and also because it sometimes takes less time to just do it than to explain it.

Aside: when the coding agent reports the number of tests

Also note, that as Pi / Qwen reported success, that not all the tests have been run (there are 998 tests in total, this is an umbrella project with make test at the root)

• ✅ All 739 tests pass
• ✅ Clean compilation (no warnings)
• ✅ Committed as 2e2086d

The rubber duck in the blog post is strong. All the tests are run, but in three rounds, as this is a small mono-repo. I am attempting to keep some things that are not specific for the domain, like flexible forms, separate, without spinning them off in to separate repos. The coding agent picked up on the last run, which has 739 tests without mentioning the other ones.

I was tempted to blame the agent for not running all tests, but they are part of a post-commit hook, so they get run deterministically, no matter what.

About the skill

It is probably overkill to use git status --porcelain a search through all files might work as well, or better. rg is not working in my agents' sandbox for some reason, I need to figure this out still. It is also specific to elixir in places. Also not sure if prioritization is necesary. I put it in, because when I see multiple things that need work in a piece of code, I want to annotate before I forget (this is analogous to putting test lists in code, but for refactorings too).

I had expected to get this to work in one prompt, but maybe my perfectionism got the better of me, once again. I thought do do this with sub-agents in one go (looking at the skill) and getting the context picked up besides my one line instruction, was interesting. See the Context Resolution section below.

Note that none of the @pi instructions will make it into the commit history, if you work one instruction at a time. Luckily the skill has an example:

# @Pi: move to Swimex.Report
defp extract_fragment(full_html, selector) do

That already needed a re-prompt, because I had specified 'pi' in the instruction to create the skill, and used 'Pi' when trying it out. Moving a function to another module is simple, but annoying:

• Copy function over to new module.
• Import function in old module
• Make sure it is referenced correctly by all call sites
• Delete original function

The At Pi skill in full

---
name: at-pi
description: Finds @pi comments in staged/unstaged changed files, presents them for prioritization, then executes each via a sub-agent (pi -p) with context from relevant imports. Use when you want to batch-execute developer instructions left as @pi annotations across changed files.
---

@pi Skill

Scans changed git files for @pi annotated instructions, prioritizes them with the user, then executes each sequentially via a sub-agent with contextual file information.

Step 1: Discover Changed Files

Run:

git status --porcelain | awk 'NR>0 {print $2}'

This returns staged + unstaged + untracked files (one path per line).

Filter to code files you care about: .ex, .exs, .heex, .js, .ts, .py, .rb, etc.

Step 2: Scan for @pi Comments

For each changed file, scan for @pi annotations. Two formats are supported:

Single-line

# @pi fix the email validation to also reject nil
def validate_email(email), do: ...

Instruction = everything after @pi on that same line.

Multi-line (block comment)

# @pi
# Fix the email validation logic here.
# It should handle: nil, empty string, and invalid format.
# See WereviewWeb.EmailValidator for reference patterns.
def validate_email(email), do: ...

Instruction = everything after @pi on the tag line (may be empty), plus all continuation lines that start with # (or // for JS/TS) until a blank line, end of file, or a non-annotation comment line.

Strip the leading # , // , or ; markers and join into one instruction string.

Parsing approach

Use grep -ni '@pi' <file> to find lines with @pi (case-insensitive — catches @Pi, @PI, @pi, etc.), then read surrounding context to capture multi-line blocks.

Context resolution (important)

When a @pi comment appears immediately above a function definition, struct, module, or code block, the instruction likely refers to that code. Read that code to identify its name, arity, and purpose. Infer this relationship and carry it into the sub-agent prompt.

For example:

# @Pi: move to Swimex.Report
defp extract_fragment(full_html, selector) do

The intent is: move extract_fragment/2 into the Swimex.Report module. The sub-agent must understand what is being referred to, not just the bare instruction text.

Step 3: Build Instruction List

For each @pi comment found, build a structured entry:

[N] FILE:PATH:LINE  "instruction text"

Where [N] is the discovery order index (1-based).

Example output:

Found 3 @pi annotations in changed files:

[1] wereview/apps/wereview/test/support/swimex_report.ex:82
    "move the output_dir logic here to use the WEREREVIEW_TEST_REPORT_DIR env var"

[2] wereview/lib/wereview_web/email_validator.ex:15
    "add regex validation for international domain names"

[3] wereview/apps/wereview_core/lib/wereview/tenants.ex:44
    "handle the case where tenant slug already exists — return {:error, :duplicate_slug}"

Step 4: Prioritize with User

For each instruction, resolve its intent before presenting:

1. If the @pi comment is immediately above a code block (function/struct/module), identify that code and describe what will happen.
2. Present a clear action statement, not just the raw instruction text.

Example output:

Found 3 @pi annotations in changed files:

[1] wereview/apps/wereview/test/support/swimex_report.ex:82
    "move the output_dir logic here to use the WEREREVIEW_TEST_REPORT_DIR env var"

[2] wereview/lib/wereview_web/email_validator.ex:15
    "add regex validation for international domain names"

[3] wereview/apps/wereview/test/wereview_web/live/session_submit_live_test.exs:7
    → The function extract_fragment/2 will be moved to Swimex.Report
    (raw instruction: "todo move to Swimex.Report")

Ask the user to reorder by specifying a sequence. For example:

Execute in this order: 3, 1, 2 (skip 2)

Or just press Enter to execute in discovery order. The user can skip any by omitting it.

Step 5: Discover Relevant Context Files

For each instruction you are about to execute:

1. Look at the file containing the @pi comment
2. Extract import/require/use/include statements from that file to find relevant neighbors
3. For Elixir (.ex / .exs):
   • import X — module being imported
   • require X — module being required
   • use X — module being used with callbacks
4. Resolve module names to actual file paths in the project
5. Read those files so you know what's available

Example for Elixir:

# From wereview/lib/wereview_web/email_validator.ex
import WereviewWeb.EmailHelpers    → wereview/lib/wereview_web/email_helpers.ex
use WereviewCore.Validation         → wereview/apps/wereview_core/lib/validation.ex

Step 6: Execute via Sub-Agent (Sequential)

For each instruction in the prioritized order:

1. Build the sub-agent prompt: Combine the original @pi instruction with context about nearby relevant files. Include:

   • The resolved intent (what code/block the annotation refers to, e.g., "move function extract_fragment/2")
   • The raw instruction text
   • The file path and line number of the annotation
   • A brief summary of what the relevant imported/required files provide (if you read them)
   • Any other contextual info you already know about the codebase

2. Spawn the sub-agent:

   pi -p "<enriched prompt>"
   

3. The sub-agent will modify files in place. You do NOT re-read or re-apply changes yourself — trust the sub-agent to make its changes.

Step 7: Show Changes and Ask Confirmation

After all instructions are executed:

1. Run git diff to show all uncommitted changes

2. Present the diff summary to the user:

   Changes after @pi execution:
   
   modified: wereview/apps/wereview/test/support/swimex_report.ex
     - removed unused @output_dir (already handled internally)
     + added email validation logic
   
   modified: wereview/lib/wereview_web/email_validator.ex
     + international domain regex support
   
   Commit these changes? (y/n)
   

3. If user says y: run git add -A && git commit -m "Execute @pi annotations"

4. If user says n: leave changes uncommitted for manual review

Error Handling

• If a file cannot be read, skip it and note the error
• If no @pi comments are found in changed files, report: "No @pi annotations found in changed files."
• If git status fails (not in a git repo), stop and inform the user
• If the user cancels mid-execution, show partial results so far and exit