mostalive/firehose
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fixed link to tag and image

Browse Source
This commit is contained in:
Firehose Bot 2026-08-12 23:48:12 +02:00
parent 3308299d3c
commit f522895e4e
1 changed files with 7 additions and 5 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

12
app/priv/blog/engineering/2026/08-08-developing-with-open-weights-models-only-three-months-in.md → app/priv/blog/engineering/2026/08-12-developing-with-open-weights-models-only-three-months-in.md
View File

@ -9,7 +9,7 @@
I blogged about [My local agentic development setup](/blog/engineering/my-local-agentic-dev-setup-today) the day after my post about cancelling my Claude Max subscription and going all in on local models went viral. I blogged about [My local agentic development setup](/blog/engineering/my-local-agentic-dev-setup-today) the day after my post about cancelling my Claude Max subscription and going all in on local models went viral.
I am still using the same flexible harness - [Pi.dev](blog/engineering/tag/pi.dev), with surprisingly few extensions (apart from yesterday, see below for prime-agent). I had expected to add web-search, but even my local models have a lot of knowledge, know how to use tools and look up urls from documentation. Enough to start building [Swim the test](/blog/engineering/swim-the-test), which is quite niche, into an existing system. I occasionally paste a hand-searched link into the prompt and that is enough. I am still using the same flexible harness - [Pi.dev](/blog/engineering/tag/pi.dev), with surprisingly few extensions (apart from yesterday, see below for prime-agent). I had expected to add web-search, but even my local models have a lot of knowledge, know how to use tools and look up urls from documentation. Enough to start building [Swim the test](/blog/engineering/swim-the-test), which is quite niche, into an existing system. I occasionally paste a hand-searched link into the prompt and that is enough.
The first six weeks were uneventful. I didn't need to create large prototypes in a day like before, that helped. I was busy preparing larger sessions (a coding dojo with 70 participants and 10 facilitators for a UK client, and a keynote about AI and TDD and a workshops on AI with TDD for Philips Image Guided Therapy). So mostly smaller things, and sometimes making small in-the-loop changes to existing code. The first six weeks were uneventful. I didn't need to create large prototypes in a day like before, that helped. I was busy preparing larger sessions (a coding dojo with 70 participants and 10 facilitators for a UK client, and a keynote about AI and TDD and a workshops on AI with TDD for Philips Image Guided Therapy). So mostly smaller things, and sometimes making small in-the-loop changes to existing code.
@ -17,15 +17,17 @@ Then in a conversation with Chris Matts he mentioned he sketches UI on a piece o
In July I went back to working on a larger piece of software, and improvements to local inference engines were coming in thick and fast. I can now run Qwen3.6 27B, their dense model, at conversational speed, and my previous go to, their 35B mixture of experts model, runs a lot faster as well. For smaller, me-in-the-loop changes with tests, the 35B model works quite well. For trickier things, or larger independent runs, where the coding agent runs through a list of tasks and executes them in sub-agents one by one, the 27B model performs better. Slower per token, faster to outcome. In July I went back to working on a larger piece of software, and improvements to local inference engines were coming in thick and fast. I can now run Qwen3.6 27B, their dense model, at conversational speed, and my previous go to, their 35B mixture of experts model, runs a lot faster as well. For smaller, me-in-the-loop changes with tests, the 35B model works quite well. For trickier things, or larger independent runs, where the coding agent runs through a list of tasks and executes them in sub-agents one by one, the 27B model performs better. Slower per token, faster to outcome.
With [mtplx](https://github.com/youssofal/mtplx) instead of LLamaCPP I can now run the dense Qwen3.6 27B at conversational speed (was below 20 tokens per second With [mtplx](https://github.com/youssofal/mtplx) instead of LLamaCPP I can now run the dense Qwen3.6 27B at conversational speed (was below 20 tokens per second, now often 30 at the start of a conversation). This means I can run more complex development tasks locally. If it slows down later, I'll go do something else, as long as it can finish the job independently.
, now often 30 at the start of a conversation). This means I can run more complex development tasks locally. If it slows down later, I'll go do something else, as long as it can finish the job independently. It managed to find something properly obscure that Deepseek v4 flash also could find, but the 35B model could not. I am keeping that private as a benchmark. Before this, none of the MLX (mac accellerated) inference engines could handle long context (for me). Mtplx does that - speed does not matter if you can't get a good outcome. As an example, it managed to find something properly obscure that Deepseek v4 flash also could find, but the 35B model could not. I am keeping that private as a benchmark. Before this, none of the MLX (mac accellerated) inference engines could handle long context (for me). Mtplx does that - speed does not matter if you can't get a good outcome.
MTPLX is also a proper consumer solution. It comes with a GUI to select models and see how it is running. It will figure out the best parameters to run a model with by running a benchmark on your machine after downloading a model, and comes with pre-selected models and quantizations. The screenshot below was after installing a smaller variation of the qwen27B model. Before this I had never seen any of them do more than 20 tokens per second. Around 30 is good enough for a conversation for me. MTPLX is also a proper consumer solution. It comes with a GUI to select models and see how it is running. It will figure out the best parameters to run a model with by running a benchmark on your machine after downloading a model, and comes with pre-selected models and quantizations. The screenshot below was after installing a smaller variation of the qwen27B model. Before this I had never seen any of them do more than 20 tokens per second. Around 30 is good enough for a conversation for me.
![Mtplx installation screenshot](../../../static/images/blog/2026/mtplx-install.png) The screenshot below was taken when I installed a smaller version of the Qwen 27B model. It shows a few sessions tested on my laptop, and which one was found to be fastest, for me.
I still have LLamaCPP, but mtplx does well enough at the moment. ![Mtplx installation screenshot](/images/blog/2026/mtplx-install.png)
I still have LLamaCPP, but mtplx does well enough at the moment. I would us LLamaCPP when I want to run multiple smaller models on my mac simultaneously - mtplx so far only runs one model at a time. No problem for 35B and 27B as more than one does not really fit in 64GB (V)RaM.
Since the school holidays were starting and I am away from my desk more often, I revived my [remote environment in a virtual machine](/blog/engineering/how-to-get-started-with-the-pi-coding-agent-on-a-vps), so I could develop on my phone. I had used DeepSeek V4 Flash in part when preparing exercises. This model can be locally on slightly more VRAM than I have.I use openrouter for it. This has worked very well for adding features to existing code, doing longer running implementation sessions etc. It is cheaper to host than Qwen 3.6 27B apparently. Since the school holidays were starting and I am away from my desk more often, I revived my [remote environment in a virtual machine](/blog/engineering/how-to-get-started-with-the-pi-coding-agent-on-a-vps), so I could develop on my phone. I had used DeepSeek V4 Flash in part when preparing exercises. This model can be locally on slightly more VRAM than I have.I use openrouter for it. This has worked very well for adding features to existing code, doing longer running implementation sessions etc. It is cheaper to host than Qwen 3.6 27B apparently.