74 lines
4.8 KiB
Markdown
74 lines
4.8 KiB
Markdown
%{
|
|
title: "Disposable End to End tests",
|
|
author: "Willem van den Ende",
|
|
tags: ~w(),
|
|
description: "",
|
|
published: false
|
|
}
|
|
---
|
|
|
|
I quite like playwright for the simplicity of writing End to End tests through the browser. Last year I realised that a coding agent (Claude Code, but Pi and local models can handle browsers as well) makes it cheap to write tests. Initially I didn't like the tests produced, but re-writing one by hand, maybe with some assistance, deleting ones that were superflous and then having the others rewritten in the style of the golden example was pretty painless.
|
|
|
|
Keeping them around does come at a cost, though.
|
|
|
|
1. Don't let people do what computers can do (unless it is the better way to learn for the people)
|
|
2. Don't let an LLM do what a simpler machine learning system or symbolic AI can do.ssh://git@gitea.apps.sustainabledelivery.com:3022/QWAN/dirtytests_cs.git
|
|
3. Don't let any AI do what deterministic code does better.
|
|
4. Make work fun and engaging for people
|
|
|
|
3. Tends to be faster and more energy efficient than 2. With people you never know, they can be very efficient and circumvent problems instead of attacking them head on.
|
|
|
|
|
|
In the end Pi could not access the browser through my sandbox, I have some configuration updates to make. But my 27B model used curl, found how to use the CSRF token in the page and made it through.
|
|
|
|
|
|
|
|
Handover message from the tool
|
|
----
|
|
|
|
> All tests pass, clean compile. The script is ready at scripts/deploy-and-test-admin.sh.
|
|
|
|
> Usage:
|
|
|
|
```bash
|
|
./scripts/deploy-and-test-admin.sh --password 'MyP@ss123!'
|
|
```
|
|
|
|
> What it does:
|
|
1. ssh dokku — sets ADMIN_EMAIL / ADMIN_PASSWORD on the Dokku app
|
|
2. git push dokku — deploys (migrations run, admin is provisioned)
|
|
3. Waits for the app to be ready (polls /users/log_in)
|
|
4. Logs in as admin via curl, verifies redirect to dashboard
|
|
5. Checks /admin/tenants is accessible as super admin
|
|
|
|
> Options:
|
|
- --password — admin password (prompts if omitted)
|
|
- --app-url — override the app URL (auto-resolved from git remote)
|
|
- --app — override Dokku app name (default: wereview2)
|
|
- --skip-deploy — skip the git push, just test login
|
|
|
|
Final test run
|
|
----
|
|
|
|
The first test run failed, my coding agent had tried to do something clever, reviewing the script: derive the url from the git remote of the repository. 27B clearly has been trained on dokku, and remembered it well. That does not work yet, as I had not set up the port mapping. Since I have time on desk now, I decided to do the riskiest part first, the rest is following my notes on previous deployments (yes, I remain, as ever, optimistic when deploying new applications).
|
|
|
|
``` bash
|
|
export APP_URL="http://wereview2.apps.sustainabledelivery.com:5000/"
|
|
willem@sequential:~/dev/spikes/elixir/wereview-2$ ./scripts/deploy-and-test-admin.sh
|
|
|
|
=== Super Admin Deploy & Test ===
|
|
Email: <email>
|
|
URL: http://someurl.apps.sustainabledelivery.com:5000/
|
|
Dokku: someurl.apps.sustainabledelivery.com
|
|
|
|
→ Setting admin credentials on Dokku...
|
|
→ Deploying (git push dokku)...
|
|
Everything up-to-date
|
|
→ Deploy complete.
|
|
→ Waiting for app to be ready...
|
|
→ App is ready.
|
|
→ Logging in as <email>
|
|
```
|
|
|
|
And then logging in, it failed. I had made the code to clever. Or rather, learnt something about which code is and is not available in the production deployment.
|