%{
  title: "AI Zombies",
  author: "Willem van den Ende",
  tags: ~w(AI Reflection),
  description: "In which the writer reflects on believing AI is conscious",
  published: false
}
---

I wrote this on LinkedIn, reposting Stuart-Winter-Tear:

I've seen very smart people see patterns where there are none. I like my systems as much as the next person, and LLMs can achieve much more than I'd expect from next token prediction. But this requires a proper harness and people operating it with both feet on the ground (and not their head in a methane powered cloud, but I digress ;-) ). Stuart Winter-Tear's short piece is well worth reading. "[..]our confidence in recognising consciousness rests on shakier ground than we like to admit."

It may look conscious to you, but it is still a next token predictor in a harness.

How can you prevent falling into this?
----

There is no failsafe recipe, I am afraid. One thing that can help is a rule of thumb I found in Matteo Vaccari's blog:

Read all the markdown that you adopt.

This is also a great way to reduce the number of things to try out. I've seen some repositories with a lot of markdown, and a general prompt, and then you can put your question or your writing to an agent to get perspectives.

There usually is too much text in these repositories, so I give up reading and put the repository aside. I want skill augmentation, not cognitive offloading.