add image to blog, fixed emacs drag and drop on linux

2026-10-21 11:19:31 +01:00
parent ebf076c450
commit 86d52aa420
3 changed files with 34 additions and 10 deletions
@@ -1,8 +1,8 @@
 %{
  title: "Video Transcriber - turn your videos into text with slides",
  author: "Willem van den Ende",
-  tags: ~w(writing local-models, open-weights AI agentic-engineering speech-to-text),
-  description: "In which the author uses a local text to speech model to draft blog posts, starting from talking while sketch noting.",
+  tags: ~w(writing local-models open-weights AI agentic-engineering speech-to-text),
+  description: "In which the author uses a local text-to-speech model to draft blog posts, starting from talking while sketch noting.",
  image: nil,
  published: true
 }
@@ -14,13 +14,29 @@ I attempted to write about Video Transcriber when I first used it nine months ag

 Instead of using AI to write my words, I am using AI to build me small tools to get my words out. These tools may or may not contain AI themselves. In today's episode, I used a Speech-to-Text model together with a deterministic [computer vision library](https://opencv.org/). I only want to record an image (a 'slide' in the domain of Video Transcriber) whenever I stop dragging the canvas around, or stop drawing for a bit. The vision library allows deterministic code to do that. It can be used instead of the existing functionality in Video Transcriber that creates a slide every 15 or 30 seconds.

-A regular pace does not seem to be a good fit for when I am sketching. I either sketch a lot, or talk a lot, but rarely at the same time.
+A regular pace does not seem to be a good fit for when I am sketching. I either sketch a lot, or talk a lot, but rarely at the same time. Extending Video Transcriber to take that into account was easier done than thought ;-). I am still surprised I can now do this kind of thing without deeply understanding the underlying technology.

+This post hinges on three things:
+
+- Using local AI to work with audio and video, and help me write
+- Using a coding agent to modify someone else's tool to taste
+- The challenge of publishing blog posts at the moment I have just done the thing
+
+It became a bit bigger than I imagined, and is less structured then when I was drafting. I did not plan to do stopped-motion detection, I swear! I hope you enjoy.
+
+Before we go on, a small sample of Video Transcribers' work. This one has timestamps to go with the text, it can also output whole paragraphs at once.
+
+![Transcript excerpt from a book review. Lines preceded by timestamps1 A paragraph of text, a heading that says 'Slide 12' with a timestamp. A partial view of a hand drawn scribble. ](/images/blog/2026/centola-barriers.png)
+
+You may notice it says "Santola" in the screenshot. That should be [Centola](https://www.damoncentola.com/). With a large Speech-To-Text model running on my laptop, this was about the only mistake. Phonetically they are the same, and I may need to work on my pronounciation. This is of course easily fixed by using search and replace.
+
+# Using local AI to work with audio and video, help me write

 ## What is Video Transcriber for?

 It is originally intended to make youtube videos easier to skim and refer back to. Reading text is a lot faster than listening to words. If you missed something, you can quickly glance back in text, whereas in video you have to scroll back and wait for the moment that you missed - it is only obvious in hindsight where that was.

+
 ## Different tools for different purposes

 For youtube videos I now use Gemini Notebook, because it already has the transcriptions. I do miss the visuals with it sometimes, and want to replace it with an open notebook that runs with local models when I want to. I hadn't used video transcriber in a while, but that changed last week.
@@ -35,14 +51,15 @@ I picked Video Transcriber back up to transcribe sketch notes. I sketch on my ta

 While I was drafting this blog post, trying to explain while I did not write about it nine months ago, I set my coding agent to work on a wish. A 6 and a half sketch noting video yielded 81 'slides' (the systems metaphor is  a transcriber for a presentation with slides). It would be nice if it only made a slide once I stopped dragging the infinite canvas around, or when I stopped writing for a bit. 

-I gave the [coding agent](/blog/engineering/tag/pi.dev) a brief prompt, and the video I made to draft a book review of Damon Centola's [Change: How to make big things happen](https://www.goodreads.com/book/show/53369466-change)
+I gave the [coding agent](/blog/engineering/tag/pi.dev) a brief prompt, and the video I made to draft a book review of Damon Centola's [Change: How to make big things happen](https://www.goodreads.com/book/show/53369466-change), together with output from Video Transcriber that made a slide every 15 seconds, for a total of 81 slides. With access to tools (ffmpeg, the OpenCV library, Python), and my question, the coding agent (Pi with GLM 5.3 flash in this case) did most of the work independently. It used Romily's demo video from the repository for quick tests, and also used my book review video to compare the resulting markdown. It did some iterations on it to find good settings, and explained three settings to me, which I had it save in a document for the future.

-Am I going to use Video Transcriber more now? Probably in the short term. In the long term, my flow of narrating could improve to the point that making a few screenshots does not feel like that much work anyomore, and the technical solution becomes less important. From what I have seen in my initial experiments, is that there are only a few significant images. Exporting the visuals from the source and drag them in at the appropriate place in the blog post is then not that much work. 
+After it more or less worked, I found that the text occasionally contained repeated phrases. The model had spotted this as well, and suggested to remove silence from the video before processing. It had found something that could do it, so  I let it rip. 

 ## Getting words out to the world

-I already invested in making it easier to put images in notes and blog posts, and export notes to blog posts. That is paying off already. Now the work is making it a habit to finish posts.
+I already invested in [making it easier to put images in notes and blog posts](/blog/engineering/removing-a-special-cause-of-variation-in-small-steps), and export notes to blog posts. That is paying off already; it is the kind of work that compounds. I have less friction in details, so I can focus on the stories I want to tell. Now the work is in making it a habit to finish and publish posts, as the drafts (with images :-) ) were piling up. 

+# The challenge of publishing blog posts at the moment I have just done the thing

 ## Why I didn't write about it nine months ago

@@ -50,8 +67,7 @@ So why didn't I write about Video Transriber earlier? I sometimes find it hard t

 Romily asked if I wanted to contribute something back, at the time, say a blog post or a pull request. I struggled with that. As a practice, [Will It Blend](https://willemvandenende.com/blog/engineering/will-it-blend) is great to get something to work, and the flow feels great, as it is very improvisational. Being able to quickly vibe code even a small improvement and use it is fantastic. A downside of it, is that I often can&rsquo;t easily explain what happened afterwards, not even with the session transcript, as the coding agent records it, in hand.

-
-## Looking back at what I did
+# Adopting someone else's tools nine months ago

 Looking back at the nine month old commits now, it was me using \`uv\` instead of \`pip\` - I standardized on uv as it felt easier and seems more modern and faster. But I am not a heavy python user like Romily, so I did not want to push my preferences at the time, and I couldn&rsquo;t quickly see how to have both co-exist in the README.

@@ -70,3 +86,8 @@ In the meantime, I would recommend checking  [Video Transcriber](https://github.
 See you next time!


+# Further Reading
+
+In [Will It Blend](https://willemvandenende.com/blog/engineering/will-it-blend)  I went into more detail in how I adjust other peoples' software to my needs. 
+
+Last year I wrote for QWAN on how I see my evolution of knowledge and tool use, in the age of Large Language models: [Stochastic to Deterministic](https://www.qwan.eu/2025/12/11/stochastic-to-deterministic.html). I do more with language and other kinds of models than before. But I still turn things I repeat into tools. Writing is still hard, but [removing accidental friction, like adding images to this blog](/blog/engineering/removing-a-special-cause-of-variation-in-small-steps) from my writing and publishing flow keeps paying off.