Wrote a two-line script that runs git add, git commit, git push inside the vault folder, then wired it to a launchd job so it runs automatically every night. First commit had way more files than I expected. Once I actually looked, the vault has quietly grown to something like 380 notes.

Second run the next morning still showed a diff, eighteen files, even though I hadn't touched a single note. Turned out .obsidian/workspace.json rewrites itself basically every time the app opens, tracking which panes and tabs were last open. Most of the rest was data.json inside each plugin's own folder, one per installed plugin, rewritten on every launch too. None of that is content. All of it was getting committed anyway, burying the actual changes.

Added a .gitignore for the .obsidian folder, except for the parts I actually care about keeping, like my custom CSS snippet and the plugin list. Diffs are readable again, just the notes that actually changed. Small fix, but exactly the kind of thing that would have made the daily backup useless within a week if I hadn't caught it on day two instead of day sixty.

First time it was a changelog line for a small fix, "Fixes #47." That repo doesn't even use GitHub Issues, I track things in a plain TODO.md file, there's no issue tracker for a #47 to exist in. Second time it was a test-run summary claiming the suite finished in 38 seconds. The actual CI log said just under 6 minutes.

Neither one felt wrong on a first read. A changelog line with an issue number looks exactly like every other changelog line. A fast test suite is exactly what I want to hear, which is probably why I almost didn't check it.

Same fix as always, just applied more consistently now. No number, no reference, no citation goes out without me pointing at the real source first, a log line, a file, an actual link. Slower to write, but at least it's true.

The progress bar took the most rework. ComfyUI doesn't always report a clean percentage while it renders, so the first version either froze on some number or jumped straight from 12% to done. Looked broken even on runs that finished fine. Now the current step stays visible the whole time. A percentage only shows up when ComfyUI actually sends one, otherwise there's just no number instead of a fake one.

Also split the docked prompt library so it only shows image prompts. It used to pull from the same store as chat and code prompts. Picking one for an image run meant scrolling past a pile of unrelated snippets first.

Reference images, LoRA support and the private vault all got the same pass. Still tuning the deviation setting that controls how close a new variant has to stay to a locked reference. Not sure yet if the default is too loose or too strict.

Turned out BenchmarkResult never had a field for the prompt in the first place. It got used during the run and then just fell out of scope the moment the run finished. Nothing overwrote it, nothing got corrupted, it simply was never saved anywhere past the moment it got used. Fixed it by adding a real prompt field to the model and threading it through save and load. The commit message ended up about as blunt as the bug itself, "Fix: benchmark prompt was never saved or shown to the user." So the prompt is visible now, live in the interface, in the result view and in every export. Markdown, HTML, CSV, JSON, all of it.

While the code was already open, I finally rebuilt the overview page into something closer to a real control center instead of a static summary. It now shows RAM and CPU, tokens per second, latency, a throughput chart, which jobs are active right now, plus an event log. All of it refreshes every few seconds. AI Control Center already does something similar for the whole machine. Wrote about that a few days ago, when it caught two stuck Ollama processes fighting over the same port. But that dashboard covers everything running on the Mac, not just Local LLM Matrix. So I stopped waiting for the standalone version and built a smaller one straight into the app, scoped to just its own runs.

A few smaller things shipped alongside that. A "reset all results" button, so starting over doesn't mean reinstalling. Ollama now gets auto-detected and can be started right from the app in the background, no terminal window popping up and stealing focus. And the recommended install path switched from manual pip plus a virtual environment to pipx, mostly because half the setup questions I was getting turned out to be venv problems in disguise.

None of this was on the roadmap for this week. It all started with one missing column in an export. v0.37.3 is live on GitHub now. Not sure what's next, probably whatever the next missing field turns out to be.

Today it caught something I would have missed for hours. The dashboard listed Ollama as running fine, green dot, port 11434 responding. But a benchmark job for Local LLM Matrix kept timing out anyway. Turned out there were two ollama serve processes alive at once, an old one left over from a crashed terminal session three days ago, plus the current one. Both bound to the same port somehow. Whichever one answered a given request depended on pure luck.

The dashboard's health check had only ever pinged the port, not the actual process table, so a zombie process that still responded to pings looked completely healthy. Added a second check that cross-references the PID behind the port against what launchd thinks should be running. Killed the stale process, response times back to normal immediately.

Small fix, but it's exactly the kind of bug this tool exists to catch. Not a benchmark number, just "why is this slow" turning into an actual answer in two minutes instead of an evening of guessing.

Didn't notice for almost an hour, until a search for that project came back nearly empty, way fewer notes than I remembered having on it. Went into the vault's built-in broken-links view, twelve notes still pointed at the filename that no longer existed.

Fixed them one by one. No bulk find-and-replace, not on markdown link syntax I didn't fully trust to run unsupervised across two hundred plus files, especially so soon after handing an agent free rein on tags in that same vault a few weeks back. Took about ten minutes, not the two seconds the rename itself felt like in Finder.

New rule for the vault, right under the tagging one. Rename inside Obsidian and nowhere else. The one spot where a native macOS feature actively made things worse instead of saving a step.

Loaded the Q5 build into Open WebUI afterward and it just sat there, spinner going, no response for almost two minutes on a short test prompt. Turned out num_ctx was still pinned to 8192 in an old Modelfile I'd copied from a smaller model months ago and never touched since. Fixed that, still slower than I expected.

Ran a real benchmark with ollama run --verbose instead of guessing. Q4_K_M was doing 22 tokens per second. Q5_K_M dropped to 6. Activity Monitor showed why, memory pressure in the red and swap climbing for most of the run. With Chrome and Slack open like normal, there wasn't enough of the 18GB left for the bigger quant plus context. macOS started paging to disk instead of just failing outright. That's the actual bottleneck, not the Modelfile bug.

Rolled back to Q4_K_M for now. Not the quality bump I was hoping for, but at least the next time I'm tempted to try a bigger quant I have real numbers to check against instead of just a feeling that it might be worth it.

Instead of doing the reorg by hand, which is exactly the thing I already gave up on once, I pointed an agent at the folder with one narrow instruction. Read each untagged note, suggest one or two tags based on the actual content, write them into the frontmatter, touch nothing else in the file.

It got through all 200 in about eighteen minutes. I spot checked maybe thirty of them, two tags were a little generic, one note about a failed benchmark run got tagged "success" which made me laugh, everything else was close enough to what I would have picked myself.

What actually worked was showing it exactly what a bad response looked like next to a good one, one pair, not a paragraph of rules. First run after that came back clean. Ran it 20 more times to make sure it wasn't luck, 19 out of 20 were valid on the first try.

Kind of obvious in hindsight. Telling a model what "not to do" in the abstract barely registers. Showing it one concrete failure case does most of the work instructions never quite manage.

Claude Code found all 7 call sites and asked before touching two of them because the imports looked ambiguous. It wrapped up in about 12 minutes, including the time I spent reading the diff.

OpenCode running a local model did the extraction fine, but missed one import in a test file. The test suite failed silently until I ran it myself. The agent had already reported success and moved on to something else.

First run choked, just not on the embedding step. A batch of older notes had a tags field written as a plain string instead of a list, my indexer assumed a list every time and threw on note 61 of 340. It failed quietly in the background too, so I didn't notice for a good twenty minutes.

Once that was fixed, search actually worked. Not fast though. Indexing the full vault takes about six minutes on my machine with nomic-embed-text running through Ollama. That number has to come down a lot before this is something I use daily instead of something I demo once and forget about.

I've migrated between three note apps over the years, and the ones that stored things in a proprietary format lost something every single time — formatting, backlinks, sometimes the whole file. The plain .md files from a project I abandoned in 2022 opened today exactly the way I left them.

There's a more practical reason too, one I only really felt today: an agent reading a folder of .md files doesn't need a parser, a plugin, or an export step, it just reads text. Same reason git handles them so well. Boring format, on purpose, and that's exactly the point.

What did happen: the site stopped being a static page and started being a place I actually update. This time the plan was to publish something rough every couple of days instead of waiting for it to be right, and so far that's the only part of the plan that's worked exactly as intended.

The thing I keep coming back to: public benchmark numbers for local models tell you almost nothing about your own setup. Local LLM Matrix exists because I got tired of trusting numbers that weren't measured on my hardware.

Scout was the earlier, narrower version, it just discovered and listed whatever Ollama models you had installed locally. Once it grew into actually running benchmark profiles and comparing results over time, the name didn't fit anymore, so it became Local LLM Matrix.

It touched twelve files. Renamed variables I liked. Reformatted things that weren't broken. I didn't recognize half the diff and had to revert most of it. Second attempt, same agent, much narrower: one file changed, three lines, done in under a minute.

The agent isn't the variable. How tightly you scope the task before handing it over is the variable.

It doesn't matter that it's messy, because it's still just plain text files on disk. No app can hold it hostage. Every note-taking app I've dropped over the years failed at exactly the moment I needed to get my data back out. Markdown has never once done that to me.

What actually fixed it: telling it exactly what "summarize" meant in this context, giving it one example of the input/output shape, and asking it to reason through the differences before writing the summary.

Turned out to be a relative path in the meta tags that worked fine locally and broke the second the site moved to its real domain. Local testing catches maybe seventy percent of what matters.

The plan is to post something here every couple of days, rough drafts of whatever I actually worked on, not polished announcements written after the fact.