Note by @fastfinge

Matt Campbell @matt@toot.cafe

2w

Question for blind people who use LLM coding agents, vibe-coding or otherwise: Have you done any useful coding projects with a local, open-weights model yet? If so, what agent harness do you use? Is there one that works with at least one local model and has a reasonably accessible UI?

1

8

0

🇨🇦Samuel Proulx🇨🇦 @fastfinge@interfree.ca

2w

@matt @jonathan859 That question isn’t all that meaningful. Kimi k3 can run on a Mac mini and is about as good as gpt4. With an OpenAI compatible endpoint you can then do whatever you want. But I suspect “run a separate computer just for AI” may not have been what you meant. But I’ve moved eighty percent of what I do to k3. But that’s more running the server myself, not running a local model on my laptop.

1

0

Matt Campbell @matt@toot.cafe

2w

@fastfinge I'm still interested in knowing how you do it. Do you actually run Kimi K3 on a Mac mini, or something else? Using Ollama or some other tool? And what coding agent do you use with it?

4

0

Bram Duvigneau @bram@bramd.nl

2w

@matt @fastfinge Having a local capable model would be nice for more experimentation in which coding tool/harness to use and probably also to run more agentic non-coding workflows that don't really fit in with Claude now

1

0

🇨🇦Samuel Proulx🇨🇦 @fastfinge@interfree.ca

2w

@bram @matt Yeah, the other reason that makes me reluctant to move all of my coding over to Hermes is memory contamination. I like that Hermes can remember and reference all of the random things hindsight has stored about me in the memory layer; it's useful when I ask it random questions at 2 AM, or want it to remind me of something, or describe an image for me, or whatever. But when I'm working on a codebase, the fact that I live in Ottawa or that I like Lit RPG sometimes gets injected into the context by the memory layer, and that is useless and actively harmful to the current task. Yeah, I could make a new Hermes profile per project, and then they'd each have different memory banks and everything else, but that's a lot of overhead. The way Claude Code and Codex handle it is much more sensible for programming tasks. But, of course, on the other hand, the fact that Hermes remembers my tooling and environment preferences is extremely useful when what I want is a quick one-off script. I don't have to remind it what I have installed, that I want uvx when possible, that we're on Debian, etc. If I start a new Claude or Codex project, they have to re-discover all that stuff every time, when all I needed was a quick 150 line script.

1

0

Bram Duvigneau @bram@bramd.nl

2w

@fastfinge @matt Which model do you use to back Hermes and how do you find the costs? I have it set up on a VM on my homeserver and was plannign to use it for non-coding automation tasks/questions, which I now put in Claude Desktop cowork. Ideally I would interface my Meta glasses with it as well to have a proper, somewhat move privacy friendly AI with good context and MCPs while walking

1

0

🇨🇦Samuel Proulx🇨🇦 @fastfinge@interfree.ca

2w

@bram @matt i run k3 locally. so no cost.

1

0

Bram Duvigneau @bram@bramd.nl

2w

@fastfinge @matt I see K3 does vision as well, have you tried it for anything image related?

1

0

🇨🇦Samuel Proulx🇨🇦 @fastfinge@interfree.ca

2w

@bram @matt No, mostly because Gemma does run on my laptop, is much faster, and does an acceptable job 99 percent of the time.

0