User avatar
Matt Campbell @matt@toot.cafe
2w
Question for blind people who use LLM coding agents, vibe-coding or otherwise: Have you done any useful coding projects with a local, open-weights model yet? If so, what agent harness do you use? Is there one that works with at least one local model and has a reasonably accessible UI?
1
8
0
0
User avatar
🇨🇦Samuel Proulx🇨🇦 @fastfinge@interfree.ca
2w
@matt @jonathan859 That question isn’t all that meaningful. Kimi k3 can run on a Mac mini and is about as good as gpt4. With an OpenAI compatible endpoint you can then do whatever you want. But I suspect “run a separate computer just for AI” may not have been what you meant. But I’ve moved eighty percent of what I do to k3. But that’s more running the server myself, not running a local model on my laptop.
1
1
0
0
User avatar
Matt Campbell @matt@toot.cafe
2w
@fastfinge I'm still interested in knowing how you do it. Do you actually run Kimi K3 on a Mac mini, or something else? Using Ollama or some other tool? And what coding agent do you use with it?
4
0
0
0
User avatar
🇨🇦Samuel Proulx🇨🇦 @fastfinge@interfree.ca
2w
@matt Yup, it runs on a mac with deltafin: github.com/gavamedia/deltafin
0
0
1
0
User avatar
🇨🇦Samuel Proulx🇨🇦 @fastfinge@interfree.ca
2w
@matt I currently use Hermes Agent with it to handle server monitoring, network diagnostics, troubleshooting, and basic admin tasks. I also use it with Hermes Agent to produce and handle batch scripts and services: a lightning strike monitor that sends me notifications when lightning is near, a security monitor that emails me new vulns for anything I run, a price checker that checks prices at my local grocery store and sends me any significant downward changes, a speedtest monitor, uptime monitors, and various other jobs. I'm working on moving stuff over to opencode for coding on Windows; right now Hermes can only run code on mac or Linux, so is of no use for the Windows projects I have in the pipeline. I do use Hermes to maintain my hermes-xmpp plugin and submit changes to the hermes-deltachat plugins with K3, because it can easily run and test both of those codebases.
1
1
2
0
User avatar
Josh @Orinks@mastodon.stickbear.me
2w
@fastfinge @matt I love Hermes, but over time I've gradually moved back to Claude Code/Codex for actual coding work. Hermes is great for jobs like monitor these server logs over a 12/24 hour period and let me know if anything abnormal happens etc
2
1
0
0
User avatar
🇨🇦Samuel Proulx🇨🇦 @fastfinge@interfree.ca
2w
@Orinks @matt The nice thing about Hermes is that you can use it on your phone, without being locked into some particular company's app. It can also access your memories from other sessions; I use hindsight for memory storage and rag. It can also be configured to shell coding tasks out to claude code or codex via the command line. The only issue I have with it is that there's no easy way to get it to run code on a remote machine. Well, okay, there is: it can ssh into another Linux machine, and I let it do that sometimes. But if I want to build and test on Windows, Hermes just can't help me. I'm looking into buzz for this purpose, but Buzz has other annoying accessibility issues.
1
1
1
0
User avatar
Josh @Orinks@mastodon.stickbear.me
2w
@fastfinge @matt Yeah, I just downloaded Buzz myself and am having trouble setting up the CLI bridges for Codex and Claude.
1
1
0
0
User avatar
🇨🇦Samuel Proulx🇨🇦 @fastfinge@interfree.ca
2w
@Orinks @matt It took me like two days to set up a relay. Getting Hermes into it was easy. But adding the other agents was super annoying. And then when you set up your own relay, notifications on the phone app don't work. If it's not self-hosted first, I'm just not interested. And it looks like buzz is only pretending.
1
0
0
0
User avatar
Josh @Orinks@mastodon.stickbear.me
2w
@fastfinge @matt Only pretending to be self-hosted? I don't recall it being self-hosted, just able to run local LLMs etc.
1
0
0
0
User avatar
🇨🇦Samuel Proulx🇨🇦 @fastfinge@interfree.ca
2w
@Orinks @matt If you don't run your own nostr relay, you depend on there servers and infrastructure for accounts, etc. You can set up your own relay though. They've just made it an annoying process so you don't.
0
0
0
0
User avatar
Alex Chapman @alexchapman@vee.seedy.cc
2w
@Orinks Yeah I decided to jump on the Hermes bandwagon, but I'm sticking with Codex and Claude Code for coding. @fastfinge @matt
0
0
0
0
User avatar
Bram Duvigneau @bram@bramd.nl
2w
@matt @fastfinge Would be nice to un it locally, but the Mac I have is a bit short on RAM for it and just an m1, the non-Macs here have quite some RAM, especially if I would delegate or cluster somehow, but I didn't find anything useful to run on CPU+IGP a hwile back. Also never did anything useful with the AMD NPU in my laptop. Given current hardware cost I decided it's not worth it to invest in anything right now, but that might change quickly if/when Claude and others force smaller users/consumers on a per-token cost as well
0
0
0
0
User avatar
Bram Duvigneau @bram@bramd.nl
2w
@matt @fastfinge Having a local capable model would be nice for more experimentation in which coding tool/harness to use and probably also to run more agentic non-coding workflows that don't really fit in with Claude now
1
0
0
0
User avatar
🇨🇦Samuel Proulx🇨🇦 @fastfinge@interfree.ca
2w
@bram @matt Yeah, the other reason that makes me reluctant to move all of my coding over to Hermes is memory contamination. I like that Hermes can remember and reference all of the random things hindsight has stored about me in the memory layer; it's useful when I ask it random questions at 2 AM, or want it to remind me of something, or describe an image for me, or whatever. But when I'm working on a codebase, the fact that I live in Ottawa or that I like Lit RPG sometimes gets injected into the context by the memory layer, and that is useless and actively harmful to the current task. Yeah, I could make a new Hermes profile per project, and then they'd each have different memory banks and everything else, but that's a lot of overhead. The way Claude Code and Codex handle it is much more sensible for programming tasks. But, of course, on the other hand, the fact that Hermes remembers my tooling and environment preferences is extremely useful when what I want is a quick one-off script. I don't have to remind it what I have installed, that I want uvx when possible, that we're on Debian, etc. If I start a new Claude or Codex project, they have to re-discover all that stuff every time, when all I needed was a quick 150 line script.
1
0
0
0
User avatar
Bram Duvigneau @bram@bramd.nl
2w
@fastfinge @matt Which model do you use to back Hermes and how do you find the costs? I have it set up on a VM on my homeserver and was plannign to use it for non-coding automation tasks/questions, which I now put in Claude Desktop cowork. Ideally I would interface my Meta glasses with it as well to have a proper, somewhat move privacy friendly AI with good context and MCPs while walking
1
0
0
0
User avatar
🇨🇦Samuel Proulx🇨🇦 @fastfinge@interfree.ca
2w
@bram @matt i run k3 locally. so no cost.
1
0
0
0
User avatar
Bram Duvigneau @bram@bramd.nl
2w
@fastfinge @matt I see K3 does vision as well, have you tried it for anything image related?
1
0
0
0
User avatar
🇨🇦Samuel Proulx🇨🇦 @fastfinge@interfree.ca
2w
@bram @matt No, mostly because Gemma does run on my laptop, is much faster, and does an acceptable job 99 percent of the time.
0
0
0
0