Egregoros

@histoire3 @lain free GPUs and RAM too to run local models

@h4890 any mac with > 32gb ram, those new unified memory systems by AMD work too. or a graphics card with at least 24gb vram (this is much faster).

@lain have a favorite that can fit in 24gb yet? I feel like staying up a little late and trying one

@harblinger gemma 4 26b-a4b, qwen 3.6 27b and if it fits / you use a smaller quantization, qwen3.6-35b-a3b. they all run well in lm studio, if you don't already have a system for running them

@lain cool thx
I'll probably just do ollama + pi.dev https://docs.ollama.com/integrations/pi

@lain how do you run the local models on macos? curious to try it out

@rin lm studio is the simplest i think. for coding things can be a mixed bag because that relies heavily on tool calls and for some reason nobody manages to support that properly on day one, so you often have to wait a week for the bugs to be ironed out

@h4890 yeah, these small ones are really useable with normal hardware and they cna do real work.

there are now 3 open weight models that are reaching the quality of the closed ones (GLM-5.1, Kimi 2.6, Deepseek V4). There are two more that can write actually good code and small enough to run locally (Gemma 4, Qwen 3.6). This is not going away, no matter if OpenAI and Anthropic survive or not.

@lain ig geohotz side of the debate is winning lol

@lain I'm impressed, it was able to write a (better?) kdenlive "earthquake" plugin than I got out of an old claude model when I first tested code generation

Still couldn't get it to be keyframable, not sure why yet

@harblinger the models are really getting better and more efficient, it's not just more compute and larger models.