pull down to refresh

130 sats \ 2 replies \ @RDClark OP 16 Oct \ parent \ on: Local models.. lived experience AI

When Hermes starts up, it sends a massive wall of instructions to the AI all at once (around 4,500 tokens). Your graphics card tries to process that entire chunk in one giant bite, takes too long, and your computer thinks the graphics card froze—so it resets it (ErrorDeviceLost).

Once that happens, the AI engine crashes in the background. Attempts 2 and 3 (EOF) only fail because Hermes is still overwhelmed

To fix it, you just need to tell Ollama to take smaller bites (num_batch) and give the model a larger memory window (num_ctx) so it doesn't choke on startup.

write
compose
25 sats \ 1 reply \ @magnolia_mayhem 17 Oct

Is this under ~/.ollama or some other location?

reply
5 sats \ 0 replies \ @RDClark OP 17 Oct

You actually don't need to go into ~/.ollama or hunt for any folders at all:

  • For Step 1: Just open your terminal and paste those commands right where it opens. Ollama automatically saves the new model inside ~/.ollama behind the scenes for you.
  • For Step 2: Hermes keeps its settings in its own folder (~/.hermes/config.yaml), or you can just type hermes model in the terminal to switch to gemma4-hermes:12b without editing any files manually.
reply