Skip to main content

Local AI smart home integration, free and open source

Local AI integration for Gladys Assistant

Run the Gladys assistant on a model at home: Ollama, LM Studio, llama.cpp, LocalAI, vLLM.

View on GitHubAll integrations

Run the Gladys assistant on a model at home instead of Gladys Plus: the chat, the scene creation, the ai.ask scene action and the weekly digest are answered by Ollama or any server that speaks the OpenAI API (LM Studio, llama.cpp, LocalAI, vLLM…). Conversations do not leave your network and no Gladys Plus account is needed.

Tested for real with Ollama 0.40.15 (CPU, small models). LM Studio, llama.cpp, LocalAI and vLLM are compatible through the OpenAI API but were not tested: feedback welcome.

What you get​

  • An AI provider for Gladys 5.3: once selected in Settings → Artificial Intelligence, every AI request of Gladys goes to your model, with its tools (device control, scenes…) and images.
  • A chat model and an optional small router model (the router sorts each message and must answer within 20 seconds).
  • With Ollama: the context window and the time a model stays in memory are set from Gladys, and the capabilities of the model are checked (a model that cannot call tools gets a clear error, not a made-up answer).
  • The connection status names the server, the models and the models in memory.
  • A dashboard widget, two scene triggers (failure, recovery) and two scene actions (load the model, free the memory).

Setting up the server​

Ollama on the machine that runs Gladys (the default: leave the address empty). Ollama only listens on 127.0.0.1, which the integrations cannot reach. Make it listen on the gateway of the Gladys integrations network, 172.30.0.1, which stays invisible from the rest of your network. On Linux:

sudo systemctl edit ollama
## add:
## [Unit]
## After=docker.service
## [Service]
## Environment="OLLAMA_HOST=172.30.0.1:11434"
sudo systemctl restart ollama
OLLAMA_HOST=172.30.0.1:11434 ollama pull qwen3:8b

The 172.30.0.1 address exists once Gladys has started an integration, hence After=docker.service.

Another machine (a PC with a graphics card): start Ollama with OLLAMA_HOST=0.0.0.0 and type http://<its address>:11434 in the address field. Ollama has no password: keep it on a trusted network.

Other servers, address to type (ending with /v1):

ServerAddressTo know
LM Studiohttp://<address>:1234/v1Developer tab, start the server, “Serve on Local Network”
llama.cpphttp://<address>:8080/v1llama-server --jinja -c 16384 --host 0.0.0.0 (tools need it)
LocalAIhttp://<address>:8080/v1API key if API_KEY is set
vLLMhttp://<address>:8000/v1--enable-auto-tool-choice and the tool parser of the model

With these servers, the context window is set on the server side (at least 16384 tokens).

Choosing a model​

The assistant needs a model that calls tools (list on ollama.com). The bigger the model, the better it creates scenes, and the more memory and time it takes. Indicative sizes with a 16k context:

MachineChat modelRouter model
Mini PC, 8 GB of RAM, no graphics cardqwen3:4b (about 4 GB, slow)qwen3:1.7b
16 GB of RAM, no graphics cardqwen3:8b (about 7 GB)qwen3:1.7b
Graphics card with 8 to 12 GBqwen3:8b, llama3.1:8bthe same model
Graphics card with 16 to 24 GB, Mac 32 GBqwen3:14b, gpt-oss:20bqwen3:4b

On a CPU, an 8B model answers in tens of seconds and loading it takes up to a minute: keep it in memory (setting below) and load it before the moments you talk to Gladys (scene action). A Raspberry Pi is too slow for the assistant.

Configuration​

  • Server address: empty for Ollama on the Gladys machine (http://172.30.0.1:11434/v1), otherwise the address of the server. /v1 is added to a bare address:port.
  • API key: only when the server asks for one.
  • Chat model: its exact name on the server (the List the models button shows them), e.g. qwen3:8b.
  • Router model: a small model for the router; empty = the chat model.
  • Context window (Ollama): 16384 by default. Creating a scene sends 10 to 15 thousand tokens; below the window, Ollama silently cuts the start of the request and the assistant loses its instructions. The logs warn when a request is larger than it.
  • Keep the model in memory (Ollama): 30 minutes by default, “Always” for instant answers.
  • Let the chat model think (Ollama): for qwen3, deepseek-r1… Off by default: much slower on a CPU.
  • Answer timeout: 110 seconds by default (Gladys waits 120). The router always has 18 seconds.

Then select Local AI in Settings → Artificial Intelligence (or with the button of the configuration page).

Actions​

  • Test the models: asks each model for a one-word answer and shows the time it took (the first one also loads the model).
  • List the models: shows the names of the models on the server, to copy into the fields.

Scenes​

Triggers:

  • Local AI: the assistant fails: server unreachable, model missing, too slow, not enough memory, model without tool calling… Once per failure, not for every message. Variables: kind, message, request (chat, router, digest), model.
  • Local AI: the assistant answers again: the first answer after a failure, with the minutes of failure.

Actions (Ollama only):

  • Local AI: load the model in memory: e.g. on waking up, so the first answer is quick. Output: the loading time.
  • Local AI: free the memory: unloads every model, e.g. at night. Output: the number of models unloaded.

Dashboard​

The Local AI widget shows the server, the chat and router models (green when in memory), the memory taken, the time of the last answer and the current failure, with two buttons (Ollama): load the model, free the memory. A setting hides the buttons.

Privacy​

Everything stays between Gladys and your server: the messages, the history, the device states read by the tools and the images of the chat. Nothing is written to the logs, at any level: only sizes and times. Ollama has no authentication: every integration installed in Gladys can reach it on 172.30.0.1.

Troubleshooting​

  • “Cannot reach the model server”: Ollama listens on 127.0.0.1 only (see above), or it started before Docker. From the Gladys machine, curl http://172.30.0.1:11434/api/version must answer.
  • “… is not on the server”: pull it (ollama pull <model>) or copy its exact name with List the models.
  • “… cannot call tools”: choose a model of the tools list.
  • “… did not answer in time” or slow answers: smaller model, “Always” in memory, router on a small model.
  • “Not enough memory”: smaller model or smaller context window.
  • The assistant ignores the house or invents devices: context window too small (the logs say so), or model too small to follow the instructions.
  • Images: a model without vision answers that it cannot see them (with Ollama); to describe camera images, choose a model with both vision and tools (qwen3-vl, mistral-small3.2…).

Configuration settings​

These are the settings Local AI asks for in its configuration screen in Gladys.

SettingTypeRequiredDescription
Getting startedsectionNoInstall Ollama on the machine that runs Gladys, make it listen on 172.30.0.1 (OLLAMA_HOST=172.30.0.1:11434), pull a model that calls tools (ollama pull qwen3:8b) and type its name below; leave the address empty. Then select this integration in Settings, Artificial Intelligence. Any OpenAI-compatible server works too: give its address.
Privacy and hardwaresectionNoConversations stay on your network. Ollama has no password: every installed integration can reach it on 172.30.0.1. A small model on a CPU answers slowly and creates scenes less reliably: 16 GB of RAM for an 8B model is a good start, a graphics card makes it much faster.
Server addressstringNoEmpty: Ollama on the machine of Gladys. Otherwise the OpenAI API address of the server, e.g. http://192.168.1.20:1234/v1 for LM Studio, :8080/v1 for llama.cpp.
API key (optional)secretNoOnly when the server asks for one (LocalAI, vLLM, a reverse proxy). Sent as a Bearer token, never logged.
Chat modelstringYesAnswers the chat, creates the scenes and writes the weekly digest: it must call tools. Its exact name on the server; the “List the models” button below shows them.
Router model (optional)stringNoSorts every chat message before the answer and must reply within 20 seconds: a small model fits (qwen3:1.7b). Empty: the chat model.
Context window (tokens, Ollama)numberNoCreating a scene sends 10 to 15 thousand tokens; below, Ollama silently cuts the start of the request. More takes more memory. Other servers set it on their side.
Keep the model in memory (Ollama)selectNoAfter the last request. Loading a model takes seconds to a minute: longer means faster first answers, but the memory stays taken.
Let the chat model think (Ollama)booleanNoFor models that reason (qwen3, deepseek-r1…): better scenes, but much slower answers on a CPU. The router never thinks.
Answer timeout (seconds)numberNoFor the chat model; Gladys waits 120 seconds at most. The router always has 18 seconds.

How to install Local AI in Gladys​

  1. In Gladys, open Integrations: Local AI appears in the catalog, next to the native integrations, with a community badge.
  2. Click Install. Gladys pulls the Docker image (ghcr.io/guim31/gladys-local-ai:1.0.0), starts it in a sandbox isolated from the core, and generates the integration's interface (devices, discovery and configuration).
  3. Open the Configuration screen of the integration, fill in the settings, and save.
  4. You can also install it directly from its repository URL: https://github.com/guim31/gladys-local-ai.

Local AI requires Gladys >=5.3.0. The catalog inside Gladys refreshes every hour, so a new version becomes available at most one hour after its release.

Not running Gladys yet? It is free and open source: follow the installation guide to get started.

About external integrations​

Local AI is an external integration: a community integration packaged as a Docker container and published on GitHub, that Gladys installs in one click and runs in a sandbox isolated from its core. It is published and maintained by guim31, not by the Gladys core team.

Subscribe to the Gladys Assistant newsletter

A few emails per month about new releases and project news. Sent by Pierre-Gilles Leymarie, founder of the project. Unsubscribe anytime 🙂
Last updated on