Create a personality with AI
You can give your voice assistant personality by using an AI conversation agent.
What can I do in Assist with AI?
With a conversation agentSoftware that Home Assistant uses to understand a request and decide what to do. Home Assistant has a built-in conversation agent. Integrations can add others, including ones that use AI. Conversation agents that use AI can also use AI tools. Each assistant uses a conversation agent. [Learn more] that uses AI, your assistantA configuration of Assist with its own name, language, and conversation agent. An assistant can also use speech-to-text to understand spoken commands and text-to-speech to reply out loud. can do the following:
- Answer in the style of a character that you describe in the Instructions of the conversation agent. For example, Super Mario.
- Understand requests that you word freely, not only the built-in sentences.
- Control the entitiesAn entity represents a sensor, actor, or function in Home Assistant. Entities are used to monitor physical properties or to control other entities. An entity is usually part of a device or a service. [Learn more] that you expose to Assist, run your exposed scriptsA script is a saved list of steps that runs when you start it, for example, from a dashboard, with Assist, or from an automation. Unlike an automation, a script has no triggers, so it doesn’t start by itself. [Learn more], and answer questions about them. For example, it can turn on the lights or tell you the temperature in the living room. For this, the conversation agent needs the Device control set of AI toolsA function that conversation agents and AI apps can use to read information, control entities, or manage Home Assistant. AI tools are grouped into sets. Each set provides different functionality. Some sets only work for administrators. [Learn more]. A new conversation agent has it selected by default. In Ollama, you select it yourself. The Ollama model must also support tools. For details, refer to the Ollama integration.
For the best results, follow the best practices. For example, give your areas and entities names that are easy to say.
Watch this 1-minute clip showing how Assist is using AI to control a smart home.
What AI integrations are available?
Many AI integrations add conversation agents that use AI. Some of them use a service in the cloud. For example, OpenAI, Anthropic, and Google Gemini. Others run a model on your own hardware. For example, Ollama. For the full list of AI integrations, refer to AI integrations.
AI models change often. To compare how well different models work with Home Assistant, refer to the Home Assistant datasets reports. For example, they show how well each model controls devices and answers questions.
Creating an assistant with an AI personality
Use this procedure to create an assistantA configuration of Assist with its own name, language, and conversation agent. An assistant can also use speech-to-text to understand spoken commands and text-to-speech to reply out loud. that answers as a character. For example, Super Mario. You set up a conversation agentSoftware that Home Assistant uses to understand a request and decide what to do. Home Assistant has a built-in conversation agent. Integrations can add others, including ones that use AI. Conversation agents that use AI can also use AI tools. Each assistant uses a conversation agent. [Learn more] that uses AI, describe the character in its instructions, and create an assistant that uses this conversation agent.
Prerequisites:
- An administrator account in Home Assistant.
- An AI integration that is set up, with a conversation agent. For example, Google Gemini. It has a free-of-charge plan. For the steps, refer to the documentation of your AI integration.
- Most AI integrations add a conversation agent when you set them up. For example, OpenAI adds OpenAI Conversation.
- For a model that runs on your own hardware, set up Ollama. Ollama doesn’t add a conversation agent by itself. On its integration page, select Add conversation agent, and select a model.
- The character settings belong to the conversation agent. Every assistant that uses this conversation agent answers as the character. If another assistant already uses it, add a separate conversation agent for the character: on the page of your AI integration, select Add conversation agent.
- To let the character control your devices: entitiesAn entity represents a sensor, actor, or function in Home Assistant. Entities are used to monitor physical properties or to control other entities. An entity is usually part of a device or a service. [Learn more] that are exposed to AssistThe voice assistant built into Home Assistant. You can use Assist by talking or typing in the Home Assistant web interface and in the Home Assistant Companion app. You can also use it by talking through a voice satellite. You can set up several assistants, each with its own language and conversation agent. [Learn more]. For the steps, refer to Exposing entities to Assist.
To create an assistant with an AI personality:
-
Go to Settings > Voice assistants, and select Add assistant.
-
Under Configuration, enter a Name for the assistant, and select the Language.
- For example, enter
Super Mario.
- For example, enter
-
Under Conversation agent, select the conversation agent of your AI integration.
-
Next to it, select Agent settings
, and describe the character. - In the Instructions field, describe how the character should answer. For example, enter
You are Super Mario from Mario Bros. Be funny. - Under AI tools, select Device control to let the character control your devices. For example, it can turn on the lights. To only talk to the character, clear all sets of AI toolsA function that conversation agents and AI apps can use to read information, control entities, or manage Home Assistant.
AI tools are grouped into sets. Each set provides different functionality. Some sets only work for administrators. [Learn more].
- Device control can only control entitiesAn entity represents a sensor, actor, or function in Home Assistant. Entities are used to monitor physical properties or to control other entities. An entity is usually part of a device or a service. [Learn more] that are exposed to Assist.
- Don’t select Home Assistant if voice satellitesA standalone device with a microphone that you use to talk to Assist. Some voice satellites also have a speaker to play Assist’s replies. Unlike a browser or the Home Assistant Companion app, a voice satellite isn’t signed in to a user account. This means Home Assistant can’t tell who is speaking. Each voice satellite uses at least one assistant. or users who aren’t administrators use this assistant. Only administrators can use the Home Assistant AI tools. Requests from voice satellites and from other users then fail, and Device control stops working too. To use the Home Assistant AI tools yourself, set up a separate assistant for them. For the steps, refer to Setting up a separate assistant for the Home Assistant AI tools.
- To see more options, clear Recommended model settings. For example, for the options of OpenAI, refer to Model.
- Select Submit. If the dialog shows more options, select Submit again.
- In the Instructions field, describe how the character should answer. For example, enter
-
Decide whether to keep Prefer handling commands locally turned on. It’s turned on by default.

- If you keep it turned on, the built-in Home Assistant conversation agent handles each command first. If it understands the command, it carries it out and answers right away. For example, it turns on the lights. The answer is fast, but it doesn’t have the personality. Commands that it doesn’t understand go to your AI conversation agent. Keep it turned on if you want fast answers, and you don’t mind that the character doesn’t always answer.
- If you turn it off, every command goes to your AI conversation agent. The answer takes longer, but the character always answers. For example, turn it off if your personality is Santa Claus and should never break character.
-
To talk to the character by voice, under Speech-to-text and Text-to-speech, select the engines that turn your voice into text and the answer into speech.
- Your AI conversation agent doesn’t turn speech into text or text into speech. The engines that you select here do that. For example, if you have a Home Assistant Cloud subscription, select Home Assistant Cloud.
- Under Text-to-speech, you can select a voice that fits the character.
-
Select Create.
-
To test the assistant, on the Assist card, select the three dots
menu of the new assistant, and then select Start conversation. -
Ask the character something. For example, ask
Who are you?- Result: The character answers in its own style.
If the conversation agent doesn’t answer, refer to the troubleshooting section of your AI integration. For example, refer to OpenAI troubleshooting.
Tutorial: Setting up Assist with OpenAI
Step-by-step tutorial with some background information, from the Home Assistant Release Party 2024.6 live stream.
Using the AI voice assistant on your devices
To learn how to use the AI assistant with your devices, refer to one of the following tutorials, depending on the hardware you want to use to interact with it:
Demos
Check this interview with an AI Mario personality