Assist - Talk to your smart home with Home Assistant
Assist is the voice assistant built into Home Assistant. It lets you control your smart home with natural language, and it can run fully on your own hardware, so your voice commands stay private. Assist works in the Home Assistant Companion app, on dedicated voice hardware like the Home Assistant Voice Preview Edition, and on devices you build yourself with ESPHome.
Look for the Assist icon at the top right of your dashboard to try it out right away.
Assist is built on an open voice foundation and powered by knowledge contributed by our community. It can work locally or, if you prefer, use one of the latest large language models to handle more conversational requests.
Getting started
When you configure voice assistant hardware made for Home Assistant, it will use a wizard to help you configure your system and get started to use voice.
Our recommended voice assistant hardware is the Home Assistant Voice Preview Edition.
In case your hardware does not support our wizard, do not worry. Here are two detailed guides based on how you plan to process your voice (Locally, or using Home Assistant Cloud voice services)
- I plan to process my voice locally
- I plan to use Home Assistant Cloud (recommended as it is the simplest)
How Assist works
When you talk or type to Assist, an assistant handles your request. The assistant uses a conversation agent to understand the request and decide what to do.
Assistants
An assistant is a setup of Assist with its own name, language, and conversation agent. An assistant can also use speech-to-textTechnology that turns spoken words into text. Assist uses speech-to-text when you talk to it. Integrations provide speech-to-text. Some run locally, and others run in the cloud. [Learn more] to understand spoken requests, and text-to-speechTechnology that turns text into spoken words. Assist uses text-to-speech to reply out loud. Home Assistant can also use it to make announcements on media players and voice satellites. Integrations provide text-to-speech. Some run locally, and others run in the cloud. [Learn more] to reply out loud. For example, Home Assistant Cloud provides both. You can also run both locally, on your own hardware. You can set up several assistants under Settings > Voice assistants. For example, you can have one assistant in English and one in German.
Conversation agents
A conversation agent understands your request and decides what to do. The built-in conversation agent is called Home Assistant. It runs locally and understands the built-in sentences and your own custom sentences. For example, it understands “Turn on the kitchen lights”.
AI integrations add conversation agents that use AI. For example, OpenAI and Ollama add them. These conversation agents understand more freely worded requests, and they can use AI toolsA function that conversation agents and AI apps can use to read information, control entities, or manage Home Assistant. AI tools are grouped into sets. Each set provides different functionality. Some sets only work for administrators. [Learn more]. For example, with AI tools, a conversation agent can tell you the temperature in the living room. For the AI tools that Home Assistant provides, refer to the LLM integration.
If you turn on Prefer handling commands locally in the assistant, the built-in conversation agent tries each command first. Only commands that it doesn’t understand go to the AI conversation agent.
Voice satellites
A voice satellite is a device with a microphone that you use to talk to Assist. For example, Home Assistant Voice Preview Edition is a voice satellite. Other examples are analog phones and devices that you build yourself with ESPHome. For more options, refer to Expand and experiment. Many voice satellites start listening when you say a wake word, such as “Okay Nabu”.
Unlike a browser or the Home Assistant Companion appThe official Home Assistant app for Android and Apple devices. On phones, tablets, and Macs, it shows your Home Assistant interface, like a browser does. It can also share data from your device’s sensors with Home Assistant and receive notifications. The Home Assistant Companion app also works on smartwatches and in cars, with Android Auto, Android Automotive OS, or Apple CarPlay. [Learn more], a voice satellite isn’t signed in to a user account. So its assistant can’t use AI toolsA function that conversation agents and AI apps can use to read information, control entities, or manage Home Assistant. AI tools are grouped into sets. Each set provides different functionality. Some sets only work for administrators. [Learn more] that only work for administrators. For example, it can turn on a light, but it can’t read your logs. If the conversation agent of that assistant also has the Home Assistant set of AI tools selected, every request from the voice satellite fails. In that case, the voice satellite can’t turn on a light either.
Expand and experiment
Once your setup is up and running and you follow the best practices, check all the possibilities we found for Expanding your Assist setup, and further experiment with different setups like wake words. Do you want to talk to Super Mario? Or another figure? If you want Assist to respond in a fun way, you can create an assistant with an AI personality.
To further improve your setup, try building other voice assistant satellite devices that allow you to add Assist with wake words to all your rooms:
-
Enable wake word detection on your Android phone to activate Assist hands-free by saying “Hey Jarvis” or “Hey Nabu”, even when your phone is locked.
-
You can use ESPHome to create your own awesome voice assistant satellites based on inexpensive ESP32 microcontrollers, like @piitaya did with his 3D-printed R5 droid. Follow our tutorial to create your own for just $13.
-
Another alternative voice satellite solution is the experimental Linux-Voice-Assistant project that uses the ESPHome protocol. It allows you to build a Linux-based voice assistant smart speaker that runs on any x64 or ARM64 hardware capable of handling local, on-device audio processing. This approach provides greater flexibility for customization. Because it runs on a full Linux system, it also gives you access to significantly more local computing resources for additional features and other integrations on the same satellite. On HAOS, you can run it using the Assist Satellite App if your Home Assistant machine has a connected microphone and speaker.
-
If you are interested in a voice assistant that is not always listening, consider using Assist on an analog phone. It will only listen when you pick up the horn, and the responses are for your ears only. Follow our tutorial to create your own analog phone voice assistant.
Supported languages and sentences
Assist aims to support more languages than other voice assistants, but this is still a work in progress, and we need your help.
Assist already supports a wide range of languages. Use the built-in sentences to control entities and areas, or create your own sentences.
Did Assist not understand your sentence? Contribute them.
Assist was introduced in Home Assistant 2023.2.