Google Generative AI
The Google Generative AI integration adds a conversation agent powered by Google Generative AI
Controlling Home Assistant is done by providing the AI access to the Assist API of Home Assistant. You can control what devices and entities it can access from the exposed entities page. The AI is able to provide you information about your devices and control them.
This integration does not integrate with sentence triggers.
This integration requires an API key to use, which you can generate here
Configuration
To add the Google Generative AI service to your Home Assistant instance, use this My button:
If the above My button doesn’t work, you can also perform the following steps manually:
-
Browse to your Home Assistant instance.
-
In the bottom right corner, select the
Add Integration button. -
From the list, select Google Generative AI.
-
Follow the instructions on screen to complete the setup.
Generate an API Key
The Google Generative AI API key is used to authenticate requests to the Google Generative AI API. To generate an API key take the following steps:
- Visit the API Keys page
to retrieve the API key you’ll use to configure the integration.
On the same page, you can see your plan: free of charge if the associated Google Cloud project doesn’t have billing, or pay-as-you-go if the associated Google Cloud project has billing enabled.
Comparison of the plans is available at this pricing page
Options
Options for Google Generative AI can be set via the user interface, by taking the following steps:
- Browse to your Home Assistant instance.
- Go to Settings > Devices & Services.
- If multiple instances of Google Generative AI are configured, choose the instance you want to configure.
- Select the integration, then select Configure.
Instructions for the AI on how it should respond to your requests. It is written using Home Assistant Templating.
If the model is allowed to interact with Home Assistant. It can only control or provide information about entities that are exposed to it.
If you choose to not use the recommended settings, you can configure the following options:
Creativity allowed in the responses. Higher values produce a more random and varied response. A temperature of zero will be deterministic.
The maximum number of words or “tokens” that the AI model should generate.
Thresholds for different harmful categories
Talking to Super Mario
You can use an OpenAI Conversation integration to talk to Super Mario and, if you want, have him control devices in your home.
The tutorial is using OpenAI, but this could also be done with the Google Generative AI integration.
Actions
Generate content
This action isn’t tied to any integration entry, so it won’t use the model, prompt, or any of the other settings in your options. If you only want to pass text, you should use the conversation.process action.
Allows you to ask Gemini Pro or Gemini Pro Vision to generate content from a prompt consisting of text and optionally images. This action populates response data with the generated content.
| Data attribute | Optional | Description | Example |
|---|---|---|---|
prompt |
no | The prompt for generating the content. | Describe this image |
image_filename |
yes | File names for images to include in the prompt. | /tmp/image.jpg |
action: google_generative_ai_conversation.generate_content
data:
prompt: >-
Very briefly describe what you see in this image from my doorbell camera.
Your message needs to be short to fit in a phone notification. Don't
describe stationary objects or buildings.
image_filename: /tmp/doorbell_snapshot.jpg
response_variable: generated_content
The response data field text will contain the generated content.
Another example with multiple images:
action: google_generative_ai_conversation.generate_content
data:
prompt: >-
Briefly describe what happened in the following sequence of images
from my driveway camera.
image_filename:
- /tmp/driveway_snapshot1.jpg
- /tmp/driveway_snapshot2.jpg
- /tmp/driveway_snapshot3.jpg
- /tmp/driveway_snapshot4.jpg
response_variable: generated_content