Awesome Tips AI in Home Assistant: Genius or Dangerous?
I built an AI that’s 100% local and very powerful! But is AI in Home Assistant a good thing?
I built an AI that’s 100% local and very powerful! But is AI in Home Assistant a good thing?
UGREEN HomeAgent HA100 (Reserve from $50, Save 50% with Free Gifts):
UGREEN HomeAgent on Kickstarter:
#UGREENHomeAgent #UGREEN #SmartHome
Thanks to UGREEN for sponsoring a portion of this video.
Link for MCP info:
Ollama:
Claude Code info:
Home Assistant LLM/model rankings:
Paul Hibbert’s AI video:
My smart home favorites:
My smart home shirts!
USA:
International:
FOLLOW ME
Subscribe to Smart Home Solver:
Subscribe to Reed’s Smart Home:
Instagram:
0:00 Intro
0:28 The 3 paths
1:48 Connecting the dots
3:30 Cloud AI
3:58 Local AI
5:19 UGREEN AI Box
7:06 Fixing automations
8:08 Creating dashboards
8:26 Next level AI
9:19 Improving local AI
11:59 Dangerous
14:10 Uh oh…
———
MUSIC
Ending song is The End by EVA.
Deals for Days. Big home savings are happening now.
How are you letting AI into your smart home?
UGREEN HomeAgent HA100 (Reserve from $50, Save 50% with Free Gifts): https://ai.us.ugreen.com/s6shWC
UGREEN HomeAgent on Kickstarter: https://bit.ly/3U2pJgs
I’m working on a local “agentic” (kind of) AI project that will use an LLM as a modular component alongside several other modules that are deterministic in nature to strip out the possibility of hallucinated commands. So I won’t be directly connecting Home Assistant to the model, it will connect through a “bridge” module where I can define defaults for the assistant and confine its outputs to only the entities I want it to have access to. If a command doesn’t slot in, the model has to try again. I still want to learn so I will ask the LLM how to fix something but not to fix it for me.
You missed an opportunity, Paul Hibbert's "it just works" would've fitted in perfectly with Steve Jobs and Bill Gates 🤣
With Reverse Engineer Anything (REA), you can literally make any device work with HA natively these days. 😉
Loved this video! Would also love to see a deeper dive on how you are utilizing AI with your voice assistant if possible! Cheers
Had AI integrate a pigeon deterrent system for my lawn. Cameras now hunt for pigeons and fire the sprinklers.
AI is significant in my home lab now. There is nothing I can really do without it but it saves me a loooooot of time for investigations, comparisons, setup, fine tuning configs, TESTING things (which I am really lazy to do by myself). Also it helps keep my infrastructure simple as I can get rid of complex visual tools like nodered in favour of more complex but built-in tools like HA automation themselves. I'm also investingating dedicated local hardware for voice assistant processing, since performance of smart home itself should not depend on cloud services for sure. But as a tool? Yes, great one.
I use AI. Lot…. Been feeding in my logs, it identifes errors and tells me how to fix them
Caught some automations I thought were working, but were half failing in the background
I can finally live life like the Jetsons! Oh hell yeah! I feel like a kid all over again
1) I had AI help me create a bunch of python scripts that allows me to run my old Raspberry Pi Zero connected into my Police Band radio. I can actually control the Police Band radio with Home Assistant now (and see what is on the screen). (My police band radio was able to be plugged into a PC and controlled with the PC, so I came up with a way to utilize the Raspberry Pi to control it instead and integrated that into Home Assistant.
2) I created my own MCP server with a bit more functionality than the native one (didn't know about the community maintained one). I had pretty much given it the ability to do everything OpenCode can do in Home Assistant, but outside of Home Assistant (when I created it, I was still using Raspberry Pi 4 to run Home Assistant, but I was maxing it out, so I decided to do the Opencode stuff outside of Home Assistant.
3) I used AI to help generate more complex and thorough Adguard Custom Filter lists. So when I browse a website, it can figure out which hosts are required to make the page work and add those to my custom filter so certain websites work. (While still keeping non-essentially telemetry stuff blocked).
One of the things that’s been particularly awesome for me is using Claude to assess logs and traces and to refine ways of doing things. For example, we have a new tankless hot water heater and I have temperature probes on the hot manifold and the reculation loop. The heater has a mode where you can trigger a button that starts the circulation to move to pre-warm the hot manifold. Now, instead of having that run on a very base level hours per day schedule, I can have it learn habits and build a very complex set of conditions under which the heater pre-warms so we get hot water when we need but don’t waste gas or water when we don’t. That kind of stuff would’ve taken Forever prior.
One thing that I’ve done in my instance of Claude That interfaces with HA is a very specific set of parameters around conditional approval for actions so far it’s yielded really good results. I’ve also had it tried to go through my current instance of HA and gather intent language so that any changes it makes for future automations, etc. are done so with similar intense to what I have had in my work thus far. As with most things, the prompt is everything.
Great video! I've been doing something similar with Home Assistant and AI. I built a driveway monitor using an LD2450 mmWave radar with AI to identify what's actually coming up the drive, plus an automated mowing flow that runs on its own. It's been really reliable and a lot of fun to build. Thanks for the inspiration, always enjoy your content!
I prefer the using the Opencode app in HA with my local model over an MCP integration. It may be a bit more limited, but at least I am in control of what tasks are given to the AI model and I can always turn it off easily when it's not in use.
Not at all. It's just not an added level of complexity that I'm interested in. All these companies coming out and saying "Look! It has AI now!" I figure it's probably going to go about as well as Alexa+ did. I would rather my devices do what they promised us they would do (which many still don't) and the companies aren't even bothering with that. Just slapping AI on everything as misdirection. It p***es me off.
I just set up HA Voice with Claude. I plan to get off it when I get a beefier machine – an xmas present to myself if you will. Currently, I can only use gemma4:12b locally, but there’s a lag and it’s on my laptop which isn’t usually on. I haven’t done anything crazy on it yet, but looking forward to tinkering. Only so many hours in the day… It’s wild to me that these local models are good enough. Why pay for something when you can get it for free? Not looking forward to that bubble burst…
Using local models is a poor choice
You didn’t use Paul it just woooorks
When it comes to dashboards, you still need to have some kind of vision you're trying to create. Once you do you can just iterate on it with the LLM, "add translucent color to this entry" or "set a background image" or "replace the buttons with bubble card buttons" and on and on. If you just tell it to create a dashboard you're going to get basic but workable. But of course, Claude in its most powerful models outclass the free local models. There are also skill files for HA out there, which gives the AI instructions on how to work on HA.
i never wwanted to use home assisant cuz it was to hard to understand and i have no patience for it ;.. today decided to connect grok to it and give him acces … he did everything … actualy insane … and kind of scary … but it fucking work ….
… Dangerous. EXTREMELY dangerous.
The only thing i'd use a local AI for is speech to text translation, then have the text match against commands that already exist. Which in theory should enable better natural language support i.e. you don't have to remember the exact commands yourself, it'll have some inference on what you want to do.
I wish a non-chinese, non-US company had made a thing like that Ugreen box, that would make it easier to believe that privacy aspect.
I’m so far down the AI with HASS rabbit hole I can’t find my way back out. It’s rabbit holes spawning from rabbit holes spawning from rabbit holes.
I have a bit of a reserved feeling about handing my system over to an application called MCP…🤪
(Dates back to 1982)
End of line …
why isn't there a connector yet?
What sent the 'water bowl is empty' notice I saw on your ipad?
I just watched a video on how local AI isn't always safe, and then video pops up right after 😂.
Not saying you're wrong, just that the security model around these local models isn't as air tight.
So I use Claude to use the HA REST API but also with SSH access (for debugging and doing niche stuff) on a dedicated VM. It runs 24/7 and Claude does A LOT of stuff for me. Making mods for games via ssh on my steam deck, making firmware for my ESP32 devices, diagnosing my WiFi issues. Most importantly, it manages my HA automations, sets up integrations, organises things, and makes dashboards. It's so nice. I talked to Claude about the MCP, but it said that the API was more efficient (Ironic I don't use the MCP considering I work for HA 😂)
It's so fun! I just don't have local AI just yet. I recommend everyone try it at least once
Ask the clou AI to write a prompt for the local AI.