How to Control Hardware with AI: MCP Servers, CLIs and APIs (2026)
TL;DR: To control hardware with AI, whether Claude or another agent, you give it one of three handles: an MCP server with typed tools, a CLI it can run in a shell, or a local HTTP or MQTT API it can call with a short script. Home Assistant covers most smart-home gear with no code, because it ships a first-party MCP server and first-party Anthropic and Ollama conversation agents. Before you buy, check for open firmware, a local API, no required cloud account, and Home Assistant support.
What does it mean to control hardware with AI?
The model never touches a GPIO pin. It sends a command through a control layer that already exists, and the device does the rest.
That layer comes in three shapes. The hardware underneath is ordinary: an ESP32, a Zigbee sensor, a LoRa radio, a relay. What makes it AI-controllable is a documented way in that accepts a command a model can produce.
| Pattern | How the agent uses it | Works with | Example devices |
|---|---|---|---|
| MCP server | The client lists the server's typed tools and the model calls one | Claude Desktop, Claude Code, Cursor | Home Assistant, Meshtastic radios, 3D printers |
| CLI | A shell-capable agent runs a command and reads the output | Claude Code, Cursor, any agent with a terminal | Meshtastic radios, ESPHome boards, MQTT devices |
| HTTP or MQTT API | The agent writes a curl call or a five-line script | Any model that can write code | WLED, Tasmota, Zigbee2MQTT, Shelly |
Pick by how you work. Want to chat with your house? MCP. Already living in a terminal with Claude Code? CLI. Writing an automation that runs while you sleep? API.
How does an MCP server let Claude control a device?
An MCP server publishes a list of tools. Each tool has a name, a description and typed parameters, something like send_message(text, channel). The client hands that list to the model. The model picks a tool, fills in the arguments, and the server does the work.
The model only sees the tools the server chooses to publish. That is the main reason to prefer MCP for anything you would rather an agent not improvise on.
Servers worth knowing:
- Home Assistant ships one in core. The
mcp_serverintegration (source) was merged on 2025-01-02 in PR 134122. - Meshtastic publishes an official server from the project's own GitHub org: meshtastic/meshtastic-mcp, also on PyPI.
- 3D printers have a community server that covers Moonraker, OctoPrint, Bambu and Prusa machines: mcp-3D-printer-server.
When we checked on 2026-10-20, the MCP projects for WLED, Philips Hue and Flipper Zero were tiny or had gone months without a commit. For those devices, skip MCP and use their own CLI or local API, covered below.
MCP shows up on the device side too. The Xiaozhi firmware for ESP32 voice boards calls itself "An MCP-based chatbot", and its backend can be self-hosted.
Example to adapt: connect Claude Desktop to Home Assistant
The Home Assistant MCP Server docs describe one Claude route, the remote connector. A second route comes from the community, not from those docs.
- Remote connector (official). In Claude Desktop, open Settings, then Connectors, then Add, then Add custom connector, and paste your Home Assistant address ending in
/api/mcp. Claude connects from the cloud, even from Claude Desktop, so the docs say your Home Assistant instance must be reachable from the internet. - Local gateway (community). For a client without remote-server support, the community ha-mcp guide runs
mcp-proxythroughuvx, and one guide pinsmcp<2.0.0to dodge an import error. The Home Assistant docs don't cover this route, so treat it as a community setup.
# Example to adapt: the endpoint an MCP client connects to
https://YOUR-HOME-ASSISTANT-ADDRESS/api/mcp
Check the docs page for the exact steps before you expose that address to the internet.
How does an AI agent use a CLI to control hardware?
Claude Code, Cursor's agent and similar tools run shell commands. Hand them a device with a command-line tool and they need no glue code. The agent runs --help, builds the command, reads the text that comes back, and adjusts.
CLIs an agent can drive today: meshtastic (Meshtastic radios), esphome (ESPHome boards), hass-cli (Home Assistant), mosquitto_pub (any MQTT topic), usbrelay (USB HID relay boards) and kasa (TP-Link Kasa plugs, via python-kasa).
# Example to adapt: send a text over a Meshtastic radio connected to this machine
meshtastic --sendtext "Back at camp by 6"
Check the current Meshtastic CLI docs for the flags that pick which radio to talk to. For hardware, browse our Meshtastic collection and confirm Meshtastic firmware support on the product page. New to the project? Read what is Meshtastic.
The same move works for ESPHome. For example, esphome run office-sensor.yaml compiles your config and flashes the board (check the current docs). Great, until the agent flashes the garage config onto the office board. Keep the approval prompt on for anything that writes firmware.
How does an agent control hardware through an HTTP or MQTT API?
Most open firmware listens on your local network. Send it an HTTP request or publish an MQTT message, and it acts. The agent writes a curl call, a short Python script, or an automation that runs later without it.
| Firmware | Interface | Shape of the call | Docs (check the current version) |
|---|---|---|---|
| WLED | JSON over HTTP | POST /json/state | kno.wled.ge |
| Tasmota | HTTP or MQTT | GET /cm?cmnd=Power%20On | tasmota.github.io |
| Zigbee2MQTT | MQTT | JSON published to the device's /set topic | zigbee2mqtt.io |
| Shelly Gen2+ | RPC over HTTP, WebSocket or MQTT | Named RPC method | shelly-api-docs |
# Example to adapt: turn a WLED controller on at half brightness
curl -X POST "http://192.168.1.50/json/state" \
-H "Content-Type: application/json" \
-d '{"on": true, "bri": 128}'
Swap in your controller's IP, and check the WLED JSON API docs for the current field names.
This pattern has no guardrails beyond the ones you write. A script with a bad loop will toggle that relay every second until you notice. Read the script before it runs, and turn on the API's password option if it has one.
Why is Home Assistant the hub for AI control?
Home Assistant turns Zigbee, Wi-Fi, ESPHome and MQTT devices into one set of entities, then exposes those entities to AI in two first-party ways.
Conversation agents (anthropic, ollama). The Anthropic and Ollama integrations plug into Assist, Home Assistant's voice and chat pipeline. Ollama keeps the model on hardware you run yourself.
MCP server (mcp_server). The developer path for Claude Desktop, Claude Code and Cursor. Per the docs, MCP clients reach Home Assistant through the Assist API. So the agent sees the entities you expose to Assist and nothing else. Expose the lights and the sensors. Think twice about the relay on the space heater.
Every Zigbee sensor, mmWave presence sensor, Wi-Fi relay and ESPHome board you add inherits both paths with no per-device code. So once a device from our smart home collection is in Home Assistant, an agent can reach it before you write a line of code. The mmWave presence sensor guide and the ESP32-CAM guide each take a device all the way into Home Assistant, ready for either path.
Want more than Assist exposes? The community ha-mcp server adds richer tools, and home-assistant-vibecode-agent is aimed at Claude Code and Cursor users. Home Assistant needs something to run on, and a Raspberry Pi from our dev boards collection is the usual start.
Can an AI agent control a cheap robot toy?
Usually not. Many cheap robot toys, dogs especially, are IR remote only. No Wi-Fi, no serial port, no firmware you can replace. An agent has nothing to talk to. That includes some toy-tier robots we sell. Fun toys, not agent targets.
If a robot is the goal, buy one with a documented SDK or command API. The clearest example on hobby hardware is Hugging Face's LeRobot, whose SmolVLA docs show a model doing pick-and-place with a real SO-101 arm.
Our smart robot dog guide sorts toys from programmable dogs. Check the control interface on any product in the robot kits collection before you buy.
What should you check before you buy AI-controllable hardware?
Five questions. If the product page cannot answer them, the seller probably cannot either.
- Does it run open firmware? ESPHome, Tasmota, WLED or Meshtastic means a documented interface and a community that keeps it working.
- Is there a local API? HTTP, MQTT, serial or a CLI that works on your own network, with docs you can read before checkout.
- Does it need a cloud account? If every command round-trips through a vendor server, your agent stops when that server does.
- Does Home Assistant support it? A working integration gets you both AI paths with no code.
- Is the MCP server alive? If you want MCP specifically, check the last commit. We treat 90 days without one as a warning sign.
Never flashed a board? Get a first win with one of our free kit plans before the agent gets involved. The easiest one needs only your phone and an NFC sticker.
What does "AI-ready" mean at Kindly Morrow?
An AI-ready label in this store is a dated checklist, never a certification. No third party tests it.
When a product here carries the label, it names three things: the firmware version, the date we tested, and the recipe we ran (the command, the MCP config or the Home Assistant setup). Firmware moves. A recipe that worked on one release can break on the next, which is why the date is part of the claim.
If any AI-ready label, ours included, has no date and no recipe, treat it as marketing.
Related guides: Getting started with ESP32, the Home Assistant compatible devices guide, and ESP32 vs Raspberry Pi if you are still picking a board.
Frequently Asked Questions
Can Claude control my smart home?
Yes, through Home Assistant. Add the Anthropic integration to talk to your house through Assist, or add the Model Context Protocol Server integration and connect Claude Desktop, Claude Code or Cursor. Either way, Claude only reaches the entities you expose to Assist. Devices outside Home Assistant need their own MCP server, CLI or API.
Do I need to write code to control hardware with AI?
Not for the Home Assistant path or a device with a CLI. Those need configuration, not code. For the HTTP or MQTT path, the agent writes the code, usually a curl call or a short script you read before it runs.
What is the difference between an MCP server and an API?
An API is the device's own interface, such as an HTTP endpoint or an MQTT topic. An MCP server wraps actions in typed tools with names and descriptions that an AI client discovers on its own, and it limits what the model can call. A plain API works with any language, model or script, but you supply the guardrails.
Can I use a local model instead of Claude?
Yes. Home Assistant ships a first-party Ollama integration, so a model on your own hardware can be the conversation agent. The CLI and API patterns do not care which model writes the command, as long as the agent can run a shell or make a network request.
Is it safe to let an AI agent control hardware?
It is as safe as what you expose. Start with sensors and lights, not relays. Keep the approval prompt on for commands that flash firmware or switch mains power. Prefer devices that work locally, so a cloud outage cannot break your setup. And keep a log, so you can find out why the porch light came on at 3 a.m.
What hardware should I start with?
Three good first builds: a Raspberry Pi running Home Assistant with a Zigbee coordinator and one sensor, an ESP32 board running ESPHome, or a Meshtastic radio you drive with the meshtastic CLI. Each has open firmware, a local interface and an AI path you can test the same day. Browse dev boards, smart home and Meshtastic for the parts.
Written by Kindly Morrow
Kindly Morrow picks every product in the store, builds with ESP32 and Home Assistant, and writes the guides.
Guides are drafted with AI and checked against datasheets and vendor docs before they go live.
Ready to try it?
Five free plans, from a sticker that puts your phone to sleep to a screen that shows what Claude is doing. Each one lists what else you need and what it can't do.