The problem Marco solves

Many older people live alone and struggle with small physical tasks: reaching a cup, picking up something they dropped, moving an object across the table. Assistive robots are expensive, and smart assistants can talk but can't act in the physical world.

Marco pairs a desk voice assistant with a ~$200 open-source robot arm. The person just talks, in plain words, and the arm moves. No app, no screen, no buttons to learn. A stop word ("stop", "wait", "no") always halts the arm, and the arm protects its own motors.

How it works:

1. SenseCAP Watcher (the ears): reflashed with open-source XiaoZhi firmware and a custom English wake word, "Marco". Its agent knows about the arm and turns speech into tool calls.
2. MCP bridge (wireless): the agent calls 8 arm tools over MCP (move, wrist, gripper, joint, poses, stop, free-text command). No cable between the Watcher and the arm, even on different networks.
3. SO-ARM101 arm (the hands): a local web panel drives the 6 servos by voice, big buttons or sliders, in English and Spanish. A wrist camera finds a cup, centers on it and fetches it ("bring me the cup").
4. Rokid glasses: [one line on what the glasses do]

Challenges I ran into

• Custom firmware on a consumer device: the stock Watcher firmware couldn't do a custom wake word or call tools, so we backed up the full flash and moved it to XiaoZhi (ESP-IDF) without touching the factory partition that holds its keys.
• Different networks: a local webhook couldn't reach the arm behind NAT. We first relayed through ntfy.sh, then switched to the agent's MCP endpoint so the arm pulls commands directly.
• Safety on real hardware: a hard-coded pose drove the arm into its base and the elbow stalled, heating to 50 °C. We added motor protection, a stop word that always works, and slider testing before any scripted motion.
• Voice timing: the first word after the wake word got cut, and the device kept listening after replies, so the arm side ignores text that isn't a command.